Here, the sed way
sed -E ':a;N;$!ba;s#a#Z#;s#a#Z#;s#a#Z#' alphabet
Since sed normaly works on lines, any command to sed will act only on 1 line at a time. To be able to replace only the first 3 occurances we need to first make the whole file a single selection on which we will do our 3 replacements. Otherwise we will do 3 replacements on each line.
:a
creates a label
N
appends next line into patterns space
$!
skips last newline
ba
branches to label a
We have now selected the whole file and will be acting on that space instead on one line at a time, do 3 replacements of "a" with "Z".
The above command will only work on GNU sed, more general but a bit uglier version that should work on non-GNU sed:
sed -e ':a' -e 'N' -e '$!ba' -e 's#a#Z#' -e 's#a#Z#' -e 's#a#Z#' alphabet
EDIT:
As suggested in the comments, adding version which uses g command to first replace all occurrences of 'a' with 'Z' and then replace all occurrences of 'Z' after 3th with 'a' again, which effectively leads to replacing only the first 3 occurrences of 'a'. This way you can change the last number to reflect the number of substitutions you need.
sed -e ':a;N;$!ba;s#a#Z#g;s#Z#a#g4' alphabet