I just had this problem and found the answer here:
Basically, openssh used keyboard-interactive to implement challenge-repsonse. So if either of these options are set to "yes", then keyboard-interactive gets set to "yes" in the code. You have to set both to "no" in order to get the behavior you want.
I had to do:
ssh -o PasswordAuthentication=no -o KbdInteractiveAuthentication=no -o ChallengeResponseAuthentication=no
Of course, the
BatchMode=yes setting would take care of all of these for you and future proof you against any new user interactive authentication methods in the future.