If the previous key pressed was ESC, this is because there are keys that can generate escape sequences beginning with ESC O. For example depending on the cursor key mode, the up arrow key may generate the escape sequence ESC O A.
If it sees a partial escape sequence, vim will wait for a short timeout to see if another character is received that matches one of the escape sequences defined for your terminal type. If not it will treat the characters as separate keystrokes. You can test this by entering a capital A while waiting for the line to open to see if it treats it as an up arrow instead.
There are some vim settings that can be used to control this.
:set ttimeoutlen=100 will change the escape sequence timeout to 100ms.
:set noesckeys will completely disable recognition of keys sending an escape sequence when in insert mode. Also, vi-compatible mode will not have this issue because esckeys is off in that mode – that is why
vim -u NONE works.