I resolved this by checking the 'Skip unattended installation' option while creating the Ubuntu VM in VirtualBox. Using VirtualBox 7.0, the checkbox can be found on the first screen when creating a new VM. I have marked the checkbox in red in the image below. Judging from other Q&A regarding this issue, I'm guessing this works because it asks you your locale during the installation, while it doesn't if you choose for an unattended installation.
This doesn't really change anything about the installation process except that you have to enter credentials and other info while installing Ubuntu, instead of through the VirtualBox wizard before installing.