VirtualBox IS able to load from USB! However it is a bit tricky.
I was not able to boot from USB in BIOS mode, so enable the EFI mode!
Now add the wanted USB device to the config to automatically mount it on start
Now start machine
If the USB-drive is already detected, skip this step (the reset). However on some systems unfortunately right after start the USB-Stick might not be detected:
make sure that USB is connected:
If there is a general issue to mount an USB drive to a VM, see this post VirtualBox Error: USB device is busy with the previous request.
Now reset the machine (not power off) using the menu or hit Hostkey+R (ususally Hostkey is RightCTRL).
You might be lucky and the system is already booting from USB. If not you need to provide the boot file manually. To do this you should now see an additional drive (here FS0: - no other drives are attached to my VM). If there is a vdi attached (this is always detected, probably as FS0:), you might find a new like FS1:
First type the 'drive' to boot from, here
Next you need to specify the efi-file, you can also use 'ls' or 'cd' commands to browse the folders on the USB drive. Provide file name (usually located below 'EFI') e.g.
Hit Enter to boot from that EFI boot file.
It is possible to store this info permanently into the file startup.nsh, see here for a good explanation.
As an alternative type 'exit' + Enter to leave the shell and enter the EFI Menu. Now choose 'Boot Manager':
Select the drive to boot from. There are more options in the EFI settings like providing a path to the efi file manually. Unfortunately these settings are not stored permanently in NVRAM of the VM (VBox 7.06).
If you want to enter this menu directly, hit Esc or F2 immeditaly when the VM starts. The timing here is critical, it took me some attempts (resetting machine and try again).