I have a computer on which I am dual booting Windows 7 and Gentoo Linux (both 64-bit). I want to be able to load up my Linux installation in a VM while I am booted into Windows. I have installed VirtualBox and followed the instructions for creating a raw disk VMDK. When I start the VM, Linux starts booting, but then fails with the following error when unlocking my root partition:

truecrypt[441] trap invalid opcode ip:373615538e0 sp:3dd0e0dfb60 error:0 in libpixman-1.so.0[373614d6000+8d000]

Everything works fine when I boot into Linux directly.

What could cause an illegal instruction to be hit in libpixman only when booting in VirtualBox?


As a troubleshooting step, I recompiled pixman without "-march", and no longer get an illegal instruction error in that library. (The boot fails in the same spot with the same error in a different library, however.)

How can I determine the specific opcode that isn't working in VirtualBox so I can disable it in my CFLAGS without having to disable all CPU-specific optimizations?

I am still confused as to why there would be any user-mode instruction that would fail to work in a VM. Is this a known limitation? My CPU is an Intel Core i7 3720QM, and I have hardware virtualization support enabled.

  • I think your best bet is TPM not being available in Virtualbox...
    – Paul
    Jun 19, 2015 at 17:00

1 Answer 1


It turned out the problem was that "-march=native" resulted in the compiled system binaries containing SSE4 and AVX instructions, which are not supported by the version of VirtualBox I was using. Explicitly disabling those in my compiler flags and recompiling allowed the system to boot properly under VirtualBox.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .