How much RAM does your host have and how much RAM is allocated to your guest in VirtualBox? For example, if your system has 4GB RAM, you will always need minimally 2GB RAM for your host to properly run the virtual machine.
Windows 10 is known for not being efficient with RAM. A lot of background tasks (a lot more than Windows 7) require a heavy portion of your RAM. Spreading your RAM evenly over your host and your guest disregards the fact that Windows 10 needs a lot more RAM. If you don't have enough RAM for your host, the VirtualBox application will run slower (which has nothing to do with your guest). Try to give your guest some less RAM (1GB if you have 4GB in total, 1GB or 2GB if you have 8GB in total, ...). You can do that at Settings->System and move the slider at base memory.
TL;DR: Decrease the allocated RAM for your guest in VirtualBox.