Memory modules of different sizes but otherwise identical specs should have the same speed.
The system is another story. Because of the above, there's no way that that 4 8 GiB modules could be slower than 4 2 GiB modules. If your computer will be faster with 32 GiB of RAM than it would be with 8 GiB depends on whether your system will be using more than 8 GiB, i.e. the applications you will be running.
Most current consumer-level CPUs have a dual-channel memory architecture, meaning that memory modules should always be installed in pairs (2 or 4). As @David Schwartz pointed out, 2 modules should be slightly faster than 4 (although I don't think there would be a noticable difference, but using only two modules also leaves room for future upgrades.
Keeping all of this this in mind, 2 4 GiB modules make sense than 4 2 GiB modules. In addition, they're probably cheaper. The same goes for 2 8 GiB modules vs 4 4GiB modules.