The link between the router and your PC depends, roughly speaking, by the signal-to-noise ratio. The stronger the signal (and the lower the noise), the higher frequency modulation can be adopted, and the more bandwidth becomes available.
Most WiFi routers/access points will try to maintain the highest data rate they can, even if this causes errors and renegotiations, which means that you just might squeeze more bandwidth out of a continuous 54 Mbps connection than from a shaky 104 Mbps or 300 Mbps connection that can't stay up. That is why sometimes lowering the top speed of the connection results in an overall improvement in data transfer rates.
In the speed test, the data is not downloaded to your router, but rather the packets get routed from the server to your PC, and the speed test server waits for your PC to respond. So, the total speed is the speed of the slowest component in this turnabout.
Exxaggerating the times, you get:
Server --- 1 ms --> Router --- 20 ms --> Your PC
Server <-- 1 ms --- Router <-- 20 ms --- Your PC
So the server "sees" the answer to its packets after 1+20+20+1 = 42 ms, and this is the speed it has to report, even if cable speed is twenty times higher.
If you had a proxy installed near your router
Server --- 1 ms --> Router --- Proxy --- Access Point --> 20 ms --> Your PC
Server <-- 1 ms <-- Router --- Proxy
then in some configurations (not all!), if a realtime answer from your PC is not necessary, as in most HTTP calls, then the server would "see" a lag of 2 ms only. You would still see your data after the usual 21 ms delay, though.
Applications that require an answer from your PC and no other, such as most online games, would still be affected by the WiFi lag.
For the same reason, "speed pings" to your router made from outside (e.g. from your ISP) would report a fast line: and indeed, the line they see - the 1ms trait - is fast.