Bufferbloat can cause those symptoms. Run http://dslreports.com/speedtest from a desktop/laptop web browser to check your bufferbloat grade.
Bufferbloat is a widespread design flaw in routers and other networking equipment. It is when they are really dumb about managing their queue of unset packets during times when the next network link — usually your broadband Internet connection — is being fully utilized. They let their buffer/queue of packets just get longer and longer which only adds latency (lag, high ping time, sluggish response), and actually hides the congestion problem from TCP's Congestion Control algorithm, making things even worse.
The solution is known as Smart Queue Management (SQM). The best SQM algorithm at this time is Cake (FQ-CoDel is a close second). Unfortunately commercial router vendors are late to this game, so there are very few commercial products that support SQM in their factory firmware. But if your router is supported by an open source Linux-based router firmware distro like OpenWrt, you can load OpenWrt on your router and set up Cake SQM.
Because bufferbloat mostly affects latency and not bandwidth, it only delays the start of a big download or stream or speed test. Once a big download (or speedtest or stream) gets started, it goes at full bandwidth. This is why Ookla Speedtest.net's methodology of measuring latency with a quick ping on an idle network, and then measuring bandwidth, doesn't catch bufferbloat. To catch bufferbloat you must test latency DURING the bandwidth test, so that the network connection is fully utilized/congested during your latency test.
By the way, is your broadband Internet connection DOCSIS-based (that is, do you get your Internet service from a Cable TV service provider)? If so, beware that there are many DOCSIS modem/gateway boxes out there based on the known-bad Intel Puma 6 chipset that has bad latency problems. Check your model against the list at badmodems.com. If it's on the list, replace it with something else that you know for sure is not based on Intel Puma 6.