When data flows over the internet, both the upload and download are used.
If you download something, everytime a series of packages has been download, an upload package is sent back to confirm that the download was okay. If you upload something, everytime a series of packages has been uploaded the client waits to receive the confirmation package that it downloads.
Even with fast download speeds, it can happen that the upload channel is full. If that happens, the download stalls because sending that confirmation package is stalled.
Because of this principle, whenever your connection is measured (ping) the same thing applies. You send your ping to the remote server, the server responds and your client waits for the confirmation package.
What happens in your case, is that the upload channel is constantly full, causing congestion. Due to the congestion, the connection becomes slow and pings drastically increase from say 10-50ms to 300ms and more.
The best way to combat your issue is either limiting the connection so the upload and download limits are never reached, or setup a Quality of Service which prioritizes certain packages over others (and usually also maintains an overall bandwidth limit.
EDIT: That being said, it should be noted that ping itself is the time it takes from a package to go from your computer to another one. Even with slow speeds, if the computer is closeby, the ping will be low, if the computer is far away, even with fast speeds, the ping will be high. But in your case, when sometimes the connection has a low ping while other times it has a very high ping, this is definitely due to congestion.