2

Most of the time whenever I post reasonably large chunks into an SSH terminal, it will absolutely crawl through it at a rate of something like 1000 characters every 20 seconds; sometimes practically stopping.

"Reasonably large chunks" isn't some massive set of data, it's often less than a 5000 character (php, ascii) script.

Why is this?

I see bash on the target server is thrashing, using 100% cpu. What's going on?

Edit Traceroute:

traceroute to ec2-23-24-25-26.compute-1.amazonaws.com (23.24.25.26), 64 hops max, 52 byte packets
 1  192.168.1.1 (192.168.1.1)  1.704 ms  1.040 ms  0.914 ms
 2  10.240.88.1 (10.240.88.1)  11.722 ms  12.449 ms  12.633 ms
 3  bmly-core-2b-ae4-1332.network.virginmedia.net (80.3.65.233)  10.545 ms  20.922 ms  10.216 ms
 4  brnt-bb-1b-ae6-0.network.virginmedia.net (213.105.159.229)  37.524 ms  13.632 ms  15.232 ms
 5  popl-bb-1a-as7-0.network.virginmedia.net (62.255.81.237)  11.881 ms  50.011 ms  12.091 ms
 6  popl-bb-1b-ae0-0.network.virginmedia.net (213.105.174.230)  15.561 ms  19.908 ms  39.367 ms
 7  tele-ic-5-ae0-0.network.virginmedia.net (213.105.159.117)  18.477 ms  42.429 ms  12.764 ms
 8  equinix01-iad2.amazon.com (206.223.115.68)  162.399 ms  96.645 ms  108.442 ms
 9  72.21.220.51 (72.21.220.51)  92.975 ms
    72.21.220.49 (72.21.220.49)  101.136 ms
    205.251.245.7 (205.251.245.7)  96.280 ms
10  72.21.222.149 (72.21.222.149)  97.333 ms
    72.21.222.141 (72.21.222.141)  104.084 ms  112.991 ms
11  216.182.224.214 (216.182.224.214)  98.459 ms
    216.182.224.207 (216.182.224.207)  105.273 ms
    216.182.224.55 (216.182.224.55)  108.202 ms
12  * * *
4
  • 1
    What's the connection type to the host? Can you do a traceroute?
    – ott--
    Sep 23, 2012 at 16:17
  • 1
    Why are you pasting PHP over SSH? Sep 23, 2012 at 16:19
  • Is this with any server, or just one? Sep 23, 2012 at 16:22
  • It's with all of my servers on EC2. I'm pasting PHP because I have an amazing <1 second deployment system for my main cluster, but their are a few miscellaneous machines that intermittently need updating, and I have yet to automate those.
    – Max
    Sep 23, 2012 at 16:36

1 Answer 1

2

This is typically a buffer that gets filled. You can compare this to traffic on the highway, if there drive less cars than the highway is designed it goes all fine. But if there drive more cars than the limit, the traffic on the highway slows down because the highway can't handle it. The CPU simply gets higher because it's constantly emptying and filling small parts of the buffer, causing a lot more cycles. The 100ms latency can quickly add up to making it several seconds to get all the data through the buffer.

SSH is designed for executing commands, so you might want to use another way to accomplish this or try to rethink what exactly you are doing. I don't see why you should be remotely editing PHP (that doesn't make sense nowadays) or why you would copy paste a file when you can just sent it...

2
  • I can buy such a slowdown being due to various poorly designed small buffers, but I can't believe it would thrash the cpu so much. This is a server (EC2 small) that can manage tens of thousands of messages per second through rabbitmq without strain. If I paste smaller chunks repeatedly it's fine, it's only larger chunks that slow it down seemingly factorially. I think something else is at fault, likely the feeding of data from the ssh process to the bash shell.
    – Max
    Sep 23, 2012 at 17:07
  • @Alec: Quite possible, it could even be on your client. We don't know which buffer would be having problem; although the CPU would indicate it being on the server side so indeed, the only other possibility is the connection between sshd and bash. Furthermore, I think your 100% CPU is rather a single core being saturated rather than the whole CPU; so it only slows down that thread / core while still being able to massively progress other things simultaneously. That's a nice about scheduling and multiple cores, if you have one program that goes out of hand it doesn't take down your server... Sep 23, 2012 at 19:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .