0

I'm working on a new kind of proxy server using node.js. With cluster module I'm able to fork the process (which are double the number of cores). The server configuration is 8GB RAM, 8 core Intel (Xeon) processors and has bandwidth limit of 500 MB per month and it is Ubuntu Precise Machine. However not all the bandwidth we are able to use. What our users are using is only 30MB daily. We want our users to use full bandwidth, but they are not able to use and also even at 30 MB limit the speed is slow, because of node.js server is restarting every 10 minutes (average time) because of sockets hanging.

Sockets are hanging because of some uncaught exceptions in node.js and node.js tends not to respond and goes to infinite loop, then one of the cluster process dies and I see the count of clusters (everything is good at 16 processes), if the number of processes of node.js falls to 14, there is cronjob, which kills entire node.js and restarts the entire proxy script.

I have modified sysctl.conf and also limits.conf. Here are those files (sysctl.conf) http://pastebin.com/6Unvgayc and limits.conf http://pastebin.com/daLiSsYr, Please suggest me if I'm wrong in these settings. I'm new to bandwidth tuning of linux servers, please let me know, if I'm doing or missing anything

Thank You Sai

Improve this question
  • 1
    Your problem description is very vague. What is "sockets hanging" and how is that causing your node.js to restart? It sounds like you're trying to tune a broken system rather than fixing it. – David Schwartz Aug 4 '14 at 8:29
  • @DavidSchwartz, Sorry about that description, Ill edit it. Sockets are hanging because of some uncaught exceptions in node.js and node.js tends not to respond and goes to infinite loop, then one of the cluster process dies and I see the count of clusters (everything is good at 16 processes), if the number of processes of node.js falls to 14, there is cronjob, which kills the process and restarts the entire proxy script. – J Bourne Aug 4 '14 at 9:04
  • 2
    You talk about bandwidth, but all I see is volume. – Daniel B Aug 4 '14 at 9:10
  • @DanielB,Thank You for the response. What do you mean by volume? Can you elaborate please? – J Bourne Aug 4 '14 at 9:11
  • 2
    500 MB and 30 MB are both data volume specifications. Bandwidth is volume over time, aka Bits per Second. – Daniel B Aug 4 '14 at 9:25
1
+100

I suspect that there is a confusion of terms here, causing confusion about what you're trying to solve.

A Hosting Service Provider will usually offer a traffic quantity (often called bandwidth) per month. That might be 500MB, or 1TB, etc, depending on the plan offered. If that "bandwidth" is too low, you'll pay the hosting service for your excessive consumption of traffic. By and large, your goal as an app developer is to keep the amount of data sent and received, per user, down - because mostly, less data and the same or better user satisfaction and transactions means that you aren't sending excess data and incurring charges for the business (that's a very general rule and easily broken with counterexamples; the principle of not-sending unneeded extra data is good, though).

Users of the web service also need bandwidth, as more conventionally defined - the amount of data passed per unit interval, usually measured in bits per second. That's also usually subject to caps by your hosting service provider, because you're sharing with other people and because their connection to the internet will be limited, at some point. Within the hosting service (between machines) you might be looking at 1Gb/s (1,000,000,000 bits per second). Your hosting service might have a limit of 30MB/s (small hosting service, not close to the backbone). There is no clear relationship between this "instantaneous" bandwidth, the maximum burst rate that you send, and the monthly data usage (the monthly hosting bandwidth). You can burst data at high speeds, but only use a small quantity. OTOH, if your maximum speed is low, then you can't exceed the monthly bandwidth... But before that is a probklem, you'll have users complaining about performance.

And that, probably, is the real issue here. Performance. I suspect that the customers are complaining that things aren't fast enough. That leads to looking at bandwidth. But it is not the problem. The performance problem is that your servers are locking up. When a user makes a request, the server starts processing, locks up and then the client times out. The user sees bad service and you go optimise... what?

You've chosen to optimise the speed of data transmission. But that's probably not the problem. A failing service is the problem. Fix the application, and then the clients no longer timeout. It's not a matter of how much data, and how fast, as that hangs present an absolute bar to progressing the job, at any level of bandwidth.

Improve this answer
  • There is no clear relationship between this "instantaneous" bandwidth, the maximum burst rate that you send, and the monthly data usage (the monthly hosting bandwidth). You can burst data at high speeds, but only use a small quantity. OTOH, if your maximum speed is low, then you can't exceed the monthly bandwidth. This is a cool explanation, I never knew this fact...! Cool, thanks for explanation.. So even you see application errors are to be fixed...! – J Bourne Aug 13 '14 at 10:44
  • 1
    Yup, performance includes all the factors. Rendering time, caching, server response speed (or lack of response & timeout), instantaneous bandwidth (how many megabits per second you can serve at peak), server connections (e.g. HTTP 1.1 rather than 1.0), number of database lookups needed to present a page, security, etc. They all count towards the users' perceived performance. Optimising the speed of delivery now is probably most around appropriate caches, reducing file counts, and making sure the server is working properly - that's probably the one to focus on, from your own description. – JezC Aug 13 '14 at 11:09
1

If your monthly allowance is 500MB, and you're using 30MB per day, you are already exceeding your limit. 30MB x 30days = 900MB. That's not the problem here, anyway.

It appears the basic problem is the node.js application, and not the server. You shouldn't need to fork node.js, as any substantially blocking tasks should be split into an asynchronous task. You lose most of the benefits of running node.js over monolithic servers by using multiprocess as your main distribution mechanism. The main node.js event loop should always be able to receive connections, and exceptions should always be handled, or at least perform cleanup (in this case, closing sockets and returning to event loop).

In fact, if exceptions are causing any application to die, the problem is the application, and not the machine it's running on. If we are talking bandwidth throughput, an application hanging is going to severely limit the efficiency of the connections being processed.

Improve this answer
  • 1
    I think someone here is miscommunicating what the actual measure of bandwidth is. I don't mean it in a passive-aggressive way, either; someone on your staff or the service provider isn't using the correct terminology. Bandwidth is usually measured in bits per second. This is denoted with a lowercase 'b', which is distinct from an uppercase 'B' for bytes. They're also usually measured over seconds, rather than days or months. That's the root of confusion from most of the users here. I don't believe the metrics across the post are the same if you're saying the calculation is wrong. – joe Aug 12 '14 at 15:32
  • 1
    I should also point out that vm.swappiness changes in later kernel versions. Not sure if it's backported to Precise's 3.2 kernel, but setting '0' will disable swap completely, leading to the oom-killer possibly nuking a server process. You should change it to '1' in the interest of future-proofing. – joe Aug 12 '14 at 15:33
  • 1
    OK, so you're saying 30MB/s is the bandwidth at time of measuring. If your limit is 500MB/mo, then that calculation actually comes out to <18s of sustained transfer. You can see why I am extremely wary of the numbers and units of measure you are providing. They don't add up. – joe Aug 13 '14 at 14:58
  • 1
    As for vm.swappiness, if you don't know what a kernel option does, you probably shouldn't be setting it. Following guides blind will lead to situations such as the above, where the parameter has changed and you end up with unintended consequences. It controls how aggressively the kernel swaps memory. <3.5, it sets the kernel to swap as little as possible. In 3.5+, it tries to avoid swapping completely. – joe Aug 13 '14 at 15:01
  • 1
    As I mentioned in my first comment, 'B' and 'b' mean very different things. 1B=8b. 30MBps = 240Mbps. It still isn't that much: ~134s of sustained transfer. I think you are trying to compare two different metrics. Bandwidth, by definition, is a bitrate. Seems like Zabbix is reporting the bandwidth correctly: the rate of sustained data transfer. You are trying to compare that against the ISP's 'bandwidth' (a misnomer, really) limit, which is the volume of data transferred. Bandwidth isn't exactly the correct term, because it is not sustained data transfer. If it were, 500MB/mo = 1.5Kbps – joe Aug 13 '14 at 16:36
0

Bandwidth is a general term that can mean almost anything. I don't see why you would to use your bandwidth fully.

There are two types you could look for :

Amount of data transfer you can make to your site.

Amount of data that can pass at a certain time. Used in communications and networks. Having a larger bandwidth than needed allows you to smooth usage peaks.

If data is delayed or lost, your dataserver buffers could be clogged and won't accept any new request. They will be in a no-working state.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.