2

I have rented a Centos6 server and running an Apache Tomcat 7 server with an Apache httpd server proxying it.

Sometimes my server gets locked and I cannot access it via putty even ping.

My question is: how can I be sure that this is about connection problems or application servers (I currently have a hibernate problem that causes the app servers not to respond because of unneeded heavy queries)?

Thanks in advance.

(PS: Fixing the problem by restarting the server)

EDIT: Following are the applications and props on my server:

*Apache httpd server --> proxy, *Apache Tomcat server --> app server, *Mysql DBMS

And got hd images appr. 69000 (99GB) serving from one folder.

EDIT2: The system comes back normal when I restart machine and run the servers again.

  • Do the hosting company provide any remote management or 'lights out' facilities ? Is this a VPS or a physical server? – user3788685 Mar 11 '16 at 18:49
  • Yeah they suggested three things: 1) Exchange the server, but keep the drives --> just mac address config needed and server is down for 20-30 minutes 2) Exchange the server and exchange the drives --> data loss may occur 3) Running a complete hardware check --> takes 10 to 14 hours depending on the server configuration and 1 and 2 may occur And, I do want to learn if I must choose one of those three choices. If it is not certain that this is not a hardware problem then I will start to investigate the software part. – Bahadir Tasdemir Mar 11 '16 at 19:29
  • is this a physical or virtual server? & I presume you have root access – user3788685 Mar 11 '16 at 19:52
  • It is a virtual server hosted via a vendor, and yes I got root access. I thought It was a DDOS attack or something and set ip-tables and mod_evasive but nothing changed so it is not an attack. – Bahadir Tasdemir Mar 12 '16 at 13:03
  • it could be a number of things but I'll post you an answer with a few ideas & things you can do to monitor it. – user3788685 Mar 12 '16 at 15:42
1

The easiest approach is probably to set up a cron job that runs every minute and does something trivial that you can check later whether it happened or not at a specific time. For example, create a file /etc/cron.d/datelog with the contents:

* * * * * root date >> /var/log/date.log

Let this run for a while. The next time the VPS stops responding, wait a few minutes before rebooting it, then check /var/log/date.log once the VPS comes back up.

If that file has entries for the time when the VPS was unresponsive, you can be fairly sure it was running normally and for whatever reason was just not accessible over the network, so you can focus on investigating why the VPS became inaccessible over the network. If the file doesn't have entries for when the VPS was unresponsive, that does not conclusively prove that the VPS stopped or crashed, but it's a very strong indication that something of that sort happened and that you should investigate further based primarily on such a hypothesis.

This approach has the advantage of not requiring the installation or configuration of any additional software; a cron daemon is included in the standard installation on almost any distribution, and I doubt CentOS is any different in this regard, because it is useful for so many other things.

  • Thank you very much for your quickly handleable suggestion. I tried exactly what you mentioned and have seen that my server was not crashing. So I just demanded my service provider to "only migrate disks to another machine, not change any other things" and voila! The error just got disappeared! (y) – Bahadir Tasdemir Mar 17 '16 at 13:18
  • 1
    @bahadirT Glad to hear the problem was resolved! – a CVn Mar 17 '16 at 13:21
1

It could be a number of things.

First off you need to have a look in the system log files and see if there is anything of note. I'd start with /var/log/messages and look around the time you lost contact.

You have stated its a VPS - I presume you have no access to the physical underlying host or hypervisor?

It could be you are hitting a limit or a trigger of some sort and the hosting provider is suspending or killing your VM instance. You should check in the control panel - it should report the states of any VM's you may also find it offers a 'KVM' or console access to the VM's where you could try and access the instance when its in its 'broken' state.

I would try monitoring the instance from within using something like Icinga, Nagios or Cacti. They all have plugins to monitor a number of applications and systems. Install and setup to monitor your httpd, network connectivity, CPU, Memory etc. That way you will at least be able to see what state things where in before the machine became unreachable.

  • Thank you very much for your support. I just decided to use @Michaels's suggestion because it was more practical and resolved my the issue. I will have a look at to the tools that you provided in the future if they can help me on the other things (y). – Bahadir Tasdemir Mar 17 '16 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.