1

Operating in a very separated IT-as-infrastructure/Engineering-as-Software environment.

I have a server that, after random periods of uptime, suffers from severe CPU speed degradation.

The server is a largely unmodified Apache webserver running on RedHat, using AMD Opteron(tm) Processor 6176 SE.

cat /proc/cpuinfo over various periods will report roughly the spec'ed hardware speeds (2300MHz) when the server first restarts. Over time, this reported speed will degrade to ~600MHz. This is reflected in multiple processes slowing down (rendering, interpreter times, content delivery, etc). We cannot even stress these CPUs back to their limits in targeted fashion.

We've requested that our IT organization look closely at the hardware to make sure there is no failure going on. Last time they looked they determined a backup PSU was bad, but that doesn't explain why the server would degrade over time, does it? Since the PSU was replaced, we're still seeing the behavior on this server. Now that we're requesting further inquiries, IT has resorted to canned messages of "We've investigated the server and can find no issues," even while the CPUs are operating <30% of their advertised speeds.

Is there something obvious I'm missing? At this point it feels like I need to escalate this failure to management but I'm searching for all other options. I'm definitely not a SysAdmin, and don't understand what could be happening that's crippling my resources so terribly.

Improve this question
7
  • Are you able to take this server off-line for a bit? I think you're on the right track (with it sounding like a hardware issue); maybe if you can run a live USB of another OS, just to see what happens to the CPU speeds. If I had to guess, it's a motherboard issue.
    – ajgringo619
    Oct 21, 2019 at 21:55
  • I'm in the middle of setting up some application containerization for redeployment. My IT organization has neglected this server for ~10 years. I was hoping for something definitive I could take, but with everything that's stacking up, a failure in management seems like the only answer. I am hardware-ignorant, and so I was hoping for some insight on the kind of timeline to expect in case the hardware is failing. It sounds like "ASAP" is the only answer, so far.
    – David Crane
    Oct 21, 2019 at 22:10
  • IT orgs exist for one reason: to keep systems running so that users can access their resources. If they are not willing to expend any more energy to solve your problems, then you need to escalate this as far as you need to.
    – ajgringo619
    Oct 21, 2019 at 22:20
  • What information is reported by cpupower frequency-info over time? (If the server's too old to have that, then cpufreq-info instead.)
    – u1686_grawity
    Oct 21, 2019 at 22:49
  • @ajgringo619 If only I could flag comments as answers. I'm a developer, not a sys-admin. I'm not equipped for this type of work and it's taking away from what I'm best at.
    – David Crane
    Oct 22, 2019 at 0:04

2 Answers 2

Reset to default
0

Is there a means of looking at the cpu temperature, or the server temperature? If the cpu is starting to overtemp, maybe it's slowing the clock rate down to cool off a bit. Is there a fan on the cpu that's maybe not running, or not running fast enough?

Improve this answer
2
  • I've started running a few tools regularly and aggregating them. I don't even know if these tools are reporting accurately from any real sensors or not, yet. Nothing has changed in a few hours even under load. I will report back as I learn more.
    – David Crane
    Oct 22, 2019 at 0:02
  • I've come to the conclusion that the server has suffered from extreme negligence. Even though it is not virtualized, all methods of gathering additional diagnostics data are incomplete, do not have supported hardware, or the hardware is malfunctioning. Escalation is the only answer, it seems.
    – David Crane
    Oct 22, 2019 at 17:33
0

@ajgringo619 had the most accurate answer: someone hasn't been, and still isn't, fulfilling the responsibilities listed in our operating agreement, and that this situation and its associated risks should be handled by management. ¯\_(ツ)_/¯

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .