Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

We have a couple of servers which are managed by a large group of admins. They usually log in as service users (say hudson) and then switch to root to make some small fix. This means we often can't map a change made to a person.

Does anyone have a script for Unix/Linux which can tell me which user logged in as root? Logins can be from all computers on the local LAN. Remote access from outside the LAN as root is not possible; admins must first login with a LAN user and can then promote themselves to root (they all use SSH).

What I would like is a script which follows the remote logins (in the local LAN) and print the user name for a certain time. You can assume that the script can login via ssh to any computer on the local LAN as root without being asked for a password.

Background: I have a script which saves backup copies of all files edited by root. The problem is to find out who really made the change.

Security is not an issue; this is not to find hackers which might have cleaned wtmp, it's to find out who made a mistake to give feedback.

[EDIT] Some pointers: The command last helps:

> last -t 20101029174200 root
root     pts/26       :0.0             Wed Oct 20 15:36 - 15:03  (23:27)    

wtmp begins Fri Oct  1 16:34:36 2010

So root was logged in via pts/26. Who else sat on that pseudo TTY?

> last -t 20101029174200 pts/26
adigulla pts/26       :0               Mon Oct 25 09:45   still logged in   
adigulla pts/26       :0               Fri Oct 22 14:00 - 17:29  (03:29)    
adigulla pts/26       :0               Thu Oct 21 15:04 - 16:05  (01:01)    
root     pts/26       :0.0             Wed Oct 20 15:36 - 15:03  (23:27)    
adigulla pts/26       :0.0             Fri Oct 15 15:57 - 15:57  (00:00)    

wtmp begins Fri Oct  1 16:34:36 2010

Hmm... must be me. So I can follow user changes on the local machine. If I log in to a remote machine:

$ last -1 hudson 
hudson   pts/0        192.168.0.51     Fri Oct 29 17:52   still logged in   

So I get the PTY and the IP address where I came from. How can I make the connection from the output of last for hudson to the user on 192.168.0.51?

[EDIT2] Please also note that we usually change user with ssh, not sudo or su. This allows for single sign on and avoids having to tell admins any passwords. If we want to grant/revoke access to something, we simply add/remove the public key from the service account. I also know that ssh logs to syslog but the messages don't tell me which user switched to root:

sshd[7460]: Accepted publickey for root from ::1 port 36689 ssh2
share|improve this question
3  
This is why you should prefer sudo (or a similar privilege elevator) to allowing root logins. –  dmckee Oct 21 '10 at 16:41
    
Sudo solves only the first half of my problem. –  Aaron Digulla Oct 26 '10 at 8:04
    
Then what is the second half of the problem? –  Arjan Oct 30 '10 at 15:57
    
Arjan: Who was the user on the client machine? (i.e. before he logged in as "hudson" via ssh on the server and then became root) –  Aaron Digulla Oct 31 '10 at 19:18

6 Answers 6

I think the best way to give feedback to admins who mistakenly logged in as root is to disable root logins whilst still allowing su and sudo. Alternatively have root's .profile display a suitable feedback message and optionally then exit.

If anyone other than an admin is mistakenly logging in as root, you definitely need to at least change root's password :-)


OK, you have revised the question to clarify what it is you want to achieve. You need to minimise the number of services that allow logins and ensure all of them records a successful login. SSH records successful logins via syslog.

If you have several people logging in as "hudson", you need to stop that. It should be a site policy that admins each log in using a unique userid.

You need to ensure that logfile rotation is adjusted so that you can search older logs for whatever period you fell necessary.

You could certainly have a script scheduled by cron that daily greps logins from syslog and which also checks timestamps on critical configuration files.

In addition there are plenty of tools for monitoring configuration file changes. Many Unix systems have an audit subsystem that can be enabled and which could also help.

Personally, I suspect that the best way to provide feedback in the event of a mistake is not to hunt for someone to pin blame on but to explain the problem to the whole group, explain why the change was a mistake and solicit ideas for preventing mistakes.

share|improve this answer
    
Unless they're using su or sudo to change to root from their login shell, all you've got to go in is remote host (if they ssh'ed in as root) or that they logged in as root from the console. –  Ian C. Oct 21 '10 at 17:33
    
@RedGrittyBrick: That's not the point. There are companies with 10-20 sys admins. If one of them makes a mistake and that is discovered three weeks later, how do you plan to give feedback? –  Aaron Digulla Oct 22 '10 at 10:07
    
@Aaron: Ahh you are talking about general sysadmin mistakes not just the mistake of logging in as root. Mistakes that go undiscovered until no one can remember making the mistake? Anyway I'd disable all root logins and make it a policy to use sudo. I'd change root's password and seal it in an envelope in the CIO's wall-safe. Do you have a formal change management system? –  RedGrittyBrick Oct 22 '10 at 12:11
    
Sudo just solves the problem locally: I know that the user "hudson" did the change but hudson is just a service, not a real user. –  Aaron Digulla Oct 25 '10 at 12:14
    
Then isn't your "hudson" service responsible for logging who invoked the change? –  RedGrittyBrick Oct 25 '10 at 13:49

you should be able to determine who is becoming root by parsing /var/log/auth.log.

for example, if i su to root on my own box, a line like this is entered in /var/log/auth.log:

Oct 29 20:10:33 localhost su: pam_unix(su:session): session opened for user root by (uid=1000)

now looking in /etc/passwd, we see:

brad:x:1000:100:brad clawsie,,,:/home/brad:/bin/zsh

which lets us correlate uid 1000 to a name. writing a perl script to do this should be very straightforward, you're simply joining the uid across /var/log/auth.log and /etc/passwd over a certain time range, namely the mtime for the file you are investigating.

of course if someone su's to root and simply executes the touch command on the file you are researching, they will set its mtime, and you will falsely believe it is they who have edited the file.

if i were to recommend a best practice to avoid your issues in the first place, it would be to store all scripts, crons, config files...everything as files in a version control system that must be edited by a real user, then applied to the live system with some sort of deployment tool (make, apt, whatever). if people keep breaking things, have their changed signed off by a more senior developer before deployment.

your backups might not help you if you failed to make a copy at the right time...a version control system lets you revert back over as many edits as you like.

any time files on live production systems are being edited in place by root, thats a major problem. i can't tell in your question if hudson is a real person on your system or a synthetic user for a service (i.e. the hudson service). if it is a synthetic user, i would ask why this uid gets a "real" login shell at all.

share|improve this answer
    
+1 Our first attempt was to put the config files directly under version control (i.e. you could do "svn up/ci" in /etc) but that meant all changes were done by root. +1 for the idea to copy the config files into a different place so users can edit them on their own machine and deploy them. –  Aaron Digulla Oct 30 '10 at 10:13

Rename su to log_su then put actual su (now its log_su) call inside a wrapper named su

#!/usr/bin/env bash
$SU_LOG=<path to log file>
echo "User: $USER running su at $(date)" >> $SU_LOG
log_su
share|improve this answer

Both the su and sudo commands can be parametered to keep logs.

Assuming that (1) going to root is not a frequent action and that (2) usually not more than one admin becomes root at the same time, and given any file-modification time-stamp, your script can simply search in the su/sudo logs to locate the admin who was root at that time.

If you are using ssh, then you can generate an authentication key for each administrator and make them use it while logging in, rather than by the password mechanism.

As described in How can I get the comment of the current authorized_keys ssh key, one can use the comment field in the key-file to identify the logged-in person.

The rest is the same: Using the file modification time-stamp against auth.log to identify the presumed ssh session that edited that file.

share|improve this answer
    
We all use ssh to change user because that allows single sign on. –  Aaron Digulla Oct 30 '10 at 10:09
    
I added some ideas for ssh. –  harrymc Oct 30 '10 at 10:35

Linux contains an advanced audit system, which one can set to track all changes to specific files.

From Understanding Linux Audit I see that it can be set up to also track SSH logins in addition to file changes, so it could be a possible solution to your problem.

share|improve this answer
    
Interesting approach but probably too complex for our setup. –  Aaron Digulla Nov 1 '10 at 16:15
    
From the article I got the impression that it is quite simple to use. See cyberciti.biz/tips/…. –  harrymc Nov 1 '10 at 16:32
    
The link in your answer is dead now. This is why you always also post the most important part of whatever you're linking to in your answer. –  Blacklight Shining Oct 7 '13 at 13:14

Please also note that we usually change user with ssh, not sudo or su. This allows for single sign on and avoids having to tell admins any passwords.

This seems rather ridiculous. You are willing to tolerate the resource-usage of ssh-ing into localhost (network buffers, tty-allocations, useless encryption and decryption, etc.), instead of configuring your sudo to simply not ask for passwords?

gandalf     ALL = NOPASSWD: ALL

If we want to grant/revoke access to something, we simply add/remove the public key from the service account.

You could also be removing the corresponding entry from /etc/sudoers instead.

I also know that ssh logs to syslog but the messages don't tell me which user switched to root

You could also check the ~/.history (or ~/.bash_history) of all the admins' for ssh-invocations around the times logged by sshd. This is easy enough for a moderately-determined nogoodnick to evade, of course, but so is anything for a person with root-access.

Still, using properly-configured sudo is a much better option. Modern versions even support /etc/sudoers.d/, so you can drop/remove (small) files into the subdirectory instead of editing the single (large) /etc/sudoers.

share|improve this answer
    
When I wrote this, sudo wasn't very common. Today, I'd say sudo is nice if you admin just a single machine and/or if you have to run just a single command. But in my case, 90% of the time, I need to do remote work. So I ssh and then I'd have to sudo. If you switch between local and remote all the time and you have to run many commands, I still prefer ssh (or one way to do everything). –  Aaron Digulla Oct 16 at 9:41
    
Actually, I'd prefer a "this user can execute this command with root permissions" command/tool (so your actions would still be logged under your user). I don't want to become root, I just need its permissions. Kind of "you can do anything you want but whoami still returns digulla. –  Aaron Digulla Oct 16 at 9:43
    
Yes, that's exactly the semantics, sudo offers -- locally. Remote logins directly as root are a bad idea and should be discouraged. The people needing and trusted with root access should login as themselves and perform something like exec sudo tcsh. Better yet is to bite the bullet and set up Kerberos -- then you can delegate permissions with ksu. –  Mikhail T. Oct 16 at 19:15
    
Hm :-/ I'm wondering what the security implications are when we create dozens of user accounts on our many servers. Feels like one of the security dilemmas: Comfort vs. security. If I use .authorized_keys, I have a single file (with a very simple format) but I can't easily find out who connected. sshsudo looks interesting but I wonder how sshpass does its magic. –  Aaron Digulla Oct 20 at 9:23
    
For large networks with large (and thus oft-changing) userbase, the only real solution is Kerberos. You can pick the original MIT implementation, or Heimdal, or Active Directory -- and they can interoperate, so pick what best integrates with your OS collection -- but there is nothing as comprehensive and as secure. It takes some effort to set up and learn (and teach personnel) -- as does any system. But you'll get the right solution... –  Mikhail T. Oct 21 at 12:32

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.