7

We have a couple of servers which are managed by a large group of admins. They usually log in as service users (say hudson) and then switch to root to make some small fix. This means we often can't map a change made to a person.

Does anyone have a script for Unix/Linux which can tell me which user logged in as root? Logins can be from all computers on the local LAN. Remote access from outside the LAN as root is not possible; admins must first login with a LAN user and can then promote themselves to root (they all use SSH).

What I would like is a script which follows the remote logins (in the local LAN) and print the user name for a certain time. You can assume that the script can login via ssh to any computer on the local LAN as root without being asked for a password.

Background: I have a script which saves backup copies of all files edited by root. The problem is to find out who really made the change.

Security is not an issue; this is not to find hackers which might have cleaned wtmp, it's to find out who made a mistake to give feedback.

[EDIT] Some pointers: The command last helps:

> last -t 20101029174200 root
root     pts/26       :0.0             Wed Oct 20 15:36 - 15:03  (23:27)    

wtmp begins Fri Oct  1 16:34:36 2010

So root was logged in via pts/26. Who else sat on that pseudo TTY?

> last -t 20101029174200 pts/26
adigulla pts/26       :0               Mon Oct 25 09:45   still logged in   
adigulla pts/26       :0               Fri Oct 22 14:00 - 17:29  (03:29)    
adigulla pts/26       :0               Thu Oct 21 15:04 - 16:05  (01:01)    
root     pts/26       :0.0             Wed Oct 20 15:36 - 15:03  (23:27)    
adigulla pts/26       :0.0             Fri Oct 15 15:57 - 15:57  (00:00)    

wtmp begins Fri Oct  1 16:34:36 2010

Hmm... must be me. So I can follow user changes on the local machine. If I log in to a remote machine:

$ last -1 hudson 
hudson   pts/0        192.168.0.51     Fri Oct 29 17:52   still logged in   

So I get the PTY and the IP address where I came from. How can I make the connection from the output of last for hudson to the user on 192.168.0.51?

[EDIT2] Please also note that we usually change user with ssh, not sudo or su. This allows for single sign on and avoids having to tell admins any passwords. If we want to grant/revoke access to something, we simply add/remove the public key from the service account. I also know that ssh logs to syslog but the messages don't tell me which user switched to root:

sshd[7460]: Accepted publickey for root from ::1 port 36689 ssh2
4
  • 5
    This is why you should prefer sudo (or a similar privilege elevator) to allowing root logins. Oct 21, 2010 at 16:41
  • Sudo solves only the first half of my problem. Oct 26, 2010 at 8:04
  • 1
    Then what is the second half of the problem?
    – Arjan
    Oct 30, 2010 at 15:57
  • Arjan: Who was the user on the client machine? (i.e. before he logged in as "hudson" via ssh on the server and then became root) Oct 31, 2010 at 19:18

8 Answers 8

4

you should be able to determine who is becoming root by parsing /var/log/auth.log.

for example, if i su to root on my own box, a line like this is entered in /var/log/auth.log:

Oct 29 20:10:33 localhost su: pam_unix(su:session): session opened for user root by (uid=1000)

now looking in /etc/passwd, we see:

brad:x:1000:100:brad clawsie,,,:/home/brad:/bin/zsh

which lets us correlate uid 1000 to a name. writing a perl script to do this should be very straightforward, you're simply joining the uid across /var/log/auth.log and /etc/passwd over a certain time range, namely the mtime for the file you are investigating.

of course if someone su's to root and simply executes the touch command on the file you are researching, they will set its mtime, and you will falsely believe it is they who have edited the file.

if i were to recommend a best practice to avoid your issues in the first place, it would be to store all scripts, crons, config files...everything as files in a version control system that must be edited by a real user, then applied to the live system with some sort of deployment tool (make, apt, whatever). if people keep breaking things, have their changed signed off by a more senior developer before deployment.

your backups might not help you if you failed to make a copy at the right time...a version control system lets you revert back over as many edits as you like.

any time files on live production systems are being edited in place by root, thats a major problem. i can't tell in your question if hudson is a real person on your system or a synthetic user for a service (i.e. the hudson service). if it is a synthetic user, i would ask why this uid gets a "real" login shell at all.

1
  • +1 Our first attempt was to put the config files directly under version control (i.e. you could do "svn up/ci" in /etc) but that meant all changes were done by root. +1 for the idea to copy the config files into a different place so users can edit them on their own machine and deploy them. Oct 30, 2010 at 10:13
3

I think the best way to give feedback to admins who mistakenly logged in as root is to disable root logins whilst still allowing su and sudo. Alternatively have root's .profile display a suitable feedback message and optionally then exit.

If anyone other than an admin is mistakenly logging in as root, you definitely need to at least change root's password :-)


OK, you have revised the question to clarify what it is you want to achieve. You need to minimise the number of services that allow logins and ensure all of them records a successful login. SSH records successful logins via syslog.

If you have several people logging in as "hudson", you need to stop that. It should be a site policy that admins each log in using a unique userid.

You need to ensure that logfile rotation is adjusted so that you can search older logs for whatever period you fell necessary.

You could certainly have a script scheduled by cron that daily greps logins from syslog and which also checks timestamps on critical configuration files.

In addition there are plenty of tools for monitoring configuration file changes. Many Unix systems have an audit subsystem that can be enabled and which could also help.

Personally, I suspect that the best way to provide feedback in the event of a mistake is not to hunt for someone to pin blame on but to explain the problem to the whole group, explain why the change was a mistake and solicit ideas for preventing mistakes.

7
  • Unless they're using su or sudo to change to root from their login shell, all you've got to go in is remote host (if they ssh'ed in as root) or that they logged in as root from the console.
    – Ian C.
    Oct 21, 2010 at 17:33
  • 1
    @RedGrittyBrick: That's not the point. There are companies with 10-20 sys admins. If one of them makes a mistake and that is discovered three weeks later, how do you plan to give feedback? Oct 22, 2010 at 10:07
  • @Aaron: Ahh you are talking about general sysadmin mistakes not just the mistake of logging in as root. Mistakes that go undiscovered until no one can remember making the mistake? Anyway I'd disable all root logins and make it a policy to use sudo. I'd change root's password and seal it in an envelope in the CIO's wall-safe. Do you have a formal change management system? Oct 22, 2010 at 12:11
  • Sudo just solves the problem locally: I know that the user "hudson" did the change but hudson is just a service, not a real user. Oct 25, 2010 at 12:14
  • Then isn't your "hudson" service responsible for logging who invoked the change? Oct 25, 2010 at 13:49
1

Rename su to log_su then put actual su (now its log_su) call inside a wrapper named su

#!/usr/bin/env bash
$SU_LOG=<path to log file>
echo "User: $USER running su at $(date)" >> $SU_LOG
log_su
1

I have made a little program that when used, will tell you which user has logged in as root if they have used sudo su, sudo bash, or sudo -i. Just know that I have been updating it quite a bit lately because I have been adding little things here and there to make it better and more efficient. But if you are interested in using it, just visit this link: https://github.com/StrangeRanger/monitor/tree/master/RLSpy. There is a README.md file explaining the purpose of the program and its features/faults. Just know that the script will only inform you of users who have logged into root within the past 7 days.

1

check /var/log/secure logs to find out which user loggedin/switch to root user - RHEL7

0

Both the su and sudo commands can be parametered to keep logs.

Assuming that (1) going to root is not a frequent action and that (2) usually not more than one admin becomes root at the same time, and given any file-modification time-stamp, your script can simply search in the su/sudo logs to locate the admin who was root at that time.

If you are using ssh, then you can generate an authentication key for each administrator and make them use it while logging in, rather than by the password mechanism.

As described in How can I get the comment of the current authorized_keys ssh key, one can use the comment field in the key-file to identify the logged-in person.

The rest is the same: Using the file modification time-stamp against auth.log to identify the presumed ssh session that edited that file.

2
  • We all use ssh to change user because that allows single sign on. Oct 30, 2010 at 10:09
  • I added some ideas for ssh.
    – harrymc
    Oct 30, 2010 at 10:35
0

Linux contains an advanced audit system, which one can set to track all changes to specific files.

From Understanding Linux Audit I see that it can be set up to also track SSH logins in addition to file changes, so it could be a possible solution to your problem.

3
  • Interesting approach but probably too complex for our setup. Nov 1, 2010 at 16:15
  • From the article I got the impression that it is quite simple to use. See cyberciti.biz/tips/….
    – harrymc
    Nov 1, 2010 at 16:32
  • The link in your answer is dead now. This is why you always also post the most important part of whatever you're linking to in your answer. Oct 7, 2013 at 13:14
0

Please also note that we usually change user with ssh, not sudo or su. This allows for single sign on and avoids having to tell admins any passwords.

This seems rather ridiculous. You are willing to tolerate the resource-usage of ssh-ing into localhost (network buffers, tty-allocations, useless encryption and decryption, etc.), instead of configuring your sudo to simply not ask for passwords?

gandalf     ALL = NOPASSWD: ALL

If we want to grant/revoke access to something, we simply add/remove the public key from the service account.

You could also be removing the corresponding entry from /etc/sudoers instead.

I also know that ssh logs to syslog but the messages don't tell me which user switched to root

You could also check the ~/.history (or ~/.bash_history) of all the admins' for ssh-invocations around the times logged by sshd. This is easy enough for a moderately-determined nogoodnick to evade, of course, but so is anything for a person with root-access.

Still, using properly-configured sudo is a much better option. Modern versions even support /etc/sudoers.d/, so you can drop/remove (small) files into the subdirectory instead of editing the single (large) /etc/sudoers.

5
  • When I wrote this, sudo wasn't very common. Today, I'd say sudo is nice if you admin just a single machine and/or if you have to run just a single command. But in my case, 90% of the time, I need to do remote work. So I ssh and then I'd have to sudo. If you switch between local and remote all the time and you have to run many commands, I still prefer ssh (or one way to do everything). Oct 16, 2014 at 9:41
  • Actually, I'd prefer a "this user can execute this command with root permissions" command/tool (so your actions would still be logged under your user). I don't want to become root, I just need its permissions. Kind of "you can do anything you want but whoami still returns digulla. Oct 16, 2014 at 9:43
  • Yes, that's exactly the semantics, sudo offers -- locally. Remote logins directly as root are a bad idea and should be discouraged. The people needing and trusted with root access should login as themselves and perform something like exec sudo tcsh. Better yet is to bite the bullet and set up Kerberos -- then you can delegate permissions with ksu.
    – Mikhail T.
    Oct 16, 2014 at 19:15
  • Hm :-/ I'm wondering what the security implications are when we create dozens of user accounts on our many servers. Feels like one of the security dilemmas: Comfort vs. security. If I use .authorized_keys, I have a single file (with a very simple format) but I can't easily find out who connected. sshsudo looks interesting but I wonder how sshpass does its magic. Oct 20, 2014 at 9:23
  • For large networks with large (and thus oft-changing) userbase, the only real solution is Kerberos. You can pick the original MIT implementation, or Heimdal, or Active Directory -- and they can interoperate, so pick what best integrates with your OS collection -- but there is nothing as comprehensive and as secure. It takes some effort to set up and learn (and teach personnel) -- as does any system. But you'll get the right solution...
    – Mikhail T.
    Oct 21, 2014 at 12:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .