17
$ cat important_file > /dev/null &
[1] 9711
$ rm important_file 
$ killall -STOP cat

[1]+  Stopped                 cat important_file > /tmp/p
$ ls -l /proc/`pidof cat`/fd/
total 0
lrwx------ 1 vi vi 64 May 13 20:32 0 -> /dev/pts/29
l-wx------ 1 vi vi 64 May 13 20:32 1 -> /tmp/p
lrwx------ 1 vi vi 64 May 13 20:32 2 -> /dev/pts/29
lr-x------ 1 vi vi 64 May 13 20:32 3 -> /home/vi/important_file (deleted)

How to recover this important_file?

I tried something like

injcode -m dup2 -ofd=3 -ofilename=/tmp/recovered_file -oflags=O_CREAT $PID_OF_CAT

but it does nothing.

11

If /home is NFS, there will be a .nfsNNNNNNNNNN file in /home/vi that you can access/copy. If home is a local filesystem, you should be able to do the same thing via the /proc/PID/fd/3 link:

cp /proc/PID/fd/3 /tmp/recovered_file

If you want to actually undelete the file, here's a blog post on the subject.

  • 1
    OK, I was confused by that readlink /proc/13381/fd/3 -> "/home/vi/important_file (deleted)" and /home/vi/important_file\ \(deleted\) obviously does not exist. – Vi. May 14 '11 at 1:50
17

... better than copying at a given time (and gathering only that time's snapshot of the file's content) is to "tail -f" that file into a new file:

tail -c +0 -f /proc/PIDofProgram>/fd/# > /new/path/to/file

(thanks to tail's cautious programmers, that will even work with binary output.)

During its runtime, the tail -f itself keeps the file open, safely preventing it from being purged off disk when the original program ends. Thus, don't stop the tail -f immediately after your original program ends - check the tail'ed /new/path/to/file first whether it is what you want. If it isn't (or is unsatisfying for any other reason), you can copy the original file again, but this time after all writing to it has finished by "Program" and from the still-running tail -f's /proc/PIDoftail/fd/ directory.

  • 3
    What about creating a hardlink to /proc/PIDofProgram>/fd/# ? – becko Nov 14 '15 at 22:26
  • 1
    @becko Invalid cross-device link. – Kamil Maciorowski Apr 27 '17 at 14:18
9

Use lsof to find the inode number, and debugfs to recreate a hard link to it. For example:

# lsof -p 12345 | grep /var/log/messages
syslogd 12345 root    3w   REG                8,3    3000    987654 /var/log/messages (deleted)
# mount | grep var
/dev/sda2 on /var type ext3 (rw)
# debugfs -w /dev/sda2
debugfs: cd log
debugfs: ln <987654> tmp
debugfs: mi tmp
                      Mode    [0100600] 
                   User ID    [0] 
                  Group ID    [0] 
                      Size    [3181271] 
             Creation time    [1375916400] 
         Modification time    [1375916322] 
               Access time    [1375939901]
             Deletion time    [9601027] 0
                Link count    [0] 1
               Block count    [6232] 
                File flags    [0x0] 
...snip...
debugfs:  q
# mv /var/log/tmp /var/log/messages
# ls -al /var/log/messages
-rw------- 0 root root 3301 Aug  8 10:10 /var/log/messages

Before you complain, I faked the above transcript as I don't have a deleted file to hand right now ;-)

I use mi to reset the delete time and link count to sensible values (0 and 1 respectively), but it doesn't work properly - you can see the link count remains at zero in ls. I think the kernel might be caching the inode data. You should probably fsck at the earliest opportunity after using debugfs, to be on the safe side.

In my experience, you should create the link using a temporary file name and then rename to the proper name. Linking it directly to the original file name seems to cause directory corruption. YMMV!

  • Why exactly are you suggesting this if it doesn't really work properly and causes system damage? I think you should have a more vivid disclaimer within the answer that this is simply a WiP, and should not really be tried in production. – cnst Sep 4 '14 at 16:25
3

You can just cp the file, i.e:

cp /proc/<pid>/fd/<fdno> /new/path/to/file

Of course, if the file is still being modified, you'll run into trouble with this approach.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.