14

I found that if I transfer a great amount of data between two processes via pipe, some temporary file will be created by linux in /tmp directory. If the pipe operation succeeds, the corresponding temporary file will be removed by OS automatically. But if the operation failed, the tmp file remains there.

For some reason, I don't want the user have the oppotunity to get the data I transfered trhough pipe, so I don't want anything left on harddisk even if my program crashed. How can I do this?

5
  • 4
    I'm very doubtful that it is the OS creating these files, particularly, I doubt it is the pipe operation.
    – nb2580
    Dec 9, 2009 at 10:42
  • @Neil: Very good point. @OP: Are you sure the receiver isn't caching the data it receives on stdin to the tmp file? If it's not your own code and not open source, you can probably check by redirecting your output from your sender to a file, and then sending that into the receiver process as its input stream, e.g.: sender > filename then receiver < filename. I'd check for the tmp file during both ops, to see if either the sender or receiver is doing it.
    – T.J. Crowder
    Dec 9, 2009 at 10:47
  • 2
    Not an answer, but I found a lot of useful information on pipe handling here: <slacy.com/blog/2008/12/on-mkfifo-and-doing-the-impossible> . Don't know how authoritative this person is, but he specifically mentions that a mkfifo-created pipe never does any buffering (at all!) and never creates files. Dec 9, 2009 at 13:39
  • 1
    @Carl Smotricz: Link is broken, so here: slacy.com/blog/2008/12/on-mkfifo-and-doing-the-impossible
    – nalply
    Dec 9, 2009 at 14:29
  • Also, writing to a pipe will block if the pipe is too full (until someone reads from the other end).
    – Jason Orendorff
    Dec 10, 2009 at 18:19

3 Answers 3

13
  1. pipes don't store data on disk. /bin/echo foo | grep bar doesn't create any files. try strace -f sh -c '/bin/echo foo | grep bar' to see all the system calls made by a shell when running a pipeline. echo is a shell builtin, so I suggested /bin/echo to make the shell run an executable.

  2. /tmp doesn't have to be on disk. It can be mounted on tmpfs (i.e. backed by virtual memory). Note that a reboot will empty /tmp in that case, so use /var/tmp for anything you want to leave around.

If what you're doing is putting data into a file, then it's not using a pipe. If the file is a fifo, not a regular file, then it's just a named rendezvous, and doesn't contain data. Use ls -l to find out.

And note that if you're hoping to stop users from seeing what's going through pipes in processes they own, you are pretty much SOL, because strace can inspect everything a process does that interacts with anything outside the process, except for reading/writing mmapped shared memory. ltrace is even more invasive. If your program will run on systems where the local user has root, you can't stop them at all. On Unix, root can do anything, and has powerful tools for the purpose.

1

A true pipe is a block of memory in the kernel, a buffer that is read/written by some processes. It does not create files anywhere.

Some apps have options that switch between using pipes (faster, no hitting disk, takes a little more memory) and using temp files (takes a bit less memory, allows you to possibly see the temp files, a touch slower). gcc is one such application, though probably are others.

2
  • A true pipe .. does not create files anywhere. A question: suppose a program A written in C language writes a sequence of bytes to stdout, whereas a program B reads a sequence of bytes from stdin. Now let me execute the shell command A | B on two different terminals simulateously. So if no disc files are created, would a kernel need to have two separate blocks of memory for pipes in this case? How could then pipes be generalised to run, say, 10 commands of the form A | B at the same time? Oct 12, 2020 at 17:38
  • 1
    @mercury0114 the shell sees the pipe, calls the pipe(2) syscall, and hooks up A and B to the pipe in the kernel. the second time you run it, same thing happens, pipe(2) syscall, new buffer in the kernel, hook up the new version of A and new B to the pipe. Oct 15, 2020 at 21:41
0

Dirty hack: Encrypt the data before sending and decrypt it on receiving if you can change both of the processes ...

2
  • Wouldn't call it a dirty hack, actually: If the data is sensitive, seems an appropriate solution. But I'm curious about the tmp file. Is the OP right that the kernel is creating it? Or is Neil right and it's one end of the pipe or the other...
    – T.J. Crowder
    Dec 9, 2009 at 10:44
  • The kernel is not creating a temporary file. On the other hand, it's very likely that the receiving process is creating a temporary file. This is pretty common, since if you want to seek in your input you need to write it to a file first.
    – larsks
    Dec 11, 2009 at 2:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .