12

I've been looking in /dev, and stumbled upon several files, including null. I wanted to see what was in the file, so I did hx null, but nothing happened and it gave the error File null is not a regular file. What is causing this message?

12
  • 109
    What is hx? Sep 28, 2019 at 14:12
  • 5
    Many answers describe what /dev/null is, but none explain why you can't read from it and get File null is not a regular file. . Anybody care to explain that bit?
    – Jeffrey
    Sep 28, 2019 at 15:34
  • 4
    @Jeffrey, my answer quoting the man page mentions "Reads from the null special file always return end of file". It is by design. However, I do not know if there exists more detailed explanation as to why it was designed this way.
    – VL-80
    Sep 28, 2019 at 15:57
  • 5
    @VL-80 I think the point of the comment was to elaborate more what the difference between a "regular" and a "special" file is.
    – MechMK1
    Sep 28, 2019 at 16:18
  • 11
    @user3840170 github.com/krpors/hx probably Sep 29, 2019 at 9:55

7 Answers 7

67

Several objects in /dev are pseudo-devices and are handled directly by kernel functions. The more frequently used ones are:

  • /dev/null: the universal sink of infinite capacity. Used to discard output: try echo foo >/dev/null. Reading from it returns an empty stream of bytes (immediate EOF).
  • /dev/zero: an infinite source of 0x00 bytes. Often used as input to overwrite things with 0s.
  • /dev/random, /dev/urandom: infinite sources of random bytes.
14
  • 49
    There is also /dev/full, which returns ENOSPC on write, useful for testing how programs behave when they run out of disk space Sep 28, 2019 at 18:46
  • 14
    /dev/random isn't continuous: "When the entropy pool is empty, reads from /dev/random will block until additional environmental noise is gathered." (man 4 random) Sep 28, 2019 at 22:36
  • 2
    There is also the null block device (/dev/nullb*) used for benchmarking the various block-layer implementations kernel.org/doc/Documentation/block/null_blk.txt
    – hojusaram
    Sep 29, 2019 at 10:48
  • 8
    I said "The more frequently used ones". For a complete list refer to the linked Wikipedia or your friendly OS manual.
    – xenoid
    Sep 29, 2019 at 13:21
  • 2
    @xenoid This is wrong, see 2uo.de/myths-about-urandom Sep 30, 2019 at 9:24
25
+100

Indeed none of the other answers seem to actually answer the question of why hx refuses to work with /dev/null. The reason why hx reacts this way is simply that it was programmed to output this error message for device files.

From https://github.com/krpors/hx/blob/develop/editor.c#L125:

if (!S_ISREG(statbuf.st_mode)) {
        fprintf(stderr, "File '%s' is not a regular file\n", filename);
        exit(1);

That means that hx specifically refuses to work on anything that isn't a regular file. I don't think there is a very good reason for this check - without it, I would expect that /dev/null would work with hx, in the sense that hx would read an empty file, and any updates to it would be lost when saving them.

(This assumes that https://github.com/krpors/hx/ is really the hx programm the poster talks about)

3
  • 1
    Here's the commit that adds the check: github.com/krpors/hx/commit/…. We don't have any explanation beyond this, but there's the possibility that krpors added the check to protect against opening directories, which is a much easier mistake to make (since directories are everywhere instead of just in dev) and probably did crash the editor when they tried it.
    – TheHans255
    Sep 30, 2019 at 15:26
  • @TheHansinator: some programs like less have a similar check to reject device files, because they are unlikely to contain ASCII text. But you can less < /dev/foo instead of less /dev/foo if you want; it only checks file names with stat, not its stdin. IDK what hx does, whether it uses fstat. Also this would appear to block reading from a pipe. But as a hex editor, it needs a seekable and writeable file; e.g. a regular file or some device files but not a pipe, socket, pty, or directory (or whatever else I'm forgetting that an FD can be open on). Some char devices don't seek Oct 1, 2019 at 12:35
  • Shielding against writing directories might (historically speaking) be a very good thing, as older unices often don't protect things from root, e.g. allow unlink on directories or writing to directory files (and indeed, mknod was often a setuid program that manipulated the dirtectory file directly). I think one can safely assume that the reason hx did it was to improve the "user interface experience" when users accidentally specify a device (or a directory). Oct 2, 2019 at 10:25
17

/dev/null is basically a way to discard information.

The main purpose is simply to be able to redirect things into oblivion.

echo 'duck' > /dev/null

will suppress the message given by echo 'duck' as it's sent to /dev/null and therefore discarded.

This is mainly used when using commands that give you output that you don't want to see.

Reading from /dev/null immediately returns end of file — i.e., it acts like an empty file.

There is other fun things in /dev like random that will give you random data; not really what you expect from a "file" :) Worth noting is that /dev/random is true random data collected from usage in the system (time between keystrokes and things like that), and that pool can be exhausted rather fast. /dev/urandom give you what seem to be random number but is calculated by a math formula. Usually /dev/urandom is fine for your need, but for for instance very strong crypto keys it's not good enough.

You can also read from /dev/zero to get an endless stream of zero (null) bytes.

9
  • also good to know that bytes from /dev/random will be numbers only if directly interpreted as numbers, those are not ascii numbers but bytes with random value. Sep 28, 2019 at 15:59
  • 1
    @Heart depends how you interpret those bytes, for example if you do cat /dev/random you will get "garbage" as you are interpreting bytes as characters and control codes. Sep 28, 2019 at 17:10
  • 2
    @VL-80 correct but as you quoted that part as reply i got impression that you maybe missed my point which was to clarify what was written to answer as average user might think that readin numbers from "file" means that numbers have textual representation. Sep 28, 2019 at 19:56
  • 9
    This repeats a common misconception about /dev/urandom (that it is somehow "insecure" or insufficiently secure).
    – Kevin
    Sep 29, 2019 at 19:59
  • 1
    @Kevin I think this is still up for debate redhat.com/en/blog/… 2019-06-05 this was published on RedHat blog. The kernel provides two character devices /dev/random and /dev/urandom. The /dev/random device is suitable for use when very high-quality randomness is desired (for example, for key generation or one-time pads), as it will only return a maximum of the number of bits of randomness (as estimated by the random number generator) contained in the entropy pool.
    – Griffin
    Sep 30, 2019 at 6:57
11

I would like to add to already existing answers that there exists a man page man null that describes the /dev/null and /dev/zero files.

Name

null, zero - data sink

Description

Data written to a null or zero special file is discarded.

Reads from the null special file always return end of file (i.e., read(2) returns 0), whereas reads from zero always return bytes containing zero (\0 characters).

null and zero are typically created by:

mknod -m 666 /dev/null c 1 3
mknod -m 666 /dev/zero c 1 5
chown root:root /dev/null /dev/zero

Files

/dev/null
/dev/zero

Notes

If these devices are not writable and readable for all users, many programs will act strangely.

Source: https://linux.die.net/man/4/null

8

/dev/null is a character device file, i.e. an interface to a device driver.

This specific device is a dummy device (not representing a real hardware). It is specially made to be a bottomless trash can; that you can dump any program's output into it, in case you don't want them to get on display or anywhere else.

  • Content-wise, if you actually tried to read this file (yes, you can read it), it will always be equivalent to an empty 0-byte length file.

I don't know what your hx command is, but I would guess from your description that it was a command to identify file type by its content. The command I use on my GNU/Linux system for this task is file, and it stops too, if it found that the file is not a regular file...

$ file /dev/null
/dev/null: character special

But file command also provide an -s option, which force it to read regardless...

$ file -s /dev/null
/dev/null: empty

Maybe your hx command has a similar option that you can use?

The reason that file (and maybe your hx command too) don't identify non-regular files by default is hinted in the documentation of -s option in file's manual page, namely...

  • Reading these files may cause side effects, side effects that are sometimes undesirable.
    • Reading from FIFO pipe consumes data in it permanently, you cannot push the data you have read back into the pipe.
    • Reading from device file causes the device to change...
      • Reading from tape device causes tape head position to shift.
      • Reading from serial port device causes buffered input bytes to be consumed.
      • Reading from random number device causes system's entropy pool to be depleted.
      • And many more, depending on driver of the device in question.
  • Identifying these special files by content requires much more work, with much less certainty:
    • Content size won't be known in advance (or needs special system-specific ioctl() calls to obtain).
    • Seeking won't work in many cases.
    • Detection of file types that relies on end-of-file signature will require reading through the entire thing.
    • Some of these files give infinite content.

In case your program is not designed to handle the works needed, then it might not be capable of identifying content of non-regular files; and aborted as a safety precaution.

  • Should the program proceeded without this safety check, it may result in a hang, a crash, or worse, cause the system to run out of memory; depending on the design of the program in question.

If this is the case, it is a limitation of your program; use file -s for identification purpose instead.


Addendum

If your hx is some kind of hex/binary editor; then it is especially NOT advisable to use it to open special files you don't know about. Hex editors usually preload entire file into the memory for your edit operations.

For this reason, if you try to make it open an infinite-content device file (like /dev/zero or /dev/urandom) or even a large finite device (like /dev/sda), it could drive your system into an out-of-memory condition, which you might need to reset the entire machine to recover.

The safety check you just tripped is likely there to keep you safe from that scenario. (As already mentioned: device file's content size is not known in advance— the program cannot determine beforehand whether you have enough memory to load it or not)

In case you just want to "peek" at the file's header no matter what; use hexadecimal viewer instead, like hexdump (recommended), or old-school od. For example:

  • hexdump -C -n 512 /dev/null (Content will be empty)
  • hexdump -C -n 512 /dev/urandom (Content will be a different gibberish every time)

^ The -n 512 option limits the view to the first 512 bytes. But even when you omitted this, and got bombarded with endless output, all you would need to do to stop it is just pressing Ctrl+C.

5

With hx I think you mean the hex editor of krpors, available on Github.

In its source file editor.c, lines 118 ff say:

struct stat statbuf;
if (stat(filename, &statbuf) == -1) {
    perror("Cannot stat file");
    exit(1);
}

// S_ISREG is a a POSIX macro to check whether the given st_mode denotes a
// regular file. See `man 2 stat'.
if (!S_ISREG(statbuf.st_mode)) {
    fprintf(stderr, "File '%s' is not a regular file\n", filename);
    exit(1);
}

So, it is pretty obvious that asserting if this is a regular file leads to this error message.

What does stat do? It retreives a structure that holds information about the said file, see man 2 stat. The invocation of stat on /dev/null succeeds, because the file exists and is accessible.

Why is the assertion that /dev/null is a regular file (S_ISREG()), false? Because /dev/null, the so-called Null Device, is a special pseudo-device file provided by a kernel driver. Everything written to it is discarded, and reading from it immediately returns end-of-file.

The directory /dev contains all the device files. Running ls -l also shows the Null Device as a special file by denoting it with a c as character device (as opposed to - for regular file, d for directory, l for soft link and others):

$ ls -l /dev/null
crw-rw-rw- 1 root root 1, 3  1. Okt 08:46 /dev/null
2
  • In terms of stat results, /dev/null is a character device. (As opposed to block device). You can tell with ls -l which shows a c in the first column, as opposed to d for directory or l for symlink. Oct 1, 2019 at 12:38
  • @PeterCordes I expanded my answer on it Oct 1, 2019 at 15:58
2

The files in dev are called "special files" - unlike a "regular file", which stores content, and a "directory", which stores references to other files, a "special file" is one such that reading and writing to it does something arbitrary, defined by the kernel.

For instance, the /dev/sda special file reads data directly from your primary hard drive, in the exact layout it's stored on that drive. The /dev/eth0 special file corresponds to your Ethernet port - writing bytes to the file sends data out of that port, and reading that file reads from the port. Some special files don't talk directly to a device, but do something much like it - for instance, there is a special file (usually called /dev/ttyN/, where N is some number) for your terminal window, where writing to the file writes to the screen and reading from the file reads from the keyboard.

And, as other answers have explained, some special files just provide utilities that are nice to have around. /dev/null is one such file - when you read from it, you get EOF, and when you write to it, nothing happens. There's also /dev/zero, where you always get a zero byte when you read from it, /dev/random, which reads as random bytes sourced from chaotic events in the system (like hard disk spin time), and a few others, depending on your particular OS.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .