I am trying to get just the last 1024 bytes of /dev/sda2. When I do sudo tail -c 1024 /dev/sda2 | hd
, the prompt just hangs until I hit Ctrl-C. However, when I tail -c 1024 ddfilecopyofsda2 | hd
, I immediately get a nice output of the last 1024 bytes of the file. I read here (https://unix.stackexchange.com/questions/60034/what-are-character-special-and-block-special-files-in-a-unix-system) that "Block devices are normally seekable," so what am I missing?
1 Answer
Here's one way to get the last 1024 bytes of a block device:
last_bytes() { sudo dd if=$2 iflag=skip_bytes skip=$(($(sudo blockdev --getsize64 $2) - $1)) bs=1M ; } ; last_bytes 1024 DEVICE
Replace DEVICE
with the device path. In your case, you'd use /dev/sda2
.
Now for a more interesting question to answer…
Why does tail -c 1024 /dev/sda2
search the entire disk?
The reason is how tail
is implemented. When tail
knows the size of the file it's reading, it knows exactly how much to seek. Otherwise, it has to read the file or stream all the way to find out how far to count back.
With pipes, it makes sense, like cat /dev/sda2 | tail -c 1024
. tail
is receiving the contents as a stream and has no way of knowing when the data will end.
You might expect tail -c 1024 /dev/sda2
to be able to figure out the size of /dev/sda2
, but actually, when tail
looks up /dev/sda2
, it's opened as a block device instead of a regular file.
The implementation detail is that tail
calls fstat()
to get information about the file.
tail
on a regular file
Here's the relevant part of an strace
of an example of tail
opening a file:
21:30:27 open("/var/log/syslog", O_RDONLY) = 3
21:30:27 fstat(3, {st_dev=makedev(0, 22), st_ino=4715, st_mode=S_IFREG|0640, st_nlink=1, st_uid=104, st_gid=4, st_blksize=131072, st_blocks=54, st_size=175500, st_atime=2017/11/10-21:28:39.243133398, st_mtime=2017/11/10-21:30:20.438031639, st_ctime=2017/11/10-21:30:20.438031639}) = 0
21:30:27 lseek(3, 0, SEEK_CUR) = 0
21:30:27 lseek(3, 174476, SEEK_SET) = 174476
fstat()
provides st_size=175500
. Now tail
just needs to count back 1024 bytes:
175500 - 1024 = 174476
… and this is exactly what
tail
does:lseek(3, 174476, SEEK_SET) = 174476
tail
on a block device
fstat()
doesn't return the size this time!:
21:29:43 open("/dev/sda", O_RDONLY) = 3
21:29:43 fstat(3, {st_dev=makedev(0, 6), st_ino=17488, st_mode=S_IFBLK|0660, st_nlink=1, st_uid=0, st_gid=6, st_blksize=4096, st_blocks=0, st_rdev=makedev(8, 0), st_atime=2017/11/10-09:21:15.643998960, st_mtime=2017/11/10-09:21:15.555998962, st_ctime=2017/11/10-09:21:15.555998962}) = 0
With no st_size
, tail
cannot know how far to seek, so it defaults to reading through the whole block device until the end.
This is why you should generally use block device tools like dd
to manipulate block devices rather than tools intended for regular files like tail
.
You might ask, "How does blockdev --getsize64
quickly get the size of the block device?"
Here's sudo strace -vvvfts1000 blockdev --getsize64 /dev/sda
:
21:53:15 open("/dev/sda", O_RDONLY) = 3
21:53:15 ioctl(3, BLKGETSIZE64, [512110190592]) = 0
blockdev
is meant to get block device ioctls, and BLKGETSIZE64
gets the size of the block device.
As for why tail
doesn't do BLKGETSIZE64
, I don't know. The source code shows:
#define IS_TAILABLE_FILE_TYPE(Mode) \
(S_ISREG (Mode) || S_ISFIFO (Mode) || S_ISSOCK (Mode) || S_ISCHR (Mode))
I only know from that line that without S_ISBLK()
, the authors didn't mean for tail
to support block devices.
-
2I wish I could upvote this answer about 5 times! Thank you for such a detailed explanation! Nov 11, 2017 at 5:20
-
1Note GNU dd since 8.16 (2012-03-26) supports the
iflag=skip_bytes
option. This allows one to efficiently skip an arbitrary portion of the input while being independent of the block size used for I/O. One often wants to use a largebs
for efficiency. Nov 13, 2017 at 5:42 -
Thanks @pixelbeat! I've updated my answer to use
dd iflag=skip_bytes
instead.– DeltikNov 13, 2017 at 20:09