86

How can I discard the last n lines of a file with a unix command line filter?

That would be sort of the opposite of tail: tail discards the first n lines but pipes the rest through, but I want the command to pipe everything through except the last n lines.

Unfortunately I haven't found anything like that - head doesnt help, too. EDIT: At least in Solaris it does not take negative arguments.

Update: I'm mostly interested in a solution that works for big files, i.e. logfiles, where you might want to inspect what happened except in the last minutes.

3
  • FYI when using head: By placing ‘-’ in front of the number with -n option, it prints all the lines of each file but not the last N lines as shown below,
    – G Koe
    Jan 30, 2013 at 13:46
  • You can ask for whatever you want to ask for, but your last paragraph doesn’t make any sense to me. If “you want to inspect what happened in the last minutes”, tail is what you want. Mar 18, 2020 at 3:18
  • @Scott Right: I got that backwards. Fixed it. Mar 19, 2020 at 18:12

6 Answers 6

85

If you have GNU head, you can use

head -n -5 file.txt

to print all but the last 5 lines of file.txt.

If head -n takes no negative arguments, try

head -n $(( $(wc -l file.txt | awk '{print $1}') - 5 )) file.txt
5
  • 20
    (and pray that file.txt is at least six lines long...)
    – user
    Jan 30, 2013 at 14:59
  • 2
    Sadly this non-GNU version doesn't work with streams, either
    – Armand
    Apr 6, 2016 at 10:58
  • 1
    @MichaelKjörling At least on ubuntu, that's not a problem. If the files has less lines than specified in head, an empty output is returned, with no errors.
    – Alphaaa
    Jun 2, 2017 at 16:07
  • If I'm not mistaken head -n 5 will print the first 5 lines, not all but the last 5... May 17, 2018 at 13:32
  • 2
    Doesn't work on Mac
    – Danon
    Mar 27, 2020 at 17:14
35
head file.txt               # first 10 lines
tail file.txt               # last 10 lines
head -n 20 file.txt         # first 20 lines
tail -n 20 file.txt         # last 20 lines
head -20 file.txt           # first 20 lines
tail -20 file.txt           # last 20 lines
head -n -5 file.txt         # all lines except the 5 last
tail -n +5 file.txt         # all lines except the 4 first, starts at line 5
2
  • 4
    very good summary
    – ruanhao
    Aug 17, 2017 at 3:31
  • 3
    A good summary of unhelpful information. Using “negative line numbers” (i.e., line numbers preceded by -) has been suggested multiple times, and the OP has said that it does not work in Solaris. Mar 15, 2020 at 22:41
14

Here's a simple way to delete the last line, which works on BSD, etc.

sed '$d' input.txt

The expression reads "on the last line, delete it". The other lines will be printed, since that is sed's default behavior.

You could chain them together to remove multiple lines

sed '$d' input.txt | sed '$d' | sed '$d'

Which is a little heavy-handed, admittedly, but does only one scan through the file.

You can also take a look at this, for more answers: https://stackoverflow.com/questions/13380607/how-to-use-sed-to-remove-last-n-lines-of-a-file

Here's a one-liner adapted from one of my favorites there:

count=10
sed -n -e ':a' -e "1,$count"'!{P;N;D;};N;ba'

I had fun deciphering that one, and I hope you do, too (: It does buffer count lines as it scans, but otherwise is pretty efficient.

Note: the ! is quoted using single-quotes to avoid it being interpreted by Bash (if that's your shell). Plain double-quotes around the whole thing works for sh, possibly others.

2
  • For decades Unix users have been confused by man pages that said -n and weren’t clear whether they meant -n or whether they meant -<a number>. Your answer is similarly confusing, since it uses N as a variable name and also uses the N command. Mar 15, 2020 at 22:35
  • @Scott: Fair enough; I guess my brain has been rewired to parse sh strings by this point, so it didn't occur to me (: I tweaked it. Part of me wanted to use the more explicit ${var}, but then the curly braces would probably similarly confuse.
    – jwd
    Mar 20, 2020 at 23:50
12

On MAC the solution of getting all lines except the last N lines:

head -n -5 file.txt

will not work, as you will get the following error:

head: illegal line count -- -5

One of the solutions is installing coreutils and running ghead:

brew install coreutils
ghead -n -5 file.txt

The GNU Core Utilities or coreutils is a package of GNU software containing reimplementations for many of the basic tools, such as cat, ls, and rm, which are used on Unix-like operating systems.

2
  • 1
    Welcome on the SU! I suggest to write also some textual explanation, what your code is doing and how.
    – peterh
    Mar 15, 2020 at 19:06
  • @Reinstate Monica - explanation added ;)
    – MikeL
    Mar 16, 2020 at 7:50
4

I'm curious why you think head is not an option:

~$ man head
...
-n, --lines=[-]K
        print the first K lines instead of the first 10; 
        with the leading `-', print all but the last K lines of each file

This seems to fit your purpose, using, for example:

head -n -20 yourfile.txt
7
  • 8
    Note that this only applies to GNU head. BSD head doesn't have this option, so this answer will not work on Solaris or other Unixes without GNU coreutils. The OP specifically tagged this with Unix and Unix-Utils too.
    – slhck
    Jan 30, 2013 at 14:06
  • 3
    @slhck Not to mention the fact that the OP mentioned that this is for Solaris.
    – user
    Jan 30, 2013 at 15:00
  • Unfortunately someone removed my mention of Solaris. But I should have mentioned anyway that the version of head does not support that. Jan 30, 2013 at 16:47
  • 1
    Sorry all. Did not notice Solaris, nor was I aware of the various versions of head. Jan 30, 2013 at 18:00
  • 1
    @hstoerr Solaris is now in your tags :)
    – slhck
    Jan 30, 2013 at 19:50
3

Another way to do it if tail -n won't take negative arguments is

tac file.txt | tail -n +6 | tac

This would remove the last 5 lines

7
  • Thanks! That's a nifty idea nobody came up with so far. Unfortunately that would be quite inefficient for the usecase I had in mind with this question: if that's a large file, it would not only be completely read through one or more times, as with the other solutions, but it would probably also be written to disk to temporary files by tac if it doesn't fit into memory. Jan 6, 2019 at 19:14
  • @Hans-Peter Very true. Decided to write a python3 script for it. Try this github.com/atw31337/donkey. I recommend using the output options. They run much faster than using redirects.
    – atw31337
    Feb 16, 2019 at 21:06
  • Nicely written! It does, however, read through the file twice, which isn't really necessary if you buffer the last n lines, and this is a problem on large files. Personally, I don't need that anymore, but in case you have fun improving it and other people need that... There were, after all, a few likes and bookmarks on the question. Feb 19, 2019 at 8:50
  • @Hans-Peter. The buffer size would be dependent on the number of lines to be removed. This could be an issue if a very large number of lines needed to be dropped from the file. In order to avoid memory related issues, I rewrote the script to use the line count method with high n values and a buffering method with lower n values; however, after testing it on a very large file, the original line count method is still faster. It seems the buffer management overhead outweighs the line count overhead...or I'm just missing something.
    – atw31337
    Feb 26, 2019 at 22:29
  • 1
    Nice, but for BSD-variant Mac OS X there is no tac command by default. :( This kind of defeats the use case.
    – ingyhere
    Apr 16, 2019 at 4:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .