55

How can I discard the last n lines of a file with a unix command line filter?

That would be sort of the opposite of tail: tail discards the first n lines but pipes the rest through, but I want the command to pipe everything through except the last n lines.

Unfortunately I haven't found anything like that - head doesnt help, too. EDIT: At least in Solaris it does not take negative arguments.

Update: I'm mostly interested in a solution that works for big files, i.e. logfiles, where you might want to inspect what happened except in the last minutes.

3
  • FYI when using head: By placing ‘-’ in front of the number with -n option, it prints all the lines of each file but not the last N lines as shown below, – G Koe Jan 30 '13 at 13:46
  • You can ask for whatever you want to ask for, but your last paragraph doesn’t make any sense to me. If “you want to inspect what happened in the last minutes”, tail is what you want. – Scott Mar 18 '20 at 3:18
  • @Scott Right: I got that backwards. Fixed it. – Hans-Peter Störr Mar 19 '20 at 18:12
60

If you have GNU head, you can use

head -n -5 file.txt

to print all but the last 5 lines of file.txt.

If head -n takes no negative arguments, try

head -n $(( $(wc -l file.txt | awk '{print $1}') - 5 )) file.txt
5
  • 16
    (and pray that file.txt is at least six lines long...) – user Jan 30 '13 at 14:59
  • 1
    Sadly this non-GNU version doesn't work with streams, either – Armand Apr 6 '16 at 10:58
  • 1
    @MichaelKjörling At least on ubuntu, that's not a problem. If the files has less lines than specified in head, an empty output is returned, with no errors. – Alphaaa Jun 2 '17 at 16:07
  • If I'm not mistaken head -n 5 will print the first 5 lines, not all but the last 5... – pypmannetjies May 17 '18 at 13:32
  • Doesn't work on Mac – Danon Mar 27 '20 at 17:14
20
head file.txt               # first 10 lines
tail file.txt               # last 10 lines
head -n 20 file.txt         # first 20 lines
tail -n 20 file.txt         # last 20 lines
head -20 file.txt           # first 20 lines
tail -20 file.txt           # last 20 lines
head -n -5 file.txt         # all lines except the 5 last
tail -n +5 file.txt         # all lines except the 4 first, starts at line 5
2
  • 2
    very good summary – ruanhao Aug 17 '17 at 3:31
  • A good summary of unhelpful information. Using “negative line numbers” (i.e., line numbers preceded by -) has been suggested multiple times, and the OP has said that it does not work in Solaris. – Scott Mar 15 '20 at 22:41
9

Here's a simple way to delete the last line, which works on BSD, etc.

sed '$d' input.txt

The expression reads "on the last line, delete it". The other lines will be printed, since that is sed's default behavior.

You could chain them together to remove multiple lines

sed '$d' input.txt | sed '$d' | sed '$d'

Which is a little heavy-handed, admittedly, but does only one scan through the file.

You can also take a look at this, for more answers: https://stackoverflow.com/questions/13380607/how-to-use-sed-to-remove-last-n-lines-of-a-file

Here's a one-liner adapted from one of my favorites there:

count=10
sed -n -e ':a' -e "1,$count"'!{P;N;D;};N;ba'

I had fun deciphering that one, and I hope you do, too (: It does buffer N lines as it scans, but otherwise is pretty efficient.

Note: the ! is quoted using single-quotes to avoid it being interpreted by Bash (if that's your shell). Plain double-quotes around the whole thing works for sh, possibly others.

2
  • For decades Unix users have been confused by man pages that said -n and weren’t clear whether they meant -n or whether they meant -<a number>. Your answer is similarly confusing, since it uses N as a variable name and also uses the N command. – Scott Mar 15 '20 at 22:35
  • @Scott: Fair enough; I guess my brain has been rewired to parse sh strings by this point, so it didn't occur to me (: I tweaked it. Part of me wanted to use the more explicit ${var}, but then the curly braces would probably similarly confuse. – jwd Mar 20 '20 at 23:50
6

On MAC the solution of getting all lines except the last N lines:

head -n -5 file.txt

will not work, as you will get the following error:

head: illegal line count -- -5

One of the solutions is installing coreutils and running ghead:

brew install coreutils
ghead -n -5 file.txt

The GNU Core Utilities or coreutils is a package of GNU software containing reimplementations for many of the basic tools, such as cat, ls, and rm, which are used on Unix-like operating systems.

2
  • 1
    Welcome on the SU! I suggest to write also some textual explanation, what your code is doing and how. – peterh Mar 15 '20 at 19:06
  • @Reinstate Monica - explanation added ;) – MikeL Mar 16 '20 at 7:50
3

I'm curious why you think head is not an option:

~$ man head
...
-n, --lines=[-]K
        print the first K lines instead of the first 10; 
        with the leading `-', print all but the last K lines of each file

This seems to fit your purpose, using, for example:

head -n -20 yourfile.txt
7
  • 6
    Note that this only applies to GNU head. BSD head doesn't have this option, so this answer will not work on Solaris or other Unixes without GNU coreutils. The OP specifically tagged this with Unix and Unix-Utils too. – slhck Jan 30 '13 at 14:06
  • 3
    @slhck Not to mention the fact that the OP mentioned that this is for Solaris. – user Jan 30 '13 at 15:00
  • Unfortunately someone removed my mention of Solaris. But I should have mentioned anyway that the version of head does not support that. – Hans-Peter Störr Jan 30 '13 at 16:47
  • 1
    Sorry all. Did not notice Solaris, nor was I aware of the various versions of head. – Anders R. Bystrup Jan 30 '13 at 18:00
  • 1
    @hstoerr Solaris is now in your tags :) – slhck Jan 30 '13 at 19:50
2

Another way to do it if tail -n won't take negative arguments is

tac file.txt | tail -n +6 | tac

This would remove the last 5 lines

6
  • Thanks! That's a nifty idea nobody came up with so far. Unfortunately that would be quite inefficient for the usecase I had in mind with this question: if that's a large file, it would not only be completely read through one or more times, as with the other solutions, but it would probably also be written to disk to temporary files by tac if it doesn't fit into memory. – Hans-Peter Störr Jan 6 '19 at 19:14
  • @Hans-Peter Very true. Decided to write a python3 script for it. Try this github.com/atw31337/donkey. I recommend using the output options. They run much faster than using redirects. – atw31337 Feb 16 '19 at 21:06
  • Nicely written! It does, however, read through the file twice, which isn't really necessary if you buffer the last n lines, and this is a problem on large files. Personally, I don't need that anymore, but in case you have fun improving it and other people need that... There were, after all, a few likes and bookmarks on the question. – Hans-Peter Störr Feb 19 '19 at 8:50
  • @Hans-Peter. The buffer size would be dependent on the number of lines to be removed. This could be an issue if a very large number of lines needed to be dropped from the file. In order to avoid memory related issues, I rewrote the script to use the line count method with high n values and a buffering method with lower n values; however, after testing it on a very large file, the original line count method is still faster. It seems the buffer management overhead outweighs the line count overhead...or I'm just missing something. – atw31337 Feb 26 '19 at 22:29
  • 1
    Nice, but for BSD-variant Mac OS X there is no tac command by default. :( This kind of defeats the use case. – ingyhere Apr 16 '19 at 4:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.