32

How can I discard the last n lines of a file with a unix command line filter?

That would be sort of the opposite of tail: tail discards the first n lines but pipes the rest through, but I want the command to pipe everything through except the last n lines.

Unfortunately I haven't found anything like that - head doesnt help, too. EDIT: At least in Solaris it does not take negative arguments.

Update: I'm mostly interested in a solution that works for big files, i.e. logfiles, where you might want to inspect what happened in the last minutes.

  • FYI when using head: By placing ‘-’ in front of the number with -n option, it prints all the lines of each file but not the last N lines as shown below, – G Koe Jan 30 '13 at 13:46
37

If you have GNU head, you can use

head -n -5 file.txt

to print all but the last 5 lines of file.txt.

If head -n takes no negative arguments, try

head -n $(( $(wc -l file.txt | awk '{print $1}') - 5 )) file.txt
  • 11
    (and pray that file.txt is at least six lines long...) – a CVn Jan 30 '13 at 14:59
  • 1
    Sadly this non-GNU version doesn't work with streams, either – Armand Apr 6 '16 at 10:58
  • 1
    @MichaelKjörling At least on ubuntu, that's not a problem. If the files has less lines than specified in head, an empty output is returned, with no errors. – Alphaaa Jun 2 '17 at 16:07
  • If I'm not mistaken head -n 5 will print the first 5 lines, not all but the last 5... – pypmannetjies May 17 '18 at 13:32
5

Here's a simple way to delete the last line, which works on BSD, etc.

sed '$d' input.txt

The expression reads "on the last line, delete it". The other lines will be printed, since that is sed's default behavior.

You could chain them together to remove multiple lines

sed '$d' input.txt | sed '$d' | sed '$d'

Which is a little heavy-handed, admittedly, but does only one scan through the file.

You can also take a look at this, for more answers: https://stackoverflow.com/questions/13380607/how-to-use-sed-to-remove-last-n-lines-of-a-file

Here's a one-liner adapted from one of my favorites there:

N=10
sed -n -e ':a' -e "1,$N!{P;N;D;};N;ba"

I had fun deciphering that one, and I hope you do, too (: It does buffer N lines as it scans, but otherwise is pretty efficient.

4
head file.txt               # first 10 lines
tail file.txt               # last 10 lines
head -n 20 file.txt         # first 20 lines
tail -n 20 file.txt         # last 20 lines
head -20 file.txt           # first 20 lines
tail -20 file.txt           # last 20 lines
head -n -5 file.txt         # all lines except the 5 last
tail -n +5 file.txt         # all lines except the 4 first, starts at line 5
  • 1
    What does this add that wasn't answered in the accepted answer? Also, as with your other answers, a few lines of explanation of your answer would greatly improve it. – music2myear Jan 20 '17 at 22:20
  • 1
    very good summary – ruanhao Aug 17 '17 at 3:31
2

I'm curious why you think head is not an option:

~$ man head
...
-n, --lines=[-]K
        print the first K lines instead of the first 10; 
        with the leading `-', print all but the last K lines of each file

This seems to fit your purpose, using, for example:

head -n -20 yourfile.txt
  • 5
    Note that this only applies to GNU head. BSD head doesn't have this option, so this answer will not work on Solaris or other Unixes without GNU coreutils. The OP specifically tagged this with Unix and Unix-Utils too. – slhck Jan 30 '13 at 14:06
  • 2
    @slhck Not to mention the fact that the OP mentioned that this is for Solaris. – a CVn Jan 30 '13 at 15:00
  • Unfortunately someone removed my mention of Solaris. But I should have mentioned anyway that the version of head does not support that. – Hans-Peter Störr Jan 30 '13 at 16:47
  • 1
    Sorry all. Did not notice Solaris, nor was I aware of the various versions of head. – Anders R. Bystrup Jan 30 '13 at 18:00
  • 1
    @hstoerr Solaris is now in your tags :) – slhck Jan 30 '13 at 19:50
0

Another way to do it if tail -n won't take negative arguments is

tac file.txt | tail -n +6 | tac

This would remove the last 5 lines

  • Thanks! That's a nifty idea nobody came up with so far. Unfortunately that would be quite inefficient for the usecase I had in mind with this question: if that's a large file, it would not only be completely read through one or more times, as with the other solutions, but it would probably also be written to disk to temporary files by tac if it doesn't fit into memory. – Hans-Peter Störr Jan 6 at 19:14
  • @Hans-Peter Very true. Decided to write a python3 script for it. Try this github.com/atw31337/donkey. I recommend using the output options. They run much faster than using redirects. – atw31337 Feb 16 at 21:06
  • Nicely written! It does, however, read through the file twice, which isn't really necessary if you buffer the last n lines, and this is a problem on large files. Personally, I don't need that anymore, but in case you have fun improving it and other people need that... There were, after all, a few likes and bookmarks on the question. – Hans-Peter Störr Feb 19 at 8:50
  • @Hans-Peter. The buffer size would be dependent on the number of lines to be removed. This could be an issue if a very large number of lines needed to be dropped from the file. In order to avoid memory related issues, I rewrote the script to use the line count method with high n values and a buffering method with lower n values; however, after testing it on a very large file, the original line count method is still faster. It seems the buffer management overhead outweighs the line count overhead...or I'm just missing something. – atw31337 Feb 26 at 22:29
  • Nice, but for BSD-variant Mac OS X there is no tac command by default. :( This kind of defeats the use case. – ingyhere Apr 16 at 4:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.