1

I am tryting to cat and grep a huge 40 GB file. After some output, my process get killed.

Any idea what can be the issue? Using OSX

[1]    37392 broken pipe  cat output.txt |
       37393 killed       grep --color=auto -C 20 201804090004054066628236

I can confirm its because of grep memory usage. Its memory usage shoots up to almost in GB's

2

Looks like cat reported the broken pipe because grep was killed. This would typically happen if the processes consumed too much memory and the OOM (out of memory) killer was triggered. You could try without the useless cat:

grep --color=auto -C 20 201804090004054066628236 output.txt
  • 1
    grep doesn't store very much information in memory so the OOM seems somewhat unlikely, unless -C 20 collects lines which are on the order of gigabytes each. – tripleee Apr 13 '18 at 9:46
  • Yeah, but I can't see another reason why grep would be killed by something other than the user. – l0b0 Apr 13 '18 at 21:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.