0

I was messing around with my system and found a novel way to use up memory, but it seems that the less command only holds a limited amount of data before stopping/killing the command.

To test, run (careful! uses lots of system memory very fast!)

$ cat /dev/zero | less

From my testing, it looks like the command is killed after less reaches 2.5 gigabytes of memory, but I can't find anything in the man page that suggests that it would limit it in such a way.

In addition, I couldn't find any documentation via the google on the subject.

Any light to this quite surprising discovery would be great!

System Information: Quad core intel i7, 8gb ram.

$ uname -a
Linux Tyler-Work 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

$ less --version
less 458 (GNU regular expressions)
Copyright (C) 1984-2012 Mark Nudelman

less comes with NO WARRANTY, to the extent permitted by law.
For information about the terms of redistribution,
see the file named README in the less distribution.
Homepage: http://www.greenwoodsoftware.com/less

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04 LTS 
Release:    14.04
Codename:   trusty

Update:

The less process sticks around after reaching the 2.5 GB usage of ram, but it stops gathering input from the process output piped in. Checking return values indicates that the cat process is being killed by a signal from less.

$ cat /dev/zero | less; echo "${PIPESTATUS[0]} ${PIPESTATUS[1]}"
141 0

Exit status 141 suggests that less is killing the cat process via a SIGPIPE (https://stackoverflow.com/questions/19120263/why-exit-code-141-with-grep-q#answer-19120674)

Gathering information from the executable itself yeilds that the less binary is 64bits, so it shouldn't be limited in the amount of ram it can use:

$ file /bin/less
/bin/less: ELF 64-bit LSB  executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=539cf624699477e3e069b6c4e4b33842f22be2d2, stripped

Which doesn't explain why less is sending a kill signal to the cat process as this specific amount of data.

  • Technically less isn't sending the signal, the kernel is once it sees that the only reader to the pipe (less) is no more. – Rich Homolka Aug 20 '14 at 2:29
  • @RichHomolka Yeah, but the less process hangs around afterward, as evidenced by the : prompt, after which pressing q causes less to quit as normal. – Tyzoid Aug 20 '14 at 2:35
1

There could be a few things, the source code of less would give you more insight on what's happening.

Is less a 32 bit app? If so, then, depending on your kernel config, you may have 3gb, 3.5 gb, or (just under) 4 gb of user space.

less probably does a malloc() of chunks of memory, onto what's called the heap. What are heap limits? It depends on kernel config and other factors. At some point, less is probably asking for memory and the system says it can't get any. It thinks "I can't get memory, I can't go forward, I really should quit since I can't keep up with the lines".

Why at 2.5G? Dunno, sometimes your heap allocator isn't fully efficient. You may have say 3Gb free, but because you need to fit together chunks, you get effectively get lower allocation amounts, with some free space showing. I remember one program that we had that just used 2GB of a 3.5Gb heap. Small allocations fragmented the heap so much we barely got half of what we wanted. Remember that malloc needs to be both fast and good, sometimes fast wins over good. It's not Tetris where it knows how to fit things perfectly, it trades packing efficiency for time. It would be an interesting experiment to use a different allocator (Google's tcmalloc seems particularly old if you know how to use it with LD_PRELOAD).

Also, is it killed? Or does it die? Do an echo $? after it ends, and you should see a number< 127 if it does an exit itself, or something > 128 if a signal killed it. If it's killed, maybe your system memory reaper killed it, though with 8Gb of RAM, I'd be surprised if your system felt the need to.

So, in short, probably 32 bit program, + some kernel constraint, + inefficient allocator, + less' allocation pattern. But check the return value to see if possibly the Out Of Memory reaper killed it.

  • I updated my original question with responses to your prompts. – Tyzoid Aug 20 '14 at 2:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.