The manpage of
less covers the topic:
-bn or --buffers=n
Specifies the amount of buffer space less will use for each file,
in units of kilobytes (1024 bytes). By default 64K of buffer space
is used for each file (unless the file is a pipe; see the -B
option). The -b option specifies instead that n kilobytes of buf‐
fer space should be used for each file. If n is -1, buffer space
is unlimited; that is, the entire file can be read into memory.
-B or --auto-buffers
By default, when data is read from a pipe, buffers are allocated
automatically as needed. If a large amount of data is read from
the pipe, this can cause a large amount of memory to be allocated.
The -B option disables this automatic allocation of buffers for
pipes, so that only 64K (or the amount of space specified by the -b
option) is used for the pipe. Warning: use of -B can result in
erroneous display, since only the most recently viewed part of the
piped data is kept in memory; any earlier data is lost.
The manpage implies that the buffer will eventually grow as big as the whole input, if you do not restrict it by -B and -b options.