BASH history truncated to 500 lines on each login

Question

For some reason, I cannot get my system to keep my BASH history after a reboot. Here are the relevant sections of my ~/.bashrc:

shopt -s histappend
PROMPT_COMMAND='history -a; updateWindowTitle'
export HISTCONTROL=ignoredups
export HISTSIZE=9999
export HISTFILESIZE=999999
export HISTFILE="$HOME/.bash_history"

As far as I can tell those are all the necessary options (I know I used to be able to keep history across multiple reboots without all of these in the past). However, despite having added these options several reboots ago, I still loose most of my history after a reboot. It is not empty, but it does not have the 9999 lines I had before rebooting.

Before anyone complains, yes I have read these questions. I have implemented some of their suggestions as listed above, the rest were either unhelpful or not relevant:

• Bash history loss when using histappend
• How to prevent Bash from altering history?
• What determines what shows up in the bash history command?
• How do I keep my bash history across sessions?
• save bash history, regularly

On the off chance that there may be other relevant commands in there, you can view my entire ~/.bashrc here.

So, what am I missing? Why is my history not saved? If anyone thinks another file may be relevant let me know and I'll post it. I checked by running grep -i hist \.* in my $HOME which showed that the only relevant . file containing the string hist or HIST was .bashrc.

I am running Linux Mint Debian Edition, GNU bash, version 4.2.36(1)-release (x86_64-pc-linux-gnu) and my favorite terminal emulator (in case that's relevant) is terminator.

---

UPDATE:

Following @mpy's suggestion in the comments, I changed my ~/.bashrc to set HISTFILE=~/bash_history as opposed to the default ~/.bash_history and that seems to solve the problem for interactive shells. Login shells still display the same behavior, with the history truncated at 500 lines. However, there are no HIST related variables set in the relevant files:

$ for f in /etc/profile ~/.profile ~/.bash_profile ~/.bash_login; do \
   echo -ne "$f :"; echo `grep HIST $f`; \
done
/etc/profile :
/home/terdon/.profile :grep: /home/terdon/.profile: No such file or directory
/home/terdon/.bash_profile :grep: /home/terdon/.bash_profile: No such file or directory
/home/terdon/.bash_login :grep: /home/terdon/.bash_login: No such file or directory
$ grep -r HIST /etc/profile.d/  <-- returns nothing

So, why is setting HISTSIZE and HISTFILESIZE in the ~/.bashrc not enough unless I explicitly set the $HISTFILE to something other than the default ~/.bash_history?

Answer 1

The problem actually boils down to the different behavior of login and non-login shells. I had set the variables that control history in my ~/.bahsrc. This file is not read when one starts a login shell, it's only read by interactive, non-login shells (from man bash):

When bash is invoked as an interactive login shell, or as a non-interactive shell with the --login option, it first reads and executes commands from the file /etc/profile, if that file exists. After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readable. The --noprofile option may be used when the shell is started to inhibit this behavior.

[. . . ]

When an interactive shell that is not a login shell is started, bash reads and executes commands from ~/.bashrc, if that file exists. This may be inhibited by using the --norc option. The --rcfile file option will force bash to read and execute commands from file instead of ~/.bashrc.

Therefore, each time I logged in, or dropped to a tty, or used ssh, the .history file was getting truncated because I had not set it to the right size in ~/.profile as well. I finally realized this and simply set the variables in ~/.profile where they belong, instead of ~/.bashrc

So, the reason my ~/.history was getting truncated was because I had only set the HISTORY variables in a file read by interactive, non-login shells and therefore every time I ran a different type of shell the variables would be ignored and the file would be cut accordingly.

Answer 2

My suggestion is to use another file as HISTFILE, not the default ~/.bash_history.

Although I have no analytical explanation, I'll try to outline what led me to this suggestion: If you use bash as your default (login) shell and also use X (which both is very probable) you have a running bash instance right after the (graphical) login:

systemd
 ...
  |-login
  |   `-bash      <<====
  |       `-slim
  |           |-X -nolisten tcp vt07 -auth /var/run/slim.auth
  |           |  `-{X}
  |           `-fluxbox
  |               `-xterm -bg black -fg white
  |                   `-bash
 ...

I think this instance is a login shell, so it doesn't read your ~/.bashrc and hence won't know anything about the histappend option:

man bash(1): When an interactive shell that is not a login shell is started, bash reads and executes commands from /etc/bash.bashrc and ~/.bashrc, if these files exist. (...)

As long as this "parent shell" runs, everything is fine, but upon its termination (i.e. system halt) it will override ~/.bash_history (because that's the default value) and messes up your history or clips it on system start to (again default) 500 lines. (Or perhaps both...)

It strikes me, too, that it's not sufficient to include the history configuration in ~/.bashrc, as this shouldn't be such a uncommon setup. I have no explanation for that.

---

Concerning your problem, that "Login shells still display the same behavior", you can try to include the history config also in ~/.bash_profile:

man bash(1): When bash is invoked as an interactive login shell, or as a non-interactive shell with the --login option, it first reads and executes commands from the file /etc/profile, if that file exists. After reading that file, it looks for ~/.bash_profile, (...)

Unfortunately I can't post a more justified explanation with details from my own bash config, as I'm a zsh guy...

Answer 3

Since all your settings are in order according to the man page, and since the history file is not restricted by size (bytes), the only possible explanation I can think of. It has to do with how the shell dies.

According to the online reference, the graceful exit (history saved) occurs only when the shell receives SIGHUP. I can't really explain how your system propagates signals when rebooted, but I suspect that your shell quits with SIGKILL or SIGPWR.

It could be because your WM runs asynchronously (wait) and the terminal emulator spawned from the WM where bash is gets an exit-forcing signal other than SIGHUP. It could also be that the OS is to quick to send the "final kill" to all processes before the initial graceful SIGHUP manages to get to the shell via X -> WM -> xterm, possibly because X or WM takes longer to exit than it takes for the OS to be ready to go down.

I'm on deep waters with this stuff, but I think something along those lines causes the erratic behavior. I've had this issue before, and the most solid remedy is exit in bash where you want to keep history.

I noticed history -a in your question, and I can't think of why that would not be sufficient to preserve the history.

You could troubleshoot the issue by figuring out what really kills your bash and moving on to figuring out where the signal originates and fixing the issue there, or simply flush the history when you know which signal is the last (assuming disks are still online by then):

trap "echo got 1  >/tmp/sig1;  exit" SIGHUP
trap "echo got 2  >/tmp/sig2;  exit" SIGINT
trap "echo got 15 >/tmp/sig15; exit" SIGTERM
 .. and so on...

The included screenshot illustrates what I'm talking about in the second and third paragraphs. The sequence there is I shell in from left, kill the left shell from right and cat the history.

man bash

On startup, (...) The file named by the value of HISTFILE is truncated, if necessary, to contain no more than the number of lines specified by the value of HISTFILESIZE (+ default 500).

If the histappend shell option is enabled (+ default on here), the lines are appended to the history file, otherwise the history file is overwritten.

online reference

3.7.6 Signals

When Bash is interactive, in the absence of any traps, it ignores SIGTERM (so that ‘kill 0’ does not kill an interactive shell), and SIGINT is caught and handled (so that the wait builtin is interruptible). When Bash receives a SIGINT, it breaks out of any executing loops. In all cases, Bash ignores SIGQUIT. If job control is in effect (see Job Control), Bash ignores SIGTTIN, SIGTTOU, and SIGTSTP.

Non-builtin commands started by Bash have signal handlers set to the values inherited by the shell from its parent. When job control is not in effect, asynchronous commands ignore SIGINT and SIGQUIT in addition to these inherited handlers. Commands run as a result of command substitution ignore the keyboard-generated job control signals SIGTTIN, SIGTTOU, and SIGTSTP.

The shell exits by default upon receipt of a SIGHUP. Before exiting, an interactive shell resends the SIGHUP to all jobs, running or stopped. Stopped jobs are sent SIGCONT to ensure that they receive the SIGHUP. To prevent the shell from sending the SIGHUP signal to a particular job, it should be removed from the jobs table with the disown builtin (see Job Control Builtins) or marked to not receive SIGHUP using disown -h.

If the huponexit shell option has been set with shopt (see The Shopt Builtin), Bash sends a SIGHUP to all jobs when an interactive login shell exits.

If Bash is waiting for a command to complete and receives a signal for which a trap has been set, the trap will not be executed until the command completes. When Bash is waiting for an asynchronous command via the wait builtin, the reception of a signal for which a trap has been set will cause the wait builtin to return immediately with an exit status greater than 128, immediately after which the trap is executed.

demonstrative screen shot

Answer 4

Check /etc/profile and /etc/profile.d/*

Maybe there is something messing with history settings there.