I have a huge text file, far too big for the whole thing to be paged into memory. All I need to do with this text file is edit the first line (its a CSV file and I need to alter the titles).
Is there a simple way I can do this in bash?
Stack Exchange network consists of 181 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.Visit Stack Exchange
Super User is a question and answer site for computer enthusiasts and power users. It only takes a minute to sign up.Sign up to join this community
You can use
less to see what you want to edit and use
sed to make the changes. This way you edit without loading the entire file.
Another way is to split the file, edit and join again:
split -b 10000k <file> and to join: cat xa* > <file>
If your modification changes the length of the file, the whole file needs to be re-written, see for example this discussion on SO. You should probably consider saving the data to a database.
Keeping that in mind, you can stream edit the file with sed. To replace the first line, do something like this (GNU sed):
< oldfile sed '1c\new_heading' > newfile
edlinfor ms-dos or
edfor Unix didn't appear to have this problem but where only line editors... RAM at these times was a scarce and precious thing and a file size in Gigabytes was something impossible to believe! ;)
ed!!! but it would be a good fit in this case (not sure it could handle a file of this size...)