125 reputation
5
bio website
location
age
visits member for 2 years
seen Aug 25 at 13:44

Oct
11
awarded  Popular Question
Oct
17
revised Using awk to split text file every 10,000 lines
fixed for loop for large number of split files
Oct
17
awarded  Scholar
Oct
17
accepted Using awk to split text file every 10,000 lines
Oct
17
comment Using awk to split text file every 10,000 lines
Thanks Thor! I had to change the for loop because i had a lot of split files, but diff is telling me the original and the split files, then merged file are the same.
Oct
17
suggested suggested edit on Using awk to split text file every 10,000 lines
Oct
15
comment Using awk to split text file every 10,000 lines
this seems to always trim off the last 34 characters of the last line for each split file.
Oct
9
awarded  Editor
Oct
9
revised Using awk to split text file every 10,000 lines
added goals bullet list
Oct
9
comment Using awk to split text file every 10,000 lines
you guys are hilarious. I think I'll try to script something through python or something. The idea was was to read through the source file one time, splitting it as it's read. The TOTAL_LINES var will read through the entire file to get the count and then it goes through the business end of the work. Haha. I'll give you an up vote for the effort.
Oct
8
awarded  Student
Oct
8
comment Using awk to split text file every 10,000 lines
@Valor - exactly. I have 400 of these files, each is about 400GB uncompressed. :(
Oct
8
comment Using awk to split text file every 10,000 lines
yes I know about split and that isn't useful. A careful reader will notice that split generates uncompressed files. I need them compressed.
Oct
8
awarded  Supporter
Oct
8
comment Using awk to split text file every 10,000 lines
That makes sense. Hmm. Is there a way to capture those and buffer them and achieve the desired result? btw, this is Ubuntu 12.04 LTS.
Oct
8
asked Using awk to split text file every 10,000 lines