99

Is there a tool to split large text file (9 GB) into smaller files, so that I can open it and look through?

Is there anything usable from the command line that comes with Windows (XP)?

Or what's the best way to split it? Can I use 7-Zip to create separate volumes and then unzip one of them separately? Will it be readable or does it need all the other parts to unzip into the big file again?

I put together a quick 48-lines Python script that splits the large file into 0.5 GB files which are easy to open even in Vim. I've just needed to look through data towards the last part of the log (yes, it is a log file). Each record is split across multiple lines so grep would not do.

Improve this question
3
  • I see you edited to mention grep. Do you have cygwin or unxutils installed? You could have used grep -n with head and tail to see chunks of the file. Example, grep -n "something" file.txt returns 95625: something. You want to see that line and 9 lines below it for a total of 10 lines: head -n 95635 file.txt | tail -n 10.
    – John T
    Jan 11, 2010 at 4:10
  • I notice you solved your problem, If you're still around, could you post the solution so others might benefit?
    – Journeyman Geek ♦
    Oct 19, 2012 at 3:30
  • This has been discussed in much detail at Stack Overflow[1] [1]: stackoverflow.com/questions/159521/…
    – Rishi Dua
    Jun 26, 2013 at 6:33

10 Answers 10

Reset to default
50

There is a freeware Windows file splitter called HJSplit.

It is available here. The website claims it can split files of any type and size, but 9 GB is a big file.

Improve this answer
7
  • 1
    According to the publisher, it supports splitting of files "over 100 Gigabytes"
    – yosh m
    Apr 11, 2013 at 7:57
  • 8
    I was able to use this program to split a large file into smaller pieces, but only by size (kB or MB). It does not respect lines, so it's not very useful for splitting log files. It's also very slow.
    – nullability
    Mar 19, 2014 at 17:46
  • HJSplit works so ugly... It does not work with 5Gb+ files. The biggest part of file just losing at all. Chunck peases counter nofuses after 999, and something else failes if use bigger peases
    – user2602807
    Sep 28, 2016 at 12:08
  • On 64bit systems it only works with a GUI in Windows. If you want to automate things, this is not a solution
    – 576i
    Nov 30, 2017 at 10:20
  • 2
    hjsplit is a dead URL
    – Serge Pavlov
    Nov 14, 2019 at 17:57
42

The GNU Core Utils package (available here for Windows) includes the Split utility.

The --help documentation is as follows:

Usage: split [OPTION] [INPUT [PREFIX]]
Output fixed-size pieces of INPUT to PREFIXaa, PREFIXab, ...; default
size is 1000 lines, and default PREFIX is `x'.  With no INPUT, or when INPUT
is -, read standard input.

Mandatory arguments to long options are mandatory for short options too.
  -a, --suffix-length=N   use suffixes of length N (default 2)
  -b, --bytes=SIZE        put SIZE bytes per output file
  -C, --line-bytes=SIZE   put at most SIZE bytes of lines per output file
  -d, --numeric-suffixes  use numeric suffixes instead of alphabetic
  -l, --lines=NUMBER      put NUMBER lines per output file
      --verbose           print a diagnostic to standard error just
                            before each output file is opened
      --help     display this help and exit
      --version  output version information and exit

SIZE may have a multiplier suffix: b for 512, k for 1K, m for 1 Meg.

For example, to split input.txt into 100Mb chunks, only splitting at the ends of lines,

split input.txt -C 100m

will give you output files named xaa, xab, xac, etc.

Improve this answer
2
  • 2
    for example git msys download includes the utility as well.
    – eis
    May 20, 2014 at 6:55
  • 1
    Cmder is amazing for this
    – Umber Ferrule
    Mar 20, 2016 at 0:36
31

One can use 7-Zip to create segments of text file in certain size (e.g. 100 MB segments out of 1.5 GB log file).

The Key options are - Use "Store" as opposed to "Compress" - Use "Split to volumes"

You should be able to see text in .001 (.nnn) files.

enter image description here

Improve this answer
2
  • The attached image is perfect!
    – Erik
    Nov 5, 2018 at 13:04
  • Good alternative, without the need to install anything (most people have either 7zip or WinRar). I tried the different formats, and they all only store the header in the first file, the rest have no headers. The smaller headers are with zip and 7zip formats. With WinRar you can do the same thing, headers are also small, but it adds them to every file, so that's a little more effort if you need too many parts.
    – Andrew
    Mar 6, 2020 at 2:28
15

Another is GSplit - according to their site it can split very large files (larger than 4 GB <-- since they crossed the 4 GB limit, I guess they can do 9 GB as well).

But, another thing—you say you want to split it into smaller parts, so you can open it up and look at it. That sounds like a very big file, perhaps a log file.

In any case, for opening large text files, may I recommend EmEditor. They claim themselves it can open very large files (up to cca. 250 GB), and I've used it in the past for files up to 2 GB. But in any case, I think it may be a better solution than splitting.

Improve this answer
3
  • I've just tried out GSplit. It works, albeit through a confusing interface, but it doesn't offer any way to only split at a linebreak - for splitting ASCII data files, therefore, it's not too useful as the split will be half way through a line.
    – Flyto
    May 10, 2013 at 15:09
  • 1
    @Flyto It actually is possible to split by line if you choose "Split After Occurrence Number" under the Pieces > Type and Size menu. You then choose the number of lines and the line delimiter in hex. By default it has Windows format CR+LF (0x0D0x0A).
    – nullability
    Mar 19, 2014 at 17:56
  • @nullability thanks - I wouldn't have thought of that. But, I found a much simpler solution through the GNU "split" utility - see my new answer :-)
    – Flyto
    Mar 19, 2014 at 22:23
9

Check out Large Text File Viewer, it's great for things like this. Most archivers and splitters will separate the file into pieces which cannot be used to read each piece of data independently and properly, you need to extract them all to get the file back.

alt text

Large Text File Viewer is free and portable.

Improve this answer
6
  • 3
    The link seems to be broken, this works as of now: softpedia.com/get/Office-tools/Other-Office-Tools/…
    – huseyint
    Jul 11, 2012 at 20:24
  • The original (or edited?) link is working now; I'd take that link over softpedia any day! Also, just tried LTFViewer, and it's awesome, opened an 818MB file that even Notepad++ complained about opening. Not much in the way of editing features, but once I'd viewed the logs in LTFViewer, echo. > myfile.txt resolved the 818MB logfile issue :D
    – Doktor J
    Jul 1, 2013 at 19:16
  • LTFViewer is realy great ( i'm using it to ), but there's an limit. I tryed to open an 3GB SQL file with it, but it freezes. Mayby i didn't have enough patience or something...
    – Mathlight
    Aug 9, 2013 at 6:37
  • LTFViewer worked very well for me. I used it on a (relatively) smaller file, only 750MB, but it opened it in under 5 seconds.
    – Mike_OBrien
    Mar 17, 2014 at 14:52
  • You can download it from the internet archive: web.archive.org/web/20151019115332/http://swiftgear.com
    – quip
    Apr 13, 2017 at 23:28
7

You can use 7-Zip itself to split the files. (You can save as a .zip or .7z format.) When you go to create the archive there is an option called "Split volume, bytes". Just select how large you want the chunks.

And yes, you can unzip them individually if you wish.

Split files in 7-Zip

Improve this answer
1
  • 1
    I don't think you can unzip them individually, you always need to start from the first file. What you can do is what other answer states: use compression level "store", so the file goes directly into the zip or 7zip file. Only the first file will have the header that you will need to remove.
    – Andrew
    Mar 6, 2020 at 2:32
4

There's an online tool that splits text files if anyone is looking to split files quickly. http://www.textfilesplitter.com.

Works great for me. And splits files respecting lines which is what I was looking for. It also says it's all HTML5 client side so it's safe to use. I'm not sure how big it can go but I think it depends on your machine's ram.

Improve this answer
4
  • 5
    Are you really suggesting to upload a 9 GB file to have it splitted in your browser?? Really??
    – spaghettidba
    Mar 1, 2016 at 9:43
  • 1
    @spaghettidba: He says it works client side using HTML5. If that's the case, than it does not need to be uploaded. I admit, I had the same reaction as you when I first read the answer, though.
    – mwolfe02
    Apr 14, 2016 at 15:32
  • 1
    Yep, I split 100+ MB in just a couple of seconds. No way it could have uploaded worked and downloaded that fast.
    – Karl Henselin
    Nov 1, 2019 at 14:48
  • Worked for me with a 141mb file. Was quick and exactly what I was looking for, amazing!
    – Daniel Black
    Apr 28, 2022 at 21:35
0

The idea of seeing part of the file before deciding what to do with it is for me the best option.

The Large Text Viewer App can be installed on Windows through the Microsoft Store and it offers an option to cut the file in chunks of size. It may well be that it uses the same editor previously mentioned (behind the scenes), but the option to install it from a known source is better IMHO than the alternative links offered. It worked great for me.

The only issue if splitting by size is that it not necessarily breaks the file at a good place, so you may have to edit it to capture the exact content you want.

Improve this answer
1
0

I have found the program ffsj very useful. There doesn't seem to be a homepage around currently. But there is a download page here. Be careful with the download clicks, as they try to get you to download additional software, as well.

Improve this answer
2
  • Why would we trust softonic.com?
    – Peter Mortensen
    Aug 19 at 19:23
  • @PeterMortensen, Yes, indeed, you should be very careful and skip if you don't feel comfortable. This is, by far, not a recommendation for softonic.com. But at the time of my answer, it seemed to be only the site that had a valid download link for the software.
    – James John McGuire 'Jahmic'
    Aug 24 at 16:08
0

Splitting files is also a function of Total Commander, the tool I can't do my work without.

Get your 30-day trial here: https://www.ghisler.com/ Licenses are dirt-cheap, concurrent and permanent.

In Total Commander, highlight the file you want to split. Select [file][split file] from the menu. In the pop-up, select your target-directory and "bytes per file". Choose from: 1.44 MB, 1.2 MB, 720 K, 360 K, 100 MB, 250 MB, 650 MB or 700 MB. Press OK and watch the magic happen...

Improve this answer
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .