Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

What would you recommend to a non-technical Windows user for viewing (no need to edit) large spreadsheets (200MB - 5GB).

Format is CSV and I am told Excel chokes.

share|improve this question

migrated from stackoverflow.com Nov 20 '12 at 11:51

This question came from our site for professional and enthusiast programmers.

3  
I would say, any programm opening a 5GB file would choke a bit or even a bit more. I am very interested in the answers to this one - +1. –  Jook Nov 20 '12 at 10:07
1  
If doesn't read all in to memory then no need to choke. On Linux I preview large files with the 'less' command. –  hoju Nov 20 '12 at 10:11
    
Don't know if it will help for sure, but have you tried 64-bit Excel? If that's no good, might need to use a text viewer/editor afer all as mentioned below. –  Karan Nov 21 '12 at 16:31
    
Your files are probably too large for this to help, but you might want to try LibreOffice calc. I know they recently greatly expanded the number of rows they can handle. The real question is why is your data so big? Is it structured in some way that you can subdivide it into meaningful smaller chunks as @mrTomahawk suggests and use those chunks to generate summary data that can then be merged for all the chunks into something more manageable? Usually, when data gets even a fraction of this big, it should be handled by a database, not a spreadsheet. –  Joe Nov 27 '12 at 8:38
add comment

2 Answers 2

up vote 3 down vote accepted

I think you will struggle to find software that will be able to read that size of file and display it in a spreadsheet-style layout. However, I did find these related posts that may be of assistance, as I think your best bet is to view the CSV file through a text editor:

Stackoverflow - Text editor to open big (giant, huge, large) text files

SuperUser - Application to open / edit a very large CSV file (500 MB, 4 million records)?

share|improve this answer
    
Also, (from the links above), people have mentioned csved.sjfrancke.nl - might be worth checking out. –  BenjiBoyWick Nov 20 '12 at 11:45
    
I tested and found that tool unreliable. vim works well for large files as those links recommend. However not an ideal solution for spreadsheets. –  hoju Nov 22 '12 at 22:47
add comment

The biggest problem I had when encoutering a problem like this was so much the size of the csv, but instead that the file contained more records then what Excel could handle. Depending on the version of Excel your running; Excel 2003 has a limit of 65,536 rows and starting in Excel 2007, the size of the grid expanded from 65,000 rows to over one million rows.

To get around my issue, I ended up breaking up the csv file into chunks which was more manageable.

I used following Powershell script to break the file up:

$reader = New-Object System.IO.StreamReader("D:\someFile.csv")
$upperBound = 2500000
$fileCount = 1
$newFilename = "D:\someFile_{0}.csv" -f ($fileCount)
while(($line = $reader.ReadLine()) -ne $null){
    Add-Content -path $newFilename -value $line
    If((Get-ChildItem -path $newFilename).Length -ge $upperBound){
        ++$fileCount
        $newFilename = "D:\someFile_{0}.csv" -f ($fileCount)
    }
}
$reader.Close()

This will break the file up into multiple files each of them the size of whatever you set the $upperBound to, and each file will be sequentially numbered.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.