Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm looking for a way to browse through some CSV unicode-encoded files. These files have relatively large number of rows (2-3 millions) and 10 columns tops, with columns being numerical, small text fields, dates or single paragraphs of text (total weight: about 300MB per file). I need to apply basic search and filtering (“show only rows where column X has value Y”, “search for this word in this column among the filtered rows”). Ease of use is top priority, as it will be used by inexperienced users.

I don't need to edit those files, just browsing.

Spreadsheets are out — they don't handle that many rows (the best of them have a limit of 1 million rows). I thought of a tool like OpenRefine, but last time I tried to load my example file, it ate 8GB of memory and started swapping before even loading half of the file.

The existing relational database systems I know (OOo Base, Access) have too complex UI to deal with all the complexity of the relational model, which would make using it for a single table a chore for an inexperienced user.

How can I best allow these inexperienced users to browse and filter this data?

share|improve this question
1  
You will have to dump it into a db, server or serverless db. AND the UI will be a separate entity all together. For DB I would recommend SQLite sqlite.org and I personally would create a simple query app or web application. Like I said, two questions here: WHERE to put the data, HOW to get the data. –  Logman Jun 24 '13 at 2:31
    
Might be a case for QlikView. –  Axel Kemper Jun 24 '13 at 21:09

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.