Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm looking for a way to browse through some CSV unicode-encoded files. These files have relatively large number of rows (2-3 millions) and 10 columns tops, with columns being numerical, small text fields, dates or single paragraphs of text (total weight: about 300MB per file). I need to apply basic search and filtering (“show only rows where column X has value Y”, “search for this word in this column among the filtered rows”). Ease of use is top priority, as it will be used by inexperienced users.

I don't need to edit those files, just browsing.

Spreadsheets are out — they don't handle that many rows (the best of them have a limit of 1 million rows). I thought of a tool like OpenRefine, but last time I tried to load my example file, it ate 8GB of memory and started swapping before even loading half of the file.

The existing relational database systems I know (OOo Base, Access) have too complex UI to deal with all the complexity of the relational model, which would make using it for a single table a chore for an inexperienced user.

How can I best allow these inexperienced users to browse and filter this data?

share|improve this question

closed as primarily opinion-based by Raystafarian, DavidPostill, fixer1234, Art Gertner, nc4pk Mar 3 '15 at 0:09

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.If this question can be reworded to fit the rules in the help center, please edit the question.

You will have to dump it into a db, server or serverless db. AND the UI will be a separate entity all together. For DB I would recommend SQLite and I personally would create a simple query app or web application. Like I said, two questions here: WHERE to put the data, HOW to get the data. – Logman Jun 24 '13 at 2:31
Might be a case for QlikView. – Axel Kemper Jun 24 '13 at 21:09

Browse other questions tagged .