Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a project involving around 4000 flat text files which represent legal documents. I need to parse them out by named section for a text analysis project I'm starting. The structure of these documents is somewhat standard, but parsing them out into standardized sections will involve some trial and error, some python scripting, and a lot of brute force "look at the file and edit it" type work.

My questions is the following. I'm not a very advanced programmer with respect to text analysis. I do most of my work on structured data (you know, numbers and stuff) using statistical languages. What is the simplest way to set these files up so that I can pull individual sections, and categorize them in the future?

Should I mark them up with some kind of XML like structure? If so, what's the right toolset? Should I put the parts in a database, or is that overkill?

share|improve this question

closed as not constructive by Karan, Brad Patton, davidgo, Breakthrough, Nifle May 25 '13 at 7:45

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

Browse other questions tagged or ask your own question.