Is there a software that performs a textual analysis on blogs?

Question

My company is looking to create a PivotViewer visualization of a client's Wordpress 2 blog posts for the last 11 years. To do so, however, we need to edit the somewhat haphazard, incomplete, and generally poor tags for use as sortable categories. I'm looking for a tool that will analyze their blog entries and perform word counting, to give us a sense of what we're dealing with.

Ideally, it would have all of these features:

1. Word blacklisting (ignore)
2. Word stemming
3. Custom synonym merging
4. Counting all uses
5. Counting number of posts a word appears in.

I would have thought that this sort of textual analysis would be extremely common, but I haven't been able to find any software that does this sort of thing on entire blogs. Is there software available to do this?

Answer 1

The software you are looking for can have many titles, like "Content analysis", "Tag cloud" or "Meta Tags" and many more such as "text analysis" and "text mining".

There are very many software tools for these purposes, both free and commercial.

I do not have personal experience with such tools, but a good place to start is Text Analysis Tools that lists dozens of such tools, both free and commercial.

Another such list is Text Analysis, Text Mining, and Information Retrieval Software.

Answer 2

You might want to give Wolfram's Mathematica a try. You will have to do some programming, but all the tools you need are there:

• Database connectivity, so that you can fetch the blog posts
• Data manipulation functions like: Tally, GatherBy
• Linguistic analysis (English only): WordData

Answer 3

Take a look at Rapidminer or Weka

Seeing as its a clients blog, you probably have database access. Download all articles as plaintext and use one of the above programs to deal with the natural language processing questions (1,2,3, and 5).

The number of uses is hard to truly automate since it has to do with automatically determining the meaning of words using the context.

Answer 4

Some of these questions could be answered in a quick and dirty way using Google Search on your blog (easiest if it has its own domain).

Answer 5

Zemanta does analysis and can suggest tags and links. It's a wordpress plugin too.

Only problem: as it currently stands it requires manual opening and selecting and saving of each post.

There are a huge number of auto-tag plugins for wordpress though. You should search the plugin finder and give a few a try.