0

With a text file, e.g. a word file of a story, following https://superuser.com/a/1598832/109367 , a list of all distinct words could be generated.

However, this is not yet a words of the original form -- for verbs (e.g. go), there are gerund, conjugations, third-person singular verb (going, goes, went, gone); for nouns (e.g. apple) there are plural forms (apples).

How could I get the original form of a word? so that I could get a vocabulary list of a book etc.

Even better, if this solution is applicable to not only English, but also, such as French etc.

2 Answers 2

1

You haven't specified what platform you're using, but what you're looking for is a stemming algorithm. This takes inflected words and reduces them to a common stem/root.

Most Natural Language Processing (NLP) libraries contain workable stemming algorithms.

Here are a few links that may help:

1
  • is there a dumb way, say, at least for english? for example, for 100k vocabularies, providing standard conjugations for verbs and plural nouns, then list the special ones, that would be only a database less than 200k rows?
    – athos
    Nov 2, 2020 at 11:27
2

(This is not really an answer, but I am posting it as an answer because it won't let me post a comment)

...the original form of a word? ...not only English, but also, such as French etc.

There is no one-to-one correspondence of words between two languages -- neither in meaning, nor in the way words change their forms. Many languages are highly inflected, with grammatical morphology not applicable to other languages. So what you're thinking to accomplish, it's not really a matter of making lists, it requires learning and understanding the languages.

1
  • is there a dumb way, say, at least for english? for example, for 100k vocabularies, providing standard conjugations for verbs and plural nouns, then list the special ones, that would be only a database less than 200k rows?
    – athos
    Nov 2, 2020 at 11:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .