Is there any software (or pseudo-code) which can automatically scan a piece of text (either pasted into the tool, or read from a .doc/.pdf) and identify citation data using standard formats? The data would then be split up into its constituent fields and exported in XML, CSV, or some other structured data format. I have looked at cb2Bib but it was only able to extract the year from Harvard-style references, which is insufficient.
-
Do you want to scan the text itself or just the references section?– innaMAug 18, 2009 at 9:09
-
Just the references - it'd likely be a document containing personal publications.– Alistair KnockAug 19, 2009 at 9:07
-
I'm not sure if this what you might need but you can try this refhive.com– Mostafa ElmoghaziApr 3, 2019 at 21:49
8 Answers
At the moment (2017) the most active Open-Source project implementing this seem to be Anystyle Parser (last version 07-2016). It can be used through a web-interface, API, or downloaded as a RubyGem.
They explicitly mention on their website that the implementation is inspired by ParsCit (last version 2013?) and FreeCite (last commit 2009).
Also form their website:
AnyStyle Parser uses powerful machine learning heuristics based on Conditional Random Fields that can be trained by everyone using our built-in editor.
That is a realy cool feature, that makes this the most interesting implementation (imho). Training seems to be pretty straightforward, as explained in the API documentation. You just provide some manually corrected results, and and run the Anystyle.parser.train
command. I am not sure if ParsCit and FreeCite also support this, but if they don't, this seems like a huge feature-difference to me.
-
With the exception of Anystyle Parser they're all mentioned in the currently highest voted answer. What actually makes them stand out? What would be advantages or disadvantages given the original question?– SethApr 7, 2017 at 10:00
-
Ah, indeed. I'll edit and improve my answer. Thx for pointing that out.– WouterApr 7, 2017 at 11:40
-
-
1@Brandon: I have posted a HOWTO here: github.com/inukshuk/wapiti-ruby/issues/3– WouterFeb 28, 2018 at 8:24
-
1That looks great, thanks! As someone who has never touched ruby, it will be very helpful indeed.– BrandonFeb 28, 2018 at 9:27
Take a look at this list of Citation Parsers that can generate XML from input text:
http://paracite.eprints.org
http://aye.comp.nus.edu.sg/parsCit (in maintenance mode as of Aug 1, 2012)
http://opcit.eprints.org
http://search.cpan.org/~mjewell/Biblio-Citation-Parser-1.10
-
Another option is github.com/inspirehep/refextract. It's not based on ML but it worked very well on my tests.– JosirFeb 2, 2019 at 13:32
Try a tool such as Regex Buddy or Expresso.
If you're not a programmer Regular Expressions may be a bit intimidating, but they're really not that hard, especially with a decent tool like one of the above.
Here's an example of someone using Regular Expressions for extracting citations:
Mendeley should be able to do this. It can import PDFs and then export the metadata to BibTeX, RIS and EndNote XML. It is free to download and is cross-platform.
Edit: I tested this on a few documents. The PDF import does seem to work well for references that are formatted correctly. For a document I created using LaTeX, all of the references with the author in the form "Smith, J." or "J. Smith", etc., were imported fine. If the author is a company (a single word), or the reference is incomplete, it does not work as well. The extracted references can easily be edited and exported to BibTeX, etc.
-
2"This feature was removed in Mendeley 0.9.7 because it was consuming a fair amount of resources (client and server side) without providing enough value. We plan to re-introduce it in an improved form in future." ...... feedback.mendeley.com/forums/4941-mendeley-feedback/suggestions/…– icemanAug 16, 2010 at 9:50
I've seen a Westlaw program do that for legal citations, but that's probably not what you're looking for. Reference Manager might do something like that for academic formats, but I've never used it.
Try http://www.crossref.org/guestquery/#stqsearch
This one is capable of automatic parsing your reference text and offers a link to an on-line article.
Zotero is a plugin for firefox which does this for web content. Not sure if there is a similar tool for documents/pdfs
-
1I know that this isn't exactly what Zotero is designed to do, but if you pointed Firefox to a text file or html file with the relevant data, Zotero might recognize the references and then you could add it to the Zotero library and export the whole library into whatever format you like (I know Zotero supports a bunch of formats). This would be painful for a large number of files though.– nednedSep 1, 2009 at 14:11
-
I don't see how Zotero does what the OP asks. I've installed it, but there seems to be no option to parse a reference.– RikkiMar 6, 2015 at 13:03
-
Zotero parses citations from specially-coded websites, not from regular text. Apr 26, 2016 at 13:06
This probably belongs more as a comment to @Abhinav, but zotero definitely only handles structured data, as you would find described here:
An interesting hack might be to try to write a program that uses each citation as a search query in your favorite database, then uses something like zotero to generate the ref information. You could also download structured information from services like citeUlike. Let me know if you end up doings something like that! (put it up on github if you do ;).