0

I have a large text file (.bib) with lots of double entries for some fields. Essentially, the file is a .bib-file (more info here) which has roughly 1000 entries. The content is structured like this:

@Article{Apak_2011_Financialriskmanagement,
Title                    = {Financial risk management in renewable energy sector: Comparative analysis between the European Union and Turkey},
Author                   = {Apak, Sudi and Atay, Erhan and Tuncer, Güngör},
Journal                  = {Procedia - Social and Behavioral Sciences},
Pages                    = {935--945},
Volume                   = {24},
Year                     = {2011},
Doi                      = {10.1016/j.sbspro.2011.09.013},
ISSN                     = {1877-0428},
}

@Incollection{Berger_1992_OutputMeasurementin,
Title                    = {Output Measurement in the Service Sectors},
Author                   = {Berger, Allen N. and Humphrey, David B.},
Crossref                 = {Griliches_1992_OutputMeasurementinb},
Pages                    = {245--300 book},
Year                     = {1992},
Publisher                = {University of Chicago Press},
Date                    = {1992-10-04},
Booktitle                = {Output Measurement in the Service Sectors},
Editor                   = {Griliches, Zvi and Berndt, Ernst R. and Bresnahan, Timothy F. and Manser, Marilyn}
}

@Book{Bogenstahl_2012_ManagementvonNetzwerken,
  Title                    = {Management von Netzwerken},
  Author                   = {Bogenstahl, Christoph},
  Publisher                = {Gabler},
  Date                     = {2012-01-01},
  ISBN                     = {978-3-8349-3572-4},
  Series                   = {Strategisches Kompetenz-Management}
}

You'll see that there is only Year defined for the first source. The second source however, has both Year and Date defined.

EDIT: HINDSIGHT IS 20/20

I just noticed that I always need the Year field because I manage the whole file via JabRef. And for generating a bibtex-key, JabRef needs the Year-field. I mean I haven't found an option to generate the bibtex-key with the help of the yyyy-part of the Date-field so far, so I'll edit the conditions.

So, is there a way to do the following actions:

  • If there is only Date defined for a source entry, copy the first 4 digits (yyyy) to Year.
  • If there is only Year defined, copy the four digits to Date.
  • If there is both Date and Year defined, do nothing.

Now following are the old conditions, when I hadn't thought of JabRef's inner workings beforehand

Old conditions, still relevant if one would like to prepare the .bib-file for the use with biblatex and has a mixup of the Date and Yearfield in his file:

  • If there is only Date defined for an source entry, do nothing.
  • If there is both Date and Year defined, copy the content of the brackets for Date into the brackets for Year. The catch here is, Date might contain more information (by the format yyyy-mm-dd) than Year, so this is exactly why I am writing down these "conditions". Regardless of the contents of Year, Date is more important. Unless I made some errors while typing all the info in, the first 4 digits of Date should be equal to Year of course.
  • If there is only Year defined, the term Year can be simply replaced with Date.

A few notes:

  • If this info could be of any help in this case: I use Windows 7 and Xubuntu 14.04. I have Office 2010, if that could be used here... or I could gladly use some sort of tool on Xubuntu, I don't know.

  • I checked already and apparently I can't use JabRef for this, it's a bit too complicated.

4
  • I have a few questions. 1. The @Incollection record seems to be a part of the first @Article record (I judge by the comma after the last entry in @Article and after the @Article itself). Should those two be treated independently? 2. You want Date to be copied to Year. What if the date contains more than just a year? Should month and day of month be copied too? Sep 6, 2014 at 13:42
  • @AtomHeartFather Sorry about that comma, that was a little error on my part. I deleted it. Yes they should be treated independently. About your 2nd question, see op. Basically, regardless of the contents of Year, Date is more important. Unless I made some errors while typing all the info in, the first 4 digits of Date should be equal to Year of course. :)
    – henry
    Sep 6, 2014 at 13:57
  • Just posted the answer, but I see you've changed some of the requirements... So the answer is not valid anymore (I am working on it now) Sep 6, 2014 at 15:13
  • @AtomHeartFather Your reply is still good for those who do not use JabRef and want to the .bib-files for the use with biblatex. I'll edit the old requirements into the op.
    – henry
    Sep 6, 2014 at 15:16

2 Answers 2

2

I'd throw perl at this problem. http://search.cpan.org/~ambs/Text-BibTeX-0.70/lib/Text/BibTeX.pm should help. Something like:

use Text::BibTeX;

$bibfile = new Text::BibTeX::File "foo.bib";
$newfile = new Text::BibTeX::File ">newfoo.bib";

while ($entry = new Text::BibTeX::Entry $bibfile) {
    next unless $entry->parse_ok;

    if ($has_year = $entry->exists ('year')) {
        $year = $entry->get('year');
    }
    if ($has_date = $entry->exists ('date')) {
        $date = $entry->get('date');
    }
    if ($has_year and ! $has_date) {
        $entry->set('date', $year);
    }
    if ($has_date and ! $has_year) {
        $entry->set('year', substr($date, 0, 4));
    }
    $entry->write ($newfile);
}
1

NOTE: This solution is for the original set of requirements. It would need to be updated to work with the current version. And also, the perl based answer is much cleaner anyway :-)

If you don't mind creating some temporary files, this could be a starting point: Copy that into a file and set the executable flag (chmod +x file)

#!/bin/bash
INFILE=$1

# split the file first
awk '/^@/{x="tmp__"++i}{print > x;}' $INFILE

# process individual files
for file in tmp__* ; do 
    DATE=$(grep "^[[:space:]]*Date" $file | sed "s/.*{\(.*\)}.*/\1/g")
    YEAR=$(grep "^[[:space:]]*Year" $file | sed "s/.*{\(.*\)}.*/\1/g")

    # Both year and date. Substitute year with date
    if [[ -n "$DATE" && -n "$YEAR" ]] ; then
        sed -i "s/\(^[[:space:]]*Year.*\)${YEAR}\(.*\)/\1${DATE}\2/g" $file
    fi

    # Only year
    if [[ -z "$DATE" && -n "$YEAR" ]] ; then
        sed -i "s/\(^[[:space:]]*\)Year/\1Date/g" $file
    fi
done

# concatenate the files back
cat tmp__* > out.bib
rm -f tmp__*

What the script does is:

  • Takes one parameter - the input file name
  • Splits the file into multiple temporary files, each of them containing only a single record
  • iterates over the files and processes them individually according to your instructions (provided that I understood them well, that is - see below)
  • concatenates the processed files into out.bib
  • removes the temporary files.

The script does not modify the original input file, so it should be pretty safe.

I am still not completely clear about your requirements, so if you give it a go and find some cases where it does not do what you expect - feel free to let me know and I will try to improve it.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .