Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm having a problem with one segment of the following script. Each echo->grep->was written in to help me figure out where the script was failing to perform as I expect it to. It is designed to be part of a larger loop, but for sanity, I only have here the part that is relevant to my question.

I have files in /var/www/acpog/upload that are named 0SCAN2013 followed by the rest of the date (YYYYMMDDHHmmSS)

And the file ./alt_upcs.csv has 2 columns, one being sku (1419), the second being UPC (9781844484201)

If I can't find a match in the /var/www/acpog/uploads/ for sku, I want to attempt to find a match for the UPC. If that fails I want to attempt to find a match for sku with zeros padding the left hand side.

Here's the snippet:

st="0"
id="1419"
sd="SCAN2013"

echo "grep \"^$id,\" /var/www/acpog/upload/$st$sd* | cut -d',' -f 4-5 | tail -1"
grep "^$id," /var/www/acpog/upload/$st$sd* | cut -d',' -f 4-5 | tail -1
dt=$(grep "^$id," /var/www/acpog/upload/$st$sd* | cut -d',' -f 4-5 | tail -1)

if [ "$dt" == "" ]
then
  echo "grep \"^$id,\" alt_upcs.csv | cut -d',' -f 2"
  grep "^$id," alt_upcs.csv | cut -d',' -f 2
  u=$(grep "^$id," alt_upcs.csv | cut -d',' -f 2)
  for j in $(grep "^$id," alt_upcs.csv | cut -d',' -f 2)
  do
    if [ "$dt" == "" ]
    then
      echo "grep \"^$j,\" /var/www/acpog/upload/$st$sd* | cut -d',' -f 4-5 | tail -1"
      grep "$j" /var/www/acpog/upload/$st$sd* | cut -d',' -f 4-5 | tail -1
      dt=$(grep $j /var/www/acpog/upload/$st$sd* | cut -d',' -f 4-5 | tail -1)
    fi
  done
fi

if [ "$dt" == "" ]
then
  echo "grep \"^0$id,\" /var/www/acpog/upload/$st$sd* | cut -d',' -f 4-5 | tail -1"
  grep "^0$id," /var/www/acpog/upload/$st$sd* | cut -d',' -f 4-5 | tail -1
  dt=$(grep "^0$id," /var/www/acpog/upload/$st$sd* | cut -d',' -f 4-5 | tail -1)
fi

and here's the output:

grep "^1419," /var/www/acpog/upload/0SCAN2013* | cut -d',' -f 4-5 | tail -1
grep "^1419," alt_upcs.csv | cut -d',' -f 2
9781844484201
," /var/www/acpog/upload/0SCAN2013* | cut -d',' -f 4-5 | tail -1
grep "^01419," /var/www/acpog/upload/0SCAN2013* | cut -d',' -f 4-5 | tail -1

Lines 1, 2, 3, and 5 are as expected. Line 4 does not have the beginning of the line I'm trying to echo up to just before the comma. If I echo the variable $j after this, it contains the UPC as expected, but the grep command using that variable isn't returning anything either.

EDIT: Also, there are 4 files that should match in the upload directory, all starting with 0SCAN2013

Here's some further output from manually running commands from the prompt:

# echo "grep \"^$j,\" /var/www/acpog/upload/$st$sd* | cut -d',' -f 4-5 | tail -1"
grep "^9781844484201," /var/www/acpog/upload/0SCAN2013* | cut -d',' -f 4-5 | tail -1

# grep $j /var/www/acpog/upload/$st$sd*
/var/www/acpog/upload/0SCAN20130821213905:9781844484201,0,2,8:21,13:48:496

# grep $j /var/www/acpog/upload/$st$sd* | cut -d',' -f 1 | tail -1
/var/www/acpog/upload/4SCAN20130805204626:9781844484201

# grep $j /var/www/acpog/upload/$st$sd* | cut -d',' -f 2 | tail -1
2

# grep $j /var/www/acpog/upload/$st$sd* | cut -d',' -f 3 | tail -1
-1

# grep $j /var/www/acpog/upload/$st$sd* | cut -d',' -f 4-5 | tail -1
8:21,13:48:496

To test, these files should work. In the directory where the script resides: filename alt_upcs.csv containing 1 line:

1419,9781844484201

In the /var/www/acpog/upload/ directory, filename 0SCAN20130821213905 contining 1 line:

9781844484201,0,2,8:21,13:48:496
share|improve this question
    
Can you post a minimum working example of your input files so we can test it? –  terdon Sep 2 '13 at 17:04

1 Answer 1

up vote 0 down vote accepted

Your csv file has windows "CRLF" line endings. (CR is "carriage return", also written \r or \0x0D; LF is "line feed", \n or \0x0A). The unix utilities you're using don't recognize the CR as part of the line ending, so it becomes part of the looked-up UPC. That will prevent the UPC from being found. It also creates misleading output (your line 4), because the CR does a carriage-return on output, causing the part of the echo following it (starting with the ,") to overwrite the beginning.

Use dos2unix or some similar utility to fix your csv file.

share|improve this answer
    
Thanks, now I feel like a nooby for not realizing this sooner. I am used to being able to see the extra characters in vi, but my files showed fine. I created a new alt_upcs.csv file from scratch and it worked. –  David Wilkins Sep 2 '13 at 17:52
    
@DavidWilkins: vim will normally autodetect line-ending style; you need to check to see whether something like [DOS] appears after the filename in your status bar. You can use set ff=unix to dedosify the file. –  rici Sep 2 '13 at 17:54
    
Thanks, that's easier to remember than tr -d '\15\32 < dosfile > unixfile and no I don't know why my server doesn't have dos2unix. It's the first one I ever worked on that doesn't –  David Wilkins Sep 2 '13 at 18:03

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.