1

I'm trying to use Bash to complete Project Euler 13. Below is my code that I just cannot figure out what's wrong with.

#!/bin/bash                                                
sum=0                                                      
while read -r -d $'\r' line; do                            
    sum=$(echo $sum + $line | bc)                          
done <<< "$(curl -s http://pastebin.com/raw/uHZ0PZjm)"     
echo "${sum:0:10}"                                         
exit 

It used to result in two errors,

(standard_in) 1: syntax error

and

(standard_in) 1: illegal character: ^M

After some research, it seemed to be an issue with the EOF terminators. I then ran dos2unix on it and it no longer gives the second error, but is still giving the first repeatedly. It seems to be some issue with how I'm piping the data into bc, but I've no clue what or how to fix it.

The correct answer is 5537376230. Thank you much for anything you can help with!

System info is

GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu)

I'm using cmder on Windows 10.

  • 1
    It didn't throw any error on an Ubuntu LTS bash shell. It answers 5483872696 ... – Hastur Feb 16 '17 at 0:39
  • Sorry, should have specified, the correct answer is 5537376230. – Egrodo Feb 16 '17 at 1:10
  • Yes you should (no problem, this only prevented me and maybe others to go further), but there is another information inside... it didn't generate any error. Please add for completeness the OS version on which you are running the script (distro, release, version of bash...). It is a good habit in general and can help to understand. – Hastur Feb 16 '17 at 9:00
  • Okay, sorry, I will be sure to do so in the future. Thanks for the advice. – Egrodo Feb 16 '17 at 17:25
  • Nothing to be sorry about... :-). BTW it's only question of some seconds, really add the OS and the version of your system. I'm still curious to know the reason that causes the syntax error. Putting the code in a script on my system it doesn't throws any error... – Hastur Feb 16 '17 at 17:30
3

You want to set \n ( == 0x0a == LF == linefeed) as your read delimiter, not \r ( == 0x0d == CR == carriage return). Either that, or make sure to put a CR at the end of your pastebin file. It looks like your pastebin file is missing a line terminator sequence at the end of the last line, so the last line is never being fed into your script.

$ curl -s http://pastebin.com/raw/uHZ0PZjm | hexdump -C | tail -n 8
...(snip)...
000013e0  35 30 39 35 31 36 0d 0a  32 30 38 34 39 36 30 33  |509516..20849603|
000013f0  39 38 30 31 33 34 30 30  31 37 32 33 39 33 30 36  |9801340017239306|
00001400  37 31 36 36 36 38 32 33  35 35 35 32 34 35 32 35  |7166682355524525|
00001410  32 38 30 34 36 30 39 37  32 32 0d 0a 35 33 35 30  |2804609722..5350|
00001420  33 35 33 34 32 32 36 34  37 32 35 32 34 32 35 30  |3534226472524250|
00001430  38 37 34 30 35 34 30 37  35 35 39 31 37 38 39 37  |8740540755917897|
00001440  38 31 32 36 34 33 33 30  33 33 31 36 39 30        |81264330331690|
0000144e

Note that there's a 0x0d0a (CR LF) between each number, but not after the last.

$ while read -r -d $'\r' line; do echo $line; done <<< "$(curl -s http://pastebin.com/raw/uHZ0PZjm)" | tail -n 3
77158542502016545090413245809786882778948721859617
72107838435069186155435662884062257473692284509516
20849603980134001723930671666823555245252804609722

Note that the last number, 535[...]690, is missing when run through your read command. But if you switch your delimiter to the Unix-native LF (\n) newline character, the last line is included:

$ while read -r -d $'\n' line; do echo $line; done <<< "$(curl -s http://pastebin.com/raw/uHZ0PZjm)" | tail -n 3
72107838435069186155435662884062257473692284509516
20849603980134001723930671666823555245252804609722
53503534226472524250874054075591789781264330331690

Edited to add: Here's a fix that handles the CRs in the pastebin file. I told read to use CRLF as the delimiter, and used an extra echo to add a CRLF after the pastebin file.

sum=0
while read -r -d $'\r\n' line; do
    sum=$(echo $sum + $line | bc)
done <<< $(curl -s http://pastebin.com/raw/uHZ0PZjm; echo -e "\r\n")
echo "${sum:0:10}"
  • Oh god, evidently Pastebin scraps the terminator at the end no matter how I paste it, that's srange. I instead hardcoded the numbers into my script and kept the CR delimiter as you can see here: pastebin.com/raw/g2iDchun. But for some reason that returns just "0" :( When I change it to the LF delimiter \n it returns instead "(standard_in) 2: syntax error". Any ideas why this might be happening? Thanks for your help clearing up the differences between the line breaks! – Egrodo Feb 16 '17 at 1:22
  • 1
    Ah, bc is choking because your pastebin has CR's in it. When I tested my change, I tested it with a plain echo, without piping it into bc. As for your new problem, the shell isn't preserving your newlines in that string assignment, so $numbers ends up being one long line with spaces between numbers. – Spiff Feb 16 '17 at 1:40
  • 1
    @Egrodo Okay, I updated my Answer with a real, fully-tested solution. – Spiff Feb 16 '17 at 1:56
  • i see I see, I didn't realize that the shell didn't preserve line breaks, that's quite dumb. Your new solution works perfectly, thank you so much for the time you've spent helping me with this. If you don't mind, I'm trying to figure out exactly what's happening with the stuff being inputted into the while loop. So I'm standard_in'ing the contents of the pastebin, then also inputting the echo? Afaik doing the done <<< thing just says "give this input to anything in the loop asking for input". How does echo play into that? – Egrodo Feb 16 '17 at 3:06
  • 1
    Yes, the command <<< "string" thing is called a "here string", and it's about the same thing as using echo "string" | command. In your case you're using $(curl… ; echo…) to form the "here string" from the standard output of curl, followed by the standard output of echo. In shell scripting, a semicolon separates commands, allowing you to put multiple commands on one line. – Spiff Feb 16 '17 at 3:59
3

You can do this:

mapfile -t lines < <(curl -s http://pastebin.com/raw/uHZ0PZjm | sed 's/\r$//')
sum=0
for bignum in "${lines[@]}"; do 
    sum=$(bc <<< "$sum + $bignum")
done
echo "${sum:0:10}"    # ==> 5537376230

That uses:

  • a process substitution to contain the call to curl and pipe into sed to remove the carriage returns.
  • redirect that into mapfile to read the lines of the input into a shell array
  • iterate over the array and call bc with the expression passed in with a here-string

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.