How do I work with an untypeable keyboard character?

Question

I am in the process of making a bash script to scrape a weather.com to get the weather report. I need to process a line of text that has an unusual character. Here is the text:

30°FHigh 35°High 52°High 45°High 43°High

There is a small circle between the numbers and the letters. Here is the code:

#!/bin/bash

clear

lynx --dump http://www.weather.com/weather/5-day/New+York+NY+10001 | \
egrep '   Today |   Tonight ' -A 22 | awk 'BEGIN {print "\n\t\t\b\b\b\b\b\bTHE FIVE DAY\
WEATHER REPORT FOR THE NYC: 10001\n"} \
/[0-9][0-9]*[a-z|A-Z]+/{print $1"\t\t"$2" "$3"\t\t"$4" "$5"\n"}' 2>> error.txt

This gives no output.

Answer 1

The easiest way is to use a dot to match the degree symbol.

/[0-9][0-9]*.[a-z|A-Z]+/

For example

$ echo -e "30\0260FHigh" \
| awk '/[0-9][0-9]*.[a-z|A-Z]+/ { print "yes" }'

yes

(\0260 is octal for the degree symbol in your question)

If you want to match it exactly you'd have to identify it's value in the encoding used. In the text in your question it seems to be 0xb0.

 $ echo -e "30\0260FHigh" \
 | awk '/[0-9][0-9]*\xb0[a-z|A-Z]+/ {print "yes" }'

 yes

\0260 and \xb0 just illustrate two ways to indicate the same thing.

Answer 2

Modern Linux shell is very Unicode-aware. In fact, I think all the data are treated as UTF-8.

I had no problem running this code:

$ echo ° | awk '/°/{print "found it"}'
found it

If you're making a shell script, make sure you have it in UTF-8 instead of ASCII.