Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

If I have a list in bash

L="11 22 11 33 22 44"

Is there a clean way to get to "11 22 33 44" ?

share|improve this question
up vote 3 down vote accepted

One way:

$ L="11 22 11 33 22 44"
$ L=$(echo $L | tr ' ' '\n' | sort -nu)    
$ echo $L
11 22 33 44

Using tr, the numbers are split to individual lines, and sort is used to sort the numbers(-n) and remove the duplicate ones(-u).

share|improve this answer
    
You can leave off the | xargs. Because of the way the shell handles multi-line output from a $(…) command, L=$(echo $L | tr ' ' '\n' | sort –u) is enough. – Scott Dec 1 '12 at 1:42
    
@Scott : thanks scott for pointing it out, updated the solution.... – Guru Dec 1 '12 at 1:45
    
Clean and easy. 'tr' was the the part I was missing. – Travis Bear Dec 1 '12 at 1:49

A common pattern in some other programming languages is to store each value as a 'key' in a hash or dictionary or associative array, and rely on the implementation to keep hash keys unique.

declare -A items=()
for item in $L; do
    items[$item]=1
done
L=${!items[*]}

(Unfortunately, the bash implementation is about 2× slower than sort -u.)

share|improve this answer
    
+1..its always a pleasure looking at pure shell solutions.. – Guru Dec 1 '12 at 2:04

Another one I've found useful for checking the list of unique IPs connecting to your servers, from the 'netstat' command is this:

Netstat Output Example

tcp 0 0 192.168.1.196:8009 192.168.21.124:45921 TIME_WAIT
tcp 0 0 192.168.7.196:8009 192.168.21.124:51579 ESTABLISHED
tcp 0 0 192.168.7.196:8009 192.168.20.12:20213 TIME_WAIT

Command Example

In this example I'm starting by filtering the TCP port, cutting out the local information, sorting the list and then piping it to the 'uniq' command.

netstat -an | grep ":8009" | cut -d " " -f 28 | cut -d ":" -f 1 | sort -u

What I get is a list of the 2 unique foreign IP addresses connecting to my system.

192.168.21.124
192.168.21.12

If you did some fancy monitoring stuff, you can graph the total number of unique nodes connecting, compare the list to your white list of allowed IPs, add non-authorized IPs to your black list, etc.

share|improve this answer

Extending on Gurus answer above, if you have an array you can:

$ L=(11 22 11 33 22 44)
$ echo ${L[@]}
11 22 11 33 22 44
$ L=($(echo ${L[@]} | tr ' ' '\n' | sort -nu))
$ echo ${L[@]}
11 22 33 44
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.