0
votes
1answer
36 views

Script to reference another file first

Hi I have a script and in it I need to get the hostnames from IPs that the script generates. The script will generate those IPs into 1 neat file with only 1 column. It is literally just IP address (1 ...
0
votes
1answer
24 views

I'm trying to mirror dns queries generated by TCPDUMP filter and replay them with dig or any other utility

I'm not a very experienced script writer at all. I did find some useful hints in generating an stdout from tcpdump that generates a live feed if you will which is what I want, the problem is handling ...
1
vote
5answers
37 views

Turn comma-separate rows into 1 column

I have a list of users basically from /etc/group, Id like to basically format that line into one column, I just need the users. So if a field looks like this: bob,john,jane,sam,joyce Then I'd like ...
1
vote
3answers
56 views

Parsing the output of “uptime” with bash

I would like to save the output of the uptime command into a csv file in a Bash script. Since the uptime command has different output formats based on the time since the last reboot I came up with a ...
0
votes
1answer
34 views

output awk print to two separate files

Here is my current code md5sum filename.iso | awk '{print $1} {print $2}' This gives me two separate lines. The first line is the md5 of the file and the second line is the name of the file. I ...
0
votes
1answer
45 views

How to capture diff output from find command exec?

I am using Ubuntu (Precise 12.04) I am trying to write some bash script to see if file is changed via awk replace by using diff command. I did a test with the following: #! /bin/bash diff fileA.htm ...
1
vote
2answers
76 views

Replacing every 10th pipe with new line in unix

Lets say I have fields: name, number, id I have a data file: name1|number1|id1|name2|number2|id2...etc I want to replace every 3rd pipe with a new line or '\n' so I get: name1|number1|id1 ...
1
vote
2answers
39 views

How do I print multiple sections in a text file?

I have a text file looking something like this: *FILESTART line_a line_b line_c *INCLUDE file_A.key file_B.key *INCLUDE file_1.key file_2.key file_3.key *SOMETHING_ELSE line_x line_y line_z *END I ...
0
votes
0answers
94 views

stdbuf not working while unbuffer does

I am trying to apply timestamps to stdout of a process. For the proper timestamps to be applied, I attempt to unbuffer stdout of the process. This works with unbuffer but not with stdbuf as I would ...
0
votes
2answers
103 views

Replace string in string but not when its somewhere between double square bracket

I have a long string from mysql field containing the data of a mediawiki page. I have to replace a string in that mediawiki page but not when the string is in mediawiki link. The mediawiki link is ...
1
vote
2answers
62 views

Is there a way to print 2 lines (if any) before and after a target or print a placeholder if not?

I'm not native english speaker so I hope to be clear. I know about grep -C 2 "TARGET" inputfile to select 2 rows before and after the row of the TARGET, but I'm not able to use it to manage my ...
0
votes
1answer
518 views

Suppress awk warnings

On a script, I get some warning of the form awk: warning: escape sequence \( treated as plain (. Since I want to use the output of that script to do stuff, I want it to output only results and not ...
1
vote
1answer
49 views

AWK does not work correctly in my system [duplicate]

i cant run this code because of an error awk 'BEGIN {x=0;y=0} { match($13,/([0-9]+)/,a); x = x + a[1]; match($12,/([0-9]+)/,b); y = y + b[1]} END {print "rxPackets:" x " txPackets:" y}' yourfile ...
0
votes
2answers
77 views

sum numbers in different lines in shell

i have 8000 lines similar to this <Flow flowId="1" timeFirstTxPacket="+0.0ns" timeFirstRxPacket="+924100.0ns" timeLastTxPacket="+199984927558.0ns" timeLastRxPacket="+199675473275.0ns" ...
1
vote
1answer
26 views

`nawk` “doesn't work” with `make`? Why?

I have the following alias: alias mca_color='nawk '"'"'BEGIN { arr["Creating shared object"] = "1;31";} \ { l = $0; for (pattern in arr) \ { gsub(".*" pattern ".*", "\033[" arr[pattern] ...
0
votes
1answer
47 views

Move numbers in a filename to the start

I have several files in a folder, with names in the form: SOMETEXT1 number SOMETEXT2.mp3, or SOMETEXT number.mp3. I want to rename these to number SOMETEXT SOMETEXT2.mp3, or number SOMETEXT.mp3. Using ...
0
votes
4answers
134 views

Changing multiple lines in to single line with spaces in linux

I have following data with multiple lines username: joe empid: 1111 status: resigned username: tom emid: 1234 username: kate empid: 2222 status: resigned What I want is to get the data in to ...
0
votes
2answers
632 views

Attempting to read data from a text file and inputting the variable text to a script

I have a file with 25 single word entries, I need a script to read through the file one line at a time and input the variable phrase into a second script that will generate a individual file based on ...
1
vote
2answers
283 views

Change names in a column in bash

I have a tab-separated file in Linux that looks like this ~$ head list.tab "x" "1" "FUHGF.jnf.m22-1" "2" "HDFHFEY.gfs.d2-1" "3" "KJFGJF.fr.md2-1" "4" "SDFSDIB.gfd.rtl2-1" "5" "FGJFGOB.hgd.k2-5" "6" ...
0
votes
3answers
2k views

Better way to replace timestamp by date when reading a squid log

Tonight i was trying to look at my proxie's lines, and I came up with this, which seems all but KISS. How would you do it simply ? awk '{ trunc=sprintf("%.0f", $1) ;$1="" ; system("echo -n `date +%R ...
0
votes
1answer
63 views

Sed/Awk line double

I have a list like this. Bob Jim Steve Is there a way in bash to turn that into a list like this? <a href="http://example.com/Bob"> Bob <a href="http://example.com/Jim"> Jim <a ...
3
votes
3answers
294 views

Need to sed awk or grep output to extract strings

I have output which looks like this: "vancouver/news/vancouver-voices": { "status": 0, "time": 0, "timestamp": 1371298701 }, "vancouver/news/vancouvers-march-second-annual-success": { ...
4
votes
1answer
3k views

Using grep to remove lines from a file which contain a string from another file

I have a file containing words (one per line) such as Dog Fish Cat Shoes I have a secondary file in CSV format such as 1,shoes,red 2,apple,black 3,fog,blue I would like to use grep with ...
0
votes
1answer
149 views

Write output to $TMPDIR/$variable : file does not exist (should be created)

I have the following .sh script in which I am trying to do all major writing in $TMPDIR. However, when it gets to line 86, it returns the following error: ...
-1
votes
2answers
130 views

Replace all numbers following a specific string with another number using sed or awk

I have tried several things but got no luck. I have an input file that has a lot of fields like this one weight=0.123456 mixed with other fields. I want to multiply all such weights by a factor, say ...
1
vote
1answer
434 views

awk - max value of a rolling average

I have a bash command which gives me the highest value in the 9th column of a file: cat "log.txt" | grep 1923 | awk '{print $9}' | sort -n | tail -1 log.txt is a log of CPU usage for process 1923 ...
0
votes
3answers
162 views

output the items that showed up 2 times and more

How can I select the ips that showed up 4 times or more , for example here? the input file is 192.168.1.28 192.168.1.100 192.168.1.31 192.168.101.2 192.168.1.31 192.168.11.02 192.168.1.31 ...
1
vote
6answers
351 views

match string in awk

How can I search the file to find the lines that have SRC= , for example here? i mean how can i find source IP address in this file using awk for example Mar 10 03:17:12 ubuntu kernel: ...
1
vote
2answers
263 views

using awk with parallel

I have about 3,000 files that are each 300MB, and I'd like to search them for a series of substrings as quickly as possible with my 16 core server. This is what I tried but it doesnt seem to ...
0
votes
1answer
121 views

Bash script is printing wrong characters in my variable

I'm trying out the following thing in my terminal: spark 0 76 100 | awk '{print substr($0,4,3)}' outputs: ▆ If I do: GRAPH=$(spark 0 25 100 | awk '{print substr($0,4,3)}') printf "%s" $GRAPH ...
2
votes
3answers
2k views

How can cat/print a file except the last 2 lines?

How can get a file, except the last (for instance) 2 lines, with standard or GNU shell tools? (bash, tail, cat and so on.)
0
votes
2answers
374 views

creating CDF data using bash or awk or perl

we have some data like: 12 0 13 0 20 0 25 1 64 4 77 1 89 100 1201 204 I'd like to get this output: 3 0 5 1 6 4 7 100 8 204 explanation: we have 3 AS(autonomous systems) that have degree of ...
3
votes
1answer
2k views

Counting occurrences in first column of a file

We have this file: 1 2 1 3 1 2 3 3 52 1 52 300 and 1000 more. I want to count the number of times each value occurs in the first column. 1 3 3 1 52 2 This means we saw 1 three times. How ...
4
votes
3answers
89 views

Output numbers from a file in a different order

I have a file that contains: 1 2 3 4 1 3 5 4 8 3 2 1 Each line has a different number of digits. And there are 1000 more like this. I want the output like this: 1 2 2 3 3 4 4 0 1 3 3 5 5 4 4 8 8 ...
2
votes
4answers
888 views

How do you remove all occurrences of values in one list from another list?

I have a list of symbols such as... wer sfe efo How do I remove all instances of these (unique) symbols from another list of (non-unique) symbols? So in the following list, the lines starting ...
1
vote
2answers
1k views

How to extract e-mail or domain from mixed data file in linux

File content: 17541 From Email subscription@test.com Inbound Policy Manager Envelope Analysis Profiler 17541 From Email subscription@yahoo.com Inbound Policy Manager ...
1
vote
5answers
3k views

How to grep column after second pipe

Example: 3|100|test@test.com|0|0|6:1,10,11,12,13,2,3,4,5,6,9|7:1,10,11,13,16,2,4,5,6,9| Expected view after grep: test@test.com
1
vote
2answers
174 views

Using sh -c in terminal

I have this command line that I enter into terminal and it works as intended: dscl . -readall /Users UniqueID | awk '/^RecordName:/ {name=$2}; /^UniqueID: / {if ($2 > 500) print name}' What I ...
2
votes
1answer
76 views

Substitute large numbers of files according to style guide

I'd like to do a wholesale reformatting of our tests, and I'm cleaning up some inconsistent capitalization. I'm thinking of using awk to do this, since sed falls a little short, and since I need ...
1
vote
2answers
378 views

Parsing .co.uk whois with awk

I wonder if someone could tell me why these awks are not working, they should supply the registrar and the expiry date : ${AWK} -F: '/Registrar:/ && $0 != "" { getline; ...
0
votes
4answers
227 views

Bash script to get specific prior context

I am searching through some log files where there are groups of actions performed. At the start of each group there is a line that has information about the group and then lots of verbose informion ...
0
votes
1answer
391 views

IP Address dotted decimal to /8 or /16 notation using bash,sed or awk?

I have an input file that contains a list of ip addresses and the ip_counts(some parameter that I use internally.)The file looks somewhat like this. 202.124.127.26 2135869 202.124.127.25 2111217 ...
8
votes
5answers
14k views

How to get the pid of a running process using a single command that parse the output of ps?

I am looking for a single line that does return the pid of a running process. Currently I have: ps -A -o pid,cmd|grep xxx|head -n 1 And this returns the fist pid, command. I need only the first ...
0
votes
1answer
1k views

Parsing out specific URLs from local html file

I want to parse out certain URLs from a saved html-file with a simple bash-script/command, so i can download them via wget laters. What i have so far is: awk 'BEGIN{ RS="<a *href *= *\""} NR>2 ...
1
vote
1answer
833 views

Output high CPU processes using bash shell

I am trying to monitor my CentOS processes using top command to see if there are any processes where CPU usage is greater than X%, below is the command I am using to see if any processes CPU usage is ...
1
vote
6answers
5k views

Pipe output of awk to kill -9

I'm trying to kill any process that has to do with mysql. I'm piecing together a command, and so far, haven't come up with the right solution. One other consideration, is that the last process will ...
3
votes
4answers
2k views

How to remove lines from large text file using bash

I got a huge text file (log file) in my CentOS which I would like to remove top part of, probably couple of thousand lines each day. (Or probably just split into two) I have search this site and ...
4
votes
3answers
1k views

how to limit find command's output used with option -print0

I'd like to limit output of find command. In the past I used to use for this ls command, e.g: ls *tgz|head -100|xargs -i mv "{}" ../ but I got to know that the result may be unpredictable if name ...
2
votes
1answer
237 views

awk: equivalent of mysql's “\G” to print tabulated data in multiple lines

Is there an equivalent to mysql's "\G" at the end of the query but awk (or similar) that will to print tabulated data from a text file in multiple lines with the header next to it? Something that will ...
2
votes
2answers
389 views

Awk responds differently based on how an empty argument is specified

I seem to have stumbled on something which is probably a bug in awk, but it could also be a bug in my understanding of bash/awk. I was trying to debug issues where the output of a python program was ...