AWK is a text-processing language. It is mainly used to manipulate and process structured data, and to generate reports.

learn more… | top users | synonyms

0
votes
2answers
34 views

Count the number of lines in a piped input?

While running ps -eaf I see a list of running processes. Is it possible to count the number to lines using awk?
1
vote
0answers
59 views

Automatically Split a Huge TeX File Into Many Smaller Files

How would I automatically split the sections or chapters of a huge TeX file into many smaller TeX files? This somewhat works: awk -v RS='\\chapter{' '{print "\\chapter{"$0 > (NR".tex")}' ...
0
votes
4answers
198 views

Changing multiple lines in to single line with spaces in linux

I have following data with multiple lines username: joe empid: 1111 status: resigned username: tom emid: 1234 username: kate empid: 2222 status: resigned What I want is to get the data in to ...
2
votes
1answer
1k views

Round off the decimal value in awk

i want to round off the value 262.5 to 263 using awk i have tried the below echo "262.5" | awk '{printf "%.f\n", $1}' My expected answer is 263
1
vote
2answers
712 views

Sum a particular column and take the unique record of another using AWK only

I have an input file like the following: 19 | 17 | 1 | 89 | C | 9 | 100 | 9 | PL | 2175 20 | 17 | 1 | 89 | C | 9 | 100 | 9 | PL | 2175 21 | 17 | 2 | 89 | C | 9 | 100 | 9 | PL | 2175 23 | 17 | 2 | 89 ...
5
votes
4answers
10k views

How to get video duration in seconds?

How can I get video duration in seconds? What I've tried: ffmpeg -i file.flv 2>&1 | grep "Duration" Duration: 00:39:43.08, start: 0.040000, bitrate: 386 kb/s mediainfo file.flv | grep ...
-1
votes
1answer
1k views

How can I find my awk version?

If I want to know the version of awk I get the following: $ awk --version awk: not an option: --version Checking in man awk I see that my awk is mawk - pattern scanning and text processing ...
1
vote
2answers
269 views

How can I sort and de-dupe a very long list of words?

I have a 400 megabyte file. The file is a list of words, each word on a single line. End of lines are a mix of cr/lf and lf. Some of the words are unicode. I'd prefer answers for Linux or Windows ...
0
votes
2answers
676 views

Attempting to read data from a text file and inputting the variable text to a script

I have a file with 25 single word entries, I need a script to read through the file one line at a time and input the variable phrase into a second script that will generate a individual file based on ...
0
votes
2answers
116 views

Trying to get example from Awk and Sed book to work on Ubuntu 13.04

I have been developing Windows Software for years. I am trying to branch out and learn Linux. It will really help with my new job. I have picked up the book, "Awk and Sed" 2nd Edition. I am ...
0
votes
1answer
204 views

Find a pattern in a line, and append that matched pattern to the same line

I'm trying to pull a job number out of notes listed in a csv, and then append that number to the end of the line, preferably through sed, awk, grep or perl (which I have installed in Cygwin. Here's a ...
1
vote
2answers
400 views

Change names in a column in bash

I have a tab-separated file in Linux that looks like this ~$ head list.tab "x" "1" "FUHGF.jnf.m22-1" "2" "HDFHFEY.gfs.d2-1" "3" "KJFGJF.fr.md2-1" "4" "SDFSDIB.gfd.rtl2-1" "5" "FGJFGOB.hgd.k2-5" "6" ...
0
votes
3answers
2k views

Better way to replace timestamp by date when reading a squid log

Tonight i was trying to look at my proxie's lines, and I came up with this, which seems all but KISS. How would you do it simply ? awk '{ trunc=sprintf("%.0f", $1) ;$1="" ; system("echo -n `date +%R ...
0
votes
1answer
64 views

Sed/Awk line double

I have a list like this. Bob Jim Steve Is there a way in bash to turn that into a list like this? <a href="http://example.com/Bob"> Bob <a href="http://example.com/Jim"> Jim <a ...
0
votes
1answer
83 views

trying to extract two tables at a time from http files with 40 tables each

I have about 20 webpages. Each page has top banner navigation and then has information on up to 20 vehicles. There are 2 tables per vehicle. The logical flow is: page navigation, table 1 for ...
3
votes
3answers
335 views

Need to sed awk or grep output to extract strings

I have output which looks like this: "vancouver/news/vancouver-voices": { "status": 0, "time": 0, "timestamp": 1371298701 }, "vancouver/news/vancouvers-march-second-annual-success": { ...
-2
votes
1answer
77 views

Grep word in one file, and use that word to match in another file, adding the thing that follows the match

I want to grep several words in file1, and use each word to grep what follows after its match in file2. And then I want to add the string that followed the match to the word I used into file03, so ...
1
vote
1answer
1k views

using sed/awk to parse XML tags

For the record, I've spent several days working on this with no luck. I'm working with XML files with data containing something like the following: <row id="67581917031" name="4022" ...
0
votes
3answers
6k views

remove text from file

I want to remove some text from file1.txt. I put the text in the file tmp and do: grep -f tmp file.txt But it gives me only the difference. The question is how to remove the difference from ...
1
vote
1answer
329 views

How to render two columns preserving all whitespace after the first column with awk or alternative?

my output comes in human-readable figures with one space followed by directory names: I am currently using ... awk '{ printf "%-20s %-40s\n", $1, $2 }' input 1G foo 1.5M foo baz 5K foo spaces in ...
0
votes
1answer
189 views

How to split file on basis of size without breaking last line of split file

I have a big file and want to split the file on the basis of size. For e.g. Main file: 10 Mb and then I want to split it into 2 mb parts. But the issue is if I use split -b it breaks the last line ...
1
vote
1answer
140 views

sed/awk: modify source file?

I'm trying to script some changes I need to make to a couple of config files. It's worth mentioning that I'm an expert with neither sed nor awk, though I'm far better with sed. Right now, I'm doing: ...
0
votes
1answer
231 views

Using awk or cut command to get CPU percentage

I am trying to execute a Linux command through a PHP script to get the CPU percentage for each core: … with cut: $output = null; passthru("mpstat -P ALL | grep 0 | tr -s ' ' | cut -d ' ' -f 4", ...
0
votes
1answer
152 views

redirect awk information to stdout

I am trying to gather information for CPU use for all the processors in a text file. mpstat -P ALL -u 1 | awk '{print $4}' > cpu.txt && mptat -P ALL -u 1 | awk '{print $4}' | tee cpu.txt ...
1
vote
1answer
402 views

netcat output into separate files using string delimiters

I want netcat to pipe the output of a tcp/ip connection to commands that examine the output and split it into distinct files separated by the strings "msg" and "/msg" from the output, something like ...
0
votes
1answer
144 views

Counting by number of occurrences in first column

I have an input file like this: ATTACK-RESPONSES id check returned root BACKDOOR ACKcmdC trojan scan BACKDOOR hack-a-tack attempt BACKDOOR WinCrash 1.0 Server Active ICMP Destination Unreachable ...
0
votes
2answers
220 views

Computing average on a list of values for each value

I have a huge file (50k lines) with the following format: 52370581 2116 17996781 4314 47818829 2584 61421045 2158 ... 25145813 2274 The first value can be present multiple times and I want to ...
4
votes
1answer
5k views

Using grep to remove lines from a file which contain a string from another file

I have a file containing words (one per line) such as Dog Fish Cat Shoes I have a secondary file in CSV format such as 1,shoes,red 2,apple,black 3,fog,blue I would like to use grep with ...
0
votes
1answer
182 views

Write output to $TMPDIR/$variable : file does not exist (should be created)

I have the following .sh script in which I am trying to do all major writing in $TMPDIR. However, when it gets to line 86, it returns the following error: ...
9
votes
3answers
4k views

sed: how to replace line if found or append to end of file if not found?

With a single input file that only contains comments (starting with #) and VARIABLE=value lines, is it possible to replace a value for a single variable if found and, otherwise, append the pair to the ...
0
votes
1answer
446 views

Mass migration of EXIF GPS data (a.k.a. Escape from iPhoto)

I am trying to move my library of photos (over 12000) out of iPhoto for use in Adobe Lightroom and would like to include the GPS data I've added to the photos using iPhoto's in-app feature (This data ...
-1
votes
2answers
913 views

Extract IP addresses in each line of file in shell

I have this input file in ubuntu: 146.14.142.96.17747 197.102.40.184.13748: 146.14.142.96.17747 197.102.40.184.13749: 146.14.142.96.17747 197.102.40.184.13750: 146.114.142.96.17747 ...
0
votes
4answers
405 views

Regular expression to find senders IP using grep or other languages

I have input file that contains: 19:04:01.631948 IP (tos 0xc0, ttl 1, id 0, offset 0, flags [none], proto UDP (17), length 48) 181.173.82.61.1985 > 250.66.33.195.1985: HSRPv0-hello 20: ...
0
votes
3answers
770 views

Match and remove first and second pattern within xml tags

How can I match and remove first and second pattern within xml tags using sed or awk? Here is the example <data>A78-1-1134-HI-1</data> <data>T78-12-1346-AG-2</data> ...
1
vote
3answers
6k views

extract string from file using shell script or awk

i have txt file that contains some website.i'd like to extract websites how can i do this? xhr_object.open("GET",filename,false ); <li><a href="http://www.dakar.com" ...
-1
votes
2answers
155 views

Replace all numbers following a specific string with another number using sed or awk

I have tried several things but got no luck. I have an input file that has a lot of fields like this one weight=0.123456 mixed with other fields. I want to multiply all such weights by a factor, say ...
0
votes
1answer
231 views

Counting duplicates lines from a stream

I'm currently parsing apache logs with that command: tail -f /opt/apache/logs/access/gvh-access_log.1365638400 | grep specific.stuff. | awk '{print $12}' | cut -d/ -f3 > ~/logs The output is a ...
0
votes
3answers
871 views

Removing first line with pattern with _both_ awk and sed

Fairly easy. I'm looking for the simplest approaches in both sed and awk for removing the first line in a file that contains a specific string.
1
vote
1answer
288 views

error occur when i run awk command write in .sh file and run in cygwin terminal

i have cygwin terminal for run shell script(.sh ) file...it work good..but when i add awk command in shell script and then run Error occur /usr/bin/gawk.exe: error while loading shared libraries: ...
1
vote
1answer
546 views

awk - max value of a rolling average

I have a bash command which gives me the highest value in the 9th column of a file: cat "log.txt" | grep 1923 | awk '{print $9}' | sort -n | tail -1 log.txt is a log of CPU usage for process 1923 ...
0
votes
3answers
216 views

output the items that showed up 2 times and more

How can I select the ips that showed up 4 times or more , for example here? the input file is 192.168.1.28 192.168.1.100 192.168.1.31 192.168.101.2 192.168.1.31 192.168.11.02 192.168.1.31 ...
1
vote
6answers
502 views

match string in awk

How can I search the file to find the lines that have SRC= , for example here? i mean how can i find source IP address in this file using awk for example Mar 10 03:17:12 ubuntu kernel: ...
1
vote
2answers
360 views

using awk with parallel

I have about 3,000 files that are each 300MB, and I'd like to search them for a series of substrings as quickly as possible with my 16 core server. This is what I tried but it doesnt seem to ...
0
votes
1answer
1k views

Grep issue (match two strings on same line)

Here I have some grep command which is not working correctly: cat file1.txt: apples Date: Sun, 24 Feb 2013 8:14:06 -0800 peaches melons cherry sky cloud green purple yellow cat file2.txt: apples ...
0
votes
1answer
124 views

Bash script is printing wrong characters in my variable

I'm trying out the following thing in my terminal: spark 0 76 100 | awk '{print substr($0,4,3)}' outputs: ▆ If I do: GRAPH=$(spark 0 25 100 | awk '{print substr($0,4,3)}') printf "%s" $GRAPH ...
1
vote
1answer
434 views

Excel singlecolumn to multiple columns based on values

I have 1 column like below: AAA 999888 BBB 1111 BBB 2222 AAA 999777 BBB 3333 AAA 999666 BBB 4444 CCC Doit ... Consider above up to a few thousand rows. I want to convert it to below 3 column table ...
0
votes
2answers
598 views

extracting words using awk

I have a data file that is made up of lines that all contain FG=???? (where ???? represents a four digit number that is different on each line ). I would like to extract this from each line and save ...
0
votes
1answer
130 views

Can't use awk in terminal

I want to use awk command to do something, I input awk in terminal, but the terminal show me that awk: cannot open /home/user/myexe/external_symbols.awk (No such file or directory) Someone ...
0
votes
2answers
262 views

awk IGNORECASE not working. weird

One script i used for years on production machine, stopped working after a linux system re-installation. after days of digging it turned out that the special variable IGNORECASE=1 has no effect. I ...
2
votes
3answers
2k views

How can cat/print a file except the last 2 lines?

How can get a file, except the last (for instance) 2 lines, with standard or GNU shell tools? (bash, tail, cat and so on.)