AWK is a text-processing language. It is mainly used to manipulate and process structured data, and to generate reports.

learn more… | top users | synonyms

1
vote
3answers
2k views

How to search file for matching whole lines?

I have a command which sends to stdout a series of numbers, each on a new line. I need to determine whether a particular number exists in the list. The match needs to be exact, not a subset. For ...
4
votes
1answer
3k views

How to use awk separator of single quote?

I'd like to use awk to pull out information from SQL output like the following: (count(distinct serverclass)='2') And need to extract the number, which is 2 in this example. I'd like to do ...
2
votes
2answers
597 views

Grep, find line with minimum x words

Is there a way to find all lines with grep, that contains at least x words?
1
vote
1answer
338 views

mutt can't get password from an encrypted .gpg file

I am trying to get the imap/smtp password for my email accounts in mutt by parsing an earlier decrypted .gpg file via awk, and assigning the result to a variable used in .muttrc # Process the ...
3
votes
5answers
6k views

How to split a text file into multiple text files

I have a text file called entry.txt that contains the following: [ entry1 ] 1239 1240 1242 1391 1392 1394 1486 1487 1489 1600 1601 1603 1657 1658 1660 2075 2076 2078 2322 2323 2325 2740 2741 2743 ...
2
votes
2answers
193 views

Count elements in lines with awk

I have a tab-delimited text file containing data like Col 1 Col 2 1 a 1 b 1 c 1 d 1 d 2 a 2 b 3 a so on and so forth. I would like to transform this structure into ...
2
votes
1answer
595 views

How to calculate and group the time differences of specified syscalls when running strace

I'm planing to do some web performance testing (to see how much time it's spent for each file), but it could be useful for any application. I exactly know existence and how to use different ...
1
vote
2answers
509 views

Parsing .co.uk whois with awk

I wonder if someone could tell me why these awks are not working, they should supply the registrar and the expiry date : ${AWK} -F: '/Registrar:/ && $0 != "" { getline; ...
5
votes
3answers
687 views

Search for files with more than one term (grep, awk?)

I am using a command like this to find files with the word 'term' in them: grep -l term * But I now want to be able to find files which have two different words in them (let's call them termA and ...
0
votes
1answer
137 views

how to get access to environment variables from output file

I want to get variables from a output file specific location,and input file format as below: log1.txt format: [v] Output Data ...
3
votes
3answers
179 views

unix command to verify span of word in text

What unix command(s) can I use to determine the line span that a word appears in text? The "span" being equal to the line number of the last instance of a word minus the line number of the first ...
0
votes
0answers
127 views

XML text transformation

I have a large file part of which is like below: <DataGroup xsi:type="ReportDataGroup"> <SmartReportTemplate DescriptionContentType="text/plain" IsActive="true"> <Name ...
0
votes
4answers
233 views

Bash script to get specific prior context

I am searching through some log files where there are groups of actions performed. At the start of each group there is a line that has information about the group and then lots of verbose informion ...
2
votes
1answer
93 views

sanity check file to produce binary answer

What Unix command(s) can I use to determine the existence of each of several words and the non-existence of each of several words which will produce a binary answer? For example: I want to be sure ...
0
votes
1answer
503 views

IP Address dotted decimal to /8 or /16 notation using bash,sed or awk?

I have an input file that contains a list of ip addresses and the ip_counts(some parameter that I use internally.)The file looks somewhat like this. 202.124.127.26 2135869 202.124.127.25 2111217 ...
2
votes
3answers
440 views

How to substitute multiple lines between delimiters

Given the text example: .... text ,.. {{{python string1 = 'abcde' string2 = '12345' print(string1[[1:3]]) print(string2[[:-1]]) }}} .... text ,.. The [[ and ]] also appear outside of {{{ too. ...
13
votes
5answers
21k views

How to get the pid of a running process using a single command that parse the output of ps?

I am looking for a single line that does return the pid of a running process. Currently I have: ps -A -o pid,cmd|grep xxx|head -n 1 And this returns the fist pid, command. I need only the first ...
0
votes
1answer
292 views

list files with full modification date

I want to produce a list of files in a directory and feed it into a script to generate database entries including their creation dates. The database is Solr, so I need to be able to translate ISO 8601 ...
0
votes
2answers
2k views

Convert an ascii character to ascii hexadecimal representation

How do I convert an ASCII char to ASCII HEX in a AWK script? I want loop by a range of letter example for(i="a"; i<"g"; i++) print i; NOTE: I want loop by a range from hex representation and ...
2
votes
2answers
353 views

Why is awk printing this line more than once?

I have the following ldif: dn: cn=Robert Smith,ou=people,dc=example,dc=com objectclass: inetOrgPerson cn: Robert Smith cn: Robert J Smith cn: bob smith sn: smith uid: rjsmith userpassword: rJsmitH ...
4
votes
2answers
5k views

How can I identify non-ASCII characters from the shell?

Is there a simple way to print all non-ASCII characters and the line numbers on which they occur in a file using a command line utility such as grep, awk, perl, etc? I want to change the encoding of ...
0
votes
1answer
338 views

ping|awk not responsive

While trying melodyping, I noticed that ping <host> |awk '{print $1}' takes about 30 seconds until the first line is actually displayed. Even though w/o piping to awk each ping is returned in ...
0
votes
2answers
182 views

Generate new file with specific structure from text file

On Linux I have the file orig-file.txt. This file include 4 fields now, but they could be less or more (this file is generated by other application). What is the best option to translate the ...
0
votes
1answer
2k views

Parsing out specific URLs from local html file

I want to parse out certain URLs from a saved html-file with a simple bash-script/command, so i can download them via wget laters. What i have so far is: awk 'BEGIN{ RS="<a *href *= *\""} NR>2 ...
0
votes
2answers
111 views

Generate a files list and truncate it in 2 list of X Mb of files each

I need to generate a list of all files in a certain dir, and truncate it in X lists containing each one a fixed amount of data of files. E.g. I have 95 Gb of data. generate the file list ...
1
vote
1answer
949 views

Output high CPU processes using bash shell

I am trying to monitor my CentOS processes using top command to see if there are any processes where CPU usage is greater than X%, below is the command I am using to see if any processes CPU usage is ...
1
vote
1answer
30 views

Looking for a command that will return the proc, func, etc that uses another proc, func, etc

Grep-ing for a function name to see where it's being used returns a list of rows and row numbers. Is there a smart way of using grep and awk to get the name of the calling procedure instead? Thanks
5
votes
4answers
5k views

Print back-reference in regular expression

I was hoping for a way to make sed replace the entire line with the replacement (rather than just the match) so I could do something like this: sed -e "/$some_complex_regex_with_a_backref/\1/" and ...
1
vote
3answers
1k views

Inserting string from xargs into another string

Say I have some command that outputs a bunch of strings. The output from this command could look something like: ... foo bar blargh ... Say I have some set string myString$Blah I want to insert ...
1
vote
1answer
5k views

Extract substring before = sign using awk

Say I have some file with a bunch of lines in the form someString=someMoreCharacters anotherString.blah=foo=bar blah.blah.blah=foo.bar.=foobar Desired output someString anotherString.blah ...
1
vote
6answers
8k views

Pipe output of awk to kill -9

I'm trying to kill any process that has to do with mysql. I'm piecing together a command, and so far, haven't come up with the right solution. One other consideration, is that the last process will ...
2
votes
3answers
2k views

Get LVM size using AWK?

I'm in need to find out the LVM size of image vm001. Let's say I have a LVM volume called /dev/VGgroup/vm001. Now using lvdisplay I can find out the size: --- Logical volume --- LV Name ...
3
votes
4answers
3k views

How to remove lines from large text file using bash

I got a huge text file (log file) in my CentOS which I would like to remove top part of, probably couple of thousand lines each day. (Or probably just split into two) I have search this site and ...
5
votes
3answers
2k views

how to limit find command's output used with option -print0

I'd like to limit output of find command. In the past I used to use for this ls command, e.g: ls *tgz|head -100|xargs -i mv "{}" ../ but I got to know that the result may be unpredictable if name ...
2
votes
2answers
3k views

Is it possible to change the default field separator in awk?

I'm always using awk to manipulate comma separated files, so the first line in my code is always changing the field separator to "," like so: awk 'BEGIN {FS=","} $2 < 20 {print $1}' myfile.csv ...
2
votes
1answer
285 views

awk: equivalent of mysql's “\G” to print tabulated data in multiple lines

Is there an equivalent to mysql's "\G" at the end of the query but awk (or similar) that will to print tabulated data from a text file in multiple lines with the header next to it? Something that will ...
0
votes
3answers
508 views

Oneliner for multiline regex s/,\n]/]/g?

Can someone provide a simple one-liner to remove certain line breaks? In vim I use %s/,\n]/]/g This should be possible with a very simple one-liner IMHO, but how?
0
votes
0answers
187 views

How can I improve or wrap my sed-script?

I instrumented a mvn build to emit profiling data, which I want to analyse later on. Therefore, I made the sed script below, to generate json-data from the profilers output. I call the script with ...
0
votes
1answer
245 views

how to redirect output of command line execution to more than one file according to weight of output

Currently I'm using an awk line to filter a set of files according to certain criteria. How can I redirect the result output to more than one file if the result, for example if the file that would ...
2
votes
2answers
509 views

Awk responds differently based on how an empty argument is specified

I seem to have stumbled on something which is probably a bug in awk, but it could also be a bug in my understanding of bash/awk. I was trying to debug issues where the output of a python program was ...
0
votes
1answer
369 views

usermod + awk invalid syntax

I have been trying to write a command (Solaris server) that adds a user to a specific group. It's a pretty long command so I'll post the whole command here and then break it down. USER=myUser ...
14
votes
6answers
7k views

Removing ANSI color codes from text stream

Examining the output from perl -e 'use Term::ANSIColor; print color "white"; print "ABC\n"; print color "reset";' in a text editor (e.g., vi) shows the following: ^[[37mABC ^[[0m How would one ...
0
votes
1answer
295 views

awk + sed + match complex string

I need useful command to match the string "/pci.../pci.../pci..../scsi..../disk..." with awk or sed? ( I need to put this command in my ksh script ) remark 1 [...] can be any string , and “pci pci ...
5
votes
3answers
5k views

How to pipe awk output (with periodic, continuous input) to output file?

I am trying to write a command that pipes the continuous output of a free command (run every second) to an awk command that parses a specific value (available free memory) and outputs this to a file ...
3
votes
1answer
5k views

“No such file or directory” for /bin/awk on Ubuntu

How can I install awk in this version of Ubuntu 11? I installed it from Ubuntu Software Center and I cannot use it. I'm trying to run an awk script: #!/bin/awk Begin { print strftime("ora ...
0
votes
1answer
169 views

How to reconnect a screen session every time terminal is fired?

I wrote this command to reconnect an active screen session when Terminator is opened. It works find but there has got to be a better way to to it This is goes in the Terminator config file (note ...
1
vote
2answers
456 views

How to pass bash variables (passwords) to IRSSI config file?

I'm using IRSSI and I wonder how can I pass my IRC channel's passwords to the IRSSI config file (~/.irssi/config) without writing them directly in the file. In Mutt I've an awk command that queries a ...
2
votes
2answers
422 views

What's the best command to use to change a timestamp to readable date when 'tail -f'-ing a log file?

I have a program which logs like this: 1324218770.7644 INFO etc What's the best Linux shell program to pipe that through to change the timestamps to a human readable format like Sun 18 Dec 2011 ...
0
votes
2answers
2k views

how to write `awk here document`

I have a bash script: #!/bin/bash gawk -f realmap.awk realmap.log | column -ts: > realmap.csv gnuplot <<-_EOF_ set term png set out 'realmap.png' set xlabel 'index' set ...
1
vote
1answer
796 views

How to solve “Broken Pipe” error when using awk with head

I'm getting broken pipe errors from a command that does something like: ls -tr1 /a/path | awk -F '\n' -vpath=/prepend/path/ '{print path$1}' | head -n 50 Essentially I want to list (with absolute ...