AWK is a text-processing language. It is mainly used to manipulate and process structured data, and to generate reports.

learn more… | top users | synonyms

0
votes
1answer
9 views

escaping awk with remote ssh command and bash that is already escaped

Hello. ssh ufk@10.0.0.2 "bash -lc 'pm2 list | grep app | awk { print $3} '" I need to run the command I pasted here. the problem is that i'm already escaping twice... the bash with " and the pm2 ...
8
votes
6answers
13k views

How to get video duration in seconds?

How can I get video duration in seconds? What I've tried: ffmpeg -i file.flv 2>&1 | grep "Duration" Duration: 00:39:43.08, start: 0.040000, bitrate: 386 kb/s mediainfo file.flv | grep ...
0
votes
1answer
24 views

Pass a path with parenthesis and space in system call inside awk script

Inside my simple awk script, I call system command #!/bin/bash Test='/home/software/Other/new (Applet)' ls "${Test}" var=$(ls "${Test}") echo $var awk -vTest="$var" 'BEGIN { ...
1
vote
3answers
56 views

How to use grep or awk to save every line from text file as new file?

I am a scripting beginner. What i am trying to do. I have a text file that looks like this (example): hello hi there apple I am trying to make a script that saves every line as a separate new file ...
0
votes
0answers
8 views

Mac OS X Find and Replace API

Are the Find and Replace operations under the Edit menu in Acrobat or Word simply a front end for the unix awk, sed or grep commands? Where can I find more information about what's going on underneath ...
-1
votes
0answers
15 views

What does cygwin delete when I install new packages?

I would like to install new cygwin packages, however, I obviously want to keep my old ones. I selected the new packages that I wanted to use (gawk, et. al.), and installed again. However, this ...
0
votes
1answer
34 views

need to get average idle time using mpstat

I want to get average idle time of the server using mpstat. The problem I am having is %idle is not in same columns in all the versions of linux. example 1: [root@testserver ~]# mpstat 1 2 Linux ...
0
votes
3answers
49 views

Match the column heading and print the values of the column using awk

I have a data in the file in columns. I want to write a script, which displays the data of the column matching the column heading. C-1 C-2 C-3 C-4 C-5 C-6 C-7 C-8 C-9 10 30 35 20 9 65 87 30 ...
0
votes
1answer
35 views

Monitoring changes in file with bash

I'm running DHCP server on local network which calls this script every few seconds: dumpleases | awk '{print $2}' > leasesnow #get list of all IPs leased in a file, one per line ADDED=`awk ...
2
votes
3answers
352 views

Copied directory appears to become larger at destination

I have the following code as part of a shell script: while [ $(ps -ef | awk '{print $2}' | grep -F "$CPPID") ]; do sleep 10 awk -v "usbsize=$(/bin/df | grep -F $DEVICEMOUNTPOINTQ | awk ...
2
votes
3answers
49 views

Extract version string from filename

We have a list of filenames in file a.txt with their version string embedded in them. Eg: gson-2.1 xmlParserAPIs-2.4.0 acrobat-1.1 orai18n-mapping-12.1.0.2 jdbc-se2.0 eclipse-core-runtime-20070801 ...
0
votes
2answers
2k views

how to write `awk here document`

I have a bash script: #!/bin/bash gawk -f realmap.awk realmap.log | column -ts: > realmap.csv gnuplot <<-_EOF_ set term png set out 'realmap.png' set xlabel 'index' set ...
5
votes
4answers
4k views

SQL like group by and sum for text files in command line?

I have huge text files with two fields, the first is a string the second is an integer. The files are sorted by the first field. What I'd like to get in the output is one line per unique string and ...
0
votes
1answer
49 views

Using ' (' (space followed by parenthesis) as field separator in awk

In an awk script I am trying to use ' (' as the field separator. However unless I escape the bracket with double back-slash, like this: BEGIN {FS=" \\("} it does not work. If I use FS=" \(" I get ...
0
votes
1answer
50 views

Script shell get id and kill process

Please, i can't find the error in my command. ssh -t root@$machine -x "sshpass -p 'ubuntu' ssh -t root@$address -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/nul -x 'ps -ef | grep ...
1
vote
2answers
84 views

Best way to increment numbers in one line in Linux [closed]

I have one line with lot of entries among which is also time. ('DEFAULT',17,NULL,'2014-07-14 10:30:00','Something','2014-07-14 06:30:00',1),('DEFAULT',26,NULL,'2014-07-14 ...
0
votes
2answers
52 views

Extending awk one-liner from identifying matching pair of columns (row by row), to multiple columns

I am using Win7 with Cygwin. I have a one-liner that will work for identifying matches in a PAIR of columns, but when I try to introduce additional columns (i.e. >2) I cannot get it to work. So the ...
1
vote
1answer
489 views

Transform single column to multiple columns based on values

I have 1 column like below: AAA 999888 BBB 1111 BBB 2222 AAA 999777 BBB 3333 AAA 999666 BBB 4444 CCC Doit ... Consider above up to a few thousand rows. I want to convert it to below 3 column table ...
2
votes
2answers
60 views

how to insert text at the beginning of each paragraph in bash

I have a file with multiple paragraphs separated by blank line. Technically they are not paragraphs just sections of text separated by blank line. I want to number the paragraphs so to speak by ...
1
vote
2answers
456 views

Change names in a column in bash

I have a tab-separated file in Linux that looks like this ~$ head list.tab "x" "1" "FUHGF.jnf.m22-1" "2" "HDFHFEY.gfs.d2-1" "3" "KJFGJF.fr.md2-1" "4" "SDFSDIB.gfd.rtl2-1" "5" "FGJFGOB.hgd.k2-5" "6" ...
0
votes
2answers
69 views

How to copy file using awk based on time creation to some directory

I trying to copy a file to directory using awk filter based on time creation file #!/bin/bash dir=/home/resaputra23/pyexample/python2/* des=/home/resaputra23/pyexample/backup/ file1= stat $dir | awk ...
0
votes
1answer
36 views

Skip the first 6 lines/rows in a text file with awk

How can I skip the first 6 lines/rows in a text file (input.txt) and process the rest with awk? The format of my awk script (program.awk) is: BEGIN { } { process here } END { } My text file ...
1
vote
1answer
47 views

Multiple input files and output files in awk

I have a folder full of text files named 000.txt to 181.txt. How can I process all of them (on the command line) with the same awk script (program.awk) and send them to their respective output files ...
1
vote
1answer
51 views

How to parse a space delimited text file

I have a text file like this: Fam1000: CMIN|CMIN_9-RA CMIN|ABC_7-RA GCLA|EFX5.1 GCUC|GCUC_7-RA Fam1001: GCLA|EFX6.1 GCLA|EFX7.1 Fam1002: GCLA|EFX5.1 GCLA|EFX2.1 GCUC|GCUC_8-RA GCUC|GCUC_8-RA Fam1003: ...
1
vote
2answers
57 views

Awk to merge two files using hash

I have two files of the same length, and I want to swap out one files 3 columns with another file's contents. I want to print the following: f1Col1 f1Col2 f1Col3 f1Col4 f1Col5 f1Col6 f2Col1 f2Col2 ...
1
vote
1answer
28 views

How to serach for a line which contains specific string in Bash?

I've got a file which contains some amount of lines, I want to look for a specific string, and if the file contains it, then I want to print a different line from the same file. for example, let's ...
0
votes
0answers
36 views

Multiple replace - using sed or awk

I have a code snippet like below CREATE TABLE TABLE1( ID integer NOT NULL, ROW_CREATED varchar (26) NULL, BATCH_LOG_ID integer NULL, TRANSACTION_TYPE varchar(5) NULL, PARTITION_COL integer ...
1
vote
1answer
19 views

SED/AWK/PERL(GNU userland) - how to substitute pattern A but not pattern B?

First, I am using GNU userland so you can assume I use Bash, GNU coreutils... I have a HTML and I want to change all href="foobarz" into href="" However, I don't want to change any line ...
0
votes
2answers
2k views

Piping tail -f into awk

Greeting, I am trying to pipe tail -f into awk to monitor a logfile in realtime, but according to examples, there should be not problem but I can't get it to work. here is the command I'm running ...
0
votes
3answers
146 views

Replace string in string but not when its somewhere between double square bracket

I have a long string from mysql field containing the data of a mediawiki page. I have to replace a string in that mediawiki page but not when the string is in mediawiki link. The mediawiki link is ...
0
votes
0answers
21 views

Strang, same simple awk script has different results on different machines

This is the strangest mystery ever -- same simple awk script has different results on different machines. The awk script is really simple: $ cat awk_test.awk BEGIN { troubleshoot = 1 } # GET: ...
0
votes
2answers
28 views

list file and using awk to get comment

I want to get the all comment of all files in a folder. And what I have tried is ls -R | awk '/<!--/,/-->/' >> result But it shows nothing in the result file. But if I use the awk ...
0
votes
1answer
49 views

Get the Group name which has GID=0

I wanted to write a shell script in which it needs to give the group name whose GID=0.I used awk command like this, awk -F ':' {$3==0} /etc/group | cut -d ':' -f1 It works fine in redhat,aix.But ...
1
vote
1answer
98 views

piping tail -f into sed or awk

There is another question which is nearly the same, but none of the suggestions from its answer works for me: Piping tail -f into awk I'm trying to tail -f a logfile but want to replace \r with \n ...
7
votes
1answer
6k views

Awk doesn't work when inside double quotes

This command doesn't work: ssh $HOST "ls -l | awk '{print $1}'"` Above ignores the command awk. I think it might be because of the double quotes? Also, how would I add another set of double ...
0
votes
2answers
30 views

quote-enclosed fields in input to awk

I'm trying to process a webserver logfile in awk. Like 123.222.333.444 - - [24/Feb/2015:13:09:19 +0100] "GET / HTTP/1.1" 200 15852 "https://www.google.dk/" "Mozilla/5.0 (Windows NT 6.1) ...." ...
0
votes
2answers
32 views

Sort on ID, then sort on date, then remove lines which have the older date?

I have a couple of tab delimited files. Each file is structured like so: ID Title Rating Date_Rated What I want to do is merge all these files into one, ...
1
vote
2answers
308 views

bash sed/awk: replace string from a file searching in another file

i have a question about search and replacement on Debian. I have two files. One with: a:b c:d e:f and the other with: e c a` In the second file I want to replace a with b, c with d, e with f. ...
0
votes
0answers
79 views

Copy huge file skipping first line using dd in Linux

I was trying to copy the content of a huge file (10GB) to another skipping the first line (as in head -n1). I tried multiple ways of head, tail, awk and sed. Settled on using tail -n+2 > ./xab.1 ...
0
votes
1answer
43 views

Replace with sed a tag value under a specific section when matching a line

I would like to change a line when another line matchs under a section of a file using the sed command (or awk). Here is an exemple of the file : Extract of MyFic.txt : <PlayList> ... ...
0
votes
3answers
72 views

using awk on only files which START with 'xyz'

I have 3 files in a directory: xyz1.txt, xyz2.txt, rst_xyz.txt and I want to work with ONLY the first two (those which start with xyz). The command ls | awk '/xyz/{print $1}' will return all three of ...
0
votes
2answers
118 views

Unable to get accurate record count of a csv file

I have a .csv file that has some values formatted as paragraphs with line breaks or sometimes bullet points. "STAT","ID","DESC" "UPD", "1", "Updated" "CHG", "2", "Changed" "UPD", "3", "Updated. ...
0
votes
1answer
26 views

How to fetch the string “Hi there” from the below array in unix

I only want to fetch Hi there and store it in a variable VAR1. So basically VAR1 should contain Hi there VIEW_COL[1]="Hi there,USA,Y" This is what i have tried so far. Column_Display_Name=`echo ...
0
votes
2answers
117 views

Text deletion between patterns across multiple lines with respect to text inside pattern

I have a block of text I need to delete, however, only if it contains specific text inside the block: ... <script language="JavaScript"> var somethingA = 0; var somethingB = 0; var ...
0
votes
3answers
23 views

Filtering 2nd field from a data set and then using uniq on the output

I've got a dataset that goes like this: AAAAA 11111 Data1 AAAAA 11111 Data2 AAAAA 11111 Data3 AAAAA 11112 Data4 AAAAA 11112 Data5 AAAAA 11112 Data6 AAAAA 11112 Data7 AAAAA 11113 Data8 AAAAA 11114 ...
1
vote
1answer
76 views

How to Extract Multiple substrings from Log FIle

I am trying to extract the timestamp and the number string in the URL called in an apache logfile that looks like this: 123.456.78.90 - - [16/Dec/2014:06:27:30 +0100] "GET ...
0
votes
2answers
297 views

awk IGNORECASE not working. weird

One script i used for years on production machine, stopped working after a linux system re-installation. after days of digging it turned out that the special variable IGNORECASE=1 has no effect. I ...
1
vote
1answer
44 views

How to delete the 3rd occurrence of a pattern in a file, and beyond

How can I find the third occurrence of a pattern in a file (Note: not in a line) and delete the line containing the pattern and n lines below that (say, 5 lines below). e.g., COVER fafjasfj isfapifj ...
0
votes
2answers
188 views

ascii value in bash/awk

i have a doubt, i know that with this command, it prints the value of a character based on the Ascii table: printf "%d\n" "'a" ##prints 97 But i wonder how can i make my programm show me the ascii ...
1
vote
1answer
108 views

Using grep, sed or awk to find words in between? I want to be able to extract the package name until .el7

I am writing a script (bash/command-line) and I want to be able to extract the package name until .el7 x=dbus-sharp (an example package name - which changes) example text file: Building ...