AWK is a text-processing language. It is mainly used to manipulate and process structured data, and to generate reports.

learn more… | top users | synonyms

0
votes
2answers
13 views

quote-enclosed fields in input to awk

I'm trying to process a webserver logfile in awk. Like 123.222.333.444 - - [24/Feb/2015:13:09:19 +0100] "GET / HTTP/1.1" 200 15852 "https://www.google.dk/" "Mozilla/5.0 (Windows NT 6.1) ...." ...
0
votes
2answers
17 views

Sort on ID, then sort on date, then remove lines which have the older date?

I have a couple of tab delimited files. Each file is structured like so: ID Title Rating Date_Rated What I want to do is merge all these files into one, ...
0
votes
2answers
31 views

bash sed/awk: replace string from a file searching in another file

i have a question about search and replacement on Debian. I have two files. One with: a:b c:d e:f and the other with: e c a` In the second file I want to replace a with b, c with d, e with f. ...
0
votes
0answers
40 views

Copy huge file skipping first line using dd in Linux

I was trying to copy the content of a huge file (10GB) to another skipping the first line (as in head -n1). I tried multiple ways of head, tail, awk and sed. Settled on using tail -n+2 > ./xab.1 ...
0
votes
1answer
17 views

Replace with sed a tag value under a specific section when matching a line

I would like to change a line when another line matchs under a section of a file using the sed command (or awk). Here is an exemple of the file : Extract of MyFic.txt : <PlayList> ... ...
0
votes
3answers
43 views

using awk on only files which START with 'xyz'

I have 3 files in a directory: xyz1.txt, xyz2.txt, rst_xyz.txt and I want to work with ONLY the first two (those which start with xyz). The command ls | awk '/xyz/{print $1}' will return all three of ...
0
votes
2answers
49 views

Unable to get accurate record count of a csv file

I have a .csv file that has some values formatted as paragraphs with line breaks or sometimes bullet points. "STAT","ID","DESC" "UPD", "1", "Updated" "CHG", "2", "Changed" "UPD", "3", "Updated. ...
0
votes
1answer
20 views

How to fetch the string “Hi there” from the below array in unix

I only want to fetch Hi there and store it in a variable VAR1. So basically VAR1 should contain Hi there VIEW_COL[1]="Hi there,USA,Y" This is what i have tried so far. Column_Display_Name=`echo ...
0
votes
2answers
34 views

Text deletion between patterns across multiple lines with respect to text inside pattern

I have a block of text I need to delete, however, only if it contains specific text inside the block: ... <script language="JavaScript"> var somethingA = 0; var somethingB = 0; var ...
0
votes
3answers
22 views

Filtering 2nd field from a data set and then using uniq on the output

I've got a dataset that goes like this: AAAAA 11111 Data1 AAAAA 11111 Data2 AAAAA 11111 Data3 AAAAA 11112 Data4 AAAAA 11112 Data5 AAAAA 11112 Data6 AAAAA 11112 Data7 AAAAA 11113 Data8 AAAAA 11114 ...
1
vote
1answer
33 views

How to Extract Multiple substrings from Log FIle

I am trying to extract the timestamp and the number string in the URL called in an apache logfile that looks like this: 123.456.78.90 - - [16/Dec/2014:06:27:30 +0100] "GET ...
0
votes
2answers
249 views

awk IGNORECASE not working. weird

One script i used for years on production machine, stopped working after a linux system re-installation. after days of digging it turned out that the special variable IGNORECASE=1 has no effect. I ...
1
vote
1answer
39 views

How to delete the 3rd occurrence of a pattern in a file, and beyond

How can I find the third occurrence of a pattern in a file (Note: not in a line) and delete the line containing the pattern and n lines below that (say, 5 lines below). e.g., COVER fafjasfj isfapifj ...
4
votes
4answers
8k views

How to get video duration in seconds?

How can I get video duration in seconds? What I've tried: ffmpeg -i file.flv 2>&1 | grep "Duration" Duration: 00:39:43.08, start: 0.040000, bitrate: 386 kb/s mediainfo file.flv | grep ...
0
votes
2answers
89 views

ascii value in bash/awk

i have a doubt, i know that with this command, it prints the value of a character based on the Ascii table: printf "%d\n" "'a" ##prints 97 But i wonder how can i make my programm show me the ascii ...
1
vote
1answer
57 views

Using grep, sed or awk to find words in between? I want to be able to extract the package name until .el7

I am writing a script (bash/command-line) and I want to be able to extract the package name until .el7 x=dbus-sharp (an example package name - which changes) example text file: Building ...
2
votes
1answer
3k views

How to use awk separator of single quote?

I'd like to use awk to pull out information from SQL output like the following: (count(distinct serverclass)='2') And need to extract the number, which is 2 in this example. I'd like to do ...
1
vote
2answers
34 views

How can I print everything before a match?

After I run hostname I get the FQDN like so: $ hostname foo.mydomain.xyz I'd like to only get the short name foo so I can store it as a variable in a script. I've tried some ways with awk by ...
0
votes
1answer
92 views

Awk script to run a command based on condition using 'if'

I have a piece of code here that basically takes a file called 'srcip' (which is just a list of IP addresses) and compares it to my '~/db' file which is just a tab delimited IP HOSTNAME database of ...
1
vote
2answers
35 views

reverse print without trailing

I have a bash file calling a dns zone and have piped out the fqdn which I need to output in reverse. The snippet below does that fine except I get a trialing . at the end which I don't want creating ...
3
votes
3answers
2k views

How can I insert a line in the middle of a file, with awk, given a search pattern to insert AFTER or BEFORE

I want to insert some text in the middle of a file. The text to insert would be after a specific line, say, "<begin>". I don't know the line number, nor do I know the number of lines in the text ...
0
votes
1answer
130 views

Bash Sum Average numbers in files

I have a series of files with 2(two) lines. The first line contains a whole number (1-100) while the second line contains a decimal number(1.0000-99.9999). I need to get the sum of the first line and ...
2
votes
2answers
49 views

Replace Line Breaks on matching lines, using AWK

I have the following data What i'd like to do is for every line, where there is no ,"555" at the end, I want to remove the the line break, \n, and replace it with space. So, the 'matching line' ...
1
vote
1answer
30 views

Get a range of lines from a file and replace a line in that range using awk or sed or both

I would like to replace a line in a section of the smb.conf file using either awk or sed or both if need be. Here's the section in the file... [CMI] oplocks = no wide links = no writeable ...
0
votes
1answer
49 views

Extract text between two slashes and add it at the end of the line after a tab character

I have a text file looking like this: data/CON/001/raw_series_ST_MC_REG_SM.nii data/PAT/001/raw_series_ST_MC_REG_SM.nii I would like to achieve something like this: ...
0
votes
1answer
460 views

Perform a math operation on all numbers in a file

I have a list of pixel values in a file, and need each to be half its current value. The pattern for the value is always 123px where 123 varies drastically. How can I half each pixel value? I'm ...
0
votes
1answer
67 views

word boundary in a awk command not working when printing a range of lines

I am attempting to parse samba .conf file for a specific share section... [CMI] oplocks = no wide links = no writeable = yes delete readonly = yes path = /LOCALSITE/CMI ...
1
vote
1answer
77 views

Script to reference another file first

Hi I have a script and in it I need to get the hostnames from IPs that the script generates. The script will generate those IPs into 1 neat file with only 1 column. It is literally just IP address (1 ...
0
votes
1answer
55 views

How to use the awk to print the output

This command nstats gives me output like below. How can I use awk or grep to give me the uptime with only days on it? I cannot use the "" to get the output. HOSTNAME LOAD MEMORY SWAP PROCS ...
0
votes
1answer
120 views

I'm trying to mirror dns queries generated by TCPDUMP filter and replay them with dig or any other utility

I'm not a very experienced script writer at all. I did find some useful hints in generating an stdout from tcpdump that generates a live feed if you will which is what I want, the problem is handling ...
4
votes
1answer
52 views

writing an AWK command

My input file has three columns like the one below Input file: water 123 wa water 123 at water 123 te water 123 er rater 347 ra rater 347 at rater 347 te rater 347 er ...
-1
votes
2answers
33 views

how to split a hex file into two hex files at the first match of certain criteria in perl or just unix scripts

I have a HEX file containing following, I want to split them into two HEX files based on the address: file_in.hex @00000000 00 FC 00 20 89 01 00 00 D9 01 00 00 DB 01 00 00 DD 01 00 00 DF 01 00 00 E1 ...
1
vote
5answers
147 views

Turn comma-separate rows into 1 column

I have a list of users basically from /etc/group, Id like to basically format that line into one column, I just need the users. So if a field looks like this: bob,john,jane,sam,joyce Then I'd like ...
1
vote
3answers
263 views

Parsing the output of “uptime” with bash

I would like to save the output of the uptime command into a csv file in a Bash script. Since the uptime command has different output formats based on the time since the last reboot I came up with a ...
0
votes
1answer
128 views

output awk print to two separate files

Here is my current code md5sum filename.iso | awk '{print $1} {print $2}' This gives me two separate lines. The first line is the md5 of the file and the second line is the name of the file. I ...
0
votes
1answer
22 views

Sort piped to uniq unexpected output

I have a file that looks like this: "string",192.168.1.0,,TX,192.168.1.2,,,100,0,0,10,10,10,, "string",192.168.1.0,,TX,192.168.1.1,,,100,0,0,10,10,10,, ...
5
votes
3answers
1k views

how to limit find command's output used with option -print0

I'd like to limit output of find command. In the past I used to use for this ls command, e.g: ls *tgz|head -100|xargs -i mv "{}" ../ but I got to know that the result may be unpredictable if name ...
1
vote
1answer
82 views

How to convert a column of hex to dec in gawk, strtonum in gawk gives erroneous results

I am trying to script a database from a network stream. The network stream after being heavily sed'd drops a three column file that looks something like this called file.db ...
-2
votes
1answer
74 views

Grep word in one file, and use that word to match in another file, adding the thing that follows the match

I want to grep several words in file1, and use each word to grep what follows after its match in file2. And then I want to add the string that followed the match to the word I used into file03, so ...
0
votes
1answer
74 views

Replace negative values in a file with zeros

I want to replace all negative values in a file with zeros. How can I do this in awk? I tried with gsub but gsub(-*, 0) doesn't work... Any ideas? My "code" is awk '{gsub($(!/-/),"0",$2); print $1 ...
0
votes
1answer
114 views

How to capture diff output from find command exec?

I am using Ubuntu (Precise 12.04) I am trying to write some bash script to see if file is changed via awk replace by using diff command. I did a test with the following: #! /bin/bash diff fileA.htm ...
4
votes
3answers
4k views

How to pipe awk output (with periodic, continuous input) to output file?

I am trying to write a command that pipes the continuous output of a free command (run every second) to an awk command that parses a specific value (available free memory) and outputs this to a file ...
0
votes
1answer
914 views

Suppress awk warnings

On a script, I get some warning of the form awk: warning: escape sequence \( treated as plain (. Since I want to use the output of that script to do stuff, I want it to output only results and not ...
1
vote
1answer
167 views

How to replace a line in an xml file with a set of lines stored in a variable or file using unix commands?

I have a xml file in which the line that needs to be searched is: SEARCH='<?xml version="1.0" encoding="UTF-8" standalone="no"?><SSC>' This searched value needs to be replaced with ...
12
votes
6answers
5k views

Removing ANSI color codes from text stream

Examining the output from perl -e 'use Term::ANSIColor; print color "white"; print "ABC\n"; print color "reset";' in a text editor (e.g., vi) shows the following: ^[[37mABC ^[[0m How would one ...
-1
votes
1answer
37 views

editing column of a text file

abc.txt consists of two column data as 1 A1021 2 A1021 3 A1022 4 A1022 I want another test file, say pqr.txt (replacing A1021 by 1 and A1022 by 2 and so on). I used the following usual perl ...
1
vote
1answer
4k views

Extract substring before = sign using awk

Say I have some file with a bunch of lines in the form someString=someMoreCharacters anotherString.blah=foo=bar blah.blah.blah=foo.bar.=foobar Desired output someString anotherString.blah ...
0
votes
1answer
193 views

Find a pattern in a line, and append that matched pattern to the same line

I'm trying to pull a job number out of notes listed in a csv, and then append that number to the end of the line, preferably through sed, awk, grep or perl (which I have installed in Cygwin. Here's a ...
0
votes
1answer
54 views

Does an awk script or command execute itself to every line it reads?

I got an awk script named my_awk: #!usr/bin/awk -f { if ($1 == "#START") { FS=":";} else if ($1 == "#STOP") { FS = " ";} else { print $3} } And I invoked it through: cat my_file | awk -f ...
0
votes
1answer
37 views

Using Grep or AWK to find all of the occurences of a specified string

I have a massive single line of data and I am trying to find the occurrences of a specified string. I then want to export the occurrences (-/+) 20 characters to another file. Can anyone offer a ...