AWK is a text-processing language. It is mainly used to manipulate and process structured data, and to generate reports.

learn more… | top users | synonyms

0
votes
1answer
38 views

Using Grep or AWK to find all of the occurences of a specified string

I have a massive single line of data and I am trying to find the occurrences of a specified string. I then want to export the occurrences (-/+) 20 characters to another file. Can anyone offer a ...
0
votes
1answer
337 views

How to add a prefix to every row of a particular column in a csv file via command line on linux

So basically i am trying to achieve the following.    File before editing.  column-1,   column-2,  column-3,  column-4,  column-5 Row-1-c1, ...
2
votes
4answers
156 views

Extract lines based on long list of line numbers

I have a long list of line numbers (35389208) that I do not want in my file. By line number I mean the line in my file (e.g. line 277). My list of line numbers I do not want looks like: 277 278 279 ...
0
votes
1answer
191 views

Perform a GROUP BY-like command in UNIX

I am having a text file like the following: 1 2 3 4 5 6 7 8 9 ... n <-- column numbering 1 0 0 1 0 0 0 1 0 ... 0 0 1 0 0 0 0 0 0 0 ... 1 1 0 0 0 0 0 0 1 1 ... 0 0 1 1 1 0 1 0 0 0 ... 0 0 1 0 0 ...
1
vote
4answers
1k views

Randomly shuffle rows in a large text file

I have a text file of ~1GB with about 6k rows (each row is very long) and I need to randomly shuffle its rows. Is it possible? Possibly with awk?
0
votes
1answer
112 views

How to grep values under columns

How can I grep for a specific column name and display the value underneath that column. Sample data: StandByFile StandByPg StandByLSN StandByRcvBufUsed S0082160.LOG 621668 ...
1
vote
2answers
133 views

Replacing every 10th pipe with new line in unix

Lets say I have fields: name, number, id I have a data file: name1|number1|id1|name2|number2|id2...etc I want to replace every 3rd pipe with a new line or '\n' so I get: name1|number1|id1 ...
-1
votes
1answer
35 views

Which regular expressions (POSIX or extended) are allowed in awk?

I'm wondering which kind of regular expressions (POSIX as in grep, or extended in egrep) am I using when type in regular expressions in awk?
0
votes
1answer
71 views

awk used with grep - no output

I have a bashscript which uses grep with awk as following: #ec2-describe-instances | grep -i instance | awk '{print "Creating -> " $3; system("ec2-create-image " $2 " --name " $3 "-$(date +%F) ...
1
vote
2answers
44 views

How do I print multiple sections in a text file?

I have a text file looking something like this: *FILESTART line_a line_b line_c *INCLUDE file_A.key file_B.key *INCLUDE file_1.key file_2.key file_3.key *SOMETHING_ELSE line_x line_y line_z *END I ...
1
vote
1answer
25 views

Print every line but conditionally only fragments for some lines

I have a text file that looks like this: rno-miR-344-5p miRPlus_11239/mmu-miR-383/rno-miR-383 hsa-miR-301a/mmu-miR-301a/rno-miR-301a ...
0
votes
1answer
214 views

how to extract last number sequence in an IP Address (using Linux)

I want to extract the last digits of an IP Address based on the result of ifconfig command. ifconfig often includes a whole heap of output so I'm not really sure how to get the digits that I want. ...
0
votes
1answer
125 views

Regexp based editing of content in large CSV files [duplicate]

I wanted to remove certain pattern of contents based on a regex from csv file which is very large in size containing 100,000+ records, how is it possible via windows commandline? I also have sed and ...
-2
votes
2answers
101 views

Trying to count and print song file names using awk

RANK NAME BAND YEAR GENERE DOMESTIC/INTERNATIONAL 206:Reach Out, I'll Be There:The Four Tops:1978:Pop:3/2 207:Bye Bye Love:The Everly Brothers:1950:Classic:3/2 ...
0
votes
1answer
48 views

I have file called songs and i'm trying to find the longest and shortest song name using AWK

RANK NAME BAND YEAR GENERE DOMESTIC/INTERNATIONAL 206:Reach Out, I'll Be There:The Four Tops:1978:Pop:3/2 207:Bye Bye Love:The Everly Brothers:1950:Classic:3/2 ...
0
votes
2answers
2k views

Piping tail -f into awk

Greeting, I am trying to pipe tail -f into awk to monitor a logfile in realtime, but according to examples, there should be not problem but I can't get it to work. here is the command I'm running ...
2
votes
0answers
237 views

stdbuf not working while unbuffer does

I am trying to apply timestamps to stdout of a process. For the proper timestamps to be applied, I attempt to unbuffer stdout of the process. This works with unbuffer but not with stdbuf as I would ...
2
votes
1answer
36 views

Weird behavior of awk

Why do I get only "cat" when I run awk 'BEGIN { animal[three] = "hen" animal[two] = "dog" animal[one] = "cat" for (var in animal) { print animal[var] } } ?? Shouldn't it print ...
0
votes
1answer
156 views

Bash Sum Average numbers in files

I have a series of files with 2(two) lines. The first line contains a whole number (1-100) while the second line contains a decimal number(1.0000-99.9999). I need to get the sum of the first line and ...
0
votes
0answers
44 views

awk remove sign “,” in all columns, if it exists between

My data input file: 508,186,,Name,"VIA PAS, 4",vou1@mail.com, 291,345,,Name2,Votros 73,vou2@mail.com, 911,733,,Name3,zap 101,vou4@mail.com, 9000,32,,Michel,hops 12,"x1@mail.com, vou7@mail.com, ...
0
votes
1answer
141 views

How to find the matching records from 2 files in unix

I have two files which contains email_ids. 1. Test1.txt 2. Test2.txt Test1.txt contents are: abc@gmail.com xyz@gmail.com mns@gmail.com Test2.txt Contents are: jpg@gmail.com joy@yahoo.com ...
0
votes
0answers
6 views

Grep values from nethog and netperf [duplicate]

I need to monitor the network activities of a process and the bandwidth of the connection. So I decided to use nethog and netperf combine with grep and awk to write the values to a file. nethog ...
0
votes
2answers
273 views

Grep values from nethog and netperf

I need to monitor the network activities of a process and the bandwidth of the connection. So I decided to use nethog and netperf combine with grep and awk to write the values to a file. nethog ...
1
vote
2answers
93 views

moving directories after grep

I have isseued the next command in order to move some directories to the "build_tools" folder however the result was not the one I expected. ls -l | grep 2014-03 | awk '{if ($8!=build_tools) print ...
-2
votes
1answer
167 views

awk to print the line number of a closing bracket

I have a file 1.txt ejryheiur rewurie ( ejwhrejkrh rewrhwe rewkhrwe rewhkr ) dfhgsdfh rqwergjweg jhwgrh ( rehgwrwhe rhwejkr ) With this nawk script nawk ' BEGIN { ...
0
votes
3answers
134 views

Replace string in string but not when its somewhere between double square bracket

I have a long string from mysql field containing the data of a mediawiki page. I have to replace a string in that mediawiki page but not when the string is in mediawiki link. The mediawiki link is ...
0
votes
2answers
70 views

Replace words on given line numbers

I have a file 1.txt INTEGER-d_int () INTEGER-d_int () INTEGER-d_int ( ) INTEGER-d_intClass() INTEGER-d_intClass new() and I want to replace the occurrence of d_int with d_INT on the ...
0
votes
1answer
99 views

How to encrypt the mail in unix?

I want to encrypt the mail as that will be containing the confidential information. There are no attachments. Please help in providing the steps to encrypt the mail. I have got the reference of How ...
0
votes
1answer
220 views

How to replace an exact matching word using gsub

Please see the below script - On this nawk 'NR==FNR { a[$1]=$2 ; next} {for ( i in a) gsub(i,a[i])}1' file.dat 1.txt 1.txt ioiufeioru dfoiduf MO_CIF_INP438 fjkdj MO_CIF_INP dsjhdf BP_LINKED_TETES ...
3
votes
2answers
604 views

How to comment out multiple line matching a regex using sed or awk

I have a file that contains this: [...] location /static { ... multiple lines ... } [...] location /static/ { ... multiple lines ... } [...] And I want to get: [...] # ...
1
vote
2answers
68 views

Is there a way to print 2 lines (if any) before and after a target or print a placeholder if not?

I'm not native english speaker so I hope to be clear. I know about grep -C 2 "TARGET" inputfile to select 2 rows before and after the row of the TARGET, but I'm not able to use it to manage my ...
0
votes
2answers
55 views

grep out condition and add the values of the grep value

I have a scenario that a csv file has the values of : abcd FAILURE 7 abcd SUCCESS 12 efgh SUCCESS 3 efgh FAILURE 5 mnop SUCCESS 6 mnop SUCCESS 4 abcd FAILURE 5 efgh SUCCESS 2 mnop SUCCESS 1 ...
2
votes
1answer
589 views

grep the string lines from the file

csv file. 1,2,3,abc 2,3,4,def 3,4,5,abc 3,4,2,def 6,7,9,abc. I need to read that file and get the lines of file containing abc.
-1
votes
2answers
84 views

how to find number of substrings between delimiter? [closed]

I have the string following something(1)^^^something(2)^^^something(3)^^^ ... ^^^something(n) how to find number of something(s) in the string.
1
vote
1answer
2k views

Extract the contents of ELF and write to binary file

I have been trying to extract the contents of a firmware and putting it to a binary file, but with no success. I see the right hex contents, but am not sure how to laid them in bits into a file. ...
1
vote
2answers
359 views

batch find/replace using sed or awk

In every file with filename matching pattern1 I want to search for lines which start with pattern2, and add some string in a new line above those matched lines. As a simple example suppose I'm ...
1
vote
4answers
1k views

How to split CSV files as per number of rows specified?

I've CSV file (around 10,000 rows ; each row having 300 columns) stored on LINUX server. I want to break this CSV file into 500 CSV files of 20 records each. (Each having same CSV header as present in ...
0
votes
1answer
1k views

Suppress awk warnings

On a script, I get some warning of the form awk: warning: escape sequence \( treated as plain (. Since I want to use the output of that script to do stuff, I want it to output only results and not ...
0
votes
2answers
3k views

using awk with find -exec

I have a directory structure with 14 directories containing a bunch of files containing data in a three-column format (separated with tabs). I intended to use find and awk to extract the second column ...
3
votes
4answers
11k views

Print line X lines before/after found line

I want to search for a particular string in a certain file. If I found the string, I also want to print the line X lines before (or after) that line. Can this with grep or awk, or do I need a ...
1
vote
1answer
283 views

printing only first finded match

I've created a statement which is searching for specific informations in each line (in my case "AAA","BBB" or "CCC"), if there is no such information N/A is printed { k=0; for (i=1;i<=NF;i++) ...
1
vote
1answer
30 views

awk is limited to only one line operations?

Like in the subject. Let say I have a file which contains: *aaa A B C D E F 55 *ccc A B C R E T 33 and I would know if it's possibe to make a statement if $1=="*aaa" then print $4 ...
0
votes
1answer
175 views

Find number in string and replace it

I wrote C++ code and now I want to replace the number in a string by using shell scripts. The part that needs to be replaced: if (random<=90) How I can change the 90 from a shell script? I ...
1
vote
1answer
70 views

AWK does not work correctly in my system [duplicate]

i cant run this code because of an error awk 'BEGIN {x=0;y=0} { match($13,/([0-9]+)/,a); x = x + a[1]; match($12,/([0-9]+)/,b); y = y + b[1]} END {print "rxPackets:" x " txPackets:" y}' yourfile ...
0
votes
2answers
107 views

sum numbers in different lines in shell

i have 8000 lines similar to this <Flow flowId="1" timeFirstTxPacket="+0.0ns" timeFirstRxPacket="+924100.0ns" timeLastTxPacket="+199984927558.0ns" timeLastRxPacket="+199675473275.0ns" ...
-1
votes
3answers
347 views

convert text file column field with data dd.mm.yy to dd.mm.YYYY

How to format date for file content using linux command line tools? Input: aaa;1.70;ccc;20.01.13;zz;2013; uuu;2.70;lll;17.12.10;zz;2013; fff;3.70;nnn;31.01.98;zz;2013; auu;8.70;nuu;30.02.96;zz;2013; ...
1
vote
1answer
27 views

`nawk` “doesn't work” with `make`? Why?

I have the following alias: alias mca_color='nawk '"'"'BEGIN { arr["Creating shared object"] = "1;31";} \ { l = $0; for (pattern in arr) \ { gsub(".*" pattern ".*", "\033[" arr[pattern] ...
2
votes
2answers
137 views

Use tail directly on website data, or another utility needed?

I am planning to extract a single field of data from the last line of data on a website to populate an RRD database. I'm running Linux. The website is: ...
1
vote
1answer
45 views

Precising if statement in awk

Is that possible to precise conditions in "if" statement? Let's take an example from my previous post (by the way, "k" variable is tricky, thanks terdon;)) and say that I'm looking for $i which starts ...
0
votes
2answers
171 views

combining loop “for” with statement “if”

I've created a script for awk. I'm combining loop "for" with "if" statement. It's searching each line for word "ABC" which can occure in different place in the line. It looks like below: {for ...