AWK is a text-processing language. It is mainly used to manipulate and process structured data, and to generate reports.

learn more… | top users | synonyms

1
vote
1answer
20 views

UNIX - Surround all Column values of CSV with double quotes

I have data in following format : productType , Description Oven , Accurate Preheat, good start for the day hairDryers , Comes with extended warranty , 50% off Desired Output: "productType" , "...
0
votes
2answers
41 views

Split one file into multiple files

I have one file which is in Json format like below: { "sources":[{ "field1":1000, "field2":"winevent_log", "field3":"winevent_log", "field4":"os_security", ...
0
votes
2answers
31 views

Disk usage to show only select partitions?

I'd like to be able to only show disk usage for specific partitions in GB and a % after. Basically ignore swap, tmpfs, etc.. I'd also like to sum the disk usage amount and subtract from the total to ...
0
votes
2answers
40 views

How to collect specific information in a file

I have the need for different hardware information. such as serial numbers, capabilities, among many others, and for that, I found the hwinfo, but it gets information far beyond what I need. For ...
0
votes
0answers
38 views

AWK - system command

I am trying do a linux command in AWK: $0 ~ /^([[:alnum:]]+\.)+[[:alnum:]]+;version=.+$/{ split($0,path,";") gsub(/\./,"/",path[1]); split(path[2],ver,"=") pver=path[1]...
1
vote
3answers
33 views

Using a shell script to add a ‘</pre>’ after the last line in a large file

I need a SED, or AWK, command to find the last line in a file, and add </pre> to it. Such as: LINELINELINE Changes to: LINELINELINE</pre>
0
votes
1answer
54 views

How to extract specific data from the table?

Here is the table... Group Name Designation 2 (John) Front End Developer 12 (Jim) Back End Developer 8 (Jill) Full Stack Developer 21 (...
1
vote
1answer
45 views

Linux Shell script to change grep for a string, then comment out 2 lines above and 1 below the string found in the file

Looking for some help on this task. I have tried several shell scripts to do this, but not sure I am approaching this problem right. I have a file that contains several blocks of data like this: ...
-2
votes
1answer
16 views

Removing a substring between track number and title

I need help with a little bit of bash trickery. In organizing my music collection I've come across a set of tracks which are named in the format %track%. %artist% - %title%.%extension% How can I ...
1
vote
3answers
39 views

Display file contents if it has corresponding line number

I have two files in the following format: File A 4 5 8 9 File B 1 text 2 text1 3 text2 4 text3 5 text4 6 text5 7 text6 8 text7 9 text8 What I want to have for my output is just the following (the ...
0
votes
2answers
18 views

How to change a character on a certain line without delimiters with awk

I have a file, and I would like to change a single character on lines 400 through 600. The character happens to be the 22nd character on each line. It is also the 5th column, so I tried the following: ...
0
votes
3answers
47 views

Combine multiple lines in a file to a single line

I have a file as below.. "Field1"|"Field2"|"Field3"|"ufghjkrtyrtyfgh$" "Field1"|"Field2 continue on line 2 continue on line 3"|"Field3"|"ufghjkrtyrtyfgh$" "Field1"|"Field2"|"Field3"|"...
0
votes
1answer
59 views

Watch with awk command

How do you watch a command like? awk 'NR%2==0 {printf "%s %8.0f", $1, $5}' filename.txt Preceding this with "watch" gives this error: awk: cmd. line:1: fatal: cannot open file `{printf' for ...
0
votes
1answer
20 views

AWK - print file and length in two lines in END

I have to do script in AWK, which will choose the longer field in the line and after that in the end show in two line filed and under(in second line) length. I made this: ...
0
votes
1answer
31 views

Pattern search the user input in a file in Unix using awk

I am trying to write awk script to search for a pattern that is given input by user, in a file. My code look like this: awk 'BEGIN{printf "Enter : ";getline input<"/dev/tty"} /'"$input"'/ {print}' &...
-1
votes
1answer
22 views

How can I filter unique lines based on a substring?

If I have a set of URLs in a file like: http://example.com/do?foo=bar&etc=123 http://example.com/do?etc=456&foo=bar I want to filter this out so that there's only one url for each unique ...
0
votes
2answers
28 views

Insert a string or blank line after specific search criteria, in a loop

I'm wondering if someone could help me with a specific coding question. I have a DNA sequencing file that reads something like this (as an example): ...
0
votes
2answers
37 views

new column to a tab delimited file, with as predefined value

How can I add a new column to a tab delimited file, at the end of the file, containing the same values for each row. For example: infile.txt Name Age Address Sex X 12 adrs1 M Y 15 adrs2 ...
0
votes
1answer
57 views

arithmetic operation on a nested column in a tab delimited file

I have a tab-delimited text file containing Column1...Columnn columns and R1 to Rn rows. Within some of the columns, there are multiple nested fields with an identifier and then separated by semicolon ...
1
vote
1answer
30 views

chunking screen output using grep/head/awk/sed/etc

I have some output that prints an element every line. I want to be able to parse that output in chunks of N, even if the number of lines is not a multiple of N. find . -f | chunk 10 1 # shows first ...
0
votes
1answer
24 views

How to Check value for an input element after fetching contents of page using cUrl

I am trying to download a page using cUrl. I have saved the page to a file and I then grep on it to extract the value of the text field. The text field appears multiple times and I just need one value....
-1
votes
1answer
35 views

Oneliner to get the list of filenames which have last line (or line on position X) matching certain pattern?

How to get the list of filenames which have last line (or line on position X) matching certain pattern? We are looking for oneliner.
1
vote
1answer
39 views

Remove a certain part in a text file

I got the file, text.txt. In this file I have written: "test" "okay" I want to remove the " part of this file, how'd I do this?
3
votes
1answer
38 views

file splitting and compression in pipeline

So I have a massive file something like this... 1/20/2016,somerandomdata 1/20/2016,somerandomdata 1/20/2016,somerandomdata 1/20/2016,somerandomdata 1/21/2016,somerandomdata 1/21/2016,somerandomdata 1/...
1
vote
1answer
45 views

Increase efficiency of bash loop with awk

I have to write a fairly large text file(>300,000 lines) using text extracted from the second column of one file, then prepending and appending text to each line and finally writing it out to a new ...
0
votes
2answers
185 views

Rename CSV header using command line tools

Given a CSV file, h1 h2 h3 a b c z x b how to rename the first header to One and the third header to Three using command line tools, namely One h2 Three a b c z x b
0
votes
1answer
33 views

Transform CSV column by a function using command line tools

Given a csv file such as h1 h2 a 0 b 1 c 0 how to rename 0 to YES and 1 to NO in the second column.
2
votes
2answers
51 views

compare and calculate two variables with multiple values in shell script

I have written three hive queries in shell script for calculating debit-credit and target amounts which output below values: debit_amount : BOI 4760545.650000 AXIS 284.49000000 SBI 87.220000000 ...
3
votes
3answers
185 views

Filtering and splitting huge file based on column Windows

I have about 2 TB worth of data files formatted like 12/20/2015 somerandomdata 12/20/2015 somerandomdata 12/20/2015 somerandomdata 12/20/2015 somerandomdata 12/21/2015 somerandomdata 12/21/2015 ...
0
votes
1answer
27 views

A way to split a text file into arbitrary blocks based on first column?

I have a (very large) text file of the form... /folder1/fileA /folder1/fileB /folder1/fileC ... /folder999/fileA /folder999/fileB /folder999/fileC ...etc with an unknown number of entries for each ...
-1
votes
1answer
32 views

How to show a particular section from a line in linux?

Let's say I have a file with the following line: [{"name”:”10.0.0.1”,”address":"10.0.0.1”,”subscriptions["all"],"version":"0.0.0”,”timestamp":1450270166},{"name":"10.0.0.2","address":"10.0.0.2","...
-2
votes
1answer
29 views

How to find files within a size range?

Please suggest me the way to find all the zip files which are more than 60 MB but less than 70 MB in size using find command.
0
votes
2answers
80 views

Fetch required column from a file with \001 delimiter

I am storing output of pig file in a directory Which contains lines with multiple columns separated by delimiter \\u0001. Now in shell script I want to fetch 7th column field from the output. I ...
0
votes
4answers
193 views

How to replace first n no. of occurrence of a string using sed and/or awk?

I got a file alphabet where a has multiple occurrence in a line. $ cat alphabet a b c d e f g h i j k a a l m n a p q r a s t u v w a x y z a k l q z where $ cat alphabet | grep -o a | wc -l 7 ...
0
votes
1answer
42 views

bash script read line by line

Hello I am trying to create a script that reads my file take all the variable it needs then use it for another command. But it seems that the variables are not being memorized, because I tried a ...
-1
votes
1answer
49 views

separate a large 7GB tab limited file into multiple files

I want to separate a large 7GB file into multiple files (each file size= 1 GB) without changing it's format.. How is it possible ? AWK or any script is available? My file look like this: ...
0
votes
2answers
93 views

Why can't I redirect output from awk?

I've written this one liner to show MB read/written every 5 seconds. #!/bin/bash #iosum.sh iostat -dmz 5 |\ awk 'BEGIN{rx=wx=0}{if($0 == ""){printf"%.1f %.1f\n",rx,wx}else if($0~/^Device:/){rx=wx=...
1
vote
1answer
52 views

How to find the last 2 characters of Chinese names with command lines tools?

I have a list of Chinese names, some of them have 2 characters, others may have more or less, like following: I want to use some command lines tools, to find the last 2 characters of each. If doesn'...
0
votes
1answer
173 views

Merging two files in unix, with one common column that is redundant

I have two files with one common column that is redundant.File 1 has chromosomal locations and TF's, file 2 has chromosomal locations and Refseq numbers. File 1: chr1:66997824-67000456  ...
2
votes
2answers
107 views

How to add # Shebang in first line of a file which is an auto create script by another program

I am looking for a method to add a shebang #!/bin/csh -f to first line of my file , which is actually getting created by a another set of program, since this script is auto-created, it should run from ...
2
votes
3answers
136 views

How to fetch address and modify the value in key/value format file?

I'm currently writing a shell-script for my Raspberry Pi so it sets up an access-point with a unique SSID. To do this I want to use the mac-address of the WiFi dongle as my SSID. For doing that I want ...
1
vote
1answer
100 views

awk character differences when using if

I have an input file with this line(user data/columns stripped out) and several thousand more. The xCE is an unconverted hex value from the clients file. 412640 xCE When I run it thru this awk ...
2
votes
2answers
84 views

Sed/Awk save text between patterns if contains string

I'm facing an issue with mails. I need to get all messages between 2 people: somebody1@domain.com and person@domain.com. The file: From: somebody1@domain.com to: person@domain.com <body of the ...
1
vote
1answer
42 views

Generate CSV from NS Zone File

I have a large file that contains records in multilines. ex: domain1 NS ns1 domain1 NS ns2 domain1 NS ns3 domain2 NS dnsx What might be the fastest way to generate a CSV of the form domain1,ns1,...
1
vote
2answers
22 views

Conditional AND in awk with environment variables

I found this command. If a line has interested word, the command will output the entire line: temp="sample" awk -F " " -v var="$temp" '$1 == var' /root/smaple.txt smaple.txt contains: sample demo ...
1
vote
3answers
177 views

linux - replace all double newlines with a space

Is it possible to replace only double newlines in a text file using sed, awk, grep, tr, or whatever is needed? this is data this is more data What I need to do is to replace all double ...
1
vote
1answer
1k views

How to run the awk command in Windows 7?

I have the following adb + awk command which will work fine in Linux environment [Ref]. adb shell dumpsys package | awk -v RS='\n +Package' '/android\.permission\.CAMERA/{print $1}' But I need to ...
0
votes
1answer
30 views

Awk eqivalent to grep -C <num>

I want to print 5 lines of context around each line that matches "foo" but not "foobar". I can do the match easily enough via <output> | awk '/foo/ && !/foobar/' but I can't figure out ...
0
votes
1answer
65 views

awk - I can't print to file

I don't know ,why I can't print from to file. My command: awk 'BEGIN{print "af">file}' Error: fatal: expression for `>>' redirection has null string value
0
votes
2answers
58 views

AWK formatting Disk Utility command formatting incorrectly

I am trying to learn awk at the moment and so I am trying it on the disk utility command. Here is the output when I run Disk Utility list disk0 /dev/disk0 (internal, physical): #: ...