AWK is a text-processing language. It is mainly used to manipulate and process structured data, and to generate reports.

learn more… | top users | synonyms

1
vote
1answer
33 views

Remove a certain part in a text file

I got the file, text.txt. In this file I have written: "test" "okay" I want to remove the " part of this file, how'd I do this?
3
votes
1answer
28 views

file splitting and compression in pipeline

So I have a massive file something like this... 1/20/2016,somerandomdata 1/20/2016,somerandomdata 1/20/2016,somerandomdata 1/20/2016,somerandomdata 1/21/2016,somerandomdata 1/21/2016,somerandomdata ...
0
votes
0answers
15 views

Increase efficiency of bash loop with awk

I have to write a fairly large text file(>300,000 lines) using text extracted from the second column of one file, then prepending and appending text to each line and finally writing it out to a new ...
0
votes
2answers
28 views

Rename CSV header using command line tools

Given a CSV file, h1 h2 h3 a b c z x b how to rename the first header to One and the third header to Three using command line tools, namely One h2 Three a b c z x b
0
votes
1answer
22 views

Transform CSV column by a function using command line tools

Given a csv file such as h1 h2 a 0 b 1 c 0 how to rename 0 to YES and 1 to NO in the second column.
2
votes
2answers
29 views

compare and calculate two variables with multiple values in shell script

I have written three hive queries in shell script for calculating debit-credit and target amounts which output below values: debit_amount : BOI 4760545.650000 AXIS 284.49000000 SBI 87.220000000 ...
3
votes
3answers
63 views

Filtering and splitting huge file based on column Windows

I have about 2 TB worth of data files formatted like 12/20/2015 somerandomdata 12/20/2015 somerandomdata 12/20/2015 somerandomdata 12/20/2015 somerandomdata 12/21/2015 somerandomdata 12/21/2015 ...
0
votes
1answer
25 views

A way to split a text file into arbitrary blocks based on first column?

I have a (very large) text file of the form... /folder1/fileA /folder1/fileB /folder1/fileC ... /folder999/fileA /folder999/fileB /folder999/fileC ...etc with an unknown number of entries for each ...
-1
votes
1answer
30 views

How to show a particular section from a line in linux?

Let's say I have a file with the following line: ...
-2
votes
1answer
22 views

How to find files within a size range?

Please suggest me the way to find all the zip files which are more than 60 MB but less than 70 MB in size using find command.
0
votes
0answers
12 views

fetch required column with delimiter from files in shell script

I am storing output of pig file in a directory Which contains lines with multiple columns separated by delimiter “ \u0001” Now in shell script I want to fetch 7th column field from the output I ...
0
votes
4answers
67 views

How to replace first n no. of occurrence of a string using sed and/or awk?

I got a file alphabet where a has multiple occurrence in a line. $ cat alphabet a b c d e f g h i j k a a l m n a p q r a s t u v w a x y z a k l q z where $ cat alphabet | grep -o a | wc -l 7 ...
0
votes
1answer
35 views

bash script read line by line

Hello I am trying to create a script that reads my file take all the variable it needs then use it for another command. But it seems that the variables are not being memorized, because I tried a ...
-1
votes
1answer
46 views

separate a large 7GB tab limited file into multiple files

I want to separate a large 7GB file into multiple files (each file size= 1 GB) without changing it's format.. How is it possible ? AWK or any script is available? My file look like this: ...
0
votes
2answers
46 views

Why can't I redirect output from awk?

I've written this one liner to show MB read/written every 5 seconds. #!/bin/bash #iosum.sh iostat -dmz 5 |\ awk 'BEGIN{rx=wx=0}{if($0 == ""){printf"%.1f %.1f\n",rx,wx}else ...
1
vote
1answer
39 views

How to find the last 2 characters of Chinese names with command lines tools?

I have a list of Chinese names, some of them have 2 characters, others may have more or less, like following: I want to use some command lines tools, to find the last 2 characters of each. If ...
0
votes
1answer
66 views

Merging two files in unix, with one common column that is redundant

I have two files with one common column that is redundant.File 1 has chromosomal locations and TF's, file 2 has chromosomal locations and Refseq numbers. File 1: ...
2
votes
2answers
53 views

How to add # Shebang in first line of a file which is an auto create script by another program

I am looking for a method to add a shebang #!/bin/csh -f to first line of my file , which is actually getting created by a another set of program, since this script is auto-created, it should run from ...
2
votes
3answers
73 views

How to fetch address and modify the value in key/value format file?

I'm currently writing a shell-script for my Raspberry Pi so it sets up an access-point with a unique SSID. To do this I want to use the mac-address of the WiFi dongle as my SSID. For doing that I want ...
1
vote
1answer
58 views

awk character differences when using if

I have an input file with this line(user data/columns stripped out) and several thousand more. The xCE is an unconverted hex value from the clients file. 412640 xCE When I run it thru this awk ...
2
votes
2answers
61 views

Sed/Awk save text between patterns if contains string

I'm facing an issue with mails. I need to get all messages between 2 people: somebody1@domain.com and person@domain.com. The file: From: somebody1@domain.com to: person@domain.com <body of the ...
1
vote
1answer
30 views

Generate CSV from NS Zone File

I have a large file that contains records in multilines. ex: domain1 NS ns1 domain1 NS ns2 domain1 NS ns3 domain2 NS dnsx What might be the fastest way to generate a CSV of the form ...
1
vote
2answers
19 views

Conditional AND in awk with environment variables

I found this command. If a line has interested word, the command will output the entire line: temp="sample" awk -F " " -v var="$temp" '$1 == var' /root/smaple.txt smaple.txt contains: sample demo ...
1
vote
3answers
83 views

linux - replace all double newlines with a space

Is it possible to replace only double newlines in a text file using sed, awk, grep, tr, or whatever is needed? this is data this is more data What I need to do is to replace all double ...
1
vote
1answer
430 views

How to run the awk command in Windows 7?

I have the following adb + awk command which will work fine in Linux environment [Ref]. adb shell dumpsys package | awk -v RS='\n +Package' '/android\.permission\.CAMERA/{print $1}' But I need to ...
0
votes
1answer
25 views

Awk eqivalent to grep -C <num>

I want to print 5 lines of context around each line that matches "foo" but not "foobar". I can do the match easily enough via <output> | awk '/foo/ && !/foobar/' but I can't figure out ...
0
votes
1answer
29 views

awk - I can't print to file

I don't know ,why I can't print from to file. My command: awk 'BEGIN{print "af">file}' Error: fatal: expression for `>>' redirection has null string value
0
votes
2answers
48 views

AWK formatting Disk Utility command formatting incorrectly

I am trying to learn awk at the moment and so I am trying it on the disk utility command. Here is the output when I run Disk Utility list disk0 /dev/disk0 (internal, physical): #: ...
0
votes
1answer
26 views

Random function in awk like in bash

How can I random in awk. In bash is: echo $((RANDOM%(32-12+1)+12)) Is there function in awk, which gives the same result?
0
votes
1answer
37 views

awk - Assign variable result of system function

I would like to assign variable result of system function: awk 'BEGIN{x=system("cut -d \" \" -f 1 ol");print x}' Input "ol": 1 Result: 1 0
0
votes
2answers
82 views

Pass variable from awk to bash

How can I pass variable from awk to bash? I want to pass a lot of variable, so I don't use: x=$(awk '.....) I thing it's not usefull.
1
vote
2answers
39 views

AWK - Add variable and result of function to variable

I want to add variable and result of function in AWK. Input: t t t t a t a ta ata ta a a Script: { key="t" print gsub(key,"")#<-it's work b=b+gsub(key,"")#<- it's something wrong } END{ ...
0
votes
1answer
273 views

awk - awk concatenate string variable

I would like to awk concatenate string variable in awk. How can I do that? I tried: BEGIN{ t="." r=";" w=t+r print w} But I does't work. Output: 0 Or I want to add variable and result of ...
0
votes
1answer
13 views

awk print data in fixed column

I would like to print data in fixed column. I mean the column should be like in table. I used printf. Input: To find information use the search box on the top right corner of the screen, or ...
0
votes
2answers
60 views

awk and regex with {}

I want to print whole record if it contain word and some character, which are optional. Script: /mount[ein]{0,2}/{ print $0} Unfortunately it doesn't work. Without "{" and "}" it work. I tried with ...
0
votes
0answers
24 views

Counting keys in line awk

I have to check how many keys(3 last characters in word) is in the all lines, but awk sent error. I think awk don't correct check condition. I tested it changing key to "t(or somthing eles). My ...
1
vote
1answer
24 views

Display Associative array in awk

How can I display Associative array, when I don't know index? My script: { tab[$1]=tab[$1]+1 } END{ print tab[*]#display array -> it doesn't work }
0
votes
1answer
40 views

awk - field separator change in program, letters in the field

I want to change FS in awk, but not in BEGIN. I want to print word and the letter. awk 'BEGIN{FS=""}{print $1; FS=" ";print $1; FS=""}' Of course it does't work. Is there any function, way to show ...
0
votes
1answer
45 views

Why awk printing wrong string into the output file?

I'm trying to read file with awk and write the output in a new file. Content file sample: DATE_Login|09:01:20.507|2015-08-13|TIME_Session|00:32:32|MX|w32 ...
0
votes
2answers
31 views

How can I replace a newline with its escape sequence?

Using sed to make text that contains certain characters suitable for use in string literals is straightforward: sed "s/\\\\/\\\\\\\\/g" sed "s/\\\"/\\\\\\\"/g" But how can I do something similar ...
1
vote
2answers
85 views

Find duplicate column and separate them to a file or variable(Bash)

I have output like ; (The columns separated by tab \t) name1 something1 name1 something2 name1 something3 name2 something4 name2 something5 For this output I need two output (if there is ...
-1
votes
1answer
79 views

linux command to get quarter details

I need to paas current date to linux command which will give me output for quarter number I have two queries date +"%Yq$(expr $(expr $(date -d '-1 month' +%m) - 1) / 3 + 1)" result -> ...
1
vote
2answers
249 views

escaping awk with remote ssh command and bash that is already escaped

Hello. ssh ufk@10.0.0.2 "bash -lc 'pm2 list | grep app | awk { print $3} '" I need to run the command I pasted here. the problem is that i'm already escaping twice... the bash with " and the pm2 ...
0
votes
1answer
58 views

Pass a path with parenthesis and space in system call inside awk script

Inside my simple awk script, I call system command #!/bin/bash Test='/home/software/Other/new (Applet)' ls "${Test}" var=$(ls "${Test}") echo $var awk -vTest="$var" 'BEGIN { ...
0
votes
0answers
11 views

Mac OS X Find and Replace API

Are the Find and Replace operations under the Edit menu in Acrobat or Word simply a front end for the unix awk, sed or grep commands? Where can I find more information about what's going on underneath ...
2
votes
3answers
203 views

How to use grep or awk to save every line from text file as new file?

I am a scripting beginner. What i am trying to do. I have a text file that looks like this (example): hello hi there apple I am trying to make a script that saves every line as a separate new file ...
0
votes
1answer
149 views

need to get average idle time using mpstat

I want to get average idle time of the server using mpstat. The problem I am having is %idle is not in same columns in all the versions of linux. example 1: [root@testserver ~]# mpstat 1 2 Linux ...
0
votes
3answers
448 views

Match the column heading and print the values of the column using awk

I have a data in the file in columns. I want to write a script, which displays the data of the column matching the column heading. C-1 C-2 C-3 C-4 C-5 C-6 C-7 C-8 C-9 10 30 35 20 9 65 87 30 ...
0
votes
1answer
36 views

Monitoring changes in file with bash

I'm running DHCP server on local network which calls this script every few seconds: dumpleases | awk '{print $2}' > leasesnow #get list of all IPs leased in a file, one per line ADDED=`awk ...
2
votes
3answers
125 views

Extract version string from filename

We have a list of filenames in file a.txt with their version string embedded in them. Eg: gson-2.1 xmlParserAPIs-2.4.0 acrobat-1.1 orai18n-mapping-12.1.0.2 jdbc-se2.0 eclipse-core-runtime-20070801 ...