Questions tagged [awk]
AWK is a text-processing language. It is mainly used to manipulate and process structured data, and to generate reports.
530
questions
0
votes
0answers
31 views
awk with wildcard search not working
zcat * | awk -F"|" '$6==*404* {print $0;}'
This command is not working. I have field separator pipe and where after the 6th pipe i have value *404* wildcard with 404 key value and want to print the ...
2
votes
1answer
37 views
regex - replace ordered list of lines matching a string with another list
I have such confused ideas I couldn't describe well the problem synthetically, hence the title is horrible. Suggest me a better one too, while you're at it.
Basically, I have two SubRip (.srt) ...
0
votes
0answers
31 views
Given a large set of IDs, how would one extract information from another tab-separated file both ID-related and closely related data
The ID file is formatted thus:
A0A1H3TMV2
Z8X0S9
A0A2M6TTB8
A0A2Q5TOM7
A0A1U4MIN3
X9RTW0
The tab-separated file has the following format:
Query ID Neighbor ID
A0A1H3TMV2 A0A1H3TMX3
...
0
votes
1answer
21 views
Changing a field value conditionally using awk
I have a below values in file.
user1 India GJ 1000 IT contractor
user2 Ind GJ 1001 HR contractor
user3 USA CA 1002 R&D hourly
user4 India TN 1003 R&D fte
user5 Ind KA 1004 FE fte
user6 In GJ ...
0
votes
1answer
43 views
grep for pattern including carriage return on a Mac
I want to search files for colon followed by carriage return followed by dash. I am using macOS (so it's BSD grep).
I would have thought
grep -l ":\n-" filename-pattern
would work, but it doesn'...
0
votes
1answer
57 views
How to substitute every opening curly brace that changes bracket balance from 0 to 1 with sed/awk?
Example input:
void C::foo()
{ // <- this changes bracket balance from 0 to 1
if (true)
{ // balance is 2 after we process this bracket
...
} // now it is 1 again
} // now ...
0
votes
1answer
34 views
How do i filter request from a json using curl?
so I recently deployed this Go app on the web https://20200329t202730-dot-ipengine-dev.appspot.com/ and the format that it shows the IP currently is
{
"ip": "169.254.1.1"
}
How do I only get the ...
0
votes
2answers
158 views
Convert dates to epoch value on multiple columns using AWK
A,B,C,D
1,30/07/2014,data_n/a,data_n/a
2,30/07/2014,22/02/2018,data_n/a
3,26/02/2015,22/12/2016,28/07/2017
I would like to convert all dates at once to epoch format. I am trying to get the following ...
0
votes
1answer
46 views
How to edit a multiline pattern in bash (sed and awk are available)
So here is my very specific task. I have patterns of type
.*[class|namespace|struct].*
(this is one part of the pattern, to be precise),
then an arbitrary amount of newlines \n or spaces ...
1
vote
1answer
25 views
Using for loops in awk to print data
This is my original input :
Feb 27 15:26:22 ::ffff:127.0.0.1 admin@192.168.1.1 (35) /console/login | [Authentication] [User] [LoginFailed] Remote authentication failed. UserName = admin
This is the ...
0
votes
0answers
390 views
Recursive search for matches in two files
On the second day, I'm trying to find a solution.
There is a CCC directory; we include subdirectories with numbers 1 2 3 ... 500, in each subdirectory there is a file named A1, containing one word (...
0
votes
1answer
30 views
How to change the column header name in Linux
I want to switch this:
**hg19chrc snpid a1 a2 bp info or se p ngt**
chr1 rs4951859 C G 729679 0.631 0.97853 0.0173 0.2083 0
chr1 rs142557973 T C 731718 0.665 1....
0
votes
1answer
24 views
Understanding Piping in AWK
I found this example in a lecture by Brian Kernighan.
to add together the prices of food items in a two column table
#!/usr/bin/awk -f
{amount[$1] += $2}
END {for (name in amount)
print ...
0
votes
1answer
21 views
Using awk regular expression with gnu find
I am comfortable with AWK regular expressions and would like to use them in GNU find. Searching find's manpages I found there are two options called -regex and -regextype which seemed relevant.
So I ...
0
votes
0answers
18 views
awk not working with <> </>
It's pretty self-explanatory help?
awk '/<key>ProductVersion</key>/{flag=1;next}/<key>ReleaseType</key>/{flag=0}flag' /System/Library/CoreServices/SystemVersion.plist
awk: cmd....
0
votes
1answer
46 views
awk comparison on Ubuntu
I'm trying to write a script that check the disk usage of remote clients. There is a filter in script for meaningful output:
...
ssh server "sudo df -Ph 2>/dev/null \
| tail -n +2 \
| grep -v ...
0
votes
1answer
44 views
Iteratively find all lines starting with X and move above next line starting with Y
I have a file like this:
Lorem ipsum dolor sit amet.
Consectetur adipiscing elit.
MATCH1 Quis autem vel!
Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
MATCH1 Eum iure ...
0
votes
0answers
32 views
Join two files having the common column in different positions using AWK
I have two input files:
ex. File1.csv
a,b,2,c,d
w,a,1,d,v
av,er,3,sd,q
ex. File2.csv
1,5,6
3,7,5
The File1.csv is the main file. The column 3 of File1.csv needs to be joined with column 1 of ...
0
votes
3answers
43 views
how to average, max based on group time
need your help , i want grouping timestamp based on minutes and value average and max
my data below :
2019-12-27 12:01:00,047 3
2019-12-27 12:01:00,079 34
2019-12-27 12:02:00,091 4
2019-12-27 12:02:...
1
vote
3answers
52 views
how to awk from n to n elelment
grep -i 'search' var/log/sample.log | less -S | awk '{print $1,$2,$3,$31,$32,$33,$34,$35,$36,$37}'
I want to print from 31 to the end or from 31 to the 34 element only, and I don't want to list them ...
0
votes
1answer
52 views
Awk throws “No such file or directory error” where using hostname -I
This command works fine are print the hostname
awk -v MYHOST=$(hostname) '{printf(MYHOST)}'
However on running
awk -v MYHOST=$(hostname -I) '{printf(MYHOST)}'
throws a
awk: cannot open "ip ...
0
votes
2answers
96 views
Append strings to lines with looping over
I have a file :
aaa
bbb
ccc
and another one:
111
222
333
444
.
.
999
My desired output after appending should be :
aaa 111
bbb 222
ccc 333
aaa 444
bbb 555
ccc 666
.
.
.
aaa nnn
I can append ...
0
votes
0answers
34 views
Script shell, awk, lines and columns
Can you help me to format this datas in multiple columns ?
I have these datas on one column:
John
Patrick
6
4
5
7
1
2
40
15
MY CLUB
And would like to have :
John 6 5 1 40
Patrick 4 ...
2
votes
1answer
31 views
replace keywords in a file by keywords\index{keywords} from a list file
I am trying to produce index entries in a LaTeX file under Linux. I have two files: file1.txt has all the index entries
cats\index{cat}
dogs\index{dog}
elephants\index{elephant}
rats\index{rat}
The ...
0
votes
1answer
93 views
awk,sed help needed for addressing density [duplicate]
We are doing a study on livability and quality of life in high-density cities pulling information from sources that, when crunched, can provide an insight into the density ratio of apartments so as to ...
1
vote
2answers
55 views
Remove specific string from file
output.txt
1.1.1.1:22/ does not support password authentication. [ERROR] target ssh://2.2.2.2:22/ does not support password authentication. [ERROR] target ssh://3.3.3.3
i want to remove string :22/ ...
1
vote
1answer
56 views
awk + print lines only if second field match hours:min:sec
log file is like this
34.3.23.58 07:20:01 AM all 0.02 0.00 0.02 0.00 0.00 99.97
34.3.23.58 07:30:01 AM all 0.02 ...
0
votes
2answers
43 views
Regex String Number bash
I have this situation:
$FILE_NAME = '207.Skyfall.2012.iTA-ENG.Bluray.1080p.x264.mkv'
echo $FILE_NAME | sed 's/.[0-9].*//' | sed 's/\./ /g'
Result empty the regex remove all
if the file doesn't ...
0
votes
1answer
90 views
sed/awk: remove similar lines after first occurrence
i have:
constant1apple_____constant2asdfkjo___constant3
constant1apple_____constant2fdlkjef___constant3
constant1watermelonconstant2sdlfkeio__constant3
constant1banana____constant2asdfoie___constant3
...
0
votes
1answer
19 views
Print a file and substract in two rows
I would like to print and substract in each two rows from a file
toto 30
tata 20
toto 12
tata 6
toto 22
tata 8
the output :
diff : 10
diff : 6
diff : 14
I find a script :
FILENAME=$1
while read ...
0
votes
1answer
934 views
Use awk command to output of ls command
I have a e-book directory that has following format:
<authorname> <authorsurname> - <bookname>
I want to separate the books by authors. I use the below command in a for-loop for ...
0
votes
1answer
22 views
How to add a counter starting from a number other than 1?
I have a file where I need to add 200 more lines, but I'd like to know how to renumber them automatically. The file has the following structure:
col1\tcol2\tdb4444\tcol4\tcol5\tcol6\tcol7\tcol8\tcol9\...
2
votes
1answer
33 views
Most efficient way to print all lines with duplicated field?
The following file, fruit_notes.txt, has three pipe-separated columns: fruit, color, and tasting notes. I would like to print all lines that have a duplicated color field. Order is not important.
...
0
votes
2answers
51 views
Need help sed, awk or other tool that can manupulate text files
I have a text file Test.m3u that contains file names:
FileName1.mp4
FileName2.mp4
FileName3.mp4
I need to make a playlist in the following form:
#EXTM3U
#EXTINF:FileName1
PATH/FileName1.mp4
#...
0
votes
1answer
2k views
Counting total number of pages across multiple PDF files on the Windows' Powershell
I'm trying to write a single script on the Windows' Powershell so I can know the total number of pages across various pdf files in the same directory. However, I'm not getting the intended results. ...
1
vote
1answer
63 views
Sed / awk / perl text editing on n lines after match
I'm strugling with this.
I'm trying to edit the next 6 lines after a match, by adding text before each lines, and double quotes before and after the existing number on each lines.
What my file look ...
0
votes
0answers
24 views
Awk to get data from second line and column. (ksh)
I have a table with stored 2 columns.
create table for_ctl
(AS_OF_DT date,TOT_RCRD integer);
insert into for_ctl(AS_OF_DT,TOT_RCRD)
values(to_date(to_char(20190605,'YYYYMMDD')),5);
After that I ...
0
votes
1answer
229 views
awk and sed append end of line each related line
I want to do some line manipulation but am stuck in the line by line processing. My file looks like this:
10.10.10.10 ABtest
10.10.10.11 ABprod
I want to append a string at each line according to ...
0
votes
1answer
85 views
Why is my bash shell script changing spaces to linefeeds?
I started out writing a simple bash shell script to find duplicate files within a given directory using SHA1 to detect matches. Everything worked until I encountered file names with spaces. The ...
0
votes
1answer
50 views
Cygwin gawk doesn't pipe when called explicitly
That's the best description I can think of, apologies if it's a little esoteric.
I have cygwin installed on Win 10, Build 16299 but it's installed with the non-administrator option; there is no ...
0
votes
1answer
64 views
pass command line arguments in a bash script
I am trying to retrieve a line from a text file, write it to a file locally, and then combine it with the list of IPs into a new file. I am having issues escaping all of the parameters I think though,...
0
votes
1answer
101 views
Perform awk equivalant with Sublime Text
With Sublime Text 3, is it possible to modify an opened file to keep only a given column ?
For instance I have an file with this content:
a 1
b 2
c 3
d 4
I would like Sublime Text to transform this ...
3
votes
1answer
67 views
Strange awk syntax: Number with two dots?
By accident I did
gawk 'BEGIN { print 1.2.3+4; }'
and received
1.24.3
I expected a syntax error because of the two dots in the number.
What happened here ?
Is that a bug in GNU awk or is it a ...
1
vote
1answer
31 views
print the number in second col the number of times in a new line always as specified in col 1
how do I print the value in col 2 as many times as it is specified in col 1.
10 -11
9 -10
7 -9
7 -8
7 -7
12 -6
362 -1
271 1
94 2
41 3
17 4
13 5
16 6
5 7
expected output:
-11
-11
-11
-11
-11
-11
-11
...
0
votes
1answer
57 views
awk to combine files on partial match and append
In the awk below I am trying to cp and paste each matching line in f2 to $3 in f1 if $2 of f1 is in the line in f2 somewhere. There will always be a match (usually more then 1) and my actual data is ...
0
votes
1answer
65 views
awk with sed to combine lines and remove specic odd pattern from line
In the awk piped to sed below I am trying to format file by removing the odd xxxx_digits and whitespace after, then move the even xxxx_digit to the line above it and add a space between them. There ...
0
votes
1answer
52 views
awk to extract specific pattern in file based on variable match
In the awk below I am trying to extract the value of RSTRING based on where the value of another variable $VCF is found in a file. The value of $VCF is already extracted and the RSTRING is currently ...
0
votes
1answer
30 views
Merge lines which are not in proper
I have the lines similar to below.
[1] 07:37:38 [bb..], qw, Exited with errojjjvice not known
[2] 07:37:39 [bfb.], fg, , This is FILE1, Stderrle or directory
[3] 07:37:39 [SU..], b b, ,, Stderr: bash:...
0
votes
1answer
87 views
awk to delete even number file leaving odd number files
In the below awk I am trying to remove the .bam and .bam.bai if the digits after the IonCode_ are even and keep the files if the digits after the IonCode_ are odd. The awk does execute but no files ...
1
vote
1answer
59 views
awk/perl statement to validate and return fixed-structure data lines at start of file
I have a file that starts like this
## CONFIG-PARAMS-START ##
##
## text1 text2 NNNNNNNNN (arbitrary_comment) ##
## text1 text2 NNNNNNNNN (arbitrary_comment) ##
## text1 text2 NNNNNNNNN (...
