Questions tagged [awk]
AWK is a text-processing language. It is mainly used to manipulate and process structured data, and to generate reports.
629
questions
1
vote
2
answers
8k
views
How to replace a column value in a variable with a variable using awk
I have two variables:
var=x
out=1,2,3,4,5
I want to replace the third column in $out with $var. I am able to substitute x for 3 in column 3 with the following command (I speak of columns in $out as ...
1
vote
2
answers
1k
views
How to remove all spaces at the end of strings in a text file in linux
I have file named a.txt and it contains strings but got some additional spaces at the end
cat -vet a.txt displays
ab cd^M$
abcdefg ^M$ //there is a space at the end
aaaaaaa^M$
bbbbbbb^I^M$ //there ...
0
votes
3
answers
1k
views
sed + how to replace numbers with parenthesis with sed/awk/other
we want to relapse the [1023,1024,1022] as described in the following file , with [1022]
more file.exm
{"topic":"nj_hgf_dfgef","partition":0,"replicas":[1023,1024,1022]},
{"topic":"nj_hgf_dfgef","...
0
votes
1
answer
731
views
awk with wildcard search not working
zcat * | awk -F"|" '$6==*404* {print $0;}'
This command is not working. I have field separator pipe and where after the 6th pipe i have value *404* wildcard with 404 key value and want to print the ...
2
votes
1
answer
216
views
regex - replace ordered list of lines matching a string with another list
I have such confused ideas I couldn't describe well the problem synthetically, hence the title is horrible. Suggest me a better one too, while you're at it.
Basically, I have two SubRip (.srt) ...
0
votes
1
answer
66
views
Changing a field value conditionally using awk
I have a below values in file.
user1 India GJ 1000 IT contractor
user2 Ind GJ 1001 HR contractor
user3 USA CA 1002 R&D hourly
user4 India TN 1003 R&D fte
user5 Ind KA 1004 FE fte
user6 In GJ ...
0
votes
1
answer
2k
views
grep for pattern including carriage return on a Mac
I want to search files for colon followed by carriage return followed by dash. I am using macOS (so it's BSD grep).
I would have thought
grep -l ":\n-" filename-pattern
would work, but it doesn'...
0
votes
1
answer
360
views
How to substitute every opening curly brace that changes bracket balance from 0 to 1 with sed/awk?
Example input:
void C::foo()
{ // <- this changes bracket balance from 0 to 1
if (true)
{ // balance is 2 after we process this bracket
...
} // now it is 1 again
} // now ...
0
votes
1
answer
3k
views
How do i filter request from a json using curl?
so I recently deployed this Go app on the web https://20200329t202730-dot-ipengine-dev.appspot.com/ and the format that it shows the IP currently is
{
"ip": "169.254.1.1"
}
How do I only get the ...
0
votes
2
answers
713
views
Convert dates to epoch value on multiple columns using AWK
A,B,C,D
1,30/07/2014,data_n/a,data_n/a
2,30/07/2014,22/02/2018,data_n/a
3,26/02/2015,22/12/2016,28/07/2017
I would like to convert all dates at once to epoch format. I am trying to get the following ...
1
vote
1
answer
1k
views
How to edit a multiline pattern in bash (sed and awk are available)
So here is my very specific task. I have patterns of type
.*[class|namespace|struct].*
(this is one part of the pattern, to be precise),
then an arbitrary amount of newlines \n or spaces ...
1
vote
1
answer
110
views
Using for loops in awk to print data
This is my original input :
Feb 27 15:26:22 ::ffff:127.0.0.1 [email protected] (35) /console/login | [Authentication] [User] [LoginFailed] Remote authentication failed. UserName = admin
This is the ...
0
votes
1
answer
620
views
Recursive search for matches in two files
On the second day, I'm trying to find a solution.
There is a CCC directory; we include subdirectories with numbers 1 2 3 ... 500, in each subdirectory there is a file named A1, containing one word (...
0
votes
1
answer
823
views
How to change the column header name in Linux
I want to switch this:
**hg19chrc snpid a1 a2 bp info or se p ngt**
chr1 rs4951859 C G 729679 0.631 0.97853 0.0173 0.2083 0
chr1 rs142557973 T C 731718 0.665 1....
1
vote
1
answer
52
views
Understanding Piping in AWK
I found this example in a lecture by Brian Kernighan.
to add together the prices of food items in a two column table
#!/usr/bin/awk -f
{amount[$1] += $2}
END {for (name in amount)
print ...
0
votes
1
answer
199
views
Using awk regular expression with gnu find
I am comfortable with AWK regular expressions and would like to use them in GNU find. Searching find's manpages I found there are two options called -regex and -regextype which seemed relevant.
So I ...
0
votes
1
answer
64
views
awk comparison on Ubuntu
I'm trying to write a script that check the disk usage of remote clients. There is a filter in script for meaningful output:
...
ssh server "sudo df -Ph 2>/dev/null \
| tail -n +2 \
| grep -v ...
0
votes
1
answer
62
views
Iteratively find all lines starting with X and move above next line starting with Y
I have a file like this:
Lorem ipsum dolor sit amet.
Consectetur adipiscing elit.
MATCH1 Quis autem vel!
Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
MATCH1 Eum iure ...
0
votes
3
answers
53
views
how to average, max based on group time
need your help , i want grouping timestamp based on minutes and value average and max
my data below :
2019-12-27 12:01:00,047 3
2019-12-27 12:01:00,079 34
2019-12-27 12:02:00,091 4
2019-12-27 12:02:...
1
vote
3
answers
146
views
how to awk from n to n elelment
grep -i 'search' var/log/sample.log | less -S | awk '{print $1,$2,$3,$31,$32,$33,$34,$35,$36,$37}'
I want to print from 31 to the end or from 31 to the 34 element only, and I don't want to list them ...
0
votes
1
answer
2k
views
Awk throws "No such file or directory error" where using hostname -I
This command works fine are print the hostname
awk -v MYHOST=$(hostname) '{printf(MYHOST)}'
However on running
awk -v MYHOST=$(hostname -I) '{printf(MYHOST)}'
throws a
awk: cannot open "ip ...
0
votes
2
answers
473
views
Append strings to lines with looping over
I have a file :
aaa
bbb
ccc
and another one:
111
222
333
444
.
.
999
My desired output after appending should be :
aaa 111
bbb 222
ccc 333
aaa 444
bbb 555
ccc 666
.
.
.
aaa nnn
I can append ...
0
votes
0
answers
41
views
Script shell, awk, lines and columns
Can you help me to format this datas in multiple columns ?
I have these datas on one column:
John
Patrick
6
4
5
7
1
2
40
15
MY CLUB
And would like to have :
John 6 5 1 40
Patrick 4 ...
2
votes
1
answer
77
views
replace keywords in a file by keywords\index{keywords} from a list file
I am trying to produce index entries in a LaTeX file under Linux. I have two files: file1.txt has all the index entries
cats\index{cat}
dogs\index{dog}
elephants\index{elephant}
rats\index{rat}
The ...
0
votes
1
answer
99
views
awk,sed help needed for addressing density [duplicate]
We are doing a study on livability and quality of life in high-density cities pulling information from sources that, when crunched, can provide an insight into the density ratio of apartments so as to ...
1
vote
2
answers
281
views
Remove specific string from file
output.txt
1.1.1.1:22/ does not support password authentication. [ERROR] target ssh://2.2.2.2:22/ does not support password authentication. [ERROR] target ssh://3.3.3.3
i want to remove string :22/ ...
0
votes
1
answer
479
views
awk + print lines only if second field match hours:min:sec
log file is like this
34.3.23.58 07:20:01 AM all 0.02 0.00 0.02 0.00 0.00 99.97
34.3.23.58 07:30:01 AM all 0.02 ...
0
votes
2
answers
203
views
Regex String Number bash
I have this situation:
$FILE_NAME = '207.Skyfall.2012.iTA-ENG.Bluray.1080p.x264.mkv'
echo $FILE_NAME | sed 's/.[0-9].*//' | sed 's/\./ /g'
Result empty the regex remove all
if the file doesn't ...
1
vote
1
answer
817
views
sed/awk: remove similar lines after first occurrence
i have:
constant1apple_____constant2asdfkjo___constant3
constant1apple_____constant2fdlkjef___constant3
constant1watermelonconstant2sdlfkeio__constant3
constant1banana____constant2asdfoie___constant3
...
0
votes
1
answer
26
views
Print a file and substract in two rows
I would like to print and substract in each two rows from a file
toto 30
tata 20
toto 12
tata 6
toto 22
tata 8
the output :
diff : 10
diff : 6
diff : 14
I find a script :
FILENAME=$1
while read ...
1
vote
1
answer
5k
views
Use awk command to output of ls command
I have a e-book directory that has following format:
<authorname> <authorsurname> - <bookname>
I want to separate the books by authors. I use the below command in a for-loop for ...
0
votes
1
answer
31
views
How to add a counter starting from a number other than 1?
I have a file where I need to add 200 more lines, but I'd like to know how to renumber them automatically. The file has the following structure:
col1\tcol2\tdb4444\tcol4\tcol5\tcol6\tcol7\tcol8\tcol9\...
2
votes
1
answer
307
views
Most efficient way to print all lines with duplicated field?
The following file, fruit_notes.txt, has three pipe-separated columns: fruit, color, and tasting notes. I would like to print all lines that have a duplicated color field. Order is not important.
...
0
votes
2
answers
105
views
Need help sed, awk or other tool that can manupulate text files
I have a text file Test.m3u that contains file names:
FileName1.mp4
FileName2.mp4
FileName3.mp4
I need to make a playlist in the following form:
#EXTM3U
#EXTINF:FileName1
PATH/FileName1.mp4
#...
0
votes
1
answer
11k
views
Counting total number of pages across multiple PDF files on the Windows' Powershell
I'm trying to write a single script on the Windows' Powershell so I can know the total number of pages across various pdf files in the same directory. However, I'm not getting the intended results. ...
1
vote
1
answer
230
views
Sed / awk / perl text editing on n lines after match
I'm strugling with this.
I'm trying to edit the next 6 lines after a match, by adding text before each lines, and double quotes before and after the existing number on each lines.
What my file look ...
0
votes
1
answer
1k
views
awk and sed append end of line each related line
I want to do some line manipulation but am stuck in the line by line processing. My file looks like this:
10.10.10.10 ABtest
10.10.10.11 ABprod
I want to append a string at each line according to ...
0
votes
1
answer
92
views
Why is my bash shell script changing spaces to linefeeds?
I started out writing a simple bash shell script to find duplicate files within a given directory using SHA1 to detect matches. Everything worked until I encountered file names with spaces. The ...
0
votes
1
answer
151
views
Cygwin gawk doesn't pipe when called explicitly
That's the best description I can think of, apologies if it's a little esoteric.
I have cygwin installed on Win 10, Build 16299 but it's installed with the non-administrator option; there is no ...
0
votes
1
answer
248
views
pass command line arguments in a bash script
I am trying to retrieve a line from a text file, write it to a file locally, and then combine it with the list of IPs into a new file. I am having issues escaping all of the parameters I think though,...
0
votes
1
answer
397
views
Perform awk equivalant with Sublime Text
With Sublime Text 3, is it possible to modify an opened file to keep only a given column ?
For instance I have an file with this content:
a 1
b 2
c 3
d 4
I would like Sublime Text to transform this ...
3
votes
1
answer
204
views
Strange awk syntax: Number with two dots?
By accident I did
gawk 'BEGIN { print 1.2.3+4; }'
and received
1.24.3
I expected a syntax error because of the two dots in the number.
What happened here ?
Is that a bug in GNU awk or is it a ...
1
vote
1
answer
85
views
print the number in second col the number of times in a new line always as specified in col 1
how do I print the value in col 2 as many times as it is specified in col 1.
10 -11
9 -10
7 -9
7 -8
7 -7
12 -6
362 -1
271 1
94 2
41 3
17 4
13 5
16 6
5 7
expected output:
-11
-11
-11
-11
-11
-11
-11
...
0
votes
1
answer
160
views
awk to combine files on partial match and append
In the awk below I am trying to cp and paste each matching line in f2 to $3 in f1 if $2 of f1 is in the line in f2 somewhere. There will always be a match (usually more then 1) and my actual data is ...
0
votes
1
answer
196
views
awk with sed to combine lines and remove specic odd pattern from line
In the awk piped to sed below I am trying to format file by removing the odd xxxx_digits and whitespace after, then move the even xxxx_digit to the line above it and add a space between them. There ...
0
votes
1
answer
293
views
awk to extract specific pattern in file based on variable match
In the awk below I am trying to extract the value of RSTRING based on where the value of another variable $VCF is found in a file. The value of $VCF is already extracted and the RSTRING is currently ...
0
votes
1
answer
37
views
Merge lines which are not in proper
I have the lines similar to below.
[1] 07:37:38 [bb..], qw, Exited with errojjjvice not known
[2] 07:37:39 [bfb.], fg, , This is FILE1, Stderrle or directory
[3] 07:37:39 [SU..], b b, ,, Stderr: bash:...
0
votes
1
answer
561
views
awk to delete even number file leaving odd number files
In the below awk I am trying to remove the .bam and .bam.bai if the digits after the IonCode_ are even and keep the files if the digits after the IonCode_ are odd. The awk does execute but no files ...
1
vote
1
answer
77
views
awk/perl statement to validate and return fixed-structure data lines at start of file
I have a file that starts like this
## CONFIG-PARAMS-START ##
##
## text1 text2 NNNNNNNNN (arbitrary_comment) ##
## text1 text2 NNNNNNNNN (arbitrary_comment) ##
## text1 text2 NNNNNNNNN (...
0
votes
2
answers
83
views
awk script to provides a structured listing
I'm trying to create awk script that can provide a structured listing of account information for all UIDs greater than 1000.
This is what I tried:
This is the expected output
BEGIN { FS=":"}
$3>...