Questions tagged [awk]

AWK is a text-processing language. It is mainly used to manipulate and process structured data, and to generate reports.

Filter by
Sorted by
Tagged with
1 vote
2 answers
8k views

How to replace a column value in a variable with a variable using awk

I have two variables: var=x out=1,2,3,4,5 I want to replace the third column in $out with $var. I am able to substitute x for 3 in column 3 with the following command (I speak of columns in $out as ...
speed bump's user avatar
1 vote
2 answers
1k views

How to remove all spaces at the end of strings in a text file in linux

I have file named a.txt and it contains strings but got some additional spaces at the end cat -vet a.txt displays ab cd^M$ abcdefg ^M$ //there is a space at the end aaaaaaa^M$ bbbbbbb^I^M$ //there ...
Kros's user avatar
  • 11
0 votes
3 answers
1k views

sed + how to replace numbers with parenthesis with sed/awk/other

we want to relapse the [1023,1024,1022] as described in the following file , with [1022] more file.exm {"topic":"nj_hgf_dfgef","partition":0,"replicas":[1023,1024,1022]}, {"topic":"nj_hgf_dfgef","...
King David's user avatar
0 votes
1 answer
731 views

awk with wildcard search not working

zcat * | awk -F"|" '$6==*404* {print $0;}' This command is not working. I have field separator pipe and where after the 6th pipe i have value *404* wildcard with 404 key value and want to print the ...
Prithvi Jayaraman's user avatar
2 votes
1 answer
216 views

regex - replace ordered list of lines matching a string with another list

I have such confused ideas I couldn't describe well the problem synthetically, hence the title is horrible. Suggest me a better one too, while you're at it. Basically, I have two SubRip (.srt) ...
Jeffrey Lebowski's user avatar
0 votes
1 answer
66 views

Changing a field value conditionally using awk

I have a below values in file. user1 India GJ 1000 IT contractor user2 Ind GJ 1001 HR contractor user3 USA CA 1002 R&D hourly user4 India TN 1003 R&D fte user5 Ind KA 1004 FE fte user6 In GJ ...
KumarJohn's user avatar
  • 417
0 votes
1 answer
2k views

grep for pattern including carriage return on a Mac

I want to search files for colon followed by carriage return followed by dash. I am using macOS (so it's BSD grep). I would have thought grep -l ":\n-" filename-pattern would work, but it doesn'...
Scott C Wilson's user avatar
0 votes
1 answer
360 views

How to substitute every opening curly brace that changes bracket balance from 0 to 1 with sed/awk?

Example input: void C::foo() { // <- this changes bracket balance from 0 to 1 if (true) { // balance is 2 after we process this bracket ... } // now it is 1 again } // now ...
Artyom Gevorgyan's user avatar
0 votes
1 answer
3k views

How do i filter request from a json using curl?

so I recently deployed this Go app on the web https://20200329t202730-dot-ipengine-dev.appspot.com/ and the format that it shows the IP currently is { "ip": "169.254.1.1" } How do I only get the ...
Prajwal Koirala's user avatar
0 votes
2 answers
713 views

Convert dates to epoch value on multiple columns using AWK

A,B,C,D 1,30/07/2014,data_n/a,data_n/a 2,30/07/2014,22/02/2018,data_n/a 3,26/02/2015,22/12/2016,28/07/2017 I would like to convert all dates at once to epoch format. I am trying to get the following ...
IronMan's user avatar
1 vote
1 answer
1k views

How to edit a multiline pattern in bash (sed and awk are available)

So here is my very specific task. I have patterns of type .*[class|namespace|struct].* (this is one part of the pattern, to be precise), then an arbitrary amount of newlines \n or spaces ...
Artyom Gevorgyan's user avatar
1 vote
1 answer
110 views

Using for loops in awk to print data

This is my original input : Feb 27 15:26:22 ::ffff:127.0.0.1 [email protected] (35) /console/login | [Authentication] [User] [LoginFailed] Remote authentication failed. UserName = admin This is the ...
Cueisa's user avatar
  • 31
0 votes
1 answer
620 views

Recursive search for matches in two files

On the second day, I'm trying to find a solution. There is a CCC directory; we include subdirectories with numbers 1 2 3 ... 500, in each subdirectory there is a file named A1, containing one word (...
alex77's user avatar
  • 1
0 votes
1 answer
823 views

How to change the column header name in Linux

I want to switch this: **hg19chrc snpid a1 a2 bp info or se p ngt** chr1 rs4951859 C G 729679 0.631 0.97853 0.0173 0.2083 0 chr1 rs142557973 T C 731718 0.665 1....
GLR's user avatar
  • 1
1 vote
1 answer
52 views

Understanding Piping in AWK

I found this example in a lecture by Brian Kernighan. to add together the prices of food items in a two column table #!/usr/bin/awk -f {amount[$1] += $2} END {for (name in amount) print ...
smilingbuddha's user avatar
0 votes
1 answer
199 views

Using awk regular expression with gnu find

I am comfortable with AWK regular expressions and would like to use them in GNU find. Searching find's manpages I found there are two options called -regex and -regextype which seemed relevant. So I ...
smilingbuddha's user avatar
0 votes
1 answer
64 views

awk comparison on Ubuntu

I'm trying to write a script that check the disk usage of remote clients. There is a filter in script for meaningful output: ... ssh server "sudo df -Ph 2>/dev/null \ | tail -n +2 \ | grep -v ...
Jo Shepherd's user avatar
0 votes
1 answer
62 views

Iteratively find all lines starting with X and move above next line starting with Y

I have a file like this: Lorem ipsum dolor sit amet. Consectetur adipiscing elit. MATCH1 Quis autem vel! Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. MATCH1 Eum iure ...
rahim123's user avatar
  • 103
0 votes
3 answers
53 views

how to average, max based on group time

need your help , i want grouping timestamp based on minutes and value average and max my data below : 2019-12-27 12:01:00,047 3 2019-12-27 12:01:00,079 34 2019-12-27 12:02:00,091 4 2019-12-27 12:02:...
Fajar Hadiyanto's user avatar
1 vote
3 answers
146 views

how to awk from n to n elelment

grep -i 'search' var/log/sample.log | less -S | awk '{print $1,$2,$3,$31,$32,$33,$34,$35,$36,$37}' I want to print from 31 to the end or from 31 to the 34 element only, and I don't want to list them ...
Cueisa's user avatar
  • 31
0 votes
1 answer
2k views

Awk throws "No such file or directory error" where using hostname -I

This command works fine are print the hostname awk -v MYHOST=$(hostname) '{printf(MYHOST)}' However on running awk -v MYHOST=$(hostname -I) '{printf(MYHOST)}' throws a awk: cannot open "ip ...
Aninda Choudhary's user avatar
0 votes
2 answers
473 views

Append strings to lines with looping over

I have a file : aaa bbb ccc and another one: 111 222 333 444 . . 999 My desired output after appending should be : aaa 111 bbb 222 ccc 333 aaa 444 bbb 555 ccc 666 . . . aaa nnn I can append ...
Rajib's user avatar
  • 3,106
0 votes
0 answers
41 views

Script shell, awk, lines and columns

Can you help me to format this datas in multiple columns ? I have these datas on one column: John Patrick 6 4 5 7 1 2 40 15 MY CLUB And would like to have : John 6 5 1 40 Patrick 4 ...
Livemedia Livemedia's user avatar
2 votes
1 answer
77 views

replace keywords in a file by keywords\index{keywords} from a list file

I am trying to produce index entries in a LaTeX file under Linux. I have two files: file1.txt has all the index entries cats\index{cat} dogs\index{dog} elephants\index{elephant} rats\index{rat} The ...
yacine's user avatar
  • 23
0 votes
1 answer
99 views

awk,sed help needed for addressing density [duplicate]

We are doing a study on livability and quality of life in high-density cities pulling information from sources that, when crunched, can provide an insight into the density ratio of apartments so as to ...
jeffschips's user avatar
1 vote
2 answers
281 views

Remove specific string from file

output.txt 1.1.1.1:22/ does not support password authentication. [ERROR] target ssh://2.2.2.2:22/ does not support password authentication. [ERROR] target ssh://3.3.3.3 i want to remove string :22/ ...
beginner's user avatar
0 votes
1 answer
479 views

awk + print lines only if second field match hours:min:sec

log file is like this 34.3.23.58 07:20:01 AM all 0.02 0.00 0.02 0.00 0.00 99.97 34.3.23.58 07:30:01 AM all 0.02 ...
King David's user avatar
0 votes
2 answers
203 views

Regex String Number bash

I have this situation: $FILE_NAME = '207.Skyfall.2012.iTA-ENG.Bluray.1080p.x264.mkv' echo $FILE_NAME | sed 's/.[0-9].*//' | sed 's/\./ /g' Result empty the regex remove all if the file doesn't ...
michel's user avatar
  • 1
1 vote
1 answer
817 views

sed/awk: remove similar lines after first occurrence

i have: constant1apple_____constant2asdfkjo___constant3 constant1apple_____constant2fdlkjef___constant3 constant1watermelonconstant2sdlfkeio__constant3 constant1banana____constant2asdfoie___constant3 ...
cablewelo2ma's user avatar
0 votes
1 answer
26 views

Print a file and substract in two rows

I would like to print and substract in each two rows from a file toto 30 tata 20 toto 12 tata 6 toto 22 tata 8 the output : diff : 10 diff : 6 diff : 14 I find a script : FILENAME=$1 while read ...
tumareure's user avatar
1 vote
1 answer
5k views

Use awk command to output of ls command

I have a e-book directory that has following format: <authorname> <authorsurname> - <bookname> I want to separate the books by authors. I use the below command in a for-loop for ...
Jo Shepherd's user avatar
0 votes
1 answer
31 views

How to add a counter starting from a number other than 1?

I have a file where I need to add 200 more lines, but I'd like to know how to renumber them automatically. The file has the following structure: col1\tcol2\tdb4444\tcol4\tcol5\tcol6\tcol7\tcol8\tcol9\...
Primo's user avatar
  • 11
2 votes
1 answer
307 views

Most efficient way to print all lines with duplicated field?

The following file, fruit_notes.txt, has three pipe-separated columns: fruit, color, and tasting notes. I would like to print all lines that have a duplicated color field. Order is not important. ...
Sagebrush Gardener's user avatar
0 votes
2 answers
105 views

Need help sed, awk or other tool that can manupulate text files

I have a text file Test.m3u that contains file names: FileName1.mp4 FileName2.mp4 FileName3.mp4 I need to make a playlist in the following form: #EXTM3U #EXTINF:FileName1 PATH/FileName1.mp4 #...
PinPiguin's user avatar
  • 121
0 votes
1 answer
11k views

Counting total number of pages across multiple PDF files on the Windows' Powershell

I'm trying to write a single script on the Windows' Powershell so I can know the total number of pages across various pdf files in the same directory. However, I'm not getting the intended results. ...
136's user avatar
  • 103
1 vote
1 answer
230 views

Sed / awk / perl text editing on n lines after match

I'm strugling with this. I'm trying to edit the next 6 lines after a match, by adding text before each lines, and double quotes before and after the existing number on each lines. What my file look ...
Roxxxance's user avatar
0 votes
1 answer
1k views

awk and sed append end of line each related line

I want to do some line manipulation but am stuck in the line by line processing. My file looks like this: 10.10.10.10 ABtest 10.10.10.11 ABprod I want to append a string at each line according to ...
arifisik's user avatar
  • 302
0 votes
1 answer
92 views

Why is my bash shell script changing spaces to linefeeds?

I started out writing a simple bash shell script to find duplicate files within a given directory using SHA1 to detect matches. Everything worked until I encountered file names with spaces. The ...
dacracot's user avatar
  • 333
0 votes
1 answer
151 views

Cygwin gawk doesn't pipe when called explicitly

That's the best description I can think of, apologies if it's a little esoteric. I have cygwin installed on Win 10, Build 16299 but it's installed with the non-administrator option; there is no ...
Pete M's user avatar
  • 1
0 votes
1 answer
248 views

pass command line arguments in a bash script

I am trying to retrieve a line from a text file, write it to a file locally, and then combine it with the list of IPs into a new file. I am having issues escaping all of the parameters I think though,...
Chris Schneider's user avatar
0 votes
1 answer
397 views

Perform awk equivalant with Sublime Text

With Sublime Text 3, is it possible to modify an opened file to keep only a given column ? For instance I have an file with this content: a 1 b 2 c 3 d 4 I would like Sublime Text to transform this ...
Pierre's user avatar
  • 175
3 votes
1 answer
204 views

Strange awk syntax: Number with two dots?

By accident I did gawk 'BEGIN { print 1.2.3+4; }' and received 1.24.3 I expected a syntax error because of the two dots in the number. What happened here ? Is that a bug in GNU awk or is it a ...
Juergen's user avatar
  • 572
1 vote
1 answer
85 views

print the number in second col the number of times in a new line always as specified in col 1

how do I print the value in col 2 as many times as it is specified in col 1. 10 -11 9 -10 7 -9 7 -8 7 -7 12 -6 362 -1 271 1 94 2 41 3 17 4 13 5 16 6 5 7 expected output: -11 -11 -11 -11 -11 -11 -11 ...
Gavin's user avatar
  • 13
0 votes
1 answer
160 views

awk to combine files on partial match and append

In the awk below I am trying to cp and paste each matching line in f2 to $3 in f1 if $2 of f1 is in the line in f2 somewhere. There will always be a match (usually more then 1) and my actual data is ...
justaguy's user avatar
  • 229
0 votes
1 answer
196 views

awk with sed to combine lines and remove specic odd pattern from line

In the awk piped to sed below I am trying to format file by removing the odd xxxx_digits and whitespace after, then move the even xxxx_digit to the line above it and add a space between them. There ...
justaguy's user avatar
  • 229
0 votes
1 answer
293 views

awk to extract specific pattern in file based on variable match

In the awk below I am trying to extract the value of RSTRING based on where the value of another variable $VCF is found in a file. The value of $VCF is already extracted and the RSTRING is currently ...
justaguy's user avatar
  • 229
0 votes
1 answer
37 views

Merge lines which are not in proper

I have the lines similar to below. [1] 07:37:38 [bb..], qw, Exited with errojjjvice not known [2] 07:37:39 [bfb.], fg, , This is FILE1, Stderrle or directory [3] 07:37:39 [SU..], b b, ,, Stderr: bash:...
user1012653's user avatar
0 votes
1 answer
561 views

awk to delete even number file leaving odd number files

In the below awk I am trying to remove the .bam and .bam.bai if the digits after the IonCode_ are even and keep the files if the digits after the IonCode_ are odd. The awk does execute but no files ...
justaguy's user avatar
  • 229
1 vote
1 answer
77 views

awk/perl statement to validate and return fixed-structure data lines at start of file

I have a file that starts like this ## CONFIG-PARAMS-START ## ## ## text1 text2 NNNNNNNNN (arbitrary_comment) ## ## text1 text2 NNNNNNNNN (arbitrary_comment) ## ## text1 text2 NNNNNNNNN (...
Stilez's user avatar
  • 1,685
0 votes
2 answers
83 views

awk script to provides a structured listing

I'm trying to create awk script that can provide a structured listing of account information for all UIDs greater than 1000. This is what I tried: This is the expected output BEGIN { FS=":"} $3>...
Luay Aq's user avatar

1 2
3
4 5
13