Questions tagged [awk]

AWK is a text-processing language. It is mainly used to manipulate and process structured data, and to generate reports.

Filter by
Sorted by
Tagged with
0
votes
1answer
16 views

sed/awk: convertion of multi-column data

I am operating with multi-column data file procuded by some AWK script: # output.csv lig12, dG(rescored) 1, 0.596625 2, 1.05873 3, 1.11285 4, 0.697402 I need to modify this output.csv to one line ...
1
vote
1answer
43 views

Use sed/awk to add specific text from one file to beginning of another file

I'm trying to use sed/awk to copy a range of text from the beginning of file1 to a specific line. Then add that output to the beginning of file 2 for a script I'm writing. The following command does ...
0
votes
2answers
16 views

Extracting data from text file by specific characters

I have a program that reads from a sensor and prints on stdout. The interesting data for me is just the X,Y and Z values: [2021-05-07 10:47:41.993] [info] Acceleromter (avrg 100): X=+6696.080 Y=-5577....
0
votes
2answers
63 views

awk Removing Double Quotes from CSV File Quoted Strings separated by comma

I have a csv file I would like to remove any rouge " or , in it. The problem is all my fields look something like this "***********","*********", where * = can be any ...
0
votes
0answers
28 views

Error filtering 2 files using AWK for a given condition

First of all, thank you for your help. I have a problem filtering 2 files using AWK conditionals. The two files I want to filter are these one: Fasta.fa >SiiA lcl|NC_003197.2_prot_NP_463122....
0
votes
0answers
16 views

Get Characters up to the last given pattern

need help. I want to get all the CREATE statements on a file until ; but the only problem if there is a ; in the comment.. so I need to get the CREATE statement until the ); pattern... .... CREATE ...
0
votes
1answer
63 views

Shuffle rows independently in a large file

I need to generate a random shuffle a very large csv (where I don't know in advance how many columns would be) in this way. So I have to go from this form 1,a,... 2,b,... 3,c,... to something like ...
1
vote
2answers
23 views

Extract key from key-value pair

I have a file looking like that. mke_version = "3.3.7" msr_version = "2.8.5" mcr_version = "19.03.14" I want to print from file only key ...
0
votes
1answer
19 views

Combining two txt files containing name and last name on command line

I have two text files. First file have 20 Names listed in seperate lines and 2nd file have 20 surnames listed in seperated lines. How I can combine those files. So final text file have both first name ...
1
vote
1answer
18 views

Print sections in text file using awk and shell variables

I have a text file, materials.txt, with a large number of material definitions similar to this: *FILESTART line_a line_b line_c *MAT_PLASTIC 1 property1 property2 property3 property4 ...
0
votes
0answers
59 views

How to slice array of awk-output in bash?

Summary Depending on what bash or awk version I have, I get different results when I try to slice an array that holds awk output. On older versions of bash and awk the "slice operation" ( e....
0
votes
1answer
25 views

Replacing words with specified words using AWK

My code generates several json file all of which look similar to { "instance": "sample_instance_05", "steps": [ { "129": "North&...
13
votes
5answers
3k views

Add to every end of line in Notepad++

I have a long text file gallery-dl -g -i w4b027.txt > gallery-dl -g -i a4b028.txt > gallery-dl -g -i b4b029.txt > gallery-dl -g -i c4b030.txt > gallery-dl -g -i d4b031.txt > gallery-dl -...
0
votes
1answer
25 views

(awk) writing content between pattern 1 and second occurrence of pattern 2 into separate files

errors=$(mktemp -d) || exit 1 cat "$logFile" | awk -v d=$errors '/ ERROR /, /********************************************************************************/ {if ($0 ~ / ERROR /) {n++} ; ...
1
vote
0answers
44 views

grep and awk are not displaying output in terminal

When trying to grep Record in .txt file simply $ grep 'Record' file.txt $ it goes to the next line without any output or error (I've tried to write output in file, but it is empty) I've tried also to ...
0
votes
1answer
24 views

awk multiple commands and multiple args

I'm trying to turn this output: ❯ hg branches -T'{node} {branch}\n' 4b0826975cb29da26a6dd13f463241463c43c6a7 maria 5c31751cdb774f8b6337b6f64232b15903530997 default Into these commands: hg bookmark -r ...
2
votes
1answer
69 views

Shell special chars blowing up regexp(s) in all: sed, (g)awk, and perl [closed]

I would like to change the PS1 in my ~/.bashrc programmatically with any of the the most popular regexp tools { sed, awk or perl }. However, I have problems with special characters. Note that in the ...
2
votes
2answers
112 views

Filtering a CSV and adding a new tag column in powershell

I have a CSV-file in.txt with 3 columns (Doc_Title, Doc_Date, Doc_URL) and I want to use PS to add another column Doc_Category using another CSV-file tag.txt with 2 columns (keyword, tag). I already ...
0
votes
1answer
25 views

Extract values following specific substrings in string containing spaces using awk

I have this single line string. WordPress version: 5.5.3 Database revision: 48748 TinyMCE version: 4.9100 (49100-20200624) Package language: en_US I would like to extract 5.5.3 and en_US This is the ...
0
votes
3answers
119 views

find duplicate record in Text file

I have text file with this kind of structure mv /XXX/20000/XXX-18245 /XXX/20000/XXX-28042 mv /XXX/10000/XXX-9942 /XXX/10000/XXX-18166 mv /XXX/10000/XXX-9962 /XXX/10000/XXX-18189 mv /XXX/20000/XXX-...
2
votes
2answers
198 views

Remove lines where either column is empty

I have a large colon-delimited text file containing two columns like this: valueA:valueB valueC: valueD:valueE :valueG If there is no data to the left or right of the colon, I want to remove the ...
3
votes
1answer
204 views

Remove spaces from first column of delimited file

I have a very large colon-delimited file with two columns: dhsj fdksjdk:dndkdlkfkd shfdosod dkjsosjd:dkj dosjeod dhswisoaidodisbw: dhsoi wjdos fjfdododjfideefkjso sn:fhfdkd dkdjdf If the first ...
0
votes
0answers
20 views

Reading output of shell command (awk)

I need advice on creating a script using awk, it's that I have a text file where I have a keyword and if it's at the beginning of a line, I move the whole line to the end of the line. example: end ...
0
votes
1answer
31 views

Transpose vertically with a catch

I've been working through some issues preparing data for logstash ingestion. I have a decent chunk of the process worked out but I'm wondering if there's a better way to do this. Basically I need to ...
2
votes
2answers
3k views

Why am I getting this error in my script? awk: script.awk:19: “ syntax error

I have 2 text files 1.txt AA;00000; BB;11111; GG;22222; 2.txt KK;WW;55555;11111; KK;FF;ZZ;11111; KK;RR;YY;11111; I try to generate this 3.txt output: AA;00000; BB;11111;KK;WW;55555;KK;FF;ZZ;KK;RR;YY;...
1
vote
0answers
33 views

merging with format ;string;

I repost my question hoping to provide now a minimal reproducible example Is basically doing a merge of the two files with format NAME;URL;? Not exactly, string is not required to be a URL, it is ...
0
votes
0answers
43 views

When script carries common strings from rows, lines are truncated

I have this script BEGIN { OFS=FS=";" } ARGIND == 1 { map[$2] = ($2 in map ? map[$2] OFS : "") $1 next } { r = $0; for (i=1; i<=NF; ++i) if ($i in map) ...
0
votes
0answers
22 views

Cat awk command additional code implementation

I want to understand how to use awk cat command because I think to do not understand In this page he suggest me to use his code You should process g2.txt first like this: cat join2.awk BEGIN { OFS=...
0
votes
0answers
26 views

Not the same output of small bash-script on different Linux systems

I have a small script, that makes JOIN of LSBLK and DF commands. Script: lsblk -n -b --output KNAME,NAME,SIZE,MOUNTPOINT | egrep -v "fd0|ram|loop|sr0|hdc|cdrom|[SWAP]" | while read kname ...
0
votes
1answer
37 views

Helping with Join of output from DF and LSBLK Linux commands via bash

I already asked this question, but without update. I think this information should be related to this site. I need to merge two outputs in Linux. This: lsblk -n -b --output KNAME,NAME,SIZE,MOUNTPOINT |...
0
votes
1answer
39 views

How would I use sed and/or awk/grep to edit the line following a matched string?

I have a large text file that needs some changes. I need to do this by first locating lines that have a particular common string, and then editing the line directly after that. So for example, if I ...
0
votes
2answers
42 views

How to decrypt octal encoding from zfs diff output?

zfs diff replaces some of the characters in files names with \xxxx (in an octal encoding). Can I decrypt it back with awk or sed or any other common utility (without resorting to a perl or a python ...
0
votes
1answer
15 views

Tail log and output top processes upon encountering string fails

I am running the following command to tail a log file and output top processes to a file upon encountering a given string: tail -f mylogfile.log | awk ' /server on fire/ { system("...
1
vote
1answer
52 views

Putty formatting

When I run the following one-liner, I get run-on output in PuTTY like so: (\ added below to make it easier to read, which is not included in the actual one-liner script) fileName=10M.csv ; \ for k ...
0
votes
0answers
351 views

How to use variables in awk if condition in shell script?

When I tried to cut columns with key words from a csv file, I applied a variable from the for loop of a shell script in the awk if condition. But the if cannot recognize my string. So there's no ...
0
votes
1answer
30 views

grep for same occurrence more than once

Is it possible to grep or awk for more than 1 occurrence of the same string from a file? awk '/$STRING|$STRING/' file grep -e $STRING -e $STRING file I tried other options but still not able to get ...
1
vote
1answer
56 views

AWK/SED/GREP - print M lines after the Nth occurrence of a match

I am looking to display M lines after the Nth occurrence of a match. Actually I need to do this in order to split a file containing Nth <regex> always followed by M lines into Nth different ...
-1
votes
2answers
12 views

How to remove 3 last numbers, seperated by space, in each line of a file - and remove everything exept said three numbers

I need to split a file containing data of this type; Jazz Guitar 1 16 3 4 Carlos Guitar1 16 3 5 Overdrive Gtr1 16 3 6 Western Gtr&Whis 16 3 7 Pat Guitar Synth 16 3 8 Shadow Guitar 16 3 9 Clean El....
0
votes
1answer
32 views

The start of string token ^ is ignored into an awk script?

For learning purpose I made a very simple awk script. It should find the name and the size of some binaries inside /usr/bin. Here's an example of the script : #!/usr/bin/awk -f func round(n) { ...
2
votes
1answer
323 views

Remove all characters after first non-numeric character using sed or awk

TL;DR: How can I create an expression using either sed or awk that will remove all characters after (and including) the first character that is not either numeric (i.e. [0-9]) or a period (.). I've ...
1
vote
2answers
40 views

Missing value when summing in awk

I have two-column files, with first column sorted, such that 0 107 1 1 1 141 2 22 3 1 3 222 4 65 5 1 5 53 6 79 I want an output with no repeated entries in the first column, and as the second column ...
0
votes
2answers
18 views

How to execute bash command on nth flat file column and print it out to output?

I need just basename cmd to be executed on 2'nd column [ora@rac TEST]$ cat test.log | awk '{print $1 " " $2}' 9780c69e13e3dda64c3c6ddce2b37803 /test/test_202007140422_01.zip ...
0
votes
2answers
34 views

How can I change the end of a line depending on what is in next line?

I have a file full of text like: CREATE TABLE `wild_adventure_game` ( `game_instance_id` bigint(20) unsigned NOT NULL, `free_game_config_id` bigint(20) unsigned DEFAULT NULL, `respin_instance_id`...
1
vote
2answers
67 views

pattern match special charecters and display contents between the pattern

My apache log format is as below 192.168.10.1 <"www.mytest.com"> <-"-"-> <--“192.168.10.90"--> [2020-07-10 03:35:46 PDT] >1008< 2470 3006 "https&...
2
votes
2answers
29 views

Combine strings with same first word preserving their order

I have a file that contain many lines consecutive and non consecutive with same first word and i want to merge all those lines.Not finding any way. please suggest someone. eg. cat file X blah blah ...
1
vote
1answer
47 views

recursively sort files with string “SENT_” somewhere in filename by the substring immediately after SENT_, and then display them. Linux

So I wish to run this from any directory, say ~/ and have it work for subdirectories. Example would be; ls ~/: /home/me/FILES/DIR_1/DIR_a/SENT_2222_.... /home/me/FILES/DIR2/SENT_3333.... /home/me/...
2
votes
3answers
47 views

AWK is printing all columns

I want to isolate the second column. If I do: echo "1 2 3" | awk "{ print $2 }" the result is: 1 2 3 The expected result is: 2 I've tried with -F' ' option, but the result is ...
0
votes
3answers
188 views

Bash case statement using awk commands in a for loop not working

I have the following bash script: #!/bin/bash OUTPUT=NULL,2,3,NULL,5,6,7,8 pvar1=$(echo $OUTPUT | awk -F ',' '{print $1}') pvar2=$(echo $OUTPUT | awk -F ',' '{print $2}') pvar3=$(echo $OUTPUT | awk -...
0
votes
2answers
34 views

How to change third column of a variable so it is encased with single quotation marks using AWK or SED

I have a variable $out: out=1,2,3,4,5 I want to change the third column so the value gets single quotation marks as in (I talk of columns here as $out represents the values of an insert statement for ...
1
vote
2answers
2k views

How to replace a column value in a variable with a variable using awk

I have two variables: var=x out=1,2,3,4,5 I want to replace the third column in $out with $var. I am able to substitute x for 3 in column 3 with the following command (I speak of columns in $out as ...

1
2 3 4 5
12