AWK is a text-processing language. It is mainly used to manipulate and process structured data, and to generate reports.

learn more… | top users | synonyms

0
votes
2answers
37 views

Split one file into multiple files

I have one file which is in Json format like below: { "sources":[{ "field1":1000, "field2":"winevent_log", "field3":"winevent_log", "field4":"os_security", ...
0
votes
2answers
30 views

Disk usage to show only select partitions?

I'd like to be able to only show disk usage for specific partitions in GB and a % after. Basically ignore swap, tmpfs, etc.. I'd also like to sum the disk usage amount and subtract from the total to ...
0
votes
2answers
40 views

How to collect specific information in a file

I have the need for different hardware information. such as serial numbers, capabilities, among many others, and for that, I found the hwinfo, but it gets information far beyond what I need. For ...
1
vote
3answers
31 views

Using a shell script to add a ‘</pre>’ after the last line in a large file

I need a SED, or AWK, command to find the last line in a file, and add </pre> to it. Such as: LINELINELINE Changes to: LINELINELINE</pre>
0
votes
0answers
38 views

AWK - system command

I am trying do a linux command in AWK: $0 ~ /^([[:alnum:]]+\.)+[[:alnum:]]+;version=.+$/{ split($0,path,";") gsub(/\./,"/",path[1]); split(path[2],ver,"=") pver=path[1]...
0
votes
1answer
53 views

How to extract specific data from the table?

Here is the table... Group Name Designation 2 (John) Front End Developer 12 (Jim) Back End Developer 8 (Jill) Full Stack Developer 21 (...
1
vote
1answer
43 views

Linux Shell script to change grep for a string, then comment out 2 lines above and 1 below the string found in the file

Looking for some help on this task. I have tried several shell scripts to do this, but not sure I am approaching this problem right. I have a file that contains several blocks of data like this: ...
1
vote
3answers
39 views

Display file contents if it has corresponding line number

I have two files in the following format: File A 4 5 8 9 File B 1 text 2 text1 3 text2 4 text3 5 text4 6 text5 7 text6 8 text7 9 text8 What I want to have for my output is just the following (the ...
-2
votes
1answer
15 views

Removing a substring between track number and title

I need help with a little bit of bash trickery. In organizing my music collection I've come across a set of tracks which are named in the format %track%. %artist% - %title%.%extension% How can I ...
0
votes
3answers
46 views

Combine multiple lines in a file to a single line

I have a file as below.. "Field1"|"Field2"|"Field3"|"ufghjkrtyrtyfgh$" "Field1"|"Field2 continue on line 2 continue on line 3"|"Field3"|"ufghjkrtyrtyfgh$" "Field1"|"Field2"|"Field3"|"...
0
votes
2answers
18 views

How to change a character on a certain line without delimiters with awk

I have a file, and I would like to change a single character on lines 400 through 600. The character happens to be the 22nd character on each line. It is also the 5th column, so I tried the following: ...
0
votes
1answer
20 views

AWK - print file and length in two lines in END

I have to do script in AWK, which will choose the longer field in the line and after that in the end show in two line filed and under(in second line) length. I made this: ...
0
votes
1answer
56 views

arithmetic operation on a nested column in a tab delimited file

I have a tab-delimited text file containing Column1...Columnn columns and R1 to Rn rows. Within some of the columns, there are multiple nested fields with an identifier and then separated by semicolon ...
3
votes
5answers
423 views

Extract lines based on long list of line numbers

I have a long list of line numbers (35389208) that I do not want in my file. By line number I mean the line in my file (e.g. line 277). My list of line numbers I do not want looks like: 277 278 279 ...
0
votes
1answer
56 views

Watch with awk command

How do you watch a command like? awk 'NR%2==0 {printf "%s %8.0f", $1, $5}' filename.txt Preceding this with "watch" gives this error: awk: cmd. line:1: fatal: cannot open file `{printf' for ...
0
votes
1answer
29 views

Pattern search the user input in a file in Unix using awk

I am trying to write awk script to search for a pattern that is given input by user, in a file. My code look like this: awk 'BEGIN{printf "Enter : ";getline input<"/dev/tty"} /'"$input"'/ {print}' &...
0
votes
3answers
1k views

Executing multiple awk commands in one file

I am trying to extract lines from a file and save the output to a new file. I have been using the code: cat datafile | awk -f pullhh.txt > fuel where the file pullhh.txt contains the code /...
2
votes
3answers
717 views

using nawk, how to sum all numbers in $1, $2, and $3 fields?

Using nawk, how to sum all numbers in $1, $2, and $3 fields? 8 6 2 7 3 5 1 9 4 I want the result to be sum = 45
16
votes
7answers
8k views

sed: how to replace line if found or append to end of file if not found?

With a single input file that only contains comments (starting with #) and VARIABLE=value lines, is it possible to replace a value for a single variable if found and, otherwise, append the pair to the ...
3
votes
5answers
4k views

How to remove lines from large text file using bash

I got a huge text file (log file) in my CentOS which I would like to remove top part of, probably couple of thousand lines each day. (Or probably just split into two) I have search this site and ...
0
votes
2answers
79 views

Fetch required column from a file with \001 delimiter

I am storing output of pig file in a directory Which contains lines with multiple columns separated by delimiter \\u0001. Now in shell script I want to fetch 7th column field from the output. I ...
-1
votes
1answer
21 views

How can I filter unique lines based on a substring?

If I have a set of URLs in a file like: http://example.com/do?foo=bar&etc=123 http://example.com/do?etc=456&foo=bar I want to filter this out so that there's only one url for each unique ...
0
votes
2answers
28 views

Insert a string or blank line after specific search criteria, in a loop

I'm wondering if someone could help me with a specific coding question. I have a DNA sequencing file that reads something like this (as an example): ...
1
vote
2answers
465 views

printing only first finded match

I've created a statement which is searching for specific informations in each line (in my case "AAA","BBB" or "CCC"), if there is no such information N/A is printed { k=0; for (i=1;i<=NF;i++) ...
0
votes
0answers
36 views

Counting keys in line awk

I have to check how many keys(3 last characters in word) is in the all lines, but awk sent error. I think awk don't correct check condition. I tested it changing key to "t(or somthing eles). My script:...
1
vote
1answer
39 views

Remove a certain part in a text file

I got the file, text.txt. In this file I have written: "test" "okay" I want to remove the " part of this file, how'd I do this?
0
votes
1answer
138 views

XML text transformation

I have a large file part of which is like below: <DataGroup xsi:type="ReportDataGroup"> <SmartReportTemplate DescriptionContentType="text/plain" IsActive="true"> <Name ns1:translate="...
0
votes
2answers
35 views

new column to a tab delimited file, with as predefined value

How can I add a new column to a tab delimited file, at the end of the file, containing the same values for each row. For example: infile.txt Name Age Address Sex X 12 adrs1 M Y 15 adrs2 ...
0
votes
3answers
369 views

Text deletion between patterns across multiple lines with respect to text inside pattern

I have a block of text I need to delete, however, only if it contains specific text inside the block: ... <script language="JavaScript"> var somethingA = 0; var somethingB = 0; var ...
1
vote
2answers
392 views

How to replace a line in an xml file with a set of lines stored in a variable or file using unix commands?

I have a xml file in which the line that needs to be searched is: SEARCH='<?xml version="1.0" encoding="UTF-8" standalone="no"?><SSC>' This searched value needs to be replaced with ...
0
votes
1answer
362 views

Reset awk field separator

On my local system, it appears that the configuration for awk has somehow been changed. Running the following command: echo "Hi there" | awk '{print $2}' On my machine this prints: Hi there And ...
0
votes
1answer
53 views

several awk commands consecutively

I'm using awk command to extract some datas. The point is that I need to use awk command several times to extract all informations I need and each time when I'm using awk I need create a temp file. My ...
-1
votes
1answer
35 views

Oneliner to get the list of filenames which have last line (or line on position X) matching certain pattern?

How to get the list of filenames which have last line (or line on position X) matching certain pattern? We are looking for oneliner.
0
votes
1answer
387 views

Awk script to run a command based on condition using 'if'

I have a piece of code here that basically takes a file called 'srcip' (which is just a list of IP addresses) and compares it to my '~/db' file which is just a tab delimited IP HOSTNAME database of ...
1
vote
1answer
30 views

chunking screen output using grep/head/awk/sed/etc

I have some output that prints an element every line. I want to be able to parse that output in chunks of N, even if the number of lines is not a multiple of N. find . -f | chunk 10 1 # shows first ...
3
votes
4answers
24k views

Print line X lines before/after found line

I want to search for a particular string in a certain file. If I found the string, I also want to print the line X lines before (or after) that line. Can this with grep or awk, or do I need a ...
14
votes
6answers
30k views

How to get the pid of a running process using a single command that parse the output of ps?

I am looking for a single line that does return the pid of a running process. Currently I have: ps -A -o pid,cmd|grep xxx|head -n 1 And this returns the fist pid, command. I need only the first ...
1
vote
1answer
45 views

Increase efficiency of bash loop with awk

I have to write a fairly large text file(>300,000 lines) using text extracted from the second column of one file, then prepending and appending text to each line and finally writing it out to a new ...
0
votes
1answer
23 views

How to Check value for an input element after fetching contents of page using cUrl

I am trying to download a page using cUrl. I have saved the page to a file and I then grep on it to extract the value of the text field. The text field appears multiple times and I just need one value....
17
votes
6answers
12k views

Removing ANSI color codes from text stream

Examining the output from perl -e 'use Term::ANSIColor; print color "white"; print "ABC\n"; print color "reset";' in a text editor (e.g., vi) shows the following: ^[[37mABC ^[[0m How would one ...
5
votes
3answers
786 views

Search for files with more than one term (grep, awk?)

I am using a command like this to find files with the word 'term' in them: grep -l term * But I now want to be able to find files which have two different words in them (let's call them termA and ...
3
votes
1answer
38 views

file splitting and compression in pipeline

So I have a massive file something like this... 1/20/2016,somerandomdata 1/20/2016,somerandomdata 1/20/2016,somerandomdata 1/20/2016,somerandomdata 1/21/2016,somerandomdata 1/21/2016,somerandomdata 1/...
1
vote
1answer
50 views

How to search for a line which contains specific string in Bash?

I've got a file which contains some amount of lines, I want to look for a specific string, and if the file contains it, then I want to print a different line from the same file. for example, let's ...
0
votes
2answers
171 views

Rename CSV header using command line tools

Given a CSV file, h1 h2 h3 a b c z x b how to rename the first header to One and the third header to Three using command line tools, namely One h2 Three a b c z x b
2
votes
2answers
321 views

How to convert a column of hex to dec in gawk, strtonum in gawk gives erroneous results

I am trying to script a database from a network stream. The network stream after being heavily sed'd drops a three column file that looks something like this called file.db 123.123.123.123,...
0
votes
1answer
32 views

Transform CSV column by a function using command line tools

Given a csv file such as h1 h2 a 0 b 1 c 0 how to rename 0 to YES and 1 to NO in the second column.
2
votes
2answers
479 views

How can I sort and de-dupe a very long list of words?

I have a 400 megabyte file. The file is a list of words, each word on a single line. End of lines are a mix of cr/lf and lf. Some of the words are unicode. I'd prefer answers for Linux or Windows ...
2
votes
2answers
50 views

compare and calculate two variables with multiple values in shell script

I have written three hive queries in shell script for calculating debit-credit and target amounts which output below values: debit_amount : BOI 4760545.650000 AXIS 284.49000000 SBI 87.220000000 ...
0
votes
1answer
45 views

awk - field separator change in program, letters in the field

I want to change FS in awk, but not in BEGIN. I want to print word and the letter. awk 'BEGIN{FS=""}{print $1; FS=" ";print $1; FS=""}' Of course it does't work. Is there any function, way to show ...
0
votes
1answer
50 views

awk - Assign variable result of system function

I would like to assign variable result of system function: awk 'BEGIN{x=system("cut -d \" \" -f 1 ol");print x}' Input "ol": 1 Result: 1 0