Tagged Questions
3
votes
1answer
48 views
Using grep to remove lines from a file which contain a string from another file
I have a file containing words (one per line) such as
Dog Fish Cat Shoes
I have a secondary file in CSV format such as
1,shoes,red 2,apple,black 3,fog,blue
I would like to use grep with ...
0
votes
1answer
17 views
Write output to $TMPDIR/$variable : file does not exist (should be created)
I have the following .sh script in which I am trying to do all major writing in $TMPDIR.
However, when it gets to line 86, it returns the following error:
...
-1
votes
2answers
51 views
Replace all numbers following a specific string with another number using sed or awk
I have tried several things but got no luck. I have an input file that has a lot of fields like this one
weight=0.123456
mixed with other fields. I want to multiply all such weights by a factor, say ...
1
vote
1answer
92 views
awk - max value of a rolling average
I have a bash command which gives me the highest value in the 9th column of a file:
cat "log.txt" | grep 1923 | awk '{print $9}' | sort -n | tail -1
log.txt is a log of CPU usage for process 1923 ...
0
votes
3answers
64 views
output the items that showed up 2 times and more
How can I select the ips that showed up 4 times or more , for example here?
the input file is
192.168.1.28
192.168.1.100
192.168.1.31
192.168.101.2
192.168.1.31
192.168.11.02
192.168.1.31
...
1
vote
6answers
143 views
match string in awk
How can I search the file to find the lines that have SRC= , for example here?
i mean how can i find source IP address in this file using awk for example
Mar 10 03:17:12 ubuntu kernel: ...
1
vote
2answers
91 views
using awk with parallel
I have about 3,000 files that are each 300MB, and I'd like to search them for a series of substrings as quickly as possible with my 16 core server.
This is what I tried but it doesnt seem to ...
0
votes
1answer
69 views
Bash script is printing wrong characters in my variable
I'm trying out the following thing in my terminal:
spark 0 76 100 | awk '{print substr($0,4,3)}'
outputs:
▆
If I do:
GRAPH=$(spark 0 25 100 | awk '{print substr($0,4,3)}')
printf "%s" $GRAPH
...
2
votes
3answers
186 views
How can cat/print a file except the last 2 lines?
How can get a file, except the last (for instance) 2 lines, with standard or GNU shell tools? (bash, tail, cat and so on.)
0
votes
2answers
125 views
creating CDF data using bash or awk or perl
we have some data like:
12 0
13 0
20 0
25 1
64 4
77 1
89 100
1201 204
I'd like to get this output:
3 0
5 1
6 4
7 100
8 204
explanation: we have 3 AS(autonomous systems) that have degree of ...
1
vote
1answer
125 views
Counting occurrences in first column of a file
We have this file:
1 2
1 3
1 2
3 3
52 1
52 300
and 1000 more.
I want to count the number of times each value occurs in the first column.
1 3
3 1
52 2
This means we saw 1 three times.
How ...
4
votes
3answers
83 views
Output numbers from a file in a different order
I have a file that contains:
1 2 3 4
1 3 5 4 8
3 2 1
Each line has a different number of digits. And there are 1000 more like this.
I want the output like this:
1 2
2 3
3 4
4 0
1 3
3 5
5 4
4 8
8 ...
2
votes
4answers
165 views
How do you remove all occurrences of values in one list from another list?
I have a list of symbols such as...
wer
sfe
efo
How do I remove all instances of these (unique) symbols from another list of (non-unique) symbols?
So in the following list, the lines starting ...
1
vote
2answers
266 views
How to extract e-mail or domain from mixed data file in linux
File content:
17541 From Email subscription@test.com Inbound
Policy Manager Envelope Analysis
Profiler
17541 From Email subscription@yahoo.com Inbound
Policy Manager ...
1
vote
5answers
772 views
How to grep column after second pipe
Example:
3|100|test@test.com|0|0|6:1,10,11,12,13,2,3,4,5,6,9|7:1,10,11,13,16,2,4,5,6,9|
Expected view after grep:
test@test.com
1
vote
2answers
86 views
Using sh -c in terminal
I have this command line that I enter into terminal and it works as intended:
dscl . -readall /Users UniqueID | awk '/^RecordName:/ {name=$2}; /^UniqueID: / {if ($2 > 500) print name}'
What I ...
2
votes
1answer
69 views
Substitute large numbers of files according to style guide
I'd like to do a wholesale reformatting of our tests, and I'm cleaning up some inconsistent capitalization. I'm thinking of using awk to do this, since sed falls a little short, and since I need ...
1
vote
2answers
148 views
Parsing .co.uk whois with awk
I wonder if someone could tell me why these awks are not working, they should supply the registrar and the expiry date :
${AWK} -F: '/Registrar:/ && $0 != "" { getline; ...
0
votes
4answers
188 views
Bash script to get specific prior context
I am searching through some log files where there are groups of actions performed. At the start of each group there is a line that has information about the group and then lots of verbose informion ...
0
votes
1answer
225 views
IP Address dotted decimal to /8 or /16 notation using bash,sed or awk?
I have an input file that contains a list of ip addresses and the ip_counts(some parameter that I use internally.)The file looks somewhat like this.
202.124.127.26 2135869
202.124.127.25 2111217
...
3
votes
5answers
3k views
How to get the pid of a running process using a single command that parse the output of ps?
I am looking for a single line that does return the pid of a running process.
Currently I have:
ps -A -o pid,cmd|grep xxx|head -n 1
And this returns the fist pid, command. I need only the first ...
0
votes
1answer
598 views
Parsing out specific URLs from local html file
I want to parse out certain URLs from a saved html-file with a simple bash-script/command, so i can download them via wget laters.
What i have so far is:
awk 'BEGIN{ RS="<a *href *= *\""} NR>2 ...
1
vote
1answer
533 views
Output high CPU processes using bash shell
I am trying to monitor my CentOS processes using top command to see if there are any processes where CPU usage is greater than X%, below is the command I am using to see if any processes CPU usage is ...
1
vote
5answers
1k views
Pipe output of awk to kill -9
I'm trying to kill any process that has to do with mysql. I'm piecing together a command, and so far, haven't come up with the right solution. One other consideration, is that the last process will ...
3
votes
4answers
1k views
How to remove lines from large text file using bash
I got a huge text file (log file) in my CentOS which I would like to remove top part of, probably couple of thousand lines each day. (Or probably just split into two)
I have search this site and ...
4
votes
3answers
459 views
how to limit find command's output used with option -print0
I'd like to limit output of find command. In the past I used to use for this ls command, e.g:
ls *tgz|head -100|xargs -i mv "{}" ../
but I got to know that the result may be unpredictable if name ...
2
votes
1answer
161 views
awk: equivalent of mysql's “\G” to print tabulated data in multiple lines
Is there an equivalent to mysql's "\G" at the end of the query but awk (or similar) that will to print tabulated data from a text file in multiple lines with the header next to it? Something that will ...
2
votes
2answers
232 views
Awk responds differently based on how an empty argument is specified
I seem to have stumbled on something which is probably a bug in awk, but it could also be a bug in my understanding of bash/awk.
I was trying to debug issues where the output of a python program was ...
0
votes
1answer
278 views
How to pass bash variables (passwords) to IRSSI config file?
I'm using IRSSI and I wonder how can I pass my IRC channel's passwords to the IRSSI config file (~/.irssi/config) without writing them directly in the file.
In Mutt I've an awk command that queries a ...
0
votes
1answer
430 views
“find | grep | awk” fails miserabily
I'm scanning for all shell scripts on my server, and are doing this by the following command:
find / -type f -exec file --mime-type {} \; | grep "text/x-shellscript"
This is working fine, and here ...
2
votes
2answers
493 views
Join two lines with awk or sed
If I have:
foo
bar
..and I'd like to awk/sed this to:
foo-bar
..what's the syntax?
I'm trying to use System Profiler on OS X 10.6.x:
system_profiler SPMemoryDataType | awk '/Type/ {print $2} ...
1
vote
2answers
511 views
Shell: Find and replace word
I have a string in my shell script which is in a fixed format : '[STATUS REPORT] PROJECT'.
When user executes my shell script he will be asked to provide a value for 'PROJECT'.
I would like to ...
1
vote
2answers
312 views
Sort multiple files with bash
I have a question that involves the bash scripting language.
I have multiple directories
/studentName
/studentMail
/studentNumber
In each of these directories is a file name.txt, mail.txt, ...
4
votes
1answer
416 views
how to move around files using “ls -al” result in Linux?
How can I write a script to copy files from one directory to another directory according to last modified date?
ls -al
-rw-r--r-- 1 user user 100 2011-05-26 12:33 ...
8
votes
3answers
4k views
Filtering top command output
I'm running a Mac and want to filter the output of the top command for a particular process such that only the PID, COMMAND and %CPU columns are displayed.
After getting the PID of the process, I ...
0
votes
1answer
2k views
How to construct a grep command with a variable argument in bash?
I'm trying to do something like this in bash:
grep ( date | awk '{print "2006-" $6}' ) /some/file/here
But that syntax is incorrect.
The goal is to grep /some/file/here for the pattern 2006-2011 ...
2
votes
2answers
261 views
Rename sets of files based on size
Background
Rename one set of files based on a name that corresponds to another set, using a sort order based on file size to match the file names. The files from both sets have approximately the same ...
0
votes
4answers
572 views
using awk to make exact matches
i'm just wondering how can we use awk to do exact matches.
for eg
$ cal 09 09 2009
September 2009
Su Mo Tu We Th Fr Sa
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 ...
2
votes
3answers
270 views
Copied directory appears to become larger at destination
I have the following code as part of a shell script:
while [ $(ps -ef | awk '{print $2}' | grep -F "$CPPID") ]; do
sleep 10
awk -v "usbsize=$(/bin/df | grep -F $DEVICEMOUNTPOINTQ | awk ...
2
votes
2answers
175 views
change the number format
Hello
I have a lot of lines like below:
123;XXXXXXXXXXXXXXX;ABCDE;YYYYYYYY;08082010;000000000000000;03/08/10;110000;ZZZZZZZZZZZZZZZZZ;0002
I just want to change the number format (15 digit) ...
0
votes
2answers
233 views
awk + verify field if zero
hi all
To find if field 5 has I use the:
[[ -z ` echo $LINE | awk '{print $5}' ` ]]
my question if there is another elegant way to verify if field 5 is zero?
THX
Yael
0
votes
1answer
674 views
Solaris bash script / search and append field to end of line in /etc/group
I need a bash script that will modify /etc/group to append and delete NIS users to specific local groups on a Solaris 10u8 system. Preferable one or two functions with uid and groupname as varibles.
...
4
votes
2answers
2k views
SQL like group by and sum for text files in command line?
I have huge text files with two fields, the first is a string the second is an integer. The files are sorted by the first field. What I'd like to get in the output is one line per unique string and ...
0
votes
3answers
1k views
how to use grep, sed, and awk to parse tags?
I want to write a script that finds a open/close tag pair in a text file and prepends a fixed string to each line between the pair. I figure I use grep to find the tag line numbers and either awk or ...
3
votes
5answers
1k views
error when using commandline as a bash alias on linux
I want to save the following commandline sequence as a bash alias:
grep `date '+%d/%b/%Y'` access.logs | egrep 2765330645ae47d292c9ceac725d744e.py |awk '{print $1, $4, $5, $7, $8, $9, $10}' | sort ...
