Tag Info

Hot answers tagged

13

awk - the most common and will be found on most Unix-like systems, oldest version and inferior to newer ones. mawk - fast AWK implementation which it's code base is based on a byte-code interpreter. nawk - while the AWK language was being developed the authors released a new version (hence the n - new awk) to avoid confusion. Think of it like the Python ...


13

Using sed: perl -e 'use Term::ANSIColor; print color "white"; print "ABC\n"; print color "reset";' | sed 's/\x1b\[[0-9;]*m//g' \x1b is the escape special character \[ is the second character of color statement [0-9]* is the color value m is the last character of color statement sed is enough, but you can still use perl instead of sed: perl -e 'use ...


12

Variable interpolation is performed within double quotes, so here's what I think might be happening: when you type in ssh $HOST "ls -l | awk '{print $1}'", your shell (the one on your local computer, where you are running the SSH client) sees $1 within the double quotes and replaces it with the value of the variable $1, which will be blank. It isn't able to ...


10

If you just want the pid of the process you can make use of pgrep if available. pgrep <command> will return the pid of the command (or list of pids in case there are more than one instance of the command running, in which case you can make use of head or other appropriate commands) Hope this helps!


9

Use the head command from coreutils: head -n -2 See info head for more.


9

Figured it out, but please feel free to add better answers cmd = "dig +short -x " ; cmd ip_address | getline hostname; close(cmd) Then I can use the hostname anywhere in the script.


9

IMO the best way to do this would be to use the programming/scripting language you know best and: load small.txt into an in-memory hash/map/associative array keyed on the words Process huge.txt line by line, adding the column looked up from the hash and writing the result into an output file Buffer input and output so that it happens in chunks of at least ...


8

You can do this using grep's -f option (that's lower-case -f, not -F): % echo -e 'Dog\nFish\nCat\nShoes' > ./file1.txt % echo -e '1,shoes,red\n2,apple,black\n3,fog,blue' > ./file2.csv # Grab all lines from the CSV that match a pattern from file1: % grep -if ./file1.txt ./file2.csv 1,shoes,red # Grab all lines from the CSV that DON'T match a ...


7

This can probably be shortened. It's not a single sed command and it also uses grep, but this seems to be basically what you're wanting. It's a single line, and it edits the file in-place (no temp files). grep -q "^FOOBAR=" file && sed "s/^FOOBAR=.*/FOOBAR=newvalue/" -i file || sed "$ a\FOOBAR=newvalue" -i file


7

Yes, try doing this, and pick your preferred method =) : With grep: echo "ixi" | grep -oP "^.\K." With cut: echo "ixi" | cut -c2 With bash parameter expansion : x='ixi'; echo ${x:1:1} With sed: echo "ixi" | sed 's/.\(.\)./\1/' or echo "ixi" | sed 's/\(^.\|.$\)//g' With perl: echo "ixi" | perl -lne 'print $& if /^.\K./' With ruby: ...


7

To build on Michael Borgwardt's answer: as long as both files are sorted, you can put them together by basically performing one step of a mergesort. It'll be a little different than standard mergesort because you only want to keep one of the files. This will, of course, have to be implemented in your favorite programming language. Here's a sketch of the ...


7

The Markdown in Python implementation has support for extensions one of which includes Table of Contents generation. Additionally Pandoc (which is a Haskell markup->PDF has support for markdown (in addition to a bunch of other formats) and can output pretty HTML, LaTeX, PDFs, etc.


7

I've just invested some time to try this and came up with the following script: #!/bin/bash # set folder where files are located SOURCE_FOLDER=/path/to/source # define folder to which the files have to be copied TARGET_FOLDER=/home/abcd # #### cd "${SOURCE_FOLDER}" for FILE in *; do # everything which is not a normal file if [ ! -f "${FILE}" ]; ...


6

For basic tasks like this, you can use for /f. Instead of command | awk '{ print $4 }' try this: for /f "tokens=4" %a in ('command') do echo %a Note that you have to use %%a instead of %a in batch files.


5

nawk 'BEGIN { sum=0; } {sum += $1 + $2 + $3;} END { printf "%d\n", sum } ' a.in


5

using awk: awk '{x+=$0;}END{print x}' RS="[ \n]" file This will work irrespective of the number of rows or columns. By using the record separator(RS) as space or a newline, every value is split into a separate line, and hence can be easily added.


5

To dedupe, you need to sort -u – it will only output unique lines. This is an option specified by POSIX, so you'll probably find it on any system. sort -u wordlist.txt | awk 'length($0) > 7' Use double quotes for awk if you're on Windows. Note that the sorting step is not optional, as uniq requires duplicate lines to be adjacent in order to remove ...


5

grep will do this for you, with options -A (after) and -B (before). An example I often use is: sudo lspci -vnn | grep -i net -A 12 because this command will show 12 lines after the match, which includes the driver used to control the network (-i net) card. In general, the command is: grep tex_to_search -A n -B m file.extension which will output m ...


5

grep ",abc" file.csv Sounds enough? Or to be sure it finishes with abc: grep ",abc$" file.csv (in this case the last one in your example won't match due to the full stop at the end)


5

awk is not in /bin, it's in /usr/bin. To find out where you have awk, you can run either of the following: which -a awk type awk Then, change your shebang line accordingly: #!/usr/bin/awk … or, even better: #!/usr/bin/env awk The latter will just use the version of awk for the current environment, and is portable across different systems that have ...


5

What is displayed as ^[ is not ^ and [; it is the ASCII ESC character, produced by Esc or Ctrl[ (the ^ notation means the Ctrl key). ESC is 0x1B hexadecimal or 033 octal, so you have to use \x1B or \033 in your regexes: perl -pe 's/\033\[37m//g; s/\033[0m//g' perl -pe 's/\033\[\d*(;\d*)*m//g'


5

sed --in-place $filter $file


5

Probably the easiest thing to do here is to make grep not match its own command: ps -A | grep [m]ysql | awk '{print $1}' | xargs kill -9 The brackets around the m make it a character set that only includes m, which doesn't change the pattern but won't match itself in the ps output. This is sort of the canonical way to avoid this issue. I'd also like to ...


5

I'm a little green on Linux, so I'm unsure exactly why it doesn't work. But you can use a function instead. In .profile/.bashrc create a new function: downloads() { grep `date '+%d/%b/%Y'` access.logs | egrep 2765330645ae47d292c9ceac725d744e.py |awk '{print $1, $4, $5, $7, $8, $9, $10}' | sort |uniq -c -w15 |sort -n; } That works exactly as an alias.


5

You can just use awk. It is defined by POSIX and therefore has to exist on all POSIX-conformant systems. The -F parameter is mandated by that as well.


5

Frankly, by the time a command gets that big, I'd make it into a script and not an alias. One advantage of a script is that you make it work with more files than just 'access.logs'. That command sequence involves both single quotes and back-quotes - that always adds to the fun. Generally, you are better off using $(command args) in place of back-quotes. ...


5

I don't know what OS you're on, but on FreeBSD 7.0+ grep has a -o option to return only the part that matches the pattern. So you could grep "marker-1234" filter_log | grep -oE "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" Returns a list of just IP addresses from the 'filter_log"... This works on my system, but again, I don't know what your version of ...


5

top -pid 3907 -stats "pid,command,cpu" -pid 3907: your process ID -stats pid,command,cpu: only show process ID, name and CPU% No need to run awk on the output. If you want to post-process the output, use -l 0 to run in logging mode (0 means indefinitely, everything else limits number of samples). Output will look like this (two repetitions): ...


5

Try this: awk -F , '{ printf "%i,%08i,%s\n" , $1 , $2 , $3 }' file



Only top voted, non community-wiki answers of a minimum length are eligible