Hot answers tagged

24

Using sed: perl -e 'use Term::ANSIColor; print color "white"; print "ABC\n"; print color "reset";' | sed 's/\x1b\[[0-9;]*m//g' \x1b is the escape special character \[ is the second character of color statement [0-9]* is the color value m is the last character of color statement sed is enough, but you can still use perl instead of sed: perl -e 'use ...


19

awk - the most common and will be found on most Unix-like systems, oldest version and inferior to newer ones. mawk - fast AWK implementation which it's code base is based on a byte-code interpreter. nawk - while the AWK language was being developed the authors released a new version (hence the n - new awk) to avoid confusion. Think of it like the Python 3....


16

If you just want the pid of the process you can make use of pgrep if available. pgrep <command> will return the pid of the command (or list of pids in case there are more than one instance of the command running, in which case you can make use of head or other appropriate commands) Hope this helps!


16

If you have ffmpeg, you should also have ffprobe: ffprobe -i input.file -show_format | grep duration ffprobe -i input.file -show_format -v quiet | sed -n 's/duration=//p' This will also give fractions of seconds, if that's a problem you can further process that away with sed.


15

Just use ffprobe directly. No need for sed, grep, etc. There are several "durations" you can acquire (depending on your input). Format (container) duration $ ffprobe -v error -show_entries format=duration \ -of default=noprint_wrappers=1:nokey=1 input.mp4 30.024000 Adding the -sexagesimal option will use the HOURS:MM:SS.MICROSECONDS time unit format: ...


14

Variable interpolation is performed within double quotes, so here's what I think might be happening: when you type in ssh $HOST "ls -l | awk '{print $1}'", your shell (the one on your local computer, where you are running the SSH client) sees $1 within the double quotes and replaces it with the value of the variable $1, which will be blank. It isn't able to ...


13

This can probably be shortened. It's not a single sed command and it also uses grep, but this seems to be basically what you're wanting. It's a single line, and it edits the file in-place (no temp files). grep -q "^FOOBAR=" file && sed "s/^FOOBAR=.*/FOOBAR=newvalue/" -i file || sed "$ a\FOOBAR=newvalue" -i file


11

Probably the easiest thing to do here is to make grep not match its own command: ps -A | grep [m]ysql | awk '{print $1}' | xargs kill -9 The brackets around the m make it a character set that only includes m, which doesn't change the pattern but won't match itself in the ps output. This is sort of the canonical way to avoid this issue. I'd also like to ...


11

To get minutes, you have to divide 2383 seconds by 60. 39.7167 and then multiply the fractional part .7167 by 60 to get the remaining seconds. 43.002 So it's 39 minutes, 43 seconds. The application appears to be giving you an accurate value.


10

Use the head command from coreutils: head -n -2 See info head for more.


9

The Markdown in Python implementation has support for extensions one of which includes Table of Contents generation. Additionally Pandoc (which is a Haskell markup->PDF has support for markdown (in addition to a bunch of other formats) and can output pretty HTML, LaTeX, PDFs, etc.


9

IMO the best way to do this would be to use the programming/scripting language you know best and: load small.txt into an in-memory hash/map/associative array keyed on the words Process huge.txt line by line, adding the column looked up from the hash and writing the result into an output file Buffer input and output so that it happens in chunks of at least ...


9

Figured it out, but please feel free to add better answers cmd = "dig +short -x " ; cmd ip_address | getline hostname; close(cmd) Then I can use the hostname anywhere in the script.


9

To insert a single quote character in a single-quoted string, end the current string, write "'" or \' and begin the string again. In your example, that's awk 'BEGIN{FS="'"'"'"}{print $2}' or awk 'BEGIN{FS="'\''"}{print $2}' However, using the -F switch to specify the field separator will result in more legible code: awk -F\' '{print $2}'


9

You can do this using grep's -f option (that's lower-case -f, not -F): % echo -e 'Dog\nFish\nCat\nShoes' > ./file1.txt % echo -e '1,shoes,red\n2,apple,black\n3,fog,blue' > ./file2.csv # Grab all lines from the CSV that match a pattern from file1: % grep -if ./file1.txt ./file2.csv 1,shoes,red # Grab all lines from the CSV that DON'T match a ...


9

grep will do this for you, with options -A (after) and -B (before). An example I often use is: sudo lspci -vnn | grep -i net -A 12 because this command will show 12 lines after the match, which includes the driver used to control the network (-i net) card. In general, the command is: grep text_to_search -A n -B m file.extension which will output m ...


8

I have found out a better escape sequence remover. Check this: perl -pe 's/\x1b\[[0-9;]*[mG]//g'


8

Yes, try doing this, and pick your preferred method =) : With grep: echo "ixi" | grep -oP "^.\K." With cut: echo "ixi" | cut -c2 With bash parameter expansion : x='ixi'; echo ${x:1:1} With sed: echo "ixi" | sed 's/.\(.\)./\1/' or echo "ixi" | sed 's/\(^.\|.$\)//g' With perl: echo "ixi" | perl -lne 'print $& if /^.\K./' With ruby: ...


8

If the input is sorted, you can use uniq: <infile cut -d' ' -f1 | uniq -c If not, sort it first: <infile cut -d' ' -f1 | sort -n | uniq -c Output: 3 1 1 3 2 52 The output is swapped compared to your requirement, you can use awk '{ print $2, $1 }' to change that. 1 3 3 1 52 2 There's also the awk ...


8

It's actually quite simple with sed: if a line matches just copy it to the hold space then substitute the value. On the la$t line exchange hold space and pattern space then check if the latter is empty. If it's not empty, it means the substitution was already made so nothing to do. If it's empty, that means no match was found so replace the pattern space ...


7

You can add a number at the end of the substitute command. For example, the following will substitute the second occurrence of old with the string new on each line of file: sed 's/old/new/2' file So, instead of your proposed solution, you can use: sed 's/ /|/2' For more information, see e.g. this sed tutorial.


7

To build on Michael Borgwardt's answer: as long as both files are sorted, you can put them together by basically performing one step of a mergesort. It'll be a little different than standard mergesort because you only want to keep one of the files. This will, of course, have to be implemented in your favorite programming language. Here's a sketch of the ...


7

I don't know what OS you're on, but on FreeBSD 7.0+ grep has a -o option to return only the part that matches the pattern. So you could grep "marker-1234" filter_log | grep -oE "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" Returns a list of just IP addresses from the 'filter_log"... This works on my system, but again, I don't know what your version of ...


7

I've just invested some time to try this and came up with the following script: #!/bin/bash # set folder where files are located SOURCE_FOLDER=/path/to/source # define folder to which the files have to be copied TARGET_FOLDER=/home/abcd # #### cd "${SOURCE_FOLDER}" for FILE in *; do # everything which is not a normal file if [ ! -f "${FILE}" ]; ...


7

$ perl -ne 'print "$. $_" if m/[\x80-\xFF]/' utf8.txt 2 Pour être ou ne pas être 4 Byť či nebyť 5 是或不 or $ grep -n -P '[\x80-\xFF]' utf8.txt 2:Pour être ou ne pas être 4:Byť či nebyť 5:是或不 where utf8.txt is $ cat utf8.txt To be or not to be. Pour être ou ne pas être Om of niet zijn Byť či nebyť 是或不


7

For basic tasks like this, you can use for /f. Instead of command | awk '{ print $4 }' try this: for /f "tokens=4" %a in ('command') do echo %a Note that you have to use %%a instead of %a in batch files.


7

You can use shuf command line GNU core utility to do this. The utility is pretty fast and would take less than a minute for shuffling 1 GB file. For your case below command might just work. As shuf will read complete input before opening the output file. $ shuf -o File.txt < File.txt You can follow the GNU Documentation for this tool shuf


6

I'm a little green on Linux, so I'm unsure exactly why it doesn't work. But you can use a function instead. In .profile/.bashrc create a new function: downloads() { grep `date '+%d/%b/%Y'` access.logs | egrep 2765330645ae47d292c9ceac725d744e.py |awk '{print $1, $4, $5, $7, $8, $9, $10}' | sort |uniq -c -w15 |sort -n; } That works exactly as an alias.


6

Try this: awk -F , '{ printf "%i,%08i,%s\n" , $1 , $2 , $3 }' file



Only top voted, non community-wiki answers of a minimum length are eligible