"sed" ("stream editor") is a Unix utility that parses and transforms text files.

learn more… | top users | synonyms

1
vote
2answers
48 views

Best way to increment numbers in one line in Linux

I have one line with lot of entries among which is also time. ('DEFAULT',17,NULL,'2014-07-14 10:30:00','Something','2014-07-14 06:30:00',1),('DEFAULT',26,NULL,'2014-07-14 ...
4
votes
3answers
4k views

How append a string at the end of all lines?

I'm trying to add a string at the end of all lines in a text file, but I have a mistake somewhere. Example: I've this in a text file: begin--fr.a2dfp.net begin--m.fr.a2dfp.net begin--mfr.a2dfp.net ...
2
votes
2answers
29 views

how to insert text at the beginning of each paragraph in bash

I have a file with multiple paragraphs separated by blank line. Technically they are not paragraphs just sections of text separated by blank line. I want to number the paragraphs so to speak by ...
13
votes
4answers
9k views

Newlines in sed on Mac OS X

I find that \n doesn't work in sed under Mac OS X. Specifically, say I want to break the words separated by a single space into lines: # input foo bar I use, echo "foo bar" | sed 's/ /\n/' But ...
26
votes
10answers
15k views

How to Remove the Last 2 Lines of a Very Large File

I have a very large file ~400G, and I need to remove the last 2 lines from it. I tried to use sed, but it ran for hours before I gave up. Is there a quick way of doing this, or am I stuck with sed?
1
vote
2answers
424 views

Change names in a column in bash

I have a tab-separated file in Linux that looks like this ~$ head list.tab "x" "1" "FUHGF.jnf.m22-1" "2" "HDFHFEY.gfs.d2-1" "3" "KJFGJF.fr.md2-1" "4" "SDFSDIB.gfd.rtl2-1" "5" "FGJFGOB.hgd.k2-5" "6" ...
0
votes
0answers
32 views

Removing special characters (spaces, newlines, tabs) from a text file

I have a flat file (1252 ANSI-Latin 1 encoding) that I wrote out from querying a third party's SQL database that has within it extraneous characters. To make it easier to reformat, I am writing it ...
2
votes
2answers
90 views

How to remove string following a specific word

I am trying to remove string starting with a specific word in my manifest file. The file containing lines looks like: <assemblyIdentity type="win32" name="Microsoft.VC90.CRT" version="9.0.21022.8" ...
1
vote
1answer
45 views

How do I edit an xmlTV file using sed (or a similar tool)?

I have an xmlTV file that looks like this: <channel id="10125.dvb.guide" <!-- number="62" type="0x1" flags="0xf" bouquet="4097" region="4a" sid="10125" -->> <display-name>ITV ...
3
votes
1answer
8k views

Multi-Line Sed Replace

Consider the following text (incidentally, part of a MySQL dump): CREATE TABLE `table` ( `id` int(10) NOT NULL auto_increment, `name` varchar(100) NOT NULL default '', `description` text NOT ...
0
votes
2answers
27 views

grep and sed with pipe from tail -f appears to be caching

I am trying to get a working system to monitor special logs. I usually just want a very specific pattern which I extract using grep and a pipe from tail -f. I have noticed that grep does not output ...
0
votes
2answers
32 views

Why am I getting mv errors with this sed command?

Given the following directory: [~/test]find . . ./something_else.txt ./univision.rbw ./something_else.rb ./testtest ./testtest/weewee.rb ./testtest/weewee.txt ./univision.rb Running the following ...
2
votes
1answer
25 views

grep -l output filenames with spaces

I want to loop over files with for f in `grep -rsl "foo" . `: do sed -i -- "s/foo/bar/g" $f; done; But since filenames contain spaces, a filename is split whenever a space is found. How can I ...
0
votes
0answers
25 views

Backrefence syntax in powershell's sed

How do I back reference a capture group in powershell's sed? I can't seem to find the syntax for it. IE I expect echo "hello" | sed "s|(h)ello|world\1|" to output worldh, but instead I get the ...
0
votes
1answer
19 views

Remove a specific string common in multiple lines in a CSV file using shell script

I have a CSV file which contains 65000 lines (size approximately 28 MB). On each of the lines they have a path in the beginning in the format of: c:\abc\bcd\def\123\456. Now let's say the path ...
1
vote
1answer
26 views

How to serach for a line which contains specific string in Bash?

I've got a file which contains some amount of lines, I want to look for a specific string, and if the file contains it, then I want to print a different line from the same file. for example, let's ...
14
votes
4answers
10k views

How to use sed to remove null bytes?

What is the sed incantation to remove null bytes from a file? I'm trying: s/\000//g but that is stripping out strings of zeroes. s/\x00//g seems to have no effect. I'm trying to do this in a ...
1
vote
2answers
18 views

How to add the filepath to a specified line of a file using bash script?

I have the following bash code: echo "Enter an input path" read -e input_path OLDTEXT="\put(0,0){\includegraphics{/" NEWTEXT="\put(0,0){\includegraphics{$input_path/" find . ...
0
votes
0answers
33 views

sed gives “illegal byte sequence” error when trying to replace a character

I am using the “Terminal” in Mac OS X 10.10 (Yosemite) and I am trying to delete all occurrences of a from a file, by using sed: sed 's/a//g' file Which gives me the following error: sed: RE ...
2
votes
1answer
18 views

How to substitute all occurrences after a certain number with sed in OS X

I am trying to modify a bad .csv file, where the last 'column' is broken into a different number of columns due to extra , characters. I want to use sed to keep the first 4 commas, and replace the ...
0
votes
1answer
16 views

sed adds characters where it shouldn't add

I have the following line (in a file called 1atp.pdb: MASTER 421 1 5 16 9 0 12 6 3070 2 66 29 And I'm using sed: grep MASTER 1atp.pdb | sed 's/[0-9]*/xddddd/' ...
0
votes
0answers
25 views

Multiple replace - using sed or awk

I have a code snippet like below CREATE TABLE TABLE1( ID integer NOT NULL, ROW_CREATED varchar (26) NULL, BATCH_LOG_ID integer NULL, TRANSACTION_TYPE varchar(5) NULL, PARTITION_COL integer ...
1
vote
1answer
15 views

SED/AWK/PERL(GNU userland) - how to substitute pattern A but not pattern B?

First, I am using GNU userland so you can assume I use Bash, GNU coreutils... I have a HTML and I want to change all href="foobarz" into href="" However, I don't want to change any line ...
0
votes
3answers
135 views

Replace string in string but not when its somewhere between double square bracket

I have a long string from mysql field containing the data of a mediawiki page. I have to replace a string in that mediawiki page but not when the string is in mediawiki link. The mediawiki link is ...
1
vote
2answers
28 views

How to make sed print to the end of line from socat

I have a server which sends data in a UDP Broadcast on port 10552. The data is 7 numbers separated by commas as so. 5.351204,0.001968,-0.000473,-0.999222,0.000451,0.001455,1.084310 I fetch this ...
0
votes
1answer
87 views

How can I delete the content between two tags in a large xml file (>2GB)

The File in Question is XML but I want a Shell Script to remove all the binary (Base64) encoded images that are embedded. The files are generally huge (>2GB). I want to delete everything between: ...
1
vote
1answer
8 views

Cygwin: during sed operation changes are not save

While I'm using sed I'm able to see correct results in terminal but in file changes are not visible. I used following comand: sed -e 's/MN2/MN9/g' 01_valid.txt on terminal screen I see changes in ...
3
votes
2answers
922 views

sed, insert file before last line

The following inserting file method for sed used to be working, even before last line, but not any more. Is it a bug in the current sed? Demo of inserting file method with sed: mkdir /tmp/test ...
0
votes
2answers
31 views

Append text to lines between two patterns every time they appear in a file

This will be a rather hard one, i guess. I have to fiddle around with inline documentation that is extracted into html files to be used as online documentation, but these parts of the files should be ...
0
votes
2answers
216 views

sed: What does exit code 2 mean?

Case I am trying to replace all occurrences of <value>*</value> into <value>file:\\</value> in a designated file test.txt. I work on Windows and use sed I have installed with ...
0
votes
2answers
34 views

How to make a sed command to trim whitespace from the end of a string or normalize whitespace in the string?

I have some data like the following "brand": "Some brand name ", "otherbrand": "Something else ", "brand":"Some third way" although a little bit more unstructured like that, however one ...
20
votes
4answers
4k views

Substitution in text file **without** regular expressions

I need to substitute some text inside a text file with a replacement. Usually I would do something like sed -i 's/text/replacement/g' path/to/the/file The problem is that both text and replacement ...
0
votes
1answer
35 views

sed regex pattern matched greedy although it should not

I stumbled upon a behaviour I can not explain, hope some of you guys can help me out. I try to generate a sort of documentation from a bigger Ant project, as such I use sed to filter the information ...
2
votes
1answer
23 views

Prepending a ‘.’ to each domain name in a list but skip raw IP addresses

We are starting a new proxy server at work and have a list of domains that are to be blocked; the list is downloaded from an external site. To place them in Squid format we have to add a . to the ...
1
vote
1answer
52 views

piping tail -f into sed or awk

There is another question which is nearly the same, but none of the suggestions from its answer works for me: Piping tail -f into awk I'm trying to tail -f a logfile but want to replace \r with \n ...
0
votes
1answer
92 views

GnuWin32 / sed unexpected behavior in Powershell

I am using GnuWin32 tools on Windows Command Line/Powershell. What is see: 11:15 enlil D:\Users\x> Get-ChildItem .gitconfig | sed "s/ */ /g" D i r e c t o r y : D : \ U s e r s \ x M o d ...
0
votes
1answer
30 views

Removing lines in text document that end the same

We have a script that downloads a list of domains for our squid box to block, but we keep getting warnings like the following: 2015/03/02 17:08:47| WARNING: You should probably remove '.artnau.com' ...
0
votes
2answers
35 views

How do I print the line only with the last occurrence of a regex?

I have a (large) file that has multiple occurrences of the expression SCF Done and I am (for now) only interested in the last of these lines. Currently I am using a piped command: sed -n '/SCF ...
5
votes
2answers
8k views

How to match digits followed by a dot using sed?

I'm trying to use sed to substitute all the patterns with digits followed immediately by a dot (such as 3., 355.) by an empty string. So I try: sed 's/\d+\.//g' file.txt But it doesn't work. Why is ...
2
votes
2answers
7k views

How do I encrypt Samsung 840 SSD with Bitlocker?

I need help encrypting my Samsung 840 Evo with Bitlocker. Here is my hardware: Samsung Evo 840 500GB hard drive (firmware EXT0BB6Q) Asus P8z77-v motherboard (BIOS v. 2104) Asus TPM chip - purchased ...
0
votes
2answers
24 views

Sort on ID, then sort on date, then remove lines which have the older date?

I have a couple of tab delimited files. Each file is structured like so: ID Title Rating Date_Rated What I want to do is merge all these files into one, ...
0
votes
1answer
36 views

Escape string input to sed

I'm having some problems with replacing a string in a file using sed with a string that contains newlines. The following will fail completely. I assume I've got to escape the replace_with string, but ...
0
votes
1answer
45 views

How to extract a substring and add it to original string with sed?

I am trying to write an sh script witch receives a string as input, extracts a particular substring and adds it after an specified expression to the input string. I tried a lot with sed -i'N ... but ...
1
vote
1answer
32 views

How to specify a sed expression to remove non-printing octal characters from a text file

I am new to sed and am cleaning up a very large file. The cleaned document looks right, but when I read it into my R, it shows that each line is appended with \016\t at the end of each line. I have ...
0
votes
0answers
62 views

Copy huge file skipping first line using dd in Linux

I was trying to copy the content of a huge file (10GB) to another skipping the first line (as in head -n1). I tried multiple ways of head, tail, awk and sed. Settled on using tail -n+2 > ./xab.1 ...
3
votes
3answers
599 views

Flatten a folder structure to a file name in Bash

I'm trying to convert files from this structure: foldera/folderb/filec.png to: foldera_folderb_filec.png I'm halfway there: find . -type f -name "*.png" | sed 's@^./@@g;s@/@-@g' which ...
0
votes
2answers
119 views

use sed to replace part of a string

I am planning to replace certain strings in a file; so I am using sed. I would like to have the string replaced, while instead, sed read the string and append the text in the middle of the string. ...
3
votes
1answer
44 views

How do I prepend a string to program output without waiting for the whole line?

I have a script that runs a command on a remote server using SSH. I want to prepend the string Remote: to every line of the output, but I don't want each line to be delayed until the whole line is ...
0
votes
1answer
36 views

Replace with sed a tag value under a specific section when matching a line

I would like to change a line when another line matchs under a section of a file using the sed command (or awk). Here is an exemple of the file : Extract of MyFic.txt : <PlayList> ... ...
0
votes
4answers
115 views

How do I DELETE a comma and a space from a numeric string using a bash script

Folks, I have asked this question 5 times on here and have received 5 separate bits of code from super users that have all *failed. Problem: I have stock information being brought back into a txt ...