Questions tagged [sed]
"sed" ("stream editor") is a Unix utility that parses and transforms text files.
1,041
questions
1
vote
1
answer
1k
views
Replace in file with Huge String (Argument list too long)
I have a file in which I want to replace a string with a huge string literal in the form "19209302","19209329" such 100 thousand of entries.
I am using sed and I get "Argument list to long" error, ...
0
votes
1
answer
2k
views
How would I write this VIM command for `sed`?
I have a .csv with special characters that are wrecking an import I need to do.
In VIM, I discovered away to remove the special characters with :
:%s/\r//g
This removes ^M from some of my broken ...
1
vote
2
answers
1k
views
Best way to increment numbers in one line in Linux [closed]
I have one line with lot of entries among which is also time.
('DEFAULT',17,NULL,'2014-07-14 10:30:00','Something','2014-07-14 06:30:00',1),('DEFAULT',26,NULL,'2014-07-14 12:00:00','Something2,'...
2
votes
2
answers
207
views
How to remove string following a specific word
I am trying to remove string starting with a specific word in my manifest file. The file containing lines looks like:
<assemblyIdentity type="win32" name="Microsoft.VC90.CRT" version="9.0.21022.8" ...
1
vote
1
answer
294
views
How do I edit an xmlTV file using sed (or a similar tool)?
I have an xmlTV file that looks like this:
<channel id="10125.dvb.guide" <!-- number="62" type="0x1" flags="0xf" bouquet="4097" region="4a" sid="10125" -->>
<display-name>ITV +1</...
3
votes
2
answers
4k
views
grep and sed with pipe from tail -f appears to be caching
I am trying to get a working system to monitor special logs. I usually just want a very specific pattern which I extract using grep and a pipe from tail -f. I have noticed that grep does not output ...
2
votes
2
answers
2k
views
how to insert text at the beginning of each paragraph in bash
I have a file with multiple paragraphs separated by blank line. Technically they are not paragraphs just sections of text separated by blank line.
I want to number the paragraphs so to speak by ...
0
votes
2
answers
357
views
Why am I getting mv errors with this sed command?
Given the following directory:
[~/test]find .
.
./something_else.txt
./univision.rbw
./something_else.rb
./testtest
./testtest/weewee.rb
./testtest/weewee.txt
./univision.rb
Running the following ...
6
votes
2
answers
3k
views
grep -l output filenames with spaces
I want to loop over files with
for f in `grep -rsl "foo" . `: do sed -i -- "s/foo/bar/g" $f; done;
But since filenames contain spaces, a filename is split whenever a space is found.
How can I ...
0
votes
0
answers
501
views
Backrefence syntax in powershell's sed
How do I back reference a capture group in powershell's sed? I can't seem to find the syntax for it.
IE I expect echo "hello" | sed "s|(h)ello|world\1|" to output worldh, but instead I get the ...
0
votes
1
answer
1k
views
Remove a specific string common in multiple lines in a CSV file using shell script
I have a CSV file which contains 65000 lines (size approximately 28 MB). On each of the lines they have a path in the beginning in the format of: c:\abc\bcd\def\123\456.
Now let's say the path c:\abc\...
1
vote
1
answer
1k
views
How to search for a line which contains specific string in Bash?
I've got a file which contains some amount of lines, I want to look for a specific string, and if the file contains it, then I want to print a different line from the same file.
for example, let's ...
1
vote
2
answers
74
views
How to add the filepath to a specified line of a file using bash script?
I have the following bash code:
echo "Enter an input path"
read -e input_path
OLDTEXT="\put(0,0){\includegraphics{/"
NEWTEXT="\put(0,0){\includegraphics{$input_path/"
find . -...
2
votes
1
answer
178
views
How to substitute all occurrences after a certain number with sed in OS X
I am trying to modify a bad .csv file, where the last 'column' is broken into a different number of columns due to extra , characters. I want to use sed to keep the first 4 commas, and replace the ...
0
votes
1
answer
36
views
sed adds characters where it shouldn't add
I have the following line (in a file called 1atp.pdb:
MASTER 421 1 5 16 9 0 12 6 3070 2 66 29
And I'm using sed:
grep MASTER 1atp.pdb | sed 's/[0-9]*/xddddd/'
...
0
votes
1
answer
15k
views
sed gives “illegal byte sequence” error when trying to replace a character
I am using the “Terminal” in Mac OS X 10.10 (Yosemite) and I am trying to delete all occurrences of a from a file, by using sed:
sed 's/a//g' file
Which gives me the following error:
sed: RE error: ...
0
votes
1
answer
253
views
Multiple replace - using sed or awk
I have a code snippet like below
CREATE TABLE TABLE1(
ID integer NOT NULL,
ROW_CREATED varchar (26) NULL,
BATCH_LOG_ID integer NULL,
TRANSACTION_TYPE varchar(5) NULL,
PARTITION_COL integer ...
1
vote
1
answer
69
views
SED/AWK/PERL(GNU userland) - how to substitute pattern A but not pattern B?
First, I am using GNU userland so you can assume I use Bash, GNU coreutils...
I have a HTML and I want to change all
href="foobarz"
into
href=""
However, I don't want to change any line ...
2
votes
2
answers
594
views
How to make sed print to the end of line from socat
I have a server which sends data in a UDP Broadcast on port 10552. The data is 7 numbers separated by commas as so.
5.351204,0.001968,-0.000473,-0.999222,0.000451,0.001455,1.084310
I fetch this ...
1
vote
1
answer
40
views
Cygwin: during sed operation changes are not save
While I'm using sed I'm able to see correct results in terminal but in file changes are not visible.
I used following comand:
sed -e 's/MN2/MN9/g' 01_valid.txt
on terminal screen I see changes in ...
1
vote
2
answers
5k
views
How can I delete the content between two tags in a large xml file (>2GB)
The File in Question is XML but I want a Shell Script to remove all the binary (Base64) encoded images that are embedded. The files are generally huge (>2GB).
I want to delete everything between:
&...
0
votes
2
answers
2k
views
Append text to lines between two patterns every time they appear in a file
This will be a rather hard one, i guess.
I have to fiddle around with inline documentation that is extracted into html files to be used as online documentation, but these parts of the files should be ...
0
votes
2
answers
511
views
How to make a sed command to trim whitespace from the end of a string or normalize whitespace in the string?
I have some data like the following
"brand": "Some brand name ", "otherbrand": "Something else ",
"brand":"Some third way"
although a little bit more unstructured like that, however one ...
0
votes
1
answer
1k
views
sed regex pattern matched greedy although it should not
I stumbled upon a behaviour I can not explain, hope some of you guys can help me out.
I try to generate a sort of documentation from a bigger Ant project, as such I use sed to filter the information ...
2
votes
1
answer
93
views
Prepending a ‘.’ to each domain name in a list but skip raw IP addresses
We are starting a new proxy server at work and have a list of domains that are to be blocked; the list is downloaded from an external site.
To place them in Squid format we have to add a . to the ...
2
votes
1
answer
2k
views
piping tail -f into sed or awk
There is another question which is nearly the same, but none of the suggestions from its answer works for me:
Piping tail -f into awk
I'm trying to tail -f a logfile but want to replace \r with \n ...
0
votes
1
answer
111
views
Removing lines in text document that end the same
We have a script that downloads a list of domains for our squid box to block, but we keep getting warnings like the following:
2015/03/02 17:08:47| WARNING: You should probably remove '.artnau.com' ...
0
votes
2
answers
5k
views
How do I print the line only with the last occurrence of a regex?
I have a (large) file that has multiple occurrences of the expression SCF Done and I am (for now) only interested in the last of these lines. Currently I am using a piped command:
sed -n '/SCF Done/p'...
0
votes
2
answers
163
views
Sort on ID, then sort on date, then remove lines which have the older date?
I have a couple of tab delimited files.
Each file is structured like so:
ID Title Rating Date_Rated
What I want to do is merge all these files into one, ...
0
votes
1
answer
782
views
Escape string input to sed
I'm having some problems with replacing a string in a file using sed with a string that contains newlines. The following will fail completely. I assume I've got to escape the replace_with string, but ...
1
vote
1
answer
1k
views
How to extract a substring and add it to original string with sed?
I am trying to write an sh script witch receives a string as input, extracts a particular substring and adds it after an specified expression to the input string. I tried a lot with sed -i'N ... but ...
1
vote
1
answer
2k
views
How to specify a sed expression to remove non-printing octal characters from a text file
I am new to sed and am cleaning up a very large file. The cleaned document looks right, but when I read it into my R, it shows that each line is appended with \016\t at the end of each line. I have ...
0
votes
0
answers
584
views
Copy huge file skipping first line using dd in Linux
I was trying to copy the content of a huge file (10GB) to another skipping the first line (as in head -n1). I tried multiple ways of head, tail, awk and sed. Settled on using tail -n+2 > ./xab.1
...
7
votes
2
answers
40k
views
use sed to replace part of a string
I am planning to replace certain strings in a file; so I am using sed.
I would like to have the string replaced, while instead, sed read the string and append the text in the middle of the string.
...
5
votes
1
answer
526
views
How do I prepend a string to program output without waiting for the whole line?
I have a script that runs a command on a remote server using SSH. I want to prepend the string Remote: to every line of the output, but I don't want each line to be delayed until the whole line is ...
0
votes
1
answer
1k
views
Replace with sed a tag value under a specific section when matching a line
I would like to change a line when another line matchs under a section of a file using the sed command (or awk). Here is an exemple of the file :
Extract of MyFic.txt :
<PlayList>
... ...
0
votes
4
answers
2k
views
How do I DELETE a comma and a space from a numeric string using a bash script
Folks, I have asked this question 5 times on here and have received 5 separate bits of code from super users that have all *failed.
Problem: I have stock information being brought back into a txt ...
1
vote
1
answer
409
views
Using cURL command-line tool on Mac, What can I do to this script to ask it to fetch stock data which will replace commas with NO SPACES
Example: Google Outstanding Shares are 670,000,000 and this value is collected in a txt file from Yahoo Finance API via the below script. The issue is; that numbers WITH commas or spaces are a bloody ...
2
votes
1
answer
50
views
Use argument of unrun command in Linux
If I wanted to run mv arg1 $(sed arg1 's/regex/replace/') is there a better way to to do it than to reinsert the filename into the sed command?
0
votes
1
answer
40
views
have sed translate both cases from lowercase input
I have variables from="abc" and to="xyz" defining lowercase strings.
How can I use sed so that it replaces from by to, but matching the case.
That is echo "abc def ABC DEF" | <sedcommand>
...
0
votes
1
answer
970
views
Sed replace line stored in a variable
I need to find a line that starts with mysql.unmetric.twt.url and replace the entire line with line stored in the variable ip.
ip="mysql.unmetric.twt.url=jdbc:mysql://ec2-a-b-c-d.compute-1.amazonaws....
0
votes
1
answer
155
views
Sed and Replace
I need to find a line that starts with mysql.unmetric.twt.url and replace the entire line with line stored in the variable ip
ip="mysql.unmetric.twt.url=jdbc:mysql://ec2-a-b-c-d.compute-1.amazonaws....
0
votes
1
answer
44
views
How to fetch the string "Hi there" from the below array in unix
I only want to fetch Hi there and store it in a variable VAR1. So basically VAR1 should contain Hi there
VIEW_COL[1]="Hi there,USA,Y"
This is what i have tried so far.
Column_Display_Name=`echo ${...
0
votes
1
answer
115
views
I want to append the name of the file on the top of that file using bash script
I want to add void mainfunction() on the top of a file. I am reading the actual name of the file from another file and the name of the main function from another file. I was using:
sed -i 1i void $...
0
votes
3
answers
376
views
Filtering 2nd field from a data set and then using uniq on the output
I've got a dataset that goes like this:
AAAAA 11111 Data1
AAAAA 11111 Data2
AAAAA 11111 Data3
AAAAA 11112 Data4
AAAAA 11112 Data5
AAAAA 11112 Data6
AAAAA 11112 Data7
AAAAA 11113 Data8
AAAAA 11114 ...
0
votes
4
answers
3k
views
Text deletion between patterns across multiple lines with respect to text inside pattern
I have a block of text I need to delete, however, only if it contains specific text inside the block:
...
<script language="JavaScript">
var somethingA = 0;
var somethingB = 0;
var ...
1
vote
3
answers
80
views
Two very similar lines of sed, can they be combined?
I have the following two lines in a shell script and was wondering if there was any way to condense them into one using sed. I am not sure if there is a wildcard or anything that could replace the ...
1
vote
1
answer
156
views
bash script processing files with bad characters
I received a zip file with a folder structure that include parenthesis. I've worked through two processes at the cli and now merge them into a script
The first searches through the file structure for ...
1
vote
2
answers
754
views
Replacing several strings in a file in one go
Suppose I want to replace all AAA with BBB and all BBB with AAA in a text file.
The thing which looks an obvious solution (in Perl or in Sed) is:
s/AAA/BBB/g; s/BBB/AAA/g;
But this does not work ...
3
votes
2
answers
843
views
mod_sed - apache+sed - problems with backslash
I have just started using sed and are having a problem when using sed from Apache mod_seq.
The task is to change a web service post to a file upload post. Content-Type has already been changed using ...