Questions tagged [sed]
"sed" ("stream editor") is a Unix utility that parses and transforms text files.
1,040
questions
0
votes
0
answers
37
views
Why did sed stop working with emoji overnight, but only on 1 of 3 machines?
I have 3 machine all running the same version of sed (cygwin), under TCC command line.
This morning, I woke up to sed failing to work with emoji, but only on 1 of the 3 machines.
The sed version is ...
4
votes
1
answer
9k
views
Add a new element to existing JSON array
How can I add "arn:aws:iam::xxxxxxx:role/3" line below "arn:aws:iam::xxxxxxx:role/2" adding a comma after "arn:aws:iam::xxxxxxx:role/2",.
Before:
{
"Version": "2012-10-17",
"Statement": [
{
...
0
votes
2
answers
39
views
sed search replace regexp pattern to replace repeated strings in paths
I'm having some trouble using sed to replace a particular repeated portion of a file path, and I cannot for the life of me figure out how to get it to work.
For example, say I have a list of file ...
0
votes
2
answers
4k
views
sed: can't read /etc/dbus-1/session.conf: Input/output error
Following a guide on ZDNet here, I have been trying to get ubuntu desktop working on Windows 10. though one line of code has been giving me trouble.
sudo sed -i 's/<listen>.*<\/listen>...
0
votes
2
answers
4k
views
gettting error sed: illegal option -- i?
$ find . -type f -exec sed -i 's/original_string/tobereplace_string/g' {} +
sed: illegal option -- i
here I'm replacing string using above command but getting error illegal option -- i
can you please ...
0
votes
0
answers
22
views
rstrip On Lines of text being piped through sed s
So I am trying to implement the rstrip function on lines of text being piped into sed s and am having troubles. This is what I have that I think should work:
cat file1.txt | sed 's/[[:blank:]]+$//g;' &...
0
votes
2
answers
5k
views
replace space and empty cells in csv file
I have a csv file that has 7 columns. It has empty cells and some spaces between cells. How can I replace the empty cells with NA and remove extra spaces? Thank you very much!
Here is what my file ...
0
votes
2
answers
57
views
Find and replace date with UTC date
I have file that looks like this:
[2024-01-20 15:23:00] hello world
[2024-01-20 15:42:00] bye
[2024-01-20 15:43:00] foo bar
...
Date in this file are in UTC+1 and I want to convert in to UTC
I tried ...
0
votes
1
answer
4k
views
BASH converting txt to csv
BASH/scripting newbie here.
sed 's/ \+/,/g' output3.txt > output.csv
This one gave me so many delimited.
I only one to delimit the "pipe" | symbol and leave the rest as it is. So ...
0
votes
3
answers
2k
views
awk Removing Double Quotes from CSV File Quoted Strings separated by comma
I have a csv file I would like to remove any rouge " or , in it. The problem is all my fields look something like this "***********","*********", where * = can be any ...
0
votes
1
answer
84
views
why in spite of increasing the physical size of my text file its .gz compressed size shows significant decrease in size
I have a large test file which its .gz format is about 13 GB. I have replaced a substring in the original file and compressed the file to .gz format.
To do so, I used the following Linux command:
zcat ...
3
votes
3
answers
2k
views
How to add # Shebang in first line of a file which is an auto create script by another program
I am looking for a method to add a shebang #!/bin/csh -f to first line of my file , which is actually getting created by a another set of program.
Since this script is auto-created, it should run from ...
0
votes
1
answer
41
views
How do I tell sed to stop trying to process commands?
Maybe I'm an idiot (this is the likely scenario, and I'm new to sed/bash scripting), but why does this work:
NewVersion="$(echo $NewVersionFile | sed 's/^.*-\([0-9]*\.[0-9]*\.[0-9]*\).*/\1/')&...
0
votes
3
answers
35
views
What's the history on the option separator of GNU sed?
They say GNU sed supports using any character following the command character as a separator for command options, like the following.
echo abc | sed 'y%abc%ABC%'
But I can't find the documentation ...
1
vote
1
answer
1k
views
Replace in file with Huge String (Argument list too long)
I have a file in which I want to replace a string with a huge string literal in the form "19209302","19209329" such 100 thousand of entries.
I am using sed and I get "Argument list to long" error, ...
4
votes
2
answers
21k
views
sed to remove all whitespace from a line
I want to use sed to remove all space characters from a text file. At present I am using this command:
sed 's/ //' test.txt > test2.txt
This works in the sense that it removes the first space ...
0
votes
1
answer
62
views
AWK replace some special new line to one line
i have some log like this:
2023-11-15T08:59:28.000000+00:00 database-1 # Time: 231115 8:59:28
# User@Host: rdsadmin[rdsadmin] @ localhost []
# Thread_id: 3 Schema: QC_hit: No
# Query_time: 0....
0
votes
1
answer
297
views
Helping with Join of output from DF and LSBLK Linux commands via bash
I already asked this question, but without update. I think this information should be related to this site.
I need to merge two outputs in Linux. This:
lsblk -n -b --output KNAME,NAME,SIZE,MOUNTPOINT |...
0
votes
1
answer
15k
views
sed gives “illegal byte sequence” error when trying to replace a character
I am using the “Terminal” in Mac OS X 10.10 (Yosemite) and I am trying to delete all occurrences of a from a file, by using sed:
sed 's/a//g' file
Which gives me the following error:
sed: RE error: ...
0
votes
1
answer
81
views
sed awk truncate to wrap a line and chose specific fields
I have 3 lines like this in a file
this is the first line blah1 blah2
this is the next line 0.101 0.202 0.303
this is the answer 0.404
Desired
this is the first line 0.202 0.303 -0.404
Can this be ...
0
votes
0
answers
134
views
Convert a 1 row (comma separated) text file to multiple rows
How can I convert text files with the following format:
["word01","word02","word03","word04","word05","word06","word07","...
0
votes
1
answer
61
views
How can I remove a fix substring from a UNIX/Linux output?
My question looks ridicilously simple, but:
I would like to remove an entire string, not a list of characters, so I can't use tr.
I don't know the position of my substring, so I can't use cut.
I'm ...
0
votes
1
answer
335
views
Moving files to a directory with spaces in its name
I'm trying to create a directory that has in its name the beginning of the names of the files it contains. I need this in order to archive it later. I have run into problems when moving files if the ...
1
vote
1
answer
361
views
Use sed to "format" first two letters of every word
I am trying to get to a sed command to format (HTML) the first two letters of every word from a plaintext input. I would like to input a plain text and mark the first two letters bold with the <b&...
22
votes
7
answers
29k
views
How to print last character of a file
In Unix, using a simple command like sed, is there a way to print the last character of a file?
0
votes
2
answers
72
views
How can I insert the original matched pattern (OLD STRING) into the result (NEW STRING) Using Microsoft Word's Find/Replace Feature?
The find/replace feature in Microsoft Word usually has the same resulting output string no matter what input string matches the search pattern.
INPUT
OUTPUT
(1)
1.
(2)
1.
(3)
1.
(4)
1.
(5)
1.
(...
0
votes
2
answers
923
views
Find and replace string in a file using 'sed' or 'awk'
Let's say I have the following input file:
yhara.runner.mng1.nna1.X9HCJG.1 yhara.runner.mng1.nna1.Z2HCJG.1 100.000 100
yhara.runner.mng1.nna1.X9HCJG.1 yhara.runner.mng1.nna1.AUEM0K.1 94.144 72
...
0
votes
1
answer
532
views
need POSIX solution to remove special invisible characters from file
I have a file that comes to us from a third party (Windows) and is not in our control.
I observed that it has special characters like below:
root@DKERP:~# cat -ev ~/check_disk_space.sh
M-oM-;M-?#!/bin/...
190
votes
18
answers
156k
views
Removing ANSI color codes from text stream
Examining the output from
perl -e 'use Term::ANSIColor; print color "white"; print "ABC\n"; print color "reset";'
in a text editor (e.g., vi) shows the following:
^[[37mABC
^[[0m
How would one ...
0
votes
3
answers
709
views
how to delete all words starting with a specific string in a file?
How can I delete all words in a file starting with a specific string (in this case the string "end") so that:
<region> sample=PDL UP IN-A-1.flac lokey=21 hikey=21 lovel=0 hivel=21 end=...
3
votes
2
answers
60
views
print specific line in all files in subfolders
I have a folder with several subfolders, each with several numeric subfolders.
├── child1
│ ├── 1
│ ├── 2
│ └── 3
├── child2
│ ├── 1
│ ├── 2
│ └── 3
└── child3
├── 1
├── 2
└── ...
-1
votes
1
answer
153
views
write a efficient for loop in bash for multiple files (.PDB files)
I have problem with this script:
#!/bin/bash
for filename in /home/hb/pg/ensemble/pdbs/P24941/raw_pdb/*.pdb; do
grep COMPND $filename | grep "CHAIN:" -B 1 | sed 's/COMPND//g' | sed 's/...
0
votes
1
answer
111
views
sed: return all lines between matching patterns and quit
I want to extract only the lines that will restore a specific table from a mysql backup file. The following works.
zcat /mnt/backup/full-replication.gz | sed -n -e '/DROP TABLE.*`account_codes`/,/...
3
votes
3
answers
564
views
How do I delete every ocurrence of a word that is followed by another, without altering the whitespace?
Suppose my file looks like this:
foo bar
foo bar bar
foo foo bar foo
What I want to do is delete every foo that is followed by a bar (i.e. there is only whitespace between them), without ...
0
votes
0
answers
95
views
How do I perform this sed command with bash?
I have a file full of usernames and I want to use sed to change the usernames to PLAYER 1-20 how can I do this with a bash script and sed while incrementing?
I have this so far:
NAME=$(grep -q 'Seat' $...
2
votes
0
answers
45
views
SFTP user login details real-time filtering
I have enabled the SFTP login log into the default logfile /var/log/syslog and tried to filter the login time of each user and insert it into the database.
But the filtering is not worked as I ...
1
vote
1
answer
1k
views
Replacing a string when it is not followed by string with Sed
I am looking for a sed script to replace a string when it is not followed by a string in the content of a file.
Currently I am at this approach
sed -i -E "s/${filehash}(?!.de)/${filehash}.de/g&...
3
votes
1
answer
1k
views
Remove spaces from first column of colon-delimited file
I have a very large colon-delimited file with two columns:
dhsj fdksjdk:dndkdlkfkd
shfdosod dkjsosjd:dkj dosjeod
dhswisoaidodisbw: dhsoi wjdos
fjfdododjfideefkjso sn:fhfdkd dkdjdf
If the first ...
130
votes
18
answers
54k
views
Substitution in text file **without** regular expressions
I need to substitute some text inside a text file with a replacement. Usually I would do something like
sed -i 's/text/replacement/g' path/to/the/file
The problem is that both text and replacement ...
1
vote
0
answers
290
views
Sed editing of large text file much faster than native bash substring replacement?
I have a ~19MB text file which I would like to perform many find/replace operations on. I originally wrote the script to loop over inplace sed operations e.g.
sed -i "s|$pattern|$replacement|g&...
2
votes
3
answers
954
views
Is there a UNIX command that can replace an XML element? (Like a sed for an XPath?)
Is there a pipe-friendly program that takes:
An XML document
An XPath expression indicating the target element(s)
The desired replacement
? Something like sed but knowledgeable about XML (...
4
votes
3
answers
9k
views
Find and replace text in a file after match of pattern only for first occurrence using sed
Let's say I have the following input file:
Server 'Test AB'
option type 'ss'
option port '1234'
option timeout '60'
Server 'Test CD'
option type 'ss'
option port '1234'
option ...
2
votes
2
answers
111
views
What options are being used by sed to parse a file and write to new file
I am debugging a LINUX script that was written by someone else. In the script we have a line of code like below
sed -i "\#</resources>#{h;s#.*#cat /opt/abc/new.xml#e;G}" /opt/abc/...
13
votes
3
answers
30k
views
use sed to replace nbsp, 160, Hex 00a0, Octal 240, non-breaking space
I am having some problems with config files which have the non-breaking space chars in them.
How should I specify that character with sed so I can replace it with a space.
sed -n 's/ / /g'
...
1
vote
0
answers
211
views
pipe awk output in sed command
I have multiple lines in a file names.txt, let's say
my name is Jim your name please
what is good Harry potter
how is he Jhony and me
and so on , more than 500 lines.
I use awk command to search/ ...
0
votes
0
answers
234
views
bash loop to create m3u file with sed awk grep
I am trying to auto create m3u file for my IPTV software with the help of sed/awk or grep command,but don't have much experience in this. expert help required to do this. working on Windows-10 PC ...
0
votes
1
answer
212
views
Trying to search & replace within brackets using sed
I have a json file which among other settings, contains the following data:
"school": {
"name": "qwerty",
"key": "user",
"secret"...
0
votes
1
answer
978
views
sed: unterminated `s' command – What does it mean?
I'm just learning this sed tool. So bear with me and appreciate your help.
This works fine.
$ date | sed 's/ /_/g'
Mon_Mar_20_11:02:26_MST_2023
But not this one.
$ date | sed 's/MST/EDT'
sed: -e ...
0
votes
2
answers
456
views
How to combine tee, sed and grep?
I have a command after which I place | tee >(sed $'s/\033[[][^A-Za-z]*[A-Za-z]//g' >> ~user/filepath/file.txt) The tee is redirected to sed $'s/\033[[][^A-Za-z]*[A-Za-z]//g' which removes the ...
0
votes
1
answer
126
views
sed remove double quote pair spanning lines
In my OpenDKIM key file, I have extra quotes that can't go in "proper" DNS records. (I'm surprised the OpenDKIM devs allow this in their file, but that's another story.)
In RegEx, it is two ...