Questions tagged [sed]

"sed" ("stream editor") is a Unix utility that parses and transforms text files.

Filter by
Sorted by
Tagged with
0 votes
0 answers
37 views

Why did sed stop working with emoji overnight, but only on 1 of 3 machines?

I have 3 machine all running the same version of sed (cygwin), under TCC command line. This morning, I woke up to sed failing to work with emoji, but only on 1 of the 3 machines. The sed version is ...
4 votes
1 answer
9k views

Add a new element to existing JSON array

How can I add "arn:aws:iam::xxxxxxx:role/3" line below "arn:aws:iam::xxxxxxx:role/2" adding a comma after "arn:aws:iam::xxxxxxx:role/2",. Before: { "Version": "2012-10-17", "Statement": [ { ...
0 votes
2 answers
39 views

sed search replace regexp pattern to replace repeated strings in paths

I'm having some trouble using sed to replace a particular repeated portion of a file path, and I cannot for the life of me figure out how to get it to work. For example, say I have a list of file ...
0 votes
2 answers
4k views

sed: can't read /etc/dbus-1/session.conf: Input/output error

Following a guide on ZDNet here, I have been trying to get ubuntu desktop working on Windows 10. though one line of code has been giving me trouble. sudo sed -i 's/<listen>.*<\/listen>...
0 votes
2 answers
4k views

gettting error sed: illegal option -- i?

$ find . -type f -exec sed -i 's/original_string/tobereplace_string/g' {} + sed: illegal option -- i here I'm replacing string using above command but getting error illegal option -- i can you please ...
0 votes
0 answers
22 views

rstrip On Lines of text being piped through sed s

So I am trying to implement the rstrip function on lines of text being piped into sed s and am having troubles. This is what I have that I think should work: cat file1.txt | sed 's/[[:blank:]]+$//g;' &...
0 votes
2 answers
5k views

replace space and empty cells in csv file

I have a csv file that has 7 columns. It has empty cells and some spaces between cells. How can I replace the empty cells with NA and remove extra spaces? Thank you very much! Here is what my file ...
0 votes
2 answers
57 views

Find and replace date with UTC date

I have file that looks like this: [2024-01-20 15:23:00] hello world [2024-01-20 15:42:00] bye [2024-01-20 15:43:00] foo bar ... Date in this file are in UTC+1 and I want to convert in to UTC I tried ...
0 votes
1 answer
4k views

BASH converting txt to csv

BASH/scripting newbie here. sed 's/ \+/,/g' output3.txt > output.csv This one gave me so many delimited. I only one to delimit the "pipe" | symbol and leave the rest as it is. So ...
0 votes
3 answers
2k views

awk Removing Double Quotes from CSV File Quoted Strings separated by comma

I have a csv file I would like to remove any rouge " or , in it. The problem is all my fields look something like this "***********","*********", where * = can be any ...
0 votes
1 answer
84 views

why in spite of increasing the physical size of my text file its .gz compressed size shows significant decrease in size

I have a large test file which its .gz format is about 13 GB. I have replaced a substring in the original file and compressed the file to .gz format. To do so, I used the following Linux command: zcat ...
3 votes
3 answers
2k views

How to add # Shebang in first line of a file which is an auto create script by another program

I am looking for a method to add a shebang #!/bin/csh -f to first line of my file , which is actually getting created by a another set of program. Since this script is auto-created, it should run from ...
0 votes
1 answer
41 views

How do I tell sed to stop trying to process commands?

Maybe I'm an idiot (this is the likely scenario, and I'm new to sed/bash scripting), but why does this work: NewVersion="$(echo $NewVersionFile | sed 's/^.*-\([0-9]*\.[0-9]*\.[0-9]*\).*/\1/')&...
0 votes
3 answers
35 views

What's the history on the option separator of GNU sed?

They say GNU sed supports using any character following the command character as a separator for command options, like the following. echo abc | sed 'y%abc%ABC%' But I can't find the documentation ...
1 vote
1 answer
1k views

Replace in file with Huge String (Argument list too long)

I have a file in which I want to replace a string with a huge string literal in the form "19209302","19209329" such 100 thousand of entries. I am using sed and I get "Argument list to long" error, ...
4 votes
2 answers
21k views

sed to remove all whitespace from a line

I want to use sed to remove all space characters from a text file. At present I am using this command: sed 's/ //' test.txt > test2.txt This works in the sense that it removes the first space ...
0 votes
1 answer
62 views

AWK replace some special new line to one line

i have some log like this: 2023-11-15T08:59:28.000000+00:00 database-1 # Time: 231115 8:59:28 # User@Host: rdsadmin[rdsadmin] @ localhost [] # Thread_id: 3 Schema: QC_hit: No # Query_time: 0....
0 votes
1 answer
297 views

Helping with Join of output from DF and LSBLK Linux commands via bash

I already asked this question, but without update. I think this information should be related to this site. I need to merge two outputs in Linux. This: lsblk -n -b --output KNAME,NAME,SIZE,MOUNTPOINT |...
0 votes
1 answer
15k views

sed gives “illegal byte sequence” error when trying to replace a character

I am using the “Terminal” in Mac OS X 10.10 (Yosemite) and I am trying to delete all occurrences of a from a file, by using sed: sed 's/a//g' file Which gives me the following error: sed: RE error: ...
0 votes
1 answer
81 views

sed awk truncate to wrap a line and chose specific fields

I have 3 lines like this in a file this is the first line blah1 blah2 this is the next line 0.101 0.202 0.303 this is the answer 0.404 Desired this is the first line 0.202 0.303 -0.404 Can this be ...
0 votes
0 answers
134 views

Convert a 1 row (comma separated) text file to multiple rows

How can I convert text files with the following format: ["word01","word02","word03","word04","word05","word06","word07","...
0 votes
1 answer
61 views

How can I remove a fix substring from a UNIX/Linux output?

My question looks ridicilously simple, but: I would like to remove an entire string, not a list of characters, so I can't use tr. I don't know the position of my substring, so I can't use cut. I'm ...
0 votes
1 answer
335 views

Moving files to a directory with spaces in its name

I'm trying to create a directory that has in its name the beginning of the names of the files it contains. I need this in order to archive it later. I have run into problems when moving files if the ...
1 vote
1 answer
361 views

Use sed to "format" first two letters of every word

I am trying to get to a sed command to format (HTML) the first two letters of every word from a plaintext input. I would like to input a plain text and mark the first two letters bold with the <b&...
22 votes
7 answers
29k views

How to print last character of a file

In Unix, using a simple command like sed, is there a way to print the last character of a file?
0 votes
2 answers
72 views

How can I insert the original matched pattern (OLD STRING) into the result (NEW STRING) Using Microsoft Word's Find/Replace Feature?

The find/replace feature in Microsoft Word usually has the same resulting output string no matter what input string matches the search pattern. INPUT OUTPUT (1) 1. (2) 1. (3) 1. (4) 1. (5) 1. (...
0 votes
2 answers
923 views

Find and replace string in a file using 'sed' or 'awk'

Let's say I have the following input file: yhara.runner.mng1.nna1.X9HCJG.1 yhara.runner.mng1.nna1.Z2HCJG.1 100.000 100 yhara.runner.mng1.nna1.X9HCJG.1 yhara.runner.mng1.nna1.AUEM0K.1 94.144 72 ...
0 votes
1 answer
532 views

need POSIX solution to remove special invisible characters from file

I have a file that comes to us from a third party (Windows) and is not in our control. I observed that it has special characters like below: root@DKERP:~# cat -ev ~/check_disk_space.sh M-oM-;M-?#!/bin/...
190 votes
18 answers
156k views

Removing ANSI color codes from text stream

Examining the output from perl -e 'use Term::ANSIColor; print color "white"; print "ABC\n"; print color "reset";' in a text editor (e.g., vi) shows the following: ^[[37mABC ^[[0m How would one ...
0 votes
3 answers
709 views

how to delete all words starting with a specific string in a file?

How can I delete all words in a file starting with a specific string (in this case the string "end") so that: <region> sample=PDL UP IN-A-1.flac lokey=21 hikey=21 lovel=0 hivel=21 end=...
3 votes
2 answers
60 views

print specific line in all files in subfolders

I have a folder with several subfolders, each with several numeric subfolders. ├── child1 │   ├── 1 │   ├── 2 │   └── 3 ├── child2 │   ├── 1 │   ├── 2 │   └── 3 └── child3 ├── 1 ├── 2 └── ...
-1 votes
1 answer
153 views

write a efficient for loop in bash for multiple files (.PDB files)

I have problem with this script: #!/bin/bash for filename in /home/hb/pg/ensemble/pdbs/P24941/raw_pdb/*.pdb; do grep COMPND $filename | grep "CHAIN:" -B 1 | sed 's/COMPND//g' | sed 's/...
0 votes
1 answer
111 views

sed: return all lines between matching patterns and quit

I want to extract only the lines that will restore a specific table from a mysql backup file. The following works. zcat /mnt/backup/full-replication.gz | sed -n -e '/DROP TABLE.*`account_codes`/,/...
3 votes
3 answers
564 views

How do I delete every ocurrence of a word that is followed by another, without altering the whitespace?

Suppose my file looks like this: foo bar foo bar bar foo foo bar foo What I want to do is delete every foo that is followed by a bar (i.e. there is only whitespace between them), without ...
0 votes
0 answers
95 views

How do I perform this sed command with bash?

I have a file full of usernames and I want to use sed to change the usernames to PLAYER 1-20 how can I do this with a bash script and sed while incrementing? I have this so far: NAME=$(grep -q 'Seat' $...
2 votes
0 answers
45 views

SFTP user login details real-time filtering

I have enabled the SFTP login log into the default logfile /var/log/syslog and tried to filter the login time of each user and insert it into the database. But the filtering is not worked as I ...
1 vote
1 answer
1k views

Replacing a string when it is not followed by string with Sed

I am looking for a sed script to replace a string when it is not followed by a string in the content of a file. Currently I am at this approach sed -i -E "s/${filehash}(?!.de)/${filehash}.de/g&...
3 votes
1 answer
1k views

Remove spaces from first column of colon-delimited file

I have a very large colon-delimited file with two columns: dhsj fdksjdk:dndkdlkfkd shfdosod dkjsosjd:dkj dosjeod dhswisoaidodisbw: dhsoi wjdos fjfdododjfideefkjso sn:fhfdkd dkdjdf If the first ...
130 votes
18 answers
54k views

Substitution in text file **without** regular expressions

I need to substitute some text inside a text file with a replacement. Usually I would do something like sed -i 's/text/replacement/g' path/to/the/file The problem is that both text and replacement ...
1 vote
0 answers
290 views

Sed editing of large text file much faster than native bash substring replacement?

I have a ~19MB text file which I would like to perform many find/replace operations on. I originally wrote the script to loop over inplace sed operations e.g. sed -i "s|$pattern|$replacement|g&...
2 votes
3 answers
954 views

Is there a UNIX command that can replace an XML element? (Like a sed for an XPath?)

Is there a pipe-friendly program that takes: An XML document An XPath expression indicating the target element(s) The desired replacement ? Something like sed but knowledgeable about XML (...
4 votes
3 answers
9k views

Find and replace text in a file after match of pattern only for first occurrence using sed

Let's say I have the following input file: Server 'Test AB' option type 'ss' option port '1234' option timeout '60' Server 'Test CD' option type 'ss' option port '1234' option ...
2 votes
2 answers
111 views

What options are being used by sed to parse a file and write to new file

I am debugging a LINUX script that was written by someone else. In the script we have a line of code like below sed -i "\#</resources>#{h;s#.*#cat /opt/abc/new.xml#e;G}" /opt/abc/...
13 votes
3 answers
30k views

use sed to replace nbsp, 160, Hex 00a0, Octal 240, non-breaking space

I am having some problems with config files which have the non-breaking space chars in them. How should I specify that character with sed so I can replace it with a space. sed -n 's/ / /g' ...
1 vote
0 answers
211 views

pipe awk output in sed command

I have multiple lines in a file names.txt, let's say my name is Jim your name please what is good Harry potter how is he Jhony and me and so on , more than 500 lines. I use awk command to search/ ...
0 votes
0 answers
234 views

bash loop to create m3u file with sed awk grep

I am trying to auto create m3u file for my IPTV software with the help of sed/awk or grep command,but don't have much experience in this. expert help required to do this. working on Windows-10 PC ...
0 votes
1 answer
212 views

Trying to search & replace within brackets using sed

I have a json file which among other settings, contains the following data: "school": { "name": "qwerty", "key": "user", "secret"...
0 votes
1 answer
978 views

sed: unterminated `s' command – What does it mean?

I'm just learning this sed tool. So bear with me and appreciate your help. This works fine. $ date | sed 's/ /_/g' Mon_Mar_20_11:02:26_MST_2023 But not this one. $ date | sed 's/MST/EDT' sed: -e ...
0 votes
2 answers
456 views

How to combine tee, sed and grep?

I have a command after which I place | tee >(sed $'s/\033[[][^A-Za-z]*[A-Za-z]//g' >> ~user/filepath/file.txt) The tee is redirected to sed $'s/\033[[][^A-Za-z]*[A-Za-z]//g' which removes the ...
0 votes
1 answer
126 views

sed remove double quote pair spanning lines

In my OpenDKIM key file, I have extra quotes that can't go in "proper" DNS records. (I'm surprised the OpenDKIM devs allow this in their file, but that's another story.) In RegEx, it is two ...

1
2 3 4 5
21