Questions tagged [sed]
"sed" ("stream editor") is a Unix utility that parses and transforms text files.
1,039
questions
0
votes
2
answers
39
views
sed search replace regexp pattern to replace repeated strings in paths
I'm having some trouble using sed to replace a particular repeated portion of a file path, and I cannot for the life of me figure out how to get it to work.
For example, say I have a list of file ...
0
votes
0
answers
22
views
rstrip On Lines of text being piped through sed s
So I am trying to implement the rstrip function on lines of text being piped into sed s and am having troubles. This is what I have that I think should work:
cat file1.txt | sed 's/[[:blank:]]+$//g;' &...
0
votes
2
answers
50
views
Find and replace date with UTC date
I have file that looks like this:
[2024-01-20 15:23:00] hello world
[2024-01-20 15:42:00] bye
[2024-01-20 15:43:00] foo bar
...
Date in this file are in UTC+1 and I want to convert in to UTC
I tried ...
0
votes
1
answer
82
views
why in spite of increasing the physical size of my text file its .gz compressed size shows significant decrease in size
I have a large test file which its .gz format is about 13 GB. I have replaced a substring in the original file and compressed the file to .gz format.
To do so, I used the following Linux command:
zcat ...
0
votes
1
answer
41
views
How do I tell sed to stop trying to process commands?
Maybe I'm an idiot (this is the likely scenario, and I'm new to sed/bash scripting), but why does this work:
NewVersion="$(echo $NewVersionFile | sed 's/^.*-\([0-9]*\.[0-9]*\.[0-9]*\).*/\1/')&...
0
votes
3
answers
35
views
What's the history on the option separator of GNU sed?
They say GNU sed supports using any character following the command character as a separator for command options, like the following.
echo abc | sed 'y%abc%ABC%'
But I can't find the documentation ...
0
votes
1
answer
61
views
AWK replace some special new line to one line
i have some log like this:
2023-11-15T08:59:28.000000+00:00 database-1 # Time: 231115 8:59:28
# User@Host: rdsadmin[rdsadmin] @ localhost []
# Thread_id: 3 Schema: QC_hit: No
# Query_time: 0....
0
votes
1
answer
81
views
sed awk truncate to wrap a line and chose specific fields
I have 3 lines like this in a file
this is the first line blah1 blah2
this is the next line 0.101 0.202 0.303
this is the answer 0.404
Desired
this is the first line 0.202 0.303 -0.404
Can this be ...
0
votes
0
answers
133
views
Convert a 1 row (comma separated) text file to multiple rows
How can I convert text files with the following format:
["word01","word02","word03","word04","word05","word06","word07","...
0
votes
1
answer
56
views
How can I remove a fix substring from a UNIX/Linux output?
My question looks ridicilously simple, but:
I would like to remove an entire string, not a list of characters, so I can't use tr.
I don't know the position of my substring, so I can't use cut.
I'm ...
0
votes
1
answer
319
views
Moving files to a directory with spaces in its name
I'm trying to create a directory that has in its name the beginning of the names of the files it contains. I need this in order to archive it later. I have run into problems when moving files if the ...
1
vote
1
answer
359
views
Use sed to "format" first two letters of every word
I am trying to get to a sed command to format (HTML) the first two letters of every word from a plaintext input. I would like to input a plain text and mark the first two letters bold with the <b&...
0
votes
1
answer
513
views
need POSIX solution to remove special invisible characters from file
I have a file that comes to us from a third party (Windows) and is not in our control.
I observed that it has special characters like below:
root@DKERP:~# cat -ev ~/check_disk_space.sh
M-oM-;M-?#!/bin/...
0
votes
2
answers
891
views
Find and replace string in a file using 'sed' or 'awk'
Let's say I have the following input file:
yhara.runner.mng1.nna1.X9HCJG.1 yhara.runner.mng1.nna1.Z2HCJG.1 100.000 100
yhara.runner.mng1.nna1.X9HCJG.1 yhara.runner.mng1.nna1.AUEM0K.1 94.144 72
...
0
votes
3
answers
676
views
how to delete all words starting with a specific string in a file?
How can I delete all words in a file starting with a specific string (in this case the string "end") so that:
<region> sample=PDL UP IN-A-1.flac lokey=21 hikey=21 lovel=0 hivel=21 end=...
3
votes
2
answers
60
views
print specific line in all files in subfolders
I have a folder with several subfolders, each with several numeric subfolders.
├── child1
│ ├── 1
│ ├── 2
│ └── 3
├── child2
│ ├── 1
│ ├── 2
│ └── 3
└── child3
├── 1
├── 2
└── ...
0
votes
1
answer
99
views
sed: return all lines between matching patterns and quit
I want to extract only the lines that will restore a specific table from a mysql backup file. The following works.
zcat /mnt/backup/full-replication.gz | sed -n -e '/DROP TABLE.*`account_codes`/,/...
0
votes
0
answers
92
views
How do I perform this sed command with bash?
I have a file full of usernames and I want to use sed to change the usernames to PLAYER 1-20 how can I do this with a bash script and sed while incrementing?
I have this so far:
NAME=$(grep -q 'Seat' $...
2
votes
0
answers
44
views
SFTP user login details real-time filtering
I have enabled the SFTP login log into the default logfile /var/log/syslog and tried to filter the login time of each user and insert it into the database.
But the filtering is not worked as I ...
-1
votes
1
answer
152
views
write a efficient for loop in bash for multiple files (.PDB files)
I have problem with this script:
#!/bin/bash
for filename in /home/hb/pg/ensemble/pdbs/P24941/raw_pdb/*.pdb; do
grep COMPND $filename | grep "CHAIN:" -B 1 | sed 's/COMPND//g' | sed 's/...
3
votes
3
answers
564
views
How do I delete every ocurrence of a word that is followed by another, without altering the whitespace?
Suppose my file looks like this:
foo bar
foo bar bar
foo foo bar foo
What I want to do is delete every foo that is followed by a bar (i.e. there is only whitespace between them), without ...
1
vote
0
answers
285
views
Sed editing of large text file much faster than native bash substring replacement?
I have a ~19MB text file which I would like to perform many find/replace operations on. I originally wrote the script to loop over inplace sed operations e.g.
sed -i "s|$pattern|$replacement|g&...
2
votes
2
answers
109
views
What options are being used by sed to parse a file and write to new file
I am debugging a LINUX script that was written by someone else. In the script we have a line of code like below
sed -i "\#</resources>#{h;s#.*#cat /opt/abc/new.xml#e;G}" /opt/abc/...
1
vote
0
answers
207
views
pipe awk output in sed command
I have multiple lines in a file names.txt, let's say
my name is Jim your name please
what is good Harry potter
how is he Jhony and me
and so on , more than 500 lines.
I use awk command to search/ ...
0
votes
0
answers
230
views
bash loop to create m3u file with sed awk grep
I am trying to auto create m3u file for my IPTV software with the help of sed/awk or grep command,but don't have much experience in this. expert help required to do this. working on Windows-10 PC ...
0
votes
1
answer
193
views
Trying to search & replace within brackets using sed
I have a json file which among other settings, contains the following data:
"school": {
"name": "qwerty",
"key": "user",
"secret"...
0
votes
1
answer
953
views
sed: unterminated `s' command – What does it mean?
I'm just learning this sed tool. So bear with me and appreciate your help.
This works fine.
$ date | sed 's/ /_/g'
Mon_Mar_20_11:02:26_MST_2023
But not this one.
$ date | sed 's/MST/EDT'
sed: -e ...
0
votes
2
answers
436
views
How to combine tee, sed and grep?
I have a command after which I place | tee >(sed $'s/\033[[][^A-Za-z]*[A-Za-z]//g' >> ~user/filepath/file.txt) The tee is redirected to sed $'s/\033[[][^A-Za-z]*[A-Za-z]//g' which removes the ...
0
votes
1
answer
124
views
sed remove double quote pair spanning lines
In my OpenDKIM key file, I have extra quotes that can't go in "proper" DNS records. (I'm surprised the OpenDKIM devs allow this in their file, but that's another story.)
In RegEx, it is two ...
0
votes
1
answer
43
views
Linux - Find string and then end of section
I have an XMLTV file. I need to get all of the programme sections for a specific channel. An example snippet:
<programme start="2023031305000 -0400" end="2023031305300 -0400&...
0
votes
2
answers
230
views
How I can delete characters in text file lines using sed?
I have a text file containing urls.
1,https://cinesmovies/i/3juj2i,,
1,https://cinesmovies/i/3juj2j,,
1,https://cinesmovies/i/3juj2k,,
1,https://cinesmovies/i/3juj2l,,
I want to delete first 2 ...
2
votes
1
answer
46
views
How to remove similar elements from beginnign of each line in a text file using sed or similar?
I have text like this:
UgxJam3R6oHGHJMrbHx4AaABAg,,hqJGuSXaWcY,UCkhUJbKZIZsARJx7UsX7nw,NAME,https://URLYOUDONTNEEDTOSEE,2023-03-03T22:45:53,Sometextineed,1,false,0,
and I want to use sed or something ...
1
vote
1
answer
679
views
Add new content after specific string in text file using shell script
I want to add the below snippet after the string server { in nginx.conf file.
location /nginx-status {
stub_status on;
allow all;
}
The script below adds the snippet after server { wherever that ...
1
vote
2
answers
431
views
Delete lines between two patterns (with multiple occurrences) around a third pattern (single occurrence)
I'm trying to edit an uncompressed PDF on macOS Monterey, and I want to delete objects that contain a certain pattern in their description. Such objects start with "X 0 obj" (where X is the ...
1
vote
1
answer
47
views
How do I correctly execute 'vim $(grep -rli pattern | sed 's/\(.*\)/"&"/')' in bash?
I am trying to edit files that contain text matching a regular expression with vim and use the following command in bash to do that:
vim $(grep -rli pattern | sed 's/\(.*\)/"&"/')
I ...
0
votes
2
answers
140
views
How to group a flat list into categories and create a tab-separated value file using awk?
Crosspost: https://www.nixcraft.com/t/converting-a-list-into-a-tab-separated-file-grouped-by-values/4517
I have a text file with a list of values. The goal is to create a tab separated values file, ...
0
votes
2
answers
861
views
Remove comma and next 3 characters from line with sed
I'm writing a shell script to send fail2ban logs to a postgres database.
The event timestamps in the application log annoyingly shows the milliseconds of the event separated with a comma (like this: ...
0
votes
1
answer
659
views
Find and replace text in a file only after 2 different patterns match using `sed` or `awk`
This is an extension of the question asked-and-answered here: Find and replace text in a file after match of pattern only for first occurrence using sed
The problem being: I need to perform a find-and-...
3
votes
2
answers
705
views
sed: extract and print regexp match group
I'm trying to extract the IPv4 address of a network interface on Linux.
I have a working solution:
IFACE=eth0
ip a show dev $IFACE | sed -n 's/.*inet \(.*\)\/.*/\1/p'
For the record, here is a sample ...
0
votes
1
answer
1k
views
Bash Script - sed command for color
I have a log file and I would like to highlight some key words when it display the output.
So I like the word, myaccount and myfile.txt to be a different color so it stands out. This is what I have
...
1
vote
1
answer
141
views
sed in bash: '-n' as input is treated differently?
I'm struggling with a problem with sed in bash which occurs both on macOS and Ubuntu. During a long debugging session I pinpointed it down to this minimal example:
echo "abcde" | sed 's/.*/x/...
-1
votes
1
answer
101
views
how to use 'sed' in bash to delete a line from a specfic point on?
so i have these lines for example:
name,lastname,[email protected]
name2,lastname2,[email protected]
i need to delete the '[email protected]' in every line
essentially i want to make sed do "delete ...
2
votes
1
answer
254
views
More robust than sed for find and replace
I need to replace some strings in a file programatically. As a minimum case, I'm showing here just replacement for |FNAME| and |LNAME|.
Currently I am using this for the task:
sed -e "s/|FNAME|/$...
0
votes
0
answers
99
views
sed for back slash and tab char has error "unterminated `s' command"
I have a line in the middle has a "\\t", a back slash following a tab. I need to replace it with '\t' just a tab
sed -i -r 's/\\\t/\t/g' a.txt
it gives an error:
sed: -e expression #1, ...
0
votes
2
answers
159
views
sed replacement by regular expression
I would like to achieve to generate with sed from following source:
my_component_name_with_underlines_9.5.6
my_other_component_name_with_underlines_8.7.1
following target strings:
...
1
vote
3
answers
6k
views
Remove the lines before and after the exact match - linux
I have a file like this in /etc/test/host.conf :
# this is a test
host = example.com
private = 192.168.1.1
# end of test
# this is a test
host = example.com
private = 192.168.1.2
# end of test
I ...
0
votes
1
answer
2k
views
Find and replace the exact match in a file - change the value of something in a file - linux terminal
1) I have a file : /etc/testconfig/users.conf :
john1:*:$5$CJ5q52R0DPTbnvfddsa3IB/lofFuRoFnFc3OgKxkDP520PntI8pI76
john43:*:$5$cSV2tbDHA5645gdfN5w8W2TDMddyOPMBUR9nL7hQoVwF6QddElClKji/PB
54john:*:$5$...
0
votes
1
answer
53
views
How to Change Usernames in Password Store en Masse
I have been using pass for to store my passwords via GPG. I am trying to change an old username to a new username in dozens of accounts all at once.
Each username is in a "Username:" comment ...
-1
votes
3
answers
1k
views
sed or grep command to extract all lines that match a particular date and time range
2022-09-08T08:55:10+02:00
2022-09-08T19:55:16+02:00 200 200 76 0GET /v1/heartbeat/tcanalytics HTTP/1.1
2022-09-08T23:55:17+02:00 200 200 POST /...
0
votes
1
answer
293
views
Replace Host URL including square brackets
In Ubuntu 18.04 I am trying to replace a line hosts: ["localhost:9200"] with hosts:"http:xx.xx.xx:9200" by using the below command but it's not getting updated in the target file.
...