51

With a single input file that only contains comments (starting with #) and VARIABLE=value lines, is it possible to replace a value for a single variable if found and, otherwise, append the pair to the end of file if not found?

My current method works by deleting it in a first pass, then appending it to the end of the file in a second pass, but this method messes up the line ordering (and is also two different commands):

sed -r "/^FOOBAR=.*$/d"      -i samefile &&
sed -r "$ a\FOOBAR=newvalue" -i samefile

Is there anyway to do this, ie. keeping line order, in a single sed line? If some other utility (awk, ...) does this, I'ld take it over sed.

0

11 Answers 11

56

It's actually quite simple with sed: if a line matches just copy it to the hold space then substitute the value.
On the la$t line exchange hold space and pattern space then check if the latter is empty. If it's not empty, it means the substitution was already made so nothing to do. If it's empty, that means no match was found so replace the pattern space with the desired variable=value then append to the current line in the hold buffer. Finally, exchange again:

sed '/^FOOBAR=/{h;s/=.*/=newvalue/};${x;/^$/{s//FOOBAR=newvalue/;H};x}' infile

The above is gnu sed syntax. Portable:

sed '/^FOOBAR=/{
h
s/=.*/=newvalue/
}
${
x
/^$/{
s//FOOBAR=newvalue/
H
}
x
}' infile
6
  • 1
    How would this look like if newvalue is stored in a variable? Dec 29, 2017 at 0:31
  • 1
    sed "/^${varName}=/{h;s/=.*/=${varValue}/};\${x;/^$/{s//${varName}=${varValue}/;H};x}" ${VARFILE} with variables, double quotes to allow for variable substitution and and escaping a $ that is not a substitution at this point, additionally if you value is stored in $$varName : sed "/^${varName}=/{h;s/=.*/=${!varName}/};\${x;/^$/{s//${varName}=${!varName}/;H};x}" ${VARFILE}
    – DarkMukke
    Apr 23, 2019 at 17:18
  • 7
    This solution is not 'simple' lol. But it works. sed man pages are not the easiest things to sift through, it would be really helpful if you could detail the effect of every section of the expression :) Jul 31, 2019 at 11:04
  • Great solution! If you prefer to keep existing line in place!
    – Alek
    Feb 15, 2020 at 18:45
  • 1
    the append will fail if the file has no lines like when a file was just created by touch.
    – DKebler
    Oct 30, 2020 at 19:27
36

Here is a simpler sed approach, as I don't find sed hold space easy to work with. If you are comfortable with hold space, using don_crissti's approach gives additional opportunity to preserve anything from the existing line and its position as well.

In the below approach, you just print all but the line that you want to drop and then at the end, append the replacement.

sed -n -e '/^FOOBAR=/!p' -e '$aFOOBAR=newvalue' infile

The line gets moved to the end of the file as pointed out by @alek, so make sure that this is acceptable in your case.

8
  • 2
    This version is very good if you plan to update a file that may have an existing but outdated value.
    – guillem
    Sep 30, 2018 at 14:37
  • 4
    Great answer! If you don't mind that the existing line is moved to the end. Thanks! I prefer to put it in one parameter: sed -nr '/^FOOBAR=/!p;$aFOOBAR=newvalue' infile
    – Alek
    Feb 15, 2020 at 18:38
  • 1
    This didn’t work for me as long as the file was completely empty (just created with touch) using sed (GNU sed) 4.7 Packaged by Debian. The file just stayed empty with no error message. May 21, 2020 at 5:13
  • @DanielBöhmer I just happened to hit this a couple of days back and understand that this is normal behavior for sed, considering how it is line oriented (when empty, there are no lines). One workaround is to start with a dummy line that will always be removed.
    – haridsv
    May 21, 2020 at 16:18
  • 1
    I tried this option and I couldn't get it to work with in-place editing (-i). It seemed to duplicate previous lines.
    – Danny
    Aug 3, 2021 at 13:01
20

This can probably be shortened. It's not a single sed command and it also uses grep, but this seems to be basically what you're wanting. It's a single line, and it edits the file in-place (no temp files).

grep -q "^FOOBAR=" file && sed "s/^FOOBAR=.*/FOOBAR=newvalue/" -i file || 
    sed "$ a\FOOBAR=newvalue" -i file
0
4

Simply use grep and echo to create an empty record :

grep -q '^FOOBAR=' somefile || echo 'FOOBAR=VALUE' >> somefile
sed -i 's/FOOBAR=.*$/FOOBAR=VALUE/' somefile 

Each line escapes with error code zero.

3
  • You lose line ordering while adding an extra commands and grep and echo
    – BlakBat
    Mar 8, 2016 at 15:46
  • Indeed if you insert the new items at the end of file, but keep ordering if you replace a old value. Mar 8, 2016 at 15:54
  • 3
    This solution is much more understandable to someone who doesn't have detailed knowledge of sed or grep
    – phsource
    May 29, 2021 at 22:46
2

Based on the other answers, if what you want to do is replace a variable's value if that variable is present in the file and append it to the end of the file if it is not (which is not what your posted sed commands do), you could try this:

perl -ne '$c=1 if s/^FOOBAR=.*$/FOOBAR=newvalue/;  
             print; 
             END{print "FOOBAR=newvalue" unless $c==1}' file > tmpfile && 
mv tmpfile file
1
  • You can use Perl’s -i switch to edit the file in place and save the hassle of managing a temp file May 10, 2020 at 9:35
0

It's a bit easier in awk, although the "in place editing" is not automatic:

awk -v varname="FOOBAR" -v newval="newvalue" '
    BEGIN {FS = OFS = "="}
    $1 == varname {$2 = newval; found = 1}
    {print}
    END {if (! found) {print varname, newval}}
' file > tempfile &&
mv tempfile file
2
0

As a programmer not accustomed to shell scripts it was easier to construct a more readable (for me) command by first deleting any existing match and then appending the value to the end of file

my-script.sh

CALCULATED_VALUE="some_end_result"

# delete any old value and append the new one
sed -i '' '/^MY_ENV_VAR/d' .env || true
echo "MY_ENV_VAR=${CALCULATED_VALUE}" >> .env

sed -i '' is because I'm on a macos, I think on ubuntu it'll be just sed -i


sed -i '' '/^MY_ENV_VAR/d' .env || true

Matches the line I want to update and deletes it if it exists. || true covers cases where a matching line does not exist in .env

echo "MY_ENV_VAR=${CALCULATED_VALUE}" >> .env

Adds the new line to the end of file

1
  • This is just a more convoluted way than my original post ;-) Also, '|| true' shouldn't be necessary (macOs specific?)
    – BlakBat
    May 1, 2022 at 13:08
-1

for me I tried most of the above solutions and they didn't work using sed (GNU sed) 4.4! So I tried something similar to @snapshoe solution but a bit different:

envVar=FOOBAR
envVal=newvalue
file=samefile
grep -q "^${envVar}=" ${file} && sed -i -e "s/^${envVar}=.*/${envVar}=${envVal}/g" ${file} || echo "${envVar}=${envVal}" >> ${file}

This will replace FOOBAR if found or append to end of file if not found

one line: envVar=FOOBAR && envVal=newvalue && file=samefile && grep -q "^${envVar}=" ${file} && sed -i -e "s/^${envVar}=.*/${envVar}=${envVal}/g" ${file} || echo "${envVar}=${envVal}" >> ${file}

3
  • Question asks for a one liner. Even @snapshoe didn't do it as a one-liner.
    – BlakBat
    Apr 25, 2021 at 18:06
  • @BlakBat you do realize that if you add && after each var instead of new line or if you replace the vars with string it is then one line! learn command line before down voting answers
    – Waqleh
    Apr 26, 2021 at 10:47
  • One line = One command. && means multiple commands. Learn to be agreable and not to post an anecdotal answer ("didn't work for me yada yada"). BTW, pretty much sure it works with sed 4.4 (used on debian:stretch)
    – BlakBat
    Apr 29, 2021 at 7:43
-1

sed hold can handle one entry only. I enhanced glenn jackman's answer to handle multiple entries:

awk 'BEGIN {FS = OFS = "="}{idx=0;
  varname[++idx]="FOOBAR1";newval[idx]="val1";
  varname[++idx]="FOOBAR2";newval[idx]="val2";
  ...
}
{for (i in varname){if ($1 == varname[i]){sub($2".*",newval[i]);found[i]=1;}}print}
END {for (i in varname){if (!found[i]) print varname[i], newval[i]}
}' file > tempfile && cp tempfile file && rm tempfile
1
  • This is an interesting contribution. However, beware of sub(). This will choke if the old value ($2) is a regular expression that doesn’t match itself (e.g., a[b] or 1+2), or if it is a substring of the variable name (e.g., cat=c or dog=o). Aug 24, 2021 at 6:30
-2

to do this with perl and in-place this works fine for me:

grep ^FOOBAR= my.file && perl -i -ple "s/^FOOBAR=.+/FOOBAR=newvalue/g" my.file || echo FOOBAR=newvalue >> my.file
1
  • 2
    This answer is bad in so many ways! grep and echo and perl instead of doing everything in perl? Also, writing to a file (>> my.file) as you're reading from it (grep my.file)?
    – vladr
    Nov 6, 2015 at 18:53
-2

Actually works with the following: sed -i '/^FOOBAR=/{h;s/=.*/=newvalue/};${x;/^$/{s//FOOBAR=newvalue/;H};x}' infile In the answer choosen -i is missing.

1

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .