sed replace by enviroment var content

I want to use sed to read a piece of text and replace some strings by
the value of that in the environment.
The input file looks like this:

blabla${PATH}blabla

Then sed needs to replace "${PATH}" by the value of "PATH" in the
environment

PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin

so sed should return

blabla/etc:/usr/sbin:/usr/bin/X11:/sbin:/usr/local/binblabla

(this is just an example to make it simple)
If I use:

sed -e "s|\${\(PATH\)}|${PATH}|g" < input.txt

then is will do the job.
but... (now comes the hard part)...
It needs to be generic for multiple environment variables.
So the "replace" part of the sed needs to use "\1" so it can use the
matching input.
(After this, I need to change the "search" part to search for other
names as well, but that is no problem)
So if I do:

sed -e "s|\${\(PATH\)}|${\1}|g" < input.txt

I get: "bad substitution"
But when I use:

sed -e "s|\${\(PATH\)}|\1|g" < input.txt

It DOES use the 1st parameter.
This means the "\1" and "${ }" are interfering with each other.
I've tried lots of different single and double quotes, but it just
doesn't seem to work as I want it to.

The shell should get the value of the parameter, but sed FIRST has to
put in the correct name.

I'm trying this on AIX, but the problem is probably similar on Linux.

Re: sed replace by enviroment var content

Ruud de Jong wrote:
> I want to use sed to read a piece of text and replace some strings by
> the value of that in the environment.
> The input file looks like this:
>
> blabla${PATH}blabla
>
> Then sed needs to replace "${PATH}" by the value of "PATH" in the
> environment
>
> PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
>
> so sed should return
>
> blabla/etc:/usr/sbin:/usr/bin/X11:/sbin:/usr/local/binblabla
>
> (this is just an example to make it simple)
> If I use:
>
> sed -e "s|\${\(PATH\)}|${PATH}|g" < input.txt
>
> then is will do the job.
> but... (now comes the hard part)...
> It needs to be generic for multiple environment variables.
> So the "replace" part of the sed needs to use "\1" so it can use the
> matching input.
> (After this, I need to change the "search" part to search for other
> names as well, but that is no problem)
> So if I do:
>
> sed -e "s|\${\(PATH\)}|${\1}|g" < input.txt
>
> I get: "bad substitution"
> But when I use:
>
> sed -e "s|\${\(PATH\)}|\1|g" < input.txt
>
> It DOES use the 1st parameter.
> This means the "\1" and "${ }" are interfering with each other.
> I've tried lots of different single and double quotes, but it just
> doesn't seem to work as I want it to.
>
> The shell should get the value of the parameter, but sed FIRST has to
> put in the correct name.
>
> I'm trying this on AIX, but the problem is probably similar on Linux.

Maybe you'll get a sed solution from someone, but here's how to do it in
awk for contrast:

$ cat file
blabla${PATH}blabla
foofoo${HOME}foofoo
$ PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
$ HOME=/home/morton
awk
'match($0,"[$]{.*}"){var=substr($0,RSTART+2,RLENGTH-3);gsub( "[$]{"var"}",ENVIRON[var])}1'
file
blabla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/u sr/local/binblabla
foofoo/home/mortonfoofoo

The only caveat is that the variables (PATH, HOME, etc.) must be
exported or set on the awk command line (HOME="$HOME" etc. awk ...} for
ENVIRON to work.

Ed.

Re: sed replace by enviroment var content

On 2005-12-07, Ruud de Jong wrote:
> I want to use sed to read a piece of text and replace some strings by
> the value of that in the environment.
> The input file looks like this:
>
> blabla${PATH}blabla
>
> Then sed needs to replace "${PATH}" by the value of "PATH" in the
> environment
>
> PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
>
> so sed should return
>
> blabla/etc:/usr/sbin:/usr/bin/X11:/sbin:/usr/local/binblabla
>
> (this is just an example to make it simple)
> If I use:
>
> sed -e "s|\${\(PATH\)}|${PATH}|g" < input.txt
>
> then is will do the job.
> but... (now comes the hard part)...
> It needs to be generic for multiple environment variables.
> So the "replace" part of the sed needs to use "\1" so it can use the
> matching input.
> (After this, I need to change the "search" part to search for other
> names as well, but that is no problem)
> So if I do:
>
> sed -e "s|\${\(PATH\)}|${\1}|g" < input.txt
>
> I get: "bad substitution"
> But when I use:
>
> sed -e "s|\${\(PATH\)}|\1|g" < input.txt
>
> It DOES use the 1st parameter.
> This means the "\1" and "${ }" are interfering with each other.
> I've tried lots of different single and double quotes, but it just
> doesn't seem to work as I want it to.
>
> The shell should get the value of the parameter, but sed FIRST has to
> put in the correct name.
>
> I'm trying this on AIX, but the problem is probably similar on Linux.

It's not sed, but this should work with bash3:

eval "cat <<< \"$(< "$filename")\" "


--
Chris F.A. Johnson, author |
Shell Scripting Recipes: | My code in this post, if any,
A Problem-Solution Approach | is released under the
2005, Apress | GNU General Public Licence

Re: sed replace by enviroment var content

Ed Morton wrote:
> Ruud de Jong wrote:
>
>> I want to use sed to read a piece of text and replace some strings by
>> the value of that in the environment.
>> The input file looks like this:
>>
>> blabla${PATH}blabla
>>
>> Then sed needs to replace "${PATH}" by the value of "PATH" in the
>> environment
>>
>> PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
>>
>> so sed should return
>>
>> blabla/etc:/usr/sbin:/usr/bin/X11:/sbin:/usr/local/binblabla
>>
>> (this is just an example to make it simple)
>> If I use:
>>
>> sed -e "s|\${\(PATH\)}|${PATH}|g" < input.txt
>>
>> then is will do the job.
>> but... (now comes the hard part)...
>> It needs to be generic for multiple environment variables.
>> So the "replace" part of the sed needs to use "\1" so it can use the
>> matching input.
>> (After this, I need to change the "search" part to search for other
>> names as well, but that is no problem)
>> So if I do:
>>
>> sed -e "s|\${\(PATH\)}|${\1}|g" < input.txt
>>
>> I get: "bad substitution"
>> But when I use:
>>
>> sed -e "s|\${\(PATH\)}|\1|g" < input.txt
>>
>> It DOES use the 1st parameter.
>> This means the "\1" and "${ }" are interfering with each other.
>> I've tried lots of different single and double quotes, but it just
>> doesn't seem to work as I want it to.
>>
>> The shell should get the value of the parameter, but sed FIRST has to
>> put in the correct name.
>>
>> I'm trying this on AIX, but the problem is probably similar on Linux.
>
>
> Maybe you'll get a sed solution from someone, but here's how to do it in
> awk for contrast:
>
> $ cat file
> blabla${PATH}blabla
> foofoo${HOME}foofoo
> $ PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
> $ HOME=/home/morton
> awk
> 'match($0,"[$]{.*}"){var=substr($0,RSTART+2,RLENGTH-3);gsub( "[$]{"var"}",ENVIRON[var])}1'
> file
> blabla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/u sr/local/binblabla
>
> foofoo/home/mortonfoofoo
>
> The only caveat is that the variables (PATH, HOME, etc.) must be
> exported or set on the awk command line (HOME="$HOME" etc. awk ...} for
> ENVIRON to work.
>
> Ed.
This works, but only for one variable per line. With 2 or more vars on
one line, it does not work.
The input I have, consists of one big line with multiple vars like:

$ cat file
blabla${PATH}blabla foofoo${HOME}foofoo

The caveat you mention, is not a problem (in my case).

Re: sed replace by enviroment var content

Chris F.A. Johnson wrote:
> On 2005-12-07, Ruud de Jong wrote:
> > I want to use sed to read a piece of text and replace some strings by
> > the value of that in the environment.
> > The input file looks like this:
> >
> > blabla${PATH}blabla
> >
> > Then sed needs to replace "${PATH}" by the value of "PATH" in the
> > environment
> >
> > PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
> >
> > so sed should return
> >
> > blabla/etc:/usr/sbin:/usr/bin/X11:/sbin:/usr/local/binblabla
> >
> > (this is just an example to make it simple)
> > If I use:
> >
> > sed -e "s|\${\(PATH\)}|${PATH}|g" < input.txt
> >
> > then is will do the job.
> > but... (now comes the hard part)...
> > It needs to be generic for multiple environment variables.
> > So the "replace" part of the sed needs to use "\1" so it can use the
> > matching input.
> > (After this, I need to change the "search" part to search for other
> > names as well, but that is no problem)
> > So if I do:
> >
> > sed -e "s|\${\(PATH\)}|${\1}|g" < input.txt
> >
> > I get: "bad substitution"
> > But when I use:
> >
> > sed -e "s|\${\(PATH\)}|\1|g" < input.txt
> >
> > It DOES use the 1st parameter.
> > This means the "\1" and "${ }" are interfering with each other.
> > I've tried lots of different single and double quotes, but it just
> > doesn't seem to work as I want it to.
> >
> > The shell should get the value of the parameter, but sed FIRST has to
> > put in the correct name.
> >
> > I'm trying this on AIX, but the problem is probably similar on Linux.
>
> It's not sed, but this should work with bash3:
>
> eval "cat <<< \"$(< "$filename")\" "

This should work for bash

$ while read line; do eval echo $line; done < file

Re: sed replace by enviroment var content

Ruud de Jong wrote:
> I want to use sed to read a piece of text and replace some strings by
> the value of that in the environment.
> The input file looks like this:
>
> blabla${PATH}blabla
>
> Then sed needs to replace "${PATH}" by the value of "PATH" in the
> environment

Don't use sed for this. Use this method instead (for bash) to convert
all these environment variable references (any any other it comes
across) and write to a new file:

$ while read line; do eval echo $line; done < file > newfile

Re: sed replace by enviroment var content

RolandRB wrote:
> Ruud de Jong wrote:
> > I want to use sed to read a piece of text and replace some strings by
> > the value of that in the environment.
> > The input file looks like this:
> >
> > blabla${PATH}blabla
> >
> > Then sed needs to replace "${PATH}" by the value of "PATH" in the
> > environment
>
> Don't use sed for this. Use this method instead (for bash) to convert
> all these environment variable references (any any other it comes
> across) and write to a new file:
>
> $ while read line; do eval echo $line; done < file > newfile

Echoing $line in double quotes is better.

$ while read line; do eval echo "$line"; done < file > newfile

Re: sed replace by enviroment var content

RolandRB wrote:
> RolandRB wrote:
>
>>Ruud de Jong wrote:
>>
>>>I want to use sed to read a piece of text and replace some strings by
>>>the value of that in the environment.
>>>The input file looks like this:
>>>
>>> blabla${PATH}blabla
>>>
>>>Then sed needs to replace "${PATH}" by the value of "PATH" in the
>>>environment
>>
>>Don't use sed for this. Use this method instead (for bash) to convert
>>all these environment variable references (any any other it comes
>>across) and write to a new file:
>>
>>$ while read line; do eval echo $line; done < file > newfile
>
>
> Echoing $line in double quotes is better.
>
> $ while read line; do eval echo "$line"; done < file > newfile
>
Thanks. This works. GREAT!

Re: sed replace by enviroment var content

Ruud de Jong wrote:
> Ed Morton wrote:
>
>>Ruud de Jong wrote:
>>
>>
>>>I want to use sed to read a piece of text and replace some strings by
>>>the value of that in the environment.
>>>The input file looks like this:
>>>
>>> blabla${PATH}blabla
>>>
>>>Then sed needs to replace "${PATH}" by the value of "PATH" in the
>>>environment
>>>
>>>
PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
>>>

>>Maybe you'll get a sed solution from someone, but here's how to do it in
>>awk for contrast:
>>
>>$ cat file
>>blabla${PATH}blabla
>>foofoo${HOME}foofoo
>>$ PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
>>$ HOME=/home/morton
>>awk
>>'match($0,"[$]{.*}"){var=substr($0,RSTART+2,RLENGTH-3);gsu b("[$]{"var"}",ENVIRON[var])}1'
>>file
>>blabla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin: /usr/local/binblabla
>>
>>foofoo/home/mortonfoofoo
>>
>>The only caveat is that the variables (PATH, HOME, etc.) must be
>>exported or set on the awk command line (HOME="$HOME" etc. awk ...} for
>>ENVIRON to work.
>>
>> Ed.
>
> This works, but only for one variable per line. With 2 or more vars on
> one line, it does not work.

Then just put it inside a while () loop and replace "." with "[^}]" to
get avoid greedy pattern matching:

$ cat file
blabla${PATH}blabla foofoo${HOME}foofoo
bla${PATH}bla${HOME}bla foo${HOME}foo
$ awk '{
while(match($0,"[$]{[^}]*}")){var=substr($0,RSTART+2,RLENGTH -3);gsub("[$]{"var"}",ENVIRON[var])}}1'
file
blabla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/u sr/local/binblabla
foofoo/home/mortonfoofoo
bla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/ local/binbla/home/mortonbla
foo/home/mortonfoo

> The input I have, consists of one big line with multiple vars like:
>
> $ cat file
> blabla${PATH}blabla foofoo${HOME}foofoo

Then that's the input you should've posted in the first place.

> The caveat you mention, is not a problem (in my case).

Good.

Ed.

Re: sed replace by enviroment var content

Ruud de Jong wrote:

> RolandRB wrote:
>
>>RolandRB wrote:
>>
>>
>>>Ruud de Jong wrote:
>>>
>>>
>>>>I want to use sed to read a piece of text and replace some strings by
>>>>the value of that in the environment.
>>>>The input file looks like this:
>>>>
>>>> blabla${PATH}blabla
>>>>
>>>>Then sed needs to replace "${PATH}" by the value of "PATH" in the
>>>>environment
>>>
>>>Don't use sed for this. Use this method instead (for bash) to convert
>>>all these environment variable references (any any other it comes
>>>across) and write to a new file:
>>>
>>>$ while read line; do eval echo $line; done < file > newfile
>>
>>
>>Echoing $line in double quotes is better.
>>
>>$ while read line; do eval echo "$line"; done < file > newfile
>>
>
> Thanks. This works. GREAT!

No, it doesn't. Look (and watch what happens to the "\t"):

$ cat file
blabla${PATH}bla\tbla
$ while read line; do eval echo "$line"; done < file
blabla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/u sr/local/binblatbla
$ awk '{
while(match($0,"[$]{[^}]*}")){var=substr($0,RSTART+2,RLENGTH -3);gsub("[$]{"var"}",ENVIRON[var])}}1'
file
blabla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/u sr/local/binbla\tbla

Notice that the eval solution changed "\t" to "t".

Ed.