I have a whole bunch of files.
Using csplit I have managed to cut up a bunch
of files into more useful smaller files, now named
xx01 - xxN.

Using grep -m 1 HEADER* XX01 > filename, I can get
a certain line, such as HEADER 6-15-04 which I can copy to filename.

I have been trying to use mv XX01 to use the filename
contents as a name to mv that file to, but I don't seem
to be able to find the way to do that.

I am trying to set up a batch program to grep -m 1 * . filename,
mv * contents of filename to rename all all these xx* files
to the extracted HEADER+date name.

Being no expert at Bash, I am stuck at getting mv to use
the contents of filename as the file's new name.

What do I need to know?
many thanks.


--
When I shake my killfile I can hear them buzzing.

In article <11d863e1if6afa4 [at] corp.supernews.com>,
WCB <wbarwell [at] munnged.mylinuxisp.com> wrote:
>I have a whole bunch of files.
>Using csplit I have managed to cut up a bunch
>of files into more useful smaller files, now named
>xx01 - xxN.
>
>Using grep -m 1 HEADER* XX01 > filename, I can get
>a certain line, such as HEADER 6-15-04 which I can copy to filename.
>
>I have been trying to use mv XX01 to use the filename
>contents as a name to mv that file to, but I don't seem
>to be able to find the way to do that.
>
>I am trying to set up a batch program to grep -m 1 * . filename,
>mv * contents of filename to rename all all these xx* files
>to the extracted HEADER+date name.
>
>Being no expert at Bash, I am stuck at getting mv to use
>the contents of filename as the file's new name.
>
>What do I need to know?

First, you need to know *not* to use wildcards in an un-quoted pattern to grep.
Second, you need to know how to extract the field(s) you want from the line
output by grep.
Third, you need to know how to combine the grepped field(s) and the original
filename into a symbol you can use as the 2nd parameter to mv.


something like:

for file in XX*; do
NEWNAME=`grep -m 1 HEADER $file | awk -e "{print $2;}"`
mv $file $file.$NEWNAME
done

On 2005-07-12, WCB wrote:
> I have a whole bunch of files.
> Using csplit I have managed to cut up a bunch
> of files into more useful smaller files, now named
> xx01 - xxN.
>
> Using grep -m 1 HEADER* XX01 > filename, I can get
> a certain line, such as HEADER 6-15-04 which I can copy to filename.

Why copy it to a file? Just store it in a variable:

header=$(grep -m 1 'HEADER*' xx01)

Or store the result in the positional parameters, as in the final
script at the bottom of this post.

> I have been trying to use mv XX01 to use the filename

Why do use XX01 if the file is xx01?

> contents as a name to mv that file to, but I don't seem
> to be able to find the way to do that.

If you want to assign the first line of a file to a variable, use
read:

read var < FILENAME

> I am trying to set up a batch program to grep -m 1 * . filename,
> mv * contents of filename to rename all all these xx* files
> to the extracted HEADER+date name.

I'd recommend that you remove the space, and use "HEADER_6-15-04"
instead of "HEADER 6-15-04" as the filename. Since you are using
bash (it works in ksh93 as well), this can be accomplished with:

mv xx01 "${header// /_}"

In other shells:

set -f -- $header
mv xx01 "${1}_${2}"

To process all the files (for non-GNU greps, just omit "-m 1"):

for file in xx*
do
set -- $(grep -m 1 'HEADER*' "$file")
mv "$file" "${1}_${2}"
done

--
Chris F.A. Johnson <http://cfaj.freeshell.org>
============================================================ ======
Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
<http://www.torfree.net/~chris/books/cfaj/ssr.html>

Chris F.A. Johnson wrote:

> On 2005-07-12, WCB wrote:
>> I have a whole bunch of files.
>> Using csplit I have managed to cut up a bunch
>> of files into more useful smaller files, now named
>> xx01 - xxN.
>>
>> Using grep -m 1 HEADER* XX01 > filename, I can get
>> a certain line, such as HEADER 6-15-04 which I can copy to filename.
>
> Why copy it to a file? Just store it in a variable:
>
> header=$(grep -m 1 'HEADER*' xx01)
>

OK....


> Or store the result in the positional parameters, as in the final
> script at the bottom of this post.
>
>> I have been trying to use mv XX01 to use the filename
>
> Why do use XX01 if the file is xx01?
>

Typo, I meant xx01.

I can save the header to filename and read it using cat

>> contents as a name to mv that file to, but I don't seem
>> to be able to find the way to do that.
>
> If you want to assign the first line of a file to a variable, use
> read:
>
> read var < FILENAME
>
>> I am trying to set up a batch program to grep -m 1 * . filename,
>> mv * contents of filename to rename all all these xx* files
>> to the extracted HEADER+date name.
>
> I'd recommend that you remove the space, and use "HEADER_6-15-04"
> instead of "HEADER 6-15-04" as the filename. Since you are using
> bash (it works in ksh93 as well), this can be accomplished with:
>
> mv xx01 "${header// /_}"
>

OK...



> In other shells:

I am using bash.


>
> set -f -- $header
> mv xx01 "${1}_${2}"
>
> To process all the files (for non-GNU greps, just omit "-m 1"):
>

In my set up, -m 1 simply tells bash to take only the first line with the
word HEADER*, and ignore all other occurances of that string.
Which for my particular needs works well, as sometimes here other
occurances.



> for file in xx*
> do
> set -- $(grep -m 1 'HEADER*' "$file")
> mv "$file" "${1}_${2}"
> done

Sighhh. This didn't work. It moved the first file
and ate the rest. All 200 in my test directory.





--
When I shake my killfile I can hear them buzzing.

On 2005-07-12, WCB wrote:
> Chris F.A. Johnson wrote:
>
>> On 2005-07-12, WCB wrote:
>>> I have a whole bunch of files.
>>> Using csplit I have managed to cut up a bunch
>>> of files into more useful smaller files, now named
>>> xx01 - xxN.
>>>
>>> Using grep -m 1 HEADER* XX01 > filename, I can get
>>> a certain line, such as HEADER 6-15-04 which I can copy to filename.
>>
>> Why copy it to a file? Just store it in a variable:
>>
>> header=$(grep -m 1 'HEADER*' xx01)
>
> OK....
>
>> Or store the result in the positional parameters, as in the final
>> script at the bottom of this post.
>>
>>> I have been trying to use mv XX01 to use the filename
>>
>> Why do use XX01 if the file is xx01?
>
> Typo, I meant xx01.
>
> I can save the header to filename and read it using cat

There's no need for cat (an external command); use read, which is
built into the shell, and thus much faster.

>>> contents as a name to mv that file to, but I don't seem
>>> to be able to find the way to do that.
>>
>> If you want to assign the first line of a file to a variable, use
>> read:
>>
>> read var < FILENAME
>>
>>> I am trying to set up a batch program to grep -m 1 * . filename,
>>> mv * contents of filename to rename all all these xx* files
>>> to the extracted HEADER+date name.
>>
>> I'd recommend that you remove the space, and use "HEADER_6-15-04"
>> instead of "HEADER 6-15-04" as the filename. Since you are using
>> bash (it works in ksh93 as well), this can be accomplished with:
>>
>> mv xx01 "${header// /_}"
>>
>
> OK...
>
>> In other shells:
>
> I am using bash.

The more portable commands work just as well in bash, and have the
advantage that if you ever want to run it in another shell, you
needn't rewrite it.

>> set -f -- $header
>> mv xx01 "${1}_${2}"
>>
>> To process all the files (for non-GNU greps, just omit "-m 1"):
>>
>
> In my set up, -m 1 simply tells bash to take only the first line with the
> word HEADER*, and ignore all other occurances of that string.

No, it tells grep to stop after the first match; -m is an extension
peculiar to the GNU version of grep.

> Which for my particular needs works well, as sometimes here other
> occurances.
>
>> for file in xx*
>> do
>> set -- $(grep -m 1 'HEADER*' "$file")
>> mv "$file" "${1}_${2}"
>> done
>
> Sighhh. This didn't work. It moved the first file
> and ate the rest. All 200 in my test directory.

Then either the files are not as you described, or you mistyped the
script.

Are you sure that each file has a unique HEADER line?

--
Chris F.A. Johnson <http://cfaj.freeshell.org>
============================================================ ======
Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
<http://www.torfree.net/~chris/books/cfaj/ssr.html>

Chris F.A. Johnson wrote:

>>> for file in xx*

Check..


>>> do

Check

>>> set -- $(grep -m 1 'HEADER*' "$file")
^^
Spacing!

Check....



>>> mv "$file" "${1}_${2}"

Check.....

>>> done

Check....

All correct, case and punctuation, extra care given to
spaces.



Paste..


#!bash
# mover - moves files

for file in xx*
do
set -- $(grep -m 1 'HEADER*' "$file")
mv "$file" "${1}_${2}"

done

>>> set -- $(grep -m 1 'HEADER*' "$file")
>>> mv "$file" "${1}_${2}"

The same as far as I can see.

>>
>> Sighhh. This didn't work. It moved the first file
>> and ate the rest. All 200 in my test directory.
>
> Then either the files are not as you described, or you mistyped the
> script.
>

The script is correctly entered. This is bash in Mabdrake 10.1,
so that should be pretty standard.

Each file, for example XX21 is readable and has
the header.
Grep -m 1 "HEADER*" xx21 spits out the header I want as per design.

So the files exist, and have headers and grep works.

The script has been entered correctly.
I put it in /bin and ran it as root from a terminal.

I have some 200 files in a directory named test5.
No subdirectories.

Let me ask this.

I have grep -m 1 HEADER* xx21 that works.
How would I assign output of grep to a varible called newfilename?

And feed that to mv?

for i in `ls`; do grep make varible newfilename; mv $i $newfilename done


This bloody bash book I have does not even MENTION mv, or how to set a
simple varible like this. It has hit the wall three times in the last half
hour. Make that four times. It simply does not explain varibles in a
rational, useful manner.





> Are you sure that each file has a unique HEADER line?
>

--
When I shake my killfile I can hear them buzzing.

Chris F.A. Johnson wrote:

> On 2005-07-12, WCB wrote:
>> Chris F.A. Johnson wrote:
>>
>>> On 2005-07-12, WCB wrote:
>>>> I have a whole bunch of files.
>>>> Using csplit I have managed to cut up a bunch
>>>> of files into more useful smaller files, now named
>>>> xx01 - xxN.
>>>>
>>>> Using grep -m 1 HEADER* XX01 > filename, I can get
>>>> a certain line, such as HEADER 6-15-04 which I can copy to filename.
>>>
>>> Why copy it to a file? Just store it in a variable:
>>>
>>> header=$(grep -m 1 'HEADER*' xx01)
>>
>> OK....
>>
>>> Or store the result in the positional parameters, as in the final
>>> script at the bottom of this post.
>>>
>>>> I have been trying to use mv XX01 to use the filename
>>>
>>> Why do use XX01 if the file is xx01?
>>
>> Typo, I meant xx01.
>>
>> I can save the header to filename and read it using cat
>
> There's no need for cat (an external command); use read, which is
> built into the shell, and thus much faster.
>
>>>> contents as a name to mv that file to, but I don't seem
>>>> to be able to find the way to do that.
>>>
>>> If you want to assign the first line of a file to a variable, use
>>> read:
>>>
>>> read var < FILENAME
>>>
>>>> I am trying to set up a batch program to grep -m 1 * . filename,
>>>> mv * contents of filename to rename all all these xx* files
>>>> to the extracted HEADER+date name.
>>>
>>> I'd recommend that you remove the space, and use "HEADER_6-15-04"
>>> instead of "HEADER 6-15-04" as the filename. Since you are using
>>> bash (it works in ksh93 as well), this can be accomplished with:
>>>
>>> mv xx01 "${header// /_}"
>>>
>>
>> OK...
>>
>>> In other shells:
>>
>> I am using bash.
>
> The more portable commands work just as well in bash, and have the
> advantage that if you ever want to run it in another shell, you
> needn't rewrite it.
>
>>> set -f -- $header
>>> mv xx01 "${1}_${2}"
>>>
>>> To process all the files (for non-GNU greps, just omit "-m 1"):
>>>
>>
>> In my set up, -m 1 simply tells bash to take only the first line with the
>> word HEADER*, and ignore all other occurances of that string.
>
> No, it tells grep to stop after the first match; -m is an extension
> peculiar to the GNU version of grep.
>
>> Which for my particular needs works well, as sometimes here other
>> occurances.
>>
>>> for file in xx*
>>> do
>>> set -- $(grep -m 1 'HEADER*' "$file")
>>> mv "$file" "${1}_${2}"
>>> done
>>
>> Sighhh. This didn't work. It moved the first file
>> and ate the rest. All 200 in my test directory.
>
> Then either the files are not as you described, or you mistyped the
> script.
>
> Are you sure that each file has a unique HEADER line?
>

Yes. Each file has a unique header.

On the lines of "HCO BULLETIN 12 September 78".
Some may say HCO BULLETIN, some HCO POLICY LETTER,
all are on one line with an ending date.

Grep -m 1 works well on each one as expected.
I can run this on a given file xx45 say, and it works
as expected.

I then cleaned out my test directory and loaded only five test
files.
I tested all with grep -m 1 *HEADER xxN and tested
all five files. Alll worked as expected, I got the expected
string.

Each has a unique nam xx01 - xxN.
I cut and pasted the script and it still doesn't work.

Starting with 5 files, I ended up with two.

The rest were eaten.
Not even fur left behind.
The files that were written were single files, its not
stuffing them all together into a fie.

I will try some variations.








--
When I shake my killfile I can hear them buzzing.

Chris F.A. Johnson wrote:

>>>
>>> read var < FILENAME
>>>
>>>> I am trying
>>
>> I am using bash.
>
> The more portable commands work just as well in bash, and have the
> advantage that if you ever want to run it in another shell, you
> needn't rewrite it.
>
>>> set -f -- $header
>>> mv xx01 "${1}_${2}"
>>>
>>> To process all the files (for non-GNU greps, just omit "-m 1"):
>>>
>>
>> In my set up, -m 1 simply tells bash to take only the first line with the
>> word HEADER*, and ignore all other occurances of that string.
>
> No, it tells grep to stop after the first match; -m is an extension
> peculiar to the GNU version of grep.
>
>> Which for my particular needs works well, as sometimes here other
>> occurances.
>>
>>> for file in xx*
>>> do
>>> set -- $(grep -m 1 'HEADER*' "$file")
>>> mv "$file" "${1}_${2}"
>>> done
>>
>> Sighhh. This didn't work. It moved the first file
>> and ate the rest. All 200 in my test directory.
>
> Then either the files are not as you described, or you mistyped the
> script.
>
> Are you sure that each file has a unique HEADER line?
>

Got it! What was happening is the headers were more like Bulletin of
12 May 1987, lots of white space. So what was going on was it was taking
"Bulletin of" making that "Bulletin-of" and overwriting it as they were
generally mostly "Bulletin-of".

So I'd get "Bulletin-of" and "Bulletin-Of" and little else. So the cure was
first a sed script to get rid of the leading spaces and sed 's/ /_/g' to
replace all spaces with a _. The little g makes it
Bulletin_of_May_23_1968_Revision which is unique and doesn't
over write other files.

So it was an overwriting problem due to a missing g.
Thanks for your patience and help!

And I figured out that var=$(cat somefile) gets me $var extracted from
somefile, so mv xx07 $var then works. Which is what I was looking for as
a lot of commands can now be used in simple functions now
that I finally got that figured out. Or read var < name.

This is the basic wheel none of my books deigned to explain.
Nor for that matter sed 's/^ *//'. My Bash book actually gets by without
even having mv or cp anywhere in the book.

Much less how to pass variables to them.
Damn I wish I knew this stuff so I could right a decent
beginner's book for people who want to use this stuff
quickly without frustration.

The only good thing I can say about my Bash books is that the
Perl books are worse.

Thanks again for your help.





--
When I shake my killfile I can hear them buzzing.

On 2005-07-12, WCB wrote:
> Chris F.A. Johnson wrote:
>
>>>> for file in xx*
>
> Check..
>
>
>>>> do
>
> Check
>
>>>> set -- $(grep -m 1 'HEADER*' "$file")
> ^^
> Spacing!
>
> Check....
>
>
>
>>>> mv "$file" "${1}_${2}"
>
> Check.....
>
>>>> done
>
> Check....
>
> All correct, case and punctuation, extra care given to
> spaces.
>
>
>
> Paste..
>
>
> #!bash

I think you mean:

#! /bin/bash

but it's not necessary.

> # mover - moves files
>
> for file in xx*
> do
> set -- $(grep -m 1 'HEADER*' "$file")
> mv "$file" "${1}_${2}"
>
> done
>
>>>> set -- $(grep -m 1 'HEADER*' "$file")
>>>> mv "$file" "${1}_${2}"
>
> The same as far as I can see.
>
>>> Sighhh. This didn't work. It moved the first file
>>> and ate the rest. All 200 in my test directory.
>>
>> Then either the files are not as you described, or you mistyped the
>> script.
>
> The script is correctly entered.

Don't enter it; cut and paste it.

> This is bash in Mabdrake 10.1, so that should be pretty standard.

...and that shows why.

> Each file, for example XX21 is readable and has
> the header.

If you can't be consistent (xx21, not XX21), then it is not going
to work.

> Grep -m 1 "HEADER*" xx21 spits out the header I want as per design.
>
> So the files exist, and have headers and grep works.
>
> The script has been entered correctly.
> I put it in /bin and ran it as root from a terminal.

Don't run it as root. Always test your script with a non-root user.
And run the final version as an ordinary user, too, unless there's
a real need to do it as root (which is unlikely).

> I have some 200 files in a directory named test5.
> No subdirectories.
>
> Let me ask this.
>
> I have grep -m 1 HEADER* xx21 that works.
> How would I assign output of grep to a varible called newfilename?

See my previous post.

> And feed that to mv?
>
> for i in `ls`; do grep make varible newfilename; mv $i $newfilename done

There is no need for ls, and in some cases it will break your
script. Use:

for i in *


> This bloody bash book I have does not even MENTION mv,

Well, mv is not part of bash. (What book is it?)

> or how to set a simple varible like this.

Look for "command substitution".

> It has hit the wall three times in the last half
> hour. Make that four times. It simply does not explain varibles in a
> rational, useful manner.

To see exactly what your script is doing, put "set -x" at the top.


--
Chris F.A. Johnson <http://cfaj.freeshell.org>
============================================================ ======
Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
<http://www.torfree.net/~chris/books/cfaj/ssr.html>

On 2005-07-13, WCB wrote:
> Chris F.A. Johnson wrote:
>
>>> Sighhh. This didn't work. It moved the first file
>>> and ate the rest. All 200 in my test directory.
>>
>> Then either the files are not as you described, or you mistyped the
>> script.
>>
>> Are you sure that each file has a unique HEADER line?
>>
>
> Got it! What was happening is the headers were more like Bulletin of
> 12 May 1987, lots of white space. So what was going on was it was taking
> "Bulletin of" making that "Bulletin-of" and overwriting it as they were
> generally mostly "Bulletin-of".
>
> So I'd get "Bulletin-of" and "Bulletin-Of" and little else. So the cure was
> first a sed script to get rid of the leading spaces and sed 's/ /_/g' to
> replace all spaces with a _. The little g makes it
> Bulletin_of_May_23_1968_Revision which is unique and doesn't
> over write other files.

You don't need sed if you are using bash:

file="xxx yyy zzz 111 222"
newfile=${file// /_}

> So it was an overwriting problem due to a missing g.
> Thanks for your patience and help!
>
> And I figured out that var=$(cat somefile) gets me $var extracted from
> somefile, so mv xx07 $var then works.

You don't need cat; just read:

read var < somefile

> Which is what I was looking for as
> a lot of commands can now be used in simple functions now
> that I finally got that figured out. Or read var < name.

Exactly. It's much faster than cat.

> This is the basic wheel none of my books deigned to explain.
> Nor for that matter sed 's/^ *//'. My Bash book actually gets by without
> even having mv or cp anywhere in the book.

They are not part of bash. Read the man pages for mv and cp.

> Much less how to pass variables to them.
> Damn I wish I knew this stuff so I could right a decent
> beginner's book for people who want to use this stuff
> quickly without frustration.
>
> The only good thing I can say about my Bash books is that the
> Perl books are worse.

Which bash books?

--
Chris F.A. Johnson <http://cfaj.freeshell.org>
============================================================ ======
Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
<http://www.torfree.net/~chris/books/cfaj/ssr.html>

Chris F.A. Johnson wrote:

>> This bloody bash book I have does not even MENTION mv,
>
> Well, mv is not part of bash. (What book is it?)


O'Reilly Learning the Bash shell

>
>> or how to set a simple varible like this.
>
> Look for "command substitution".

A few pages, not well written, dense with drivel
with a very high "Hunh?" factor.

Nothing at all obvious to do with what I am trying to achieve.
If I can't get there from here with mv, the $!$!$! should
tell me that right up front and what to do about it.

Instead it has all the vi and emacs stuff and other trash I don't
need, not what I do need. Incredibly frustrating.



>> It has hit the wall three times in the last half
>> hour. Make that four times. It simply does not explain varibles in a
>> rational, useful manner.
>
> To see exactly what your script is doing, put "set -x" at the top.


My problem is this. I have a script.

It gerps the string I want, dumps the white space,
add's the "_"s to get rid of internal white space.

Using echo, I can see that it lists my files, xx001, xx002,
and so on. If lists my file name string $F, Bulletin_of_3_September_1965,
and so on.

The problem now is when I get to the next to the last line.

mv $i $F

It does not actually move anything.

I can type mv xx07 Bulletin_of_12_september_1965 hit enter and it
moves the file.
The script echos the same but it does not actually move anything.

Why this does not work as it seems it would I haven't a clue.

why mv $i F$ echos to perfect but does not actually consumate
the command and move the file I do not know, and this is my source of
frustration.

These books do not tell me the syntax, why some stuff will
work and when it won't and how to emulate common things
like mv.
I could wrie functions if stuff like this worked.
Instead I write stuff that SHOULD and it doesn't.
Which is what I need.

These books offer not a single clue how to achieve it either directly or
emulating commands that do not work as expected.

I can echo my files,

xx07
Bulletin_of_12_September_1965
xx08
Bulletin-of-24_April_1965

and on and on.

I can echo mv xx07 Bulletin_of_12_september_1965
et al on and on.

But when I ls the directory, nothing has actually mv'd.
My books are totally useless as to actually writing a real
world useful script.

They offer not a clue why this will not work.

Their Perl books are also useless in similar ways.
My last O'Reilly books.

I have been googling, finding a bit here, a bit there,
cobbling it together, and watching as at the last line,
mv fails.

And I haven't the faintest clue why mv $i $F echos
properly as I expect and mv fails utterly.

This is not fun.

We have all sorts of wonderful pipeline stuff and redirection.
Doesn't work with mv. No book every warns of that, much
less explains why, much less tells what to do to get around it.
Its killing me. A simple task, so it seems just turns into a
vast nightmare of futility and extremely hard work.

I have piles and piles of books. Not a one worth dirt when
it comes to getting this done.

This is killing me.

mv $i $F
done.

Here it all fails.
--
When I shake my killfile I can hear them buzzing.

On 2005-07-13, WCB wrote:
> Chris F.A. Johnson wrote:
>
>>> This bloody bash book I have does not even MENTION mv,
>>
>> Well, mv is not part of bash. (What book is it?)
>
> O'Reilly Learning the Bash shell
>
>>> or how to set a simple varible like this.
>>
>> Look for "command substitution".
>
> A few pages, not well written, dense with drivel
> with a very high "Hunh?" factor.
>
> Nothing at all obvious to do with what I am trying to achieve.
> If I can't get there from here with mv, the $!$!$! should
> tell me that right up front and what to do about it.
>
> Instead it has all the vi and emacs stuff and other trash I don't
> need, not what I do need. Incredibly frustrating.

The first place to look is always the man page for the command. At
the command prompt, type "man mv".

>>> It has hit the wall three times in the last half
>>> hour. Make that four times. It simply does not explain varibles in a
>>> rational, useful manner.
>>
>> To see exactly what your script is doing, put "set -x" at the top.
>
>
> My problem is this. I have a script.
>
> It gerps the string I want, dumps the white space,
> add's the "_"s to get rid of internal white space.
>
> Using echo, I can see that it lists my files, xx001, xx002,
> and so on. If lists my file name string $F, Bulletin_of_3_September_1965,
> and so on.
>
> The problem now is when I get to the next to the last line.
>
> mv $i $F

Quote your variables (it shouldn't matter in this instance, but
it's a good habit to get into):

mv "$i" "$F"

> It does not actually move anything.

What _does_ happen?

> I can type mv xx07 Bulletin_of_12_september_1965 hit enter and it
> moves the file.
> The script echos the same but it does not actually move anything.
>
> Why this does not work as it seems it would I haven't a clue.
>
> why mv $i F$ echos to perfect but does not actually consumate
> the command and move the file I do not know, and this is my source of
> frustration.
>
> These books do not tell me the syntax, why some stuff will
> work and when it won't and how to emulate common things
> like mv.
> I could wrie functions if stuff like this worked.
> Instead I write stuff that SHOULD and it doesn't.
> Which is what I need.
>
> These books offer not a single clue how to achieve it either directly or
> emulating commands that do not work as expected.
>
> I can echo my files,
>
> xx07
> Bulletin_of_12_September_1965
> xx08
> Bulletin-of-24_April_1965
>
> and on and on.
>
> I can echo mv xx07 Bulletin_of_12_september_1965
> et al on and on.
[snip]
> I have been googling, finding a bit here, a bit there,
> cobbling it together, and watching as at the last line,
> mv fails.
>
> And I haven't the faintest clue why mv $i $F echos
> properly as I expect and mv fails utterly.

Utterly? What exactly does "fails utterly" mean?

Do none of the files get moved? Are the xxNN files all stll there?

Put echo at the beginning of the mv line, save the output and post
it here, along with the script you used. Insert the files, both
script output and the script itself; do not retype anything.

> I have piles and piles of books. Not a one worth dirt when
> it comes to getting this done.

I could recommend another book, but it's not an impartial
recommendation; see my .sig for details. You might find it more
useful as it contains more than 200 examples of scripts and
functions.

--
Chris F.A. Johnson <http://cfaj.freeshell.org>
============================================================ ======
Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
<http://www.torfree.net/~chris/books/cfaj/ssr.html>

Chris F.A. Johnson wrote:

> On 2005-07-13, WCB wrote:
>> Chris F.A. Johnson wrote:
>>
>> I can echo my files,
>>
>> xx07
>> Bulletin_of_12_September_1965
>> xx08
>> Bulletin-of-24_April_1965
>>
>> and on and on.
>>
>> I can echo mv xx07 Bulletin_of_12_september_1965
>> et al on and on.
> [snip]
>> I have been googling, finding a bit here, a bit there,
>> cobbling it together, and watching as at the last line,
>> mv fails.
>>
>> And I haven't the faintest clue why mv $i $F echos
>> properly as I expect and mv fails utterly.
>
> Utterly? What exactly does "fails utterly" mean?
>
> Do none of the files get moved? Are the xxNN files all stll there?

Nothing moves, xxNN files are unchanged


> Put echo at the beginning of the mv line, save the output and post
> it here, along with the script you used. Insert the files, both
> script output and the script itself; do not retype anything.

A bit hard, I wiped out all the scripts and went back to the one you
originally posted. I'd have to reconstruct it from notes I made.


#!/bin/bash
# moving - pasted


for file in xx*
do
set -- $(grep -m 1 'HCO*' "$file")
mv "$file" "${1}_${2}_${3}_${4}_${5}_${6}_${7}"

done

I expanded it to put "_" between elements

The problem it leaves artifacts at the end of file names


HCO_BULLETIN_OF_12_JUNE_1965?

It looks like a question mark in the terminal, but a
square box in KDE's Kwrite.
Apparently some non-alphanumeric character.

Sed won't touch it. Since some variation in files
means I use ${7}, I end up with an extra _ on a few.
Since Sed can't parse ? I can't remove the extra _ either
to get rid of the trailing "_".

This ALMOST works. I have been trying to parse this with
the help of Linux in a nutshell but it doesn't have depth
of explanation for me to debug this one.
The learning Bash doesn't detail why this artifact problem
might occur either.

But its getting close.

In my original script


grep -m 1 HCO* $i > filex extracted the name.
sed 's/* ^//' filex > filey got rid of leading zeros
sed 's/ /_/g' filey > filez replaced the spaces with "_".

Filez had the name so filez gave me the file name
with no spaces, no artifacts.

My original program went something like

for i in `ls`;
do

grep -m 1 *HCO $i > filex
sed 's/^ *//' filey > filey
sed 's/ /_/g' filez >filez

f=$(cat filez)

mv $i $f

done

Echo showed something like

mv xx05 HCO_Bulletin_Of_12_June_1965
and so on. just no actual move.
A real head scratcher. Maybe a subtle syntax error.

I tried modifying your script to
run grep, sed, sed on one line and failed.
Since I am not really aware of what is happening
with "${1}_${2}" et al.

--
When I shake my killfile I can hear them buzzing.

Chris F.A. Johnson wrote:

>
> I could recommend another book, but it's not an impartial
> recommendation; see my .sig for details. You might find it more
> useful as it contains more than 200 examples of scripts and
> functions.
>


More...

I set up a script

mv xx07 file7
This works
no variables

file8="xx08"
mv $file8 file8
This works
1 variable, left hand side

file="file9"
mv xx09 $file
This works
1 variable right hand side

xx="xx10"
file+"file10"
mv $xx $file
This works
2 variables both sides


echo shows my for / done
script is getting good values
for $i and $f

echo $f >> f
echo $i >> i

This shows me my xxNN and file names are
good and extraced as expected.

My echo statement shows me mv xxNN filename
is generated as expected.

The only thing not working is the for do loop
is not for some reason actually finalizing the mv $i $f
action as expected.

My first big attempt at a script to do real work and I find
a big fat bug.
I find cp is the same as mv, so it must be
the for do done structure.

It all works until I wrap in a for do done loop.

So I get to write my first bug report for a serious
Linux command it looks like. Surely I am not the
first to run across this.


What works on a stand alone basis with mv
not work in a for done loop.

And neither does cp either.
So I am not crazy, there is some sort of bug
here apparently.




--
When I shake my killfile I can hear them buzzing.

You know me. If you want a beginners guide then I can send you
something.

WCB wrote:
> Chris F.A. Johnson wrote:
>
>
>>On 2005-07-13, WCB wrote:
>>
>>>Chris F.A. Johnson wrote:
>>>
>>>I can echo my files,
>>>
>>>xx07
>>>Bulletin_of_12_September_1965
>>>xx08
>>>Bulletin-of-24_April_1965
>>>
>>>and on and on.
>>>
>>>I can echo mv xx07 Bulletin_of_12_september_1965
>>>et al on and on.
>>
>>[snip]
>>
>>>I have been googling, finding a bit here, a bit there,
>>>cobbling it together, and watching as at the last line,
>>>mv fails.
>>>
>>>And I haven't the faintest clue why mv $i $F echos
>>>properly as I expect and mv fails utterly.
>>
>> Utterly? What exactly does "fails utterly" mean?
>>
>> Do none of the files get moved? Are the xxNN files all stll there?
>
>
> Nothing moves, xxNN files are unchanged
>
>
>
>> Put echo at the beginning of the mv line, save the output and post
>> it here, along with the script you used. Insert the files, both
>> script output and the script itself; do not retype anything.
>
>
> A bit hard, I wiped out all the scripts and went back to the one you
> originally posted. I'd have to reconstruct it from notes I made.
>
>
> #!/bin/bash
> # moving - pasted
>
>
> for file in xx*
> do
> set -- $(grep -m 1 'HCO*' "$file")
> mv "$file" "${1}_${2}_${3}_${4}_${5}_${6}_${7}"
>
> done
>
> I expanded it to put "_" between elements
>
> The problem it leaves artifacts at the end of file names
>
>
> HCO_BULLETIN_OF_12_JUNE_1965?
>
> It looks like a question mark in the terminal, but a
> square box in KDE's Kwrite.
> Apparently some non-alphanumeric character.

Obvious questions:

a) does that extra character appear in the source file xxXX?
b) do you see that char when you echo "$7"?

You said that "mv" and "cp" both just silently complete without moving
the file. What happens when you:

c) echo "$?" after the command?
d) try using cat "$file" > "${1}_${2}_${3}_${4}_${5}_${6}_${7}" instead?

At this point, I think it'd be useful for you to create and post, say, 2
sample files xx01 and xx02 along with your EXACT script and EXACT stdout
and stderr along with an "ls -al" of your test directory before and
after running your script so you could copy/paste this script and post
the resulting stdout and stderr:

pwd
ls -al
for file in xx*
do
printf "---- %s:\n" "$file"
cat "$file"
printf "----\n"
set -- $(grep -m 1 'HCO*' "$file")
printf "<%s:%s:%s:%s:%s:%s:%s>\n" "$1" "$2" "$3" "$4" "$5" "$6"
"$7"
mv "$file" "mv_${1}_${2}_${3}_${4}_${5}_${6}_${7}"
printf "mv result: %s\n" "$?"

cp "$file" "cp_${1}_${2}_${3}_${4}_${5}_${6}_${7}"
printf "cp result: %s\n" "$?"

cat "$file" > "cat_${1}_${2}_${3}_${4}_${5}_${6}_${7}"
printf "cat result: %s\n" "$?"

done
ls -al

Regards,

Ed.

Chris F.A. Johnson wrote:

> Do none of the files get moved? Are the xxNN files all stll there?
>
> Put echo at the beginning of the mv line, save the output and post
> it here, along with the script you used. Insert the files, both
> script output and the script itself; do not retype anything.
>
>> I have piles and piles of books. Not a one worth dirt when
>> it comes to getting this done.
>
> I could recommend another book, but it's not an impartial
> recommendation; see my .sig for details. You might find it more
> useful as it contains more than 200 examples of scripts and
> functions.
>


The script..


#!/bin/bash
# mover2

for i in xx*;
do

(grep -m 1 HCO* $i) > x;
(sed 's/^ *//' x) > y;
(sed 's/ /-/g' y) > z;

f=$(cat z);
echo mv $i $f; >> a

done

****************

Contents of a


mv xx101 HCO-POLICY-LETTER-OF-1-DECEMBER-1962
mv xx06 HCO-BULLETIN-OF-25-JANUARY-1962
mv xx07 HCO-BULLETIN-OF-1-FEBRUARY-1962
mv xx08 HCO-INFORMATION-LETTER-OF-1-FEBRUARY-1962
mv xx09 HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
mv xx10 HCO-BULLETIN-OF-8-FEBRUARY-1962
mv xx100 HCO-BULLETIN-OF-29-NOVEMBER-AD12
mv xx101 HCO-POLICY-LETTER-OF-1-DECEMBER-1962

Its as if it needs an enter at the end of each line.

Removing the ; at ends of line does not help.



--
When I shake my killfile I can hear them buzzing.

On 2005-07-14, WCB wrote:
> Chris F.A. Johnson wrote:
>
>> Do none of the files get moved? Are the xxNN files all stll there?
>>
>> Put echo at the beginning of the mv line, save the output and post
>> it here, along with the script you used. Insert the files, both
>> script output and the script itself; do not retype anything.
>
> The script..
>
> #!/bin/bash
> # mover2
>
> for i in xx*;
> do
>
> (grep -m 1 HCO* $i) > x;
> (sed 's/^ *//' x) > y;
> (sed 's/ /-/g' y) > z;
>
> f=$(cat z);

f=$(grep -m 1 HCO* "$i" | sed -e 's/^ *//' -e 's/ /-/g')

Or:

f=$(grep -m 1 HCO* "$i")
f=${f// /-}
f=${f//--/-}

> echo mv $i $f; >> a
> done

It doesn't make a difference in this instance, but quoting your
variables is a good habit to get into:

echo mv "$i" "$f"
done > a

> ****************
>
> Contents of a
>
>
> mv xx101 HCO-POLICY-LETTER-OF-1-DECEMBER-1962
> mv xx06 HCO-BULLETIN-OF-25-JANUARY-1962
> mv xx07 HCO-BULLETIN-OF-1-FEBRUARY-1962
> mv xx08 HCO-INFORMATION-LETTER-OF-1-FEBRUARY-1962
> mv xx09 HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
> mv xx10 HCO-BULLETIN-OF-8-FEBRUARY-1962
> mv xx100 HCO-BULLETIN-OF-29-NOVEMBER-AD12
> mv xx101 HCO-POLICY-LETTER-OF-1-DECEMBER-1962

What happens if you:

bash ./a

> Its as if it needs an enter at the end of each line.

There is.

> Removing the ; at ends of line does not help.

They serve no purpose; leave them out.


What environment are you running this in?

--
Chris F.A. Johnson <http://cfaj.freeshell.org>
============================================================ ======
Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
<http://www.torfree.net/~chris/books/cfaj/ssr.html>

"WCB" <wbarwell [at] Mungggedd.mylinuxisp.com> wrote in message news:11dcq4bft75kk18 [at] corp.supernews.com...
> Chris F.A. Johnson wrote:
>
> > Do none of the files get moved? Are the xxNN files all stll there?
> >
> > Put echo at the beginning of the mv line, save the output and post
> > it here, along with the script you used. Insert the files, both
> > script output and the script itself; do not retype anything.
> >
> >> I have piles and piles of books. Not a one worth dirt when
> >> it comes to getting this done.
> >
> > I could recommend another book, but it's not an impartial
> > recommendation; see my .sig for details. You might find it more
> > useful as it contains more than 200 examples of scripts and
> > functions.
> >
>
>
> The script..
>
>
> #!/bin/bash
> # mover2
>
> for i in xx*;
> do
>
> (grep -m 1 HCO* $i) > x;
> (sed 's/^ *//' x) > y;
> (sed 's/ /-/g' y) > z;
>
> f=$(cat z);
> echo mv $i $f; >> a
>
> done
>
> ****************
>
> Contents of a
>
>
> mv xx101 HCO-POLICY-LETTER-OF-1-DECEMBER-1962
> mv xx06 HCO-BULLETIN-OF-25-JANUARY-1962
> mv xx07 HCO-BULLETIN-OF-1-FEBRUARY-1962
> mv xx08 HCO-INFORMATION-LETTER-OF-1-FEBRUARY-1962
> mv xx09 HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
> mv xx10 HCO-BULLETIN-OF-8-FEBRUARY-1962
> mv xx100 HCO-BULLETIN-OF-29-NOVEMBER-AD12
> mv xx101 HCO-POLICY-LETTER-OF-1-DECEMBER-1962
>

The first and last lines are the same.

As has been pointed out upthread, one problem is
(grep -m 1 HCO* $i) > x;
The unquoted HCO* will expand to match all the newly
created filenames beginning with HCO

In the general case, quote HCO* but in this case just
drop the * and grep for either HC or HCO depending on
which you meant.

--
John.

Ed Morton wrote:

>
>
> WCB wrote:
>> Chris F.A. Johnson wrote:
>>
>>
>>>On 2005-07-13, WCB wrote:
>>>
>>>>Chris F.A. Johnson wrote:
>>>>
>>>>I can echo my files,
>>>>
>>>>xx07
>>>>Bulletin_of_12_September_1965
>>>>xx08
>>>>Bulletin-of-24_April_1965
>>>>
>>>>and on and on.
>>>>
>>>>I can echo mv xx07 Bulletin_of_12_september_1965
>>>>et al on and on.
>>>
>>>[snip]
>>>
>>>>I have been googling, finding a bit here, a bit there,
>>>>cobbling it together, and watching as at the last line,
>>>>mv fails.
>>>>
>>>>And I haven't the faintest clue why mv $i $F echos
>>>>properly as I expect and mv fails utterly.
>>>
>>> Utterly? What exactly does "fails utterly" mean?
>>>
>>> Do none of the files get moved? Are the xxNN files all stll there?
>>
>>
>> Nothing moves, xxNN files are unchanged
>>
>>
>>
>>> Put echo at the beginning of the mv line, save the output and post
>>> it here, along with the script you used. Insert the files, both
>>> script output and the script itself; do not retype anything.
>>
>>
>> A bit hard, I wiped out all the scripts and went back to the one you
>> originally posted. I'd have to reconstruct it from notes I made.
>>
>>
>> #!/bin/bash
>> # moving - pasted
>>
>>
>> for file in xx*
>> do
>> set -- $(grep -m 1 'HCO*' "$file")
>> mv "$file" "${1}_${2}_${3}_${4}_${5}_${6}_${7}"
>>
>> done
>>
>> I expanded it to put "_" between elements
>>
>> The problem it leaves artifacts at the end of file names
>>
>>
>> HCO_BULLETIN_OF_12_JUNE_1965?
>>
>> It looks like a question mark in the terminal, but a
>> square box in KDE's Kwrite.
>> Apparently some non-alphanumeric character.
>
> Obvious questions:
>
> a) does that extra character appear in the source file xxXX?
> b) do you see that char when you echo "$7"?


No. If i echo $i >> file, file has a list xxNN files no "?"
on any item showing up in file.
Echo mv "$file" "${1}_${2}_${3}_${4}_${5}_${6}_${7}"
no odd characters showing up as attached to what is echoed
on screen. No "?". But with echo it also does not mv anything.
Remove echo and now it does, with the mystery "?".

If I echo ${6}
echo ${7}
echo

I get a list of the proper expected strings with no spurious charaters
and mv works but again, now we have "?" appended to all but the first
filename. The rest get the usual "?".

So the damage is between the final echo and done.
So it obviously has something to do with the evaluation
after the first item in the for do done loop. It looks like
mv will not work as expected in a for do loop structure.
Strangely enough (or maybe not) cp does exactly the same thing
as mv.

If I do a more normal

for i in xx*
do

grep et al
mv $i $f

done

sort of script neither mv nor cp will work
within that loop, but mv in a script outside
a structure works from a script.
mv $i $f works outside a for do done structure
as expected.

So it looks like its some sort of bug with both
mv and cp.


>
> You said that "mv" and "cp" both just silently complete without moving
> the file. What happens when you:
>
> c) echo "$?" after the command?

> d) try using cat "$file" > "${1}_${2}_${3}_${4}_${5}_${6}_${7}" instead?

This gets me something like a list of stuff like
HCO_BULLETIN_OF_12_MAY_1965? xx02
in a terminal the HCO etc is while the xx02 is green.

"$?" echos 0

>
> At this point, I think it'd be useful for you to create and post, say, 2
> sample files xx01 and xx02 along with your EXACT script and EXACT stdout
> and stderr along with an "ls -al" of your test directory before and
> after running your script so you could copy/paste this script and post
> the resulting stdout and stderr:
>


file xx01

HUBBARD COMMUNICATIONS OFFICE
Saint Hill Manor, East Grinstead, Sussex

HCO BULLETIN OF 25 JANUARY 1962

Franchise
Sthil

stuff stuff stuff stuff etc.
...


file xx02

HUBBARD COMMUNICATIONS OFFICE
Saint Hill Manor, East Grinstead, Sussex

HCO INFORMATION LETTER OF 3 FEBRUARY 1962 All

Sthil Students

stuff stuff stuff stuff etc.

.......

The idea is to key in on HCO* and make a name of that.

grep -m -1 HCO* $i > X
sed 's/^ *//' $X > Y
sed 's/ /_/g' $Y > Z

f=(cat $Z)

$f gets me what I want as a new file name.
so I can get there.

I have two scripts.

#!/bin/bash
# mover

for file in xx*
do
set -- $(grep -m 1 'HCO*' "$file")
mv "$file" "${1}_${2}_${3}_${4}_${5}_${6}_${7}"
done

*****************

#!/bin/bash
# mover2



for i in xx*;
do

(grep -m 1 HCO* $i) > x;
(sed 's/^ *//' x) > y;
(sed 's/ /-/g' y) > z;

f=$(cat z);

# echo $f >>f
# echo $i >>i
# echo mv $i $f >>a
# echo mv $i $f >> a

done

***************************************

Echoing to a file shows my extraction script
does work.



> pwd
> ls -al

my directory is called test5
run as a user all xxNN files are owned by user with 755
results are HCO_BULLETIN_OF_12_MAY_1965?
Also owned by user with 755 permissions



> for file in xx*
> do
> printf "---- %s:\n" "$file"
> cat "$file"
> printf "----\n"
> set -- $(grep -m 1 'HCO*' "$file")
> printf "<%s:%s:%s:%s:%s:%s:%s>\n" "$1" "$2" "$3" "$4" "$5" "$6"
> "$7"
> mv "$file" "mv_${1}_${2}_${3}_${4}_${5}_${6}_${7}"
> printf "mv result: %s\n" "$?"
>
> cp "$file" "cp_${1}_${2}_${3}_${4}_${5}_${6}_${7}"
> printf "cp result: %s\n" "$?"
>
> cat "$file" > "cat_${1}_${2}_${3}_${4}_${5}_${6}_${7}"
> printf "cat result: %s\n" "$?"
>
> done
> ls -al
>

I cut and pasted this and ran it.

Results are a list of things like


cat_HCO_BULLETIN_OF_12_MAY_1964?
etc
and
mv_HCO_BULLETIN_OF_12_MAY_1964?
ect

Still "?" appended to these file names.


Something in scripts with these sort
of loops or constructs does NOT like mv.



--
When I shake my killfile I can hear them buzzing.

rolandberry [at] hotmail.com wrote:

> You know me. If you want a beginners guide then I can send you
> something.

Heh! Now you know what I'm doing.

Its not that. It looks like I have found a big bug
in bash. neither mv nor cp work in a for do done loop
or similar structures.
They work fine as a single line outside such.

The one task I am trying to do seems to be stymied not by
lack of knowledge but by a bug.

Take large file oec1
Use csplit to extract files xx00 - xxNN
extract new name.
mv xxNN newname

move all files to indexing, searchable database
system with mySQL as a backend.

Bug stops mv $i $f from working.
I have found lots of online tutorials but it seems
nobody else has found this particular bug.
So far as I know anyway.

I thought it was my lack of knowledge, but its
obviously a bash bug problem with both mv and cp affected
using mv under bash as supplied with Mandrake 10.1.

So now what is needed is a clever work around.




--
When I shake my killfile I can hear them buzzing.

In article <11dddoroaj154ee [at] corp.supernews.com>,
WCB <wbarwell [at] munnged.mylinuxisp.com> wrote:
>rolandberry [at] hotmail.com wrote:
>
>> You know me. If you want a beginners guide then I can send you
>> something.
>
>Heh! Now you know what I'm doing.
>
>Its not that. It looks like I have found a big bug
>in bash. neither mv nor cp work in a for do done loop
>or similar structures.

You are wrong.

Your script contains errors.

mv and cp *DO* work inside a for do done loop.

I use them that way regularly.

You *are* wrong.
You don't understand what you're doing.


take the script where you claim 'mv' and/or 'cp' "don't work" and put
the word 'echo' in front of the mv or cp. *now* try the script.
I guarantee that you will _not_ see the mv/cp command printed out.
the problem is *elsewhere* in your script.

On 2005-07-14, WCB wrote:
> rolandberry [at] hotmail.com wrote:
>
>> You know me. If you want a beginners guide then I can send you
>> something.
>
> Heh! Now you know what I'm doing.
>
> Its not that. It looks like I have found a big bug
> in bash. neither mv nor cp work in a for do done loop
> or similar structures.
> They work fine as a single line outside such.

That is simnply not true; I use them without any problem inside a
loop.

> The one task I am trying to do seems to be stymied not by
> lack of knowledge but by a bug.
>
> Take large file oec1
> Use csplit to extract files xx00 - xxNN
> extract new name.
> mv xxNN newname
>
> move all files to indexing, searchable database
> system with mySQL as a backend.
>
> Bug stops mv $i $f from working.

But does it stop this from working:

mv "$i" "$f"


What happens with this:

n=1
for file in xx*
do
mv "$file" "NEW_$n"
n=$(( $n + 1 ))
done


> I have found lots of online tutorials but it seems
> nobody else has found this particular bug.
> So far as I know anyway.
>
> I thought it was my lack of knowledge, but its
> obviously a bash bug problem with both mv and cp affected
> using mv under bash as supplied with Mandrake 10.1.
>
> So now what is needed is a clever work around.

No, you need to find out what is wrong with your script.

--
Chris F.A. Johnson <http://cfaj.freeshell.org>
============================================================ ======
Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
<http://www.torfree.net/~chris/books/cfaj/ssr.html>

"WCB" <wbarwell [at] Mungggedd.mylinuxisp.com> wrote in message news:11dddoroaj154ee [at] corp.supernews.com...
> rolandberry [at] hotmail.com wrote:
>
> > You know me. If you want a beginners guide then I can send you
> > something.
>
> Heh! Now you know what I'm doing.
>
> Its not that. It looks like I have found a big bug
> in bash. neither mv nor cp work in a for do done loop
> or similar structures.
> They work fine as a single line outside such.
>
> The one task I am trying to do seems to be stymied not by
> lack of knowledge but by a bug.
>
> Take large file oec1
> Use csplit to extract files xx00 - xxNN
> extract new name.
> mv xxNN newname
>
> move all files to indexing, searchable database
> system with mySQL as a backend.
>
> Bug stops mv $i $f from working.

You run: grep HCO* $i
follower by a mv which creates a file called (say)
HCO-September-26

Next time round the loop you run the same grep but
this time, before the command runs, the shell expands
the HCO* to match the filename so you actually run:
grep HCO-September-26 $i
which presumably most times gives no output.

Quote HCO* as Chris FA Johnson does in his script: 'HCO*'.
Also, if you add a "set -x" to the top of the script
you will be able to see more clearly what is being run.

--
John.

On Thu, 14 Jul 2005 14:12:12 -0500, WCB
<wbarwell [at] Mungggedd.mylinuxisp.com> wrote:
>
> The one task I am trying to do seems to be stymied not by
> lack of knowledge but by a bug.
>
> Take large file oec1
> Use csplit to extract files xx00 - xxNN
> extract new name.
> mv xxNN newname
>
> move all files to indexing, searchable database
> system with mySQL as a backend.
>
> Bug stops mv $i $f from working.
> I have found lots of online tutorials but it seems
> nobody else has found this particular bug.
> So far as I know anyway.
>
You might try changing "mv" to "echo" to debug your script.


--
Tonight you will pay the wages of sin; Don't forget to leave a tip.

WCB wrote:
<snip>
> No. If i echo $i >> file, file has a list xxNN files no "?"
> on any item showing up in file.

You've been told repeatedly to quote your variables. echo $i is VERY
different from echo "$i". I repeat my suggestion that you run the script
I posted and post it's output. I believe you ran it last time, but what
you posted was snippets and your summaries of what you think happened
and so far that's not getting us anywhere. Just run the script and post
the output and we can take it from there.

FWIW, I THINK the problem is that you have control characters in your
input files. Change the "cat" in my script to "cat -v" before you run it
so we can see them, if present.

Ed.

Bill Marcum wrote:

> On Thu, 14 Jul 2005 14:12:12 -0500, WCB
> <wbarwell [at] Mungggedd.mylinuxisp.com> wrote:
>>
>> The one task I am trying to do seems to be stymied not by
>> lack of knowledge but by a bug.
>>
>> Take large file oec1
>> Use csplit to extract files xx00 - xxNN
>> extract new name.
>> mv xxNN newname
>>
>> move all files to indexing, searchable database
>> system with mySQL as a backend.
>>
>> Bug stops mv $i $f from working.
>> I have found lots of online tutorials but it seems
>> nobody else has found this particular bug.
>> So far as I know anyway.
>>
> You might try changing "mv" to "echo" to debug your script.
>
>

I did that extensively. It allowed me to document buggy behavior.

But the bug remains.

I can get stuff to work from a script not in a loop,
but inside that, it should and won't.

Its not just mv, cp does it too. Most odd.
I have a sneaking suspician mv and cp share code that is not
compatible with a for do done loop.



--
When I shake my killfile I can hear them buzzing.

On 2005-07-14, WCB wrote:
> Bill Marcum wrote:
>
>> On Thu, 14 Jul 2005 14:12:12 -0500, WCB
>> <wbarwell [at] Mungggedd.mylinuxisp.com> wrote:
>>>
>>> The one task I am trying to do seems to be stymied not by
>>> lack of knowledge but by a bug.
>>>
>>> Take large file oec1
>>> Use csplit to extract files xx00 - xxNN
>>> extract new name.
>>> mv xxNN newname
>>>
>>> move all files to indexing, searchable database
>>> system with mySQL as a backend.
>>>
>>> Bug stops mv $i $f from working.
>>> I have found lots of online tutorials but it seems
>>> nobody else has found this particular bug.
>>> So far as I know anyway.
>>>
>> You might try changing "mv" to "echo" to debug your script.
>
> I did that extensively. It allowed me to document buggy behavior.
>
> But the bug remains.

The bug is in YOUR script.

> I can get stuff to work from a script not in a loop,
> but inside that, it should and won't.
>
> Its not just mv, cp does it too. Most odd.
>
> I have a sneaking suspician mv and cp share code that is not
> compatible with a for do done loop.

There is no such bug in mv or cp. The commands cannot know that
they are being called from within a loop.


What happens when you run this script (cut and paste it, do not
retype it):

for i in xx*
do
f=$(grep -m 1 "HCO*" "$i")
f=${f// /-}
f=${f//--/-}
mv "$i" "$f"
done

--
Chris F.A. Johnson <http://cfaj.freeshell.org>
============================================================ ======
Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
<http://www.torfree.net/~chris/books/cfaj/ssr.html>

John L wrote:

>
> "WCB" <wbarwell [at] Mungggedd.mylinuxisp.com> wrote in message
> news:11dddoroaj154ee [at] corp.supernews.com...
>> rolandberry [at] hotmail.com wrote:
>>
>> > You know me. If you want a beginners guide then I can send you
>> > something.
>>
>> Heh! Now you know what I'm doing.
>>
>> Its not that. It looks like I have found a big bug
>> in bash. neither mv nor cp work in a for do done loop
>> or similar structures.
>> They work fine as a single line outside such.
>>
>> The one task I am trying to do seems to be stymied not by
>> lack of knowledge but by a bug.
>>
>> Take large file oec1
>> Use csplit to extract files xx00 - xxNN
>> extract new name.
>> mv xxNN newname
>>
>> move all files to indexing, searchable database
>> system with mySQL as a backend.
>>
>> Bug stops mv $i $f from working.
>
> You run: grep HCO* $i
> follower by a mv which creates a file called (say)
> HCO-September-26
>
> Next time round the loop you run the same grep but
> this time, before the command runs, the shell expands
> the HCO* to match the filename so you actually run:
> grep HCO-September-26 $i
> which presumably most times gives no output.

Well, no.

What I have is:

******
/bin/bash
# mover5

set -x

for i in xx*;
do

grep -m 1 HCO* $i > x
sed 's/^ *//' x > y
sed 's/ /-/g' y > z

This works, I get z which is my name
f=$(cat z)

This gives me $f


echo $i >> I
echo $f >> F
echo

I gives ...
xx06
xx07
xx08
xx09
xx10
xx100
xx101


F gives me ....
HCO-BULLETIN-OF-25-JANUARY-1962
HCO-BULLETIN-OF-1-FEBRUARY-1962
HCO-INFORMATION-LETTER-OF-1-FEBRUARY-1962
HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
HCO-BULLETIN-OF-8-FEBRUARY-1962
HCO-BULLETIN-OF-29-NOVEMBER-AD12
HCO-POLICY-LETTER-OF-1-DECEMBER-1962

**************************************

So far, so good

mv $i $f >> M

Cat M gives me

mv xx06 HCO-BULLETIN-OF-25-JANUARY-1962
mv xx07 HCO-BULLETIN-OF-1-FEBRUARY-1962
mv xx08 HCO-INFORMATION-LETTER-OF-1-FEBRUARY-1962
mv xx09 HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
mv xx10 HCO-BULLETIN-OF-8-FEBRUARY-1962
mv xx100 HCO-BULLETIN-OF-29-NOVEMBER-AD12
mv xx101 HCO-POLICY-LETTER-OF-1-DECEMBER-1962

This should work.
But nothing moves.

Why..

If I type mv xx06 HCO-BULLETIN-OF-25-JANUARY-1962 and hit enter
MV works

Or I can do it from a script,

#!/bin/bash

mv xx07 HCO-SOME-BLOODY-FILE

That works.

It is just in a do for done loop, there is no signal passed
to mv to do the deed. Its like typing it and not hitting enter.

done
# end

****************************

Here is a test script I wrote and ran

#1/bin/bash

# mover3

file1="bigfile"

file2="xx08"

mv xx09 file9

mv $file2 file8

mv xx06 $file1

f10="xx10"
f100="file10"

mv $f10 $f100

***************************

All of these ran perfectly, outside the
for do done loop!
And... cp works the same as mv!

Great in the above script, not as cp $i $f
in a for do done loop.

Its just a big old bug is all.


The little test script above tells the tale.

The echo mv $i $f >> M file shows
everything is working fine up to that point.

The for do done falls flat on its face big time.

And mv does not like attempted work arounds.
(read < $f) and other attempted work arounds failed.
I screwed with stuff like this just to see what happens
"$i", {$f}, nothing seems to do anything.
appending \r doesn't. I played with it quite a bit
and its plain old broke.

Its an extremely reliable bug.
Easily reproduced.

It just seems strange to me that no one
else has run across this
Or maybe they do and I just don't know it.
I haven't found anything on google.



--
When I shake my killfile I can hear them buzzing.

On 2005-07-14, WCB wrote:
>
> What I have is:
>
> ******
> /bin/bash
> # mover5
>
> set -x
>
> for i in xx*;
> do
>
> grep -m 1 HCO* $i > x

For the umpteenth time, _QUOTE HCO*_ and your variables:

grep -m 1 "HCO*" "$i"



--
Chris F.A. Johnson <http://cfaj.freeshell.org>
============================================================ ======
Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
<http://www.torfree.net/~chris/books/cfaj/ssr.html>

Robert Bonomi wrote:

> In article <11dddoroaj154ee [at] corp.supernews.com>,
> WCB <wbarwell [at] munnged.mylinuxisp.com> wrote:
>>rolandberry [at] hotmail.com wrote:
>>
>>> You know me. If you want a beginners guide then I can send you
>>> something.
>>
>>Heh! Now you know what I'm doing.
>>
>>Its not that. It looks like I have found a big bug
>>in bash. neither mv nor cp work in a for do done loop
>>or similar structures.
>
> You are wrong.
>
> Your script contains errors.

Noper.

>
> mv and cp *DO* work inside a for do done loop.
>
> I use them that way regularly.
>
> You *are* wrong.
> You don't understand what you're doing.
>

You are assuming we are using the same version of bash.
Yours may not be broken.

What version of bash do you have?
This one is 205 B patch level 0. Mandrake 10.0.

Here is my simple script
****************************
/bin/bash
# mover5

set -x

for i in xx*;
do

grep -m 1 HCO* $i > x
sed 's/^ *//' x > y
sed 's/ /-/g' y > z

f=$(cat z)

echo $i >>i
echo $f >>f
echo
echo mv $i $f >> M

done
# end

**************************************

i above gives me:
xx06
xx07
xx08
xx09
xx10
xx100
xx101

f gives me:
HCO-BULLETIN-OF-25-JANUARY-1962
HCO-BULLETIN-OF-1-FEBRUARY-1962
HCO-INFORMATION-LETTER-OF-1-FEBRUARY-1962
HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
HCO-BULLETIN-OF-8-FEBRUARY-1962
HCO-BULLETIN-OF-29-NOVEMBER-AD12
HCO-POLICY-LETTER-OF-1-DECEMBER-1962

So far so good. I have extracted and patched my target name file.

echo mv $i $f >> M gives me;
mv xx06 HCO-BULLETIN-OF-25-JANUARY-1962
mv xx07 HCO-BULLETIN-OF-1-FEBRUARY-1962
mv xx08 HCO-INFORMATION-LETTER-OF-1-FEBRUARY-1962
mv xx09 HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
mv xx10 HCO-BULLETIN-OF-8-FEBRUARY-1962
mv xx100 HCO-BULLETIN-OF-29-NOVEMBER-AD12
mv xx101 HCO-POLICY-LETTER-OF-1-DECEMBER-1962

So far, so good.

Well no. It does not work. Mv does not mv anything.
It should.

Here is a little test script

****************************************
#1/bin/bash

# mover3

file1="bigfile"
No variables. This works

file9="xx09"
mv $file9 File9
1 variable, left hand. Works

file8="file8"
mv xx08 $file8
1 variable right hand. Works

f10="xx10"
f100="file10"
mv $f10 $f100
Two variables. This works


***************************************

I wrote several such scripts testing mv and cp.
All work in these scripts as the example
above.

But not in a for do done loop.

Check echo mv $i $f >> M again.
This should work.


> take the script where you claim 'mv' and/or 'cp' "don't work" and put
> the word 'echo' in front of the mv or cp. *now* try the script.
> I guarantee that you will _not_ see the mv/cp command printed out.
> the problem is *elsewhere* in your script.


See above. Done did that. Collected evidence my script is working at
several points.

My script works perfectly.

$i >> I shows it works
$f >> F shows that works.
M shows its working up to that point.
All work outside a loop as my scripts showed me.

Just not in a for do done loop.

Neither with mv or cp.

This on a standard Mandrake 10.1 install
so its not like it an ancient bash version or
something off some 486 machine.

Something be broke. Not my script.

--
When I shake my killfile I can hear them buzzing.

WCB wrote:
<snip>
> Here is a test script I wrote and ran
>
> #1/bin/bash
>
> # mover3
>
> file1="bigfile"
>
> file2="xx08"
>
> mv xx09 file9
>
> mv $file2 file8
>
> mv xx06 $file1
>
> f10="xx10"
> f100="file10"
>
> mv $f10 $f100
>
> ***************************
>
> All of these ran perfectly, outside the
> for do done loop!

Good to know but you didn't run the same test with them IN a for do done
loop so it doesn't prove anything.

> And... cp works the same as mv!

I'd find that a smidge suspicious, especially when everyone in this NG
is telling you:

a) quote your variables, and
b) there is no such bug as you describe.

> Great in the above script, not as cp $i $f
> in a for do done loop.
>
> Its just a big old bug is all.

No, it isn't. Or, if it is, so far you haven't posted anything to lead
to that conclusion.

> The little test script above tells the tale.

No, it doesn't. It just shows that mv works as expected.

Look, try this (assuming you have files xx08 and xx09 lying around):

for file in xx08 xx09
do
mv "$file" "${file}_new"
done

If that fails THEN you've got some evidence that mv doesn't work in a loop.

Ed.

On 2005-07-15, WCB wrote:
> Robert Bonomi wrote:
>
>> In article <11dddoroaj154ee [at] corp.supernews.com>,
>> WCB <wbarwell [at] munnged.mylinuxisp.com> wrote:
>>>rolandberry [at] hotmail.com wrote:
>>>
>>>> You know me. If you want a beginners guide then I can send you
>>>> something.
>>>
>>>Heh! Now you know what I'm doing.
>>>
>>>Its not that. It looks like I have found a big bug
>>>in bash. neither mv nor cp work in a for do done loop
>>>or similar structures.
>>
>> You are wrong.
>>
>> Your script contains errors.
>
> Noper.

Yes!!!!!

>> mv and cp *DO* work inside a for do done loop.
>>
>> I use them that way regularly.
>>
>> You *are* wrong.
>> You don't understand what you're doing.
>>
>
> You are assuming we are using the same version of bash.
> Yours may not be broken.
>
> What version of bash do you have?
> This one is 205 B patch level 0. Mandrake 10.0.

There's no such animal; do you mean "bash 2.05b patch level 0"?

I've used every version of bash from 1.14 to 3.0; there is no such
bug in any of them.

> Here is my simple script
> ****************************
> /bin/bash
> # mover5
>
> set -x
>
> for i in xx*;
> do
>
> grep -m 1 HCO* $i > x

THERE ARE YOUR FIRST TWO MISTAKES; USE QUOTES:

grep -m 1 "HCO*" "$i" > x

> sed 's/^ *//' x > y
> sed 's/ /-/g' y > z
>
> f=$(cat z)
>
> echo $i >>i
> echo $f >>f
> echo
> echo mv $i $f >> M

USE QUOTES:

mv "$i" "$f"

--
Chris F.A. Johnson <http://cfaj.freeshell.org>
============================================================ ======
Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
<http://www.torfree.net/~chris/books/cfaj/ssr.html>

Ed Morton wrote:

> WCB wrote:
> <snip>
>> No. If i echo $i >> file, file has a list xxNN files no "?"
>> on any item showing up in file.
>
> You've been told repeatedly to quote your variables. echo $i is VERY
> different from echo "$i". I repeat my suggestion that you run the script
> I posted and post it's output. I believe you ran it last time, but what
> you posted was snippets and your summaries of what you think happened
> and so far that's not getting us anywhere. Just run the script and post
> the output and we can take it from there.
>
> FWIW, I THINK the problem is that you have control characters in your
> input files. Change the "cat" in my script to "cat -v" before you run it
> so we can see them, if present.

I tried over several days with "" and without.
I looked at the files with several editors.
No control characters, its plain old text.
I ran them through dos2 unix to be sure these
were kosher. They are plain old ascii
Not Word, not Mac not full of DOS files.

I am way ahead of you. Csplit and editors handle
it with no complains.
Its not some oddball word processor file or ebdic
or some bastard format.

My scrips run perfect,

echo mv $i $f >> M

cat M ;
mv xx06 HCO-BULLETIN-OF-25-JANUARY-1962
mv xx07 HCO-BULLETIN-OF-1-FEBRUARY-1962
mv xx08 HCO-INFORMATION-LETTER-OF-1-FEBRUARY-1962
mv xx09 HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
mv xx10 HCO-BULLETIN-OF-8-FEBRUARY-1962
mv xx100 HCO-BULLETIN-OF-29-NOVEMBER-AD12
mv xx101 HCO-POLICY-LETTER-OF-1-DECEMBER-1962

So down to the wire, its stuffing mv $i $f as expected.
Do you see any control characters in these names?
I don't.

Andddddddddddddddddddd...

If I run them OUTSIDE for do done, they work.
From a script.

Just not in that loop is all.

Bash 2.05 B patch level 0.

The next thing to do is, I think I loaded bash completion
last time around at install. I just uprme'd that away and
that is not the problem.

The fact that it works from a script without a loop
seems to me to strongly indicate it isn't the file.

Your script didn't do anything
much different from Johnson's script.

The problem there is with
"${1}_${2}_${3}_${4}_${5}_${6}" is this.
If I have six segments of my name that needs spaces
replaced with _ and I run the above ${6}, it adds a space
at the end.
If I cut out ${6}, it truncates my file, I lose the years
such as "1965". I can have truncation or a space.
since its a file NAME, sed et all won't work on it after
it becomes a file name of course.

If I have a line with 5 segments, I get a space and and extra "_".

I have no idea why the above script element insists on
adding " " to each file's name. But that is the problem, it does.
If it did not, Johnson's script would be what I needed.
If anybody can tell me why, I am all ears.

Its consistant. You can create names that should extract
and all exhibit the same problems. Whether I have 4
segments or 8.

I have no idea how to debug that particular line of code,
but that line of code, on bash 205 b patch level 0, is
not a happening thing.

I have several scripts each with fatal flaws.
None of my books gets anywhere as esoteric to
handle these sorts of bugs.


--
When I shake my killfile I can hear them buzzing.

On Thu, 14 Jul 2005 19:50:38 -0500, WCB <wbarwell [at] Mungggedd.mylinuxisp.com> wrote:
> Ed Morton wrote:
> > WCB wrote:
> > <snip>
> >> No. If i echo $i >> file, file has a list xxNN files no "?"
> >> on any item showing up in file.
> >
....
> I tried over several days with "" and without.

Now, where do we file a bug report on your lack of understanding,
and lack of desire to improve? Park your ego and read with very
great care the answers you gathered already.





And lose that redundant ';'...





Did somebody whisper "${QUOTE_YOUR_VARIABLES}_with_extra_text" too
softly?

--Grant.

On 2005-07-15, WCB wrote:
> Ed Morton wrote:
>
>> WCB wrote:
>> <snip>
>>> No. If i echo $i >> file, file has a list xxNN files no "?"
>>> on any item showing up in file.
>>
>> You've been told repeatedly to quote your variables. echo $i is VERY
>> different from echo "$i". I repeat my suggestion that you run the script
>> I posted and post it's output. I believe you ran it last time, but what
>> you posted was snippets and your summaries of what you think happened
>> and so far that's not getting us anywhere. Just run the script and post
>> the output and we can take it from there.
>>
>> FWIW, I THINK the problem is that you have control characters in your
>> input files. Change the "cat" in my script to "cat -v" before you run it
>> so we can see them, if present.
>
> I tried over several days with "" and without.
> I looked at the files with several editors.
> No control characters, its plain old text.
> I ran them through dos2 unix to be sure these
> were kosher. They are plain old ascii
> Not Word, not Mac not full of DOS files.
>
> I am way ahead of you. Csplit and editors handle
> it with no complains.
> Its not some oddball word processor file or ebdic
> or some bastard format.
>
> My scrips run perfect,
>
> echo mv $i $f >> M
>
> cat M ;
> mv xx06 HCO-BULLETIN-OF-25-JANUARY-1962
> mv xx07 HCO-BULLETIN-OF-1-FEBRUARY-1962
> mv xx08 HCO-INFORMATION-LETTER-OF-1-FEBRUARY-1962
> mv xx09 HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
> mv xx10 HCO-BULLETIN-OF-8-FEBRUARY-1962
> mv xx100 HCO-BULLETIN-OF-29-NOVEMBER-AD12
> mv xx101 HCO-POLICY-LETTER-OF-1-DECEMBER-1962
>
> So down to the wire, its stuffing mv $i $f as expected.
> Do you see any control characters in these names?
> I don't.
>
> Andddddddddddddddddddd...
>
> If I run them OUTSIDE for do done, they work.
> From a script.
>
> Just not in that loop is all.
>
> Bash 2.05 B patch level 0.
>
> The next thing to do is, I think I loaded bash completion
> last time around at install. I just uprme'd that away and
> that is not the problem.
>
> The fact that it works from a script without a loop
> seems to me to strongly indicate it isn't the file.
>
> Your script didn't do anything
> much different from Johnson's script.
>
> The problem there is with
> "${1}_${2}_${3}_${4}_${5}_${6}" is this.
> If I have six segments of my name that needs spaces
> replaced with _ and I run the above ${6}, it adds a space
> at the end.
> If I cut out ${6}, it truncates my file, I lose the years
> such as "1965". I can have truncation or a space.
> since its a file NAME, sed et all won't work on it after
> it becomes a file name of course.
>
> If I have a line with 5 segments, I get a space and and extra "_".
>
> I have no idea why the above script element insists on
> adding " " to each file's name. But that is the problem, it does.
> If it did not, Johnson's script would be what I needed.

If it does contain spaces my script will work. AS possible problem
is that leading spaces are converted to hyphens which mv then
interprets as options.

for i in xx*
do
f=$(grep -m 1 "HCO*" "$i")
while :
do
case $f in
\ *) f=${f# } ;; ## remove leading space
*\ ) f=${f% } ;; ## remove trailing space
*\ *) f=${f// /-} ;; ## convert spaces to hyphens
*--*) f=${f//--/-} ;; ## convert multiple hyphens to a single hyphen
*) break ;; ## nothing needs doing; exit loop
esac
done
mv "$i" "$f"
done

You repeatedly leave out the quotes in the snippets you have
posted. Use the above script EXACTLY as it is; copy it (either cut
and paste it, or save the message and edit it), do not retype it.

If it doesn't work, tell us EXACTLY what happens. Use set -x and
redirect stderr to a file.


--
Chris F.A. Johnson <http://cfaj.freeshell.org>
============================================================ ======
Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
<http://www.torfree.net/~chris/books/cfaj/ssr.html>

In article <11ddmh9q8t6j248 [at] corp.supernews.com>,
WCB <wbarwell [at] munnged.mylinuxisp.com> wrote:
>Bill Marcum wrote:
>
>> On Thu, 14 Jul 2005 14:12:12 -0500, WCB
>> <wbarwell [at] Mungggedd.mylinuxisp.com> wrote:
>>>
>>> The one task I am trying to do seems to be stymied not by
>>> lack of knowledge but by a bug.
>>>
>>> Take large file oec1
>>> Use csplit to extract files xx00 - xxNN
>>> extract new name.
>>> mv xxNN newname
>>>
>>> move all files to indexing, searchable database
>>> system with mySQL as a backend.
>>>
>>> Bug stops mv $i $f from working.
>>> I have found lots of online tutorials but it seems
>>> nobody else has found this particular bug.
>>> So far as I know anyway.
>>>
>> You might try changing "mv" to "echo" to debug your script.
>>
>>
>
>I did that extensively. It allowed me to document buggy behavior.
>
>But the bug remains.
>
>I can get stuff to work from a script not in a loop,
>but inside that, it should and won't.
>
>Its not just mv, cp does it too. Most odd.
>I have a sneaking suspician mv and cp share code that is not
>compatible with a for do done loop.


SORRY CHARLIE! That is "utter ignorance and total incompetence" speaking.

*ANY* executable program -- mv and cp included -- doesn't know, and CAN'T
TELL "if", "how", or from "inside what shell construct" it was invoked.

Without going through _CONSIDERABLE_ contortions, no executable program can
even tell "what program" it was invoked from. And to do even _that_, requires
a very non-trivial amount coding that is specific to the particular environment
that that particular executable is running in.

There is an old proverb in the Middle East

He who knows, and knows that he knows,
is a wise man, Follow him.
He who knows not, and knows that he knows not,
is educable, Teach him.
He who knows, and knows not that he knows,
is asleep, Awaken him.
He who KNOWS NOT, and KNOWS NOT that he knows not,
is a fool, Shun him.


You are rapidly becoming a candidate for category #4.

WCB <wbarwell [at] Mungggedd.mylinuxisp.com> writes:

> What I have is:
>
> ******
> /bin/bash
> # mover5
>
> set -x
>
> for i in xx*;
> do
>
> grep -m 1 HCO* $i > x
> sed 's/^ *//' x > y
> sed 's/ /-/g' y > z
>
> This works, I get z which is my name
> f=$(cat z)
>
> This gives me $f
>
>
> echo $i >> I
> echo $f >> F
> echo

This isn't what you ran, because it won't run. It's missing the done,
and who knows what else, not to mention the non-commented comments.

linux$ cat mover
#!/bin/bash
set -x
for i in xx*
do
grep -m 1 "HCO*" "$i" > x
sed 's/^ *//' x > y
sed 's/ /-/g' y > z
f=$(cat z)
echo "$i" >> I
echo "$f" >> F
mv "$i" "$f"
done



linux$ mover
+ grep -m 1 'HCO*' xx01
+ sed 's/^ *//' x
+ sed 's/ /-/g' y
++ cat z
+ f=HCO-BULLETIN-OF-25-JANUARY-1962
+ echo xx01
+ echo HCO-BULLETIN-OF-25-JANUARY-1962
+ mv xx01 HCO-BULLETIN-OF-25-JANUARY-1962
+ grep -m 1 'HCO*' xx02
+ sed 's/^ *//' x
+ sed 's/ /-/g' y
++ cat z
+ f=HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
+ echo xx02
+ echo HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
+ mv xx02 HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All




linux$ ls
F I y
HCO-BULLETIN-OF-25-JANUARY-1962 mover* z
HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All x




linux$ cat HCO-BULLETIN-OF-25-JANUARY-1962
HUBBARD COMMUNICATIONS OFFICE
Saint Hill Manor, East Grinstead, Sussex

HCO BULLETIN OF 25 JANUARY 1962

Franchise
Sthil

stuff stuff stuff stuff etc.
...




linux$ cat HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
HUBBARD COMMUNICATIONS OFFICE
Saint Hill Manor, East Grinstead, Sussex

HCO INFORMATION LETTER OF 3 FEBRUARY 1962 All

Sthil Students

stuff stuff stuff stuff etc.


How does that differ from what you wanted?

Joe

Chris F.A. Johnson wrote:

> On 2005-07-15, WCB wrote:
>> Ed Morton wrote:
>>
>>> WCB wrote:
>>> <snip>
>>>> No. If i echo $i >> file, file has a list xxNN files no "?"
>>>> on any item showing up in file.
>>>
>>> You've been told repeatedly to quote your variables. echo $i is VERY
>>> different from echo "$i". I repeat my suggestion that you run the script
>>> I posted and post it's output. I believe you ran it last time, but what
>>> you posted was snippets and your summaries of what you think happened
>>> and so far that's not getting us anywhere. Just run the script and post
>>> the output and we can take it from there.
>>>
>>> FWIW, I THINK the problem is that you have control characters in your
>>> input files. Change the "cat" in my script to "cat -v" before you run it
>>> so we can see them, if present.
>>
>> I tried over several days with "" and without.
>> I looked at the files with several editors.
>> No control characters, its plain old text.
>> I ran them through dos2 unix to be sure these
>> were kosher. They are plain old ascii
>> Not Word, not Mac not full of DOS files.
>>
>> I am way ahead of you. Csplit and editors handle
>> it with no complains.
>> Its not some oddball word processor file or ebdic
>> or some bastard format.
>>
>> My scrips run perfect,
>>
>> echo mv $i $f >> M
>>
>> cat M ;
>> mv xx06 HCO-BULLETIN-OF-25-JANUARY-1962
>> mv xx07 HCO-BULLETIN-OF-1-FEBRUARY-1962
>> mv xx08 HCO-INFORMATION-LETTER-OF-1-FEBRUARY-1962
>> mv xx09 HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
>> mv xx10 HCO-BULLETIN-OF-8-FEBRUARY-1962
>> mv xx100 HCO-BULLETIN-OF-29-NOVEMBER-AD12
>> mv xx101 HCO-POLICY-LETTER-OF-1-DECEMBER-1962
>>
>> So down to the wire, its stuffing mv $i $f as expected.
>> Do you see any control characters in these names?
>> I don't.
>>
>> Andddddddddddddddddddd...
>>
>> If I run them OUTSIDE for do done, they work.
>> From a script.
>>
>> Just not in that loop is all.
>>
>> Bash 2.05 B patch level 0.
>>
>> The next thing to do is, I think I loaded bash completion
>> last time around at install. I just uprme'd that away and
>> that is not the problem.
>>
>> The fact that it works from a script without a loop
>> seems to me to strongly indicate it isn't the file.
>>
>> Your script didn't do anything
>> much different from Johnson's script.
>>
>> The problem there is with
>> "${1}_${2}_${3}_${4}_${5}_${6}" is this.
>> If I have six segments of my name that needs spaces
>> replaced with _ and I run the above ${6}, it adds a space
>> at the end.
>> If I cut out ${6}, it truncates my file, I lose the years
>> such as "1965". I can have truncation or a space.
>> since its a file NAME, sed et all won't work on it after
>> it becomes a file name of course.
>>
>> If I have a line with 5 segments, I get a space and and extra "_".
>>
>> I have no idea why the above script element insists on
>> adding " " to each file's name. But that is the problem, it does.
>> If it did not, Johnson's script would be what I needed.
>
> If it does contain spaces my script will work. AS possible problem
> is that leading spaces are converted to hyphens which mv then
> interprets as options.
>
> for i in xx*
> do
> f=$(grep -m 1 "HCO*" "$i")
> while :
> do
> case $f in
> \ *) f=${f# } ;; ## remove leading space
> *\ ) f=${f% } ;; ## remove trailing space
> *\ *) f=${f// /-} ;; ## convert spaces to hyphens
> *--*) f=${f//--/-} ;; ## convert multiple hyphens to a single
> hyphen
> *) break ;; ## nothing needs doing; exit loop
> esac
> done
> mv "$i" "$f"
> done
>
> You repeatedly leave out the quotes in the snippets you have
> posted. Use the above script EXACTLY as it is; copy it (either cut
> and paste it, or save the message and edit it), do not retype it.
>
> If it doesn't work, tell us EXACTLY what happens. Use set -x and
> redirect stderr to a file.
>
>

Cut and pasted. changed mod and owner.
Loaded fresh test files in directory

I ran this script.
still it has trailing spaces.
In the terminal, they show up as ?
In Kwrite they are spaces which can
be seen and edited out in "properties"
and are seen boxes in properties.
I can easily edited it out from properties.

listing in the terminal looks like

HCO-BULLETIN-OF-10-MARCH-1965?
HCO-BULLETIN-OF-29-MARCH-1965?
HCO-BULLETIN-OF-2-APRIL-AD15?
HCO-BULLETIN-OF-4-APRIL-AD15?
HCO-BULLETIN-OF-5-APRIL-1965?
HCO-BULLETIN-OF-5-MARCH-1965?
HCO-BULLETIN-OF-7-APRIL-AD15?

Each file is a file, so its renaming a real file.







--
When I shake my killfile I can hear them buzzing.

On 2005-07-15, WCB wrote:
> Chris F.A. Johnson wrote:
>
>> On 2005-07-15, WCB wrote:
>>> Ed Morton wrote:
>>>
>>>> WCB wrote:
>>>> <snip>
>>>>> No. If i echo $i >> file, file has a list xxNN files no "?"
>>>>> on any item showing up in file.
>>>>
>>>> You've been told repeatedly to quote your variables. echo $i is VERY
>>>> different from echo "$i". I repeat my suggestion that you run the script
>>>> I posted and post it's output. I believe you ran it last time, but what
>>>> you posted was snippets and your summaries of what you think happened
>>>> and so far that's not getting us anywhere. Just run the script and post
>>>> the output and we can take it from there.
>>>>
>>>> FWIW, I THINK the problem is that you have control characters in your
>>>> input files. Change the "cat" in my script to "cat -v" before you run it
>>>> so we can see them, if present.
>>>
>>> I tried over several days with "" and without.
>>> I looked at the files with several editors.
>>> No control characters, its plain old text.
>>> I ran them through dos2 unix to be sure these
>>> were kosher. They are plain old ascii
>>> Not Word, not Mac not full of DOS files.
>>>
>>> I am way ahead of you. Csplit and editors handle
>>> it with no complains.
>>> Its not some oddball word processor file or ebdic
>>> or some bastard format.
>>>
>>> My scrips run perfect,
>>>
>>> echo mv $i $f >> M
>>>
>>> cat M ;
>>> mv xx06 HCO-BULLETIN-OF-25-JANUARY-1962
>>> mv xx07 HCO-BULLETIN-OF-1-FEBRUARY-1962
>>> mv xx08 HCO-INFORMATION-LETTER-OF-1-FEBRUARY-1962
>>> mv xx09 HCO-INFORMATION-LETTER-OF-3-FEBRUARY-1962-All
>>> mv xx10 HCO-BULLETIN-OF-8-FEBRUARY-1962
>>> mv xx100 HCO-BULLETIN-OF-29-NOVEMBER-AD12
>>> mv xx101 HCO-POLICY-LETTER-OF-1-DECEMBER-1962
>>>
>>> So down to the wire, its stuffing mv $i $f as expected.
>>> Do you see any control characters in these names?
>>> I don't.
>>>
>>> Andddddddddddddddddddd...
>>>
>>> If I run them OUTSIDE for do done, they work.
>>> From a script.
>>>
>>> Just not in that loop is all.
>>>
>>> Bash 2.05 B patch level 0.
>>>
>>> The next thing to do is, I think I loaded bash completion
>>> last time around at install. I just uprme'd that away and
>>> that is not the problem.
>>>
>>> The fact that it works from a script without a loop
>>> seems to me to strongly indicate it isn't the file.
>>>
>>> Your script didn't do anything
>>> much different from Johnson's script.
>>>
>>> The problem there is with
>>> "${1}_${2}_${3}_${4}_${5}_${6}" is this.
>>> If I have six segments of my name that needs spaces
>>> replaced with _ and I run the above ${6}, it adds a space
>>> at the end.
>>> If I cut out ${6}, it truncates my file, I lose the years
>>> such as "1965". I can have truncation or a space.
>>> since its a file NAME, sed et all won't work on it after
>>> it becomes a file name of course.
>>>
>>> If I have a line with 5 segments, I get a space and and extra "_".
>>>
>>> I have no idea why the above script element insists on
>>> adding " " to each file's name. But that is the problem, it does.
>>> If it did not, Johnson's script would be what I needed.
>>
>> If it does contain spaces my script will work. AS possible problem
>> is that leading spaces are converted to hyphens which mv then
>> interprets as options.
>>
>> for i in xx*
>> do
>> f=$(grep -m 1 "HCO*" "$i")
>> while :
>> do
>> case $f in
>> \ *) f=${f# } ;; ## remove leading space
>> *\ ) f=${f% } ;; ## remove trailing space
>> *\ *) f=${f// /-} ;; ## convert spaces to hyphens
>> *--*) f=${f//--/-} ;; ## convert multiple hyphens to a single
>> hyphen
>> *) break ;; ## nothing needs doing; exit loop
>> esac
>> done
>> mv "$i" "$f"
>> done
>>
>> You repeatedly leave out the quotes in the snippets you have
>> posted. Use the above script EXACTLY as it is; copy it (either cut
>> and paste it, or save the message and edit it), do not retype it.
>>
>> If it doesn't work, tell us EXACTLY what happens. Use set -x and
>> redirect stderr to a file.
>
> Cut and pasted. changed mod and owner.
> Loaded fresh test files in directory
>
> I ran this script.
> still it has trailing spaces.

What is "it"?

> In the terminal, they show up as ?

In what context do they (whatever "they" are) "show up"?

> In Kwrite they are spaces which can
> be seen and edited out in "properties"
> and are seen boxes in properties.

What are?

> I can easily edited it out from properties.

Properties of what?

> listing in the terminal looks like
>
> HCO-BULLETIN-OF-10-MARCH-1965?
> HCO-BULLETIN-OF-29-MARCH-1965?
> HCO-BULLETIN-OF-2-APRIL-AD15?
> HCO-BULLETIN-OF-4-APRIL-AD15?
> HCO-BULLETIN-OF-5-APRIL-1965?
> HCO-BULLETIN-OF-5-MARCH-1965?
> HCO-BULLETIN-OF-7-APRIL-AD15?
>
> Each file is a file, so its renaming a real file.

What files did you have to start with?

What files did you end up with?

Is that what you wanted? If not, what is different from what you
wanted? Please post the script you used, directly from the file.

--
Chris F.A. Johnson <http://cfaj.freeshell.org>
============================================================ ======
Shell Scripting Recipes: A Problem-Solution Approach, 2005, Apress
<http://www.torfree.net/~chris/books/cfaj/ssr.html>