Hey, all. I had a problem a month or so ago where I had three mirrors,
and one of them went AWOL after a system migration. Finally got as much
as I could back, created a RAID-5 of four of the 1.5 TB disks, put
everything on it, recovered more from backup, and was feeling pretty good.

Then the power went out.

Now, now RAID-5. Or, rather, mdadm sees that there are RAID partitions,
and starts; fdisk shows me an unpartitioned disk. Which is, essentially,
the exact same thing I had happen last time. (Except that, this time, I
*know* my mdadm.conf is correct, because it hasn't changed -- but, for the
hell of it, I did a "mdadm --detail --scan", and it matches.)

WTF? Am I too stupid for software RAID? Is there something really
obvious I'm missing? Any help at all would be most graciously accepted.

Thanks,

-Ken


--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Thu, 4 Mar 2010 11:12:29 -0500 (EST)
"Ken D'Ambrosio" <ken [at] jots.org> wrote:

> Hey, all. I had a problem a month or so ago where I had three mirrors,
> and one of them went AWOL after a system migration. Finally got as much
> as I could back, created a RAID-5 of four of the 1.5 TB disks, put
> everything on it, recovered more from backup, and was feeling pretty good.
>
> Then the power went out.
>
> Now, now RAID-5. Or, rather, mdadm sees that there are RAID partitions,
> and starts; fdisk shows me an unpartitioned disk. Which is, essentially,
> the exact same thing I had happen last time. (Except that, this time, I
> *know* my mdadm.conf is correct, because it hasn't changed -- but, for the
> hell of it, I did a "mdadm --detail --scan", and it matches.)
>
> WTF? Am I too stupid for software RAID? Is there something really
> obvious I'm missing? Any help at all would be most graciously accepted.

You missed the bit where you provide concrete information rather than
vaguaries.

I'm guessing that you created the array over whole-devices, and then
partitioned the array - is that correct?
If fdisk shows you an unpartitioned array, maybe just the partition table is
corrupt. Seems strange.

To so that I/we don't have to guess, please give exact commands that you run
and the exact output so we have access to the same information as you.

Too much data is much much better than not enough.

NeilBrown

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Thu, March 4, 2010 3:50 pm, Neil Brown wrote:
> On Thu, 4 Mar 2010 11:12:29 -0500 (EST)
> "Ken D'Ambrosio" <ken [at] jots.org> wrote:

[bad things happened to Ken's RAID-5 here]

>
> You missed the bit where you provide concrete information rather than
> vaguaries.

Humble apologies; I'm not well-enough versed with the intricacies of Linux
RAID to know what's appropriate and not.

> I'm guessing that you created the array over whole-devices, and then
> partitioned the array - is that correct? If fdisk shows you an
> unpartitioned array, maybe just the partition table is corrupt. Seems
> strange.

Actually, no. These were created using /dev/sd[abcd]2 -- I saved off
space on sd[abcd]1 for swap, /tmp, etc. Done via the Ubuntu installer, if
that makes a difference. For the record, all the /dev/sd[abcd]1 non-RAID
partitions look fine.

> To so that I/we don't have to guess, please give exact commands that you
> run and the exact output so we have access to the same information as you.

Well, I rebooted, and was surprised that nothing RAID-esque came up.
Since my OS is on one of the afore-mentioned non-RAIDed partitions, the
OS, itself, booted, but none of the RAID partitions mounted. I tried to
mount, and failed. That's when I checked the RAID device, /dev/md0.
fdisk showed it lacking any partitions, but the mdadm.conf file hadn't
been touched for a couple of weeks, so I was pretty sure nothing there had
changed. On the off chance that the SCSI drives had re-ordered
themselves, I went through all 24 permutations of
mdadm --assemble /dev/md0 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sdd2
since I wasn't sure if the drive order was significant. All of them
"worked," inasmuch as they created /dev/md0, but in all cases it was
partitionless. I also tried mdadm --detail --scan, to verify that it
matched UUIDs with those in the /etc/mdadm.conf file, and it did (the
array line looks thusly:
ARRAY /dev/md0 level=raid5 num-devices=4
UUID=1e89645a:7e24dcef:9e77d54f:077a6a6f )


> Too much data is much much better than not enough.

Granted... but, sometimes -- especially when learning -- there can be a
bad signal:noise ratio. Tends to make me hesitant when I'm a (relative)
newbie to a given topic. That being said, I *think* I've figured out what
I should be doing, but I also think I did it. Did I miss something?

Thanks!

-Ken

> NeilBrown
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo [at] vger.kernel.org More majordomo info at
> http://vger.kernel.org/majordomo-info.html
>
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is believed to be clean.
>
>



--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Thu, Mar 4, 2010 at 2:21 PM, Michael Evans <mjevans1983 [at] gmail.com> w=
rote:
> Try providing the output of;
> for ii in /dev/[sh]d[a-z] ; do parted $ii print ; done
>
> If you have any kind of partition table this should tell us what it
> is. =A0If you don't, something odd is happening.
>

Also, you'll need to run that as the root (unix term for absolute
administrator account) user. Since you mentioned installing and lack
partitions you probably need a linux on CD or usb stick distribution;
http://www.sysresccd.org/Main_Page works well for me and might help
you enough to get you in to user mode; as well as non-destructively
auto-detecting most storage setups and having very recent software.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Thu, March 4, 2010 5:21 pm, Michael Evans wrote:
> Try providing the output of;
> for ii in /dev/[sh]d[a-z] ; do parted $ii print ; done

Mea culpa; I'd said:

I went through all 24 permutations of
mdadm --assemble /dev/md0 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sdd2
since I wasn't sure if the drive order was significant. All of them
"worked," inasmuch as they created /dev/md0, but in all cases it was
partitionless.

Which I assumed implied that /dev/sd[a-d]2 was valid on all disks, though,
in hindsight, I wasn't explicit. So: /dev/sd[a-d]2 exists on all drives
as partition id "fd" (Linux raid autodetect). It's /dev/md0 that shows as
a valid, 4.4 TB disk... with no partition.

-Ken


--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Thu, Mar 4, 2010 at 3:41 PM, Ken D'Ambrosio <ken [at] jots.org> wrote:
> On Thu, March 4, 2010 5:21 pm, Michael Evans wrote:
>> Try providing the output of;
>> for ii in /dev/[sh]d[a-z] ; do parted $ii print ; done
>
> Mea culpa; I'd said:
>
> =A0I went through all 24 permutations of
> =A0mdadm --assemble /dev/md0 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sdd2
> =A0since I wasn't sure if the drive order was significant. =A0All of =
them
> =A0"worked," inasmuch as they created /dev/md0, but in all cases it w=
as
> =A0partitionless.
>
> Which I assumed implied that /dev/sd[a-d]2 was valid on all disks, th=
ough,
> in hindsight, I wasn't explicit. =A0So: /dev/sd[a-d]2 exists on all d=
rives
> as partition id "fd" (Linux raid autodetect). =A0It's /dev/md0 that s=
hows as
> a valid, 4.4 TB disk... with no partition.
>
> -Ken

It sounds like you might be interested in this script:

http://www.linuxfoundation.org/collaborate/workgroups/linux- raid/raid_r=
ecovery

It's untested, but it may produce a sequence that shows you valid data.

Also if you 'partitioned' the resulting raid device what you most
likely did was use it as an LVM physical volume, and then create
logical volumes from it. At least that's the way I'd do it.

After you have /dev/md0 (or whatever) running try running this;

vgscan ; vgdisplay

You might see your 'partitions' listed, at which point you can do a
read-only fsck, then read-only mount and determine if they are in fact
whole, or if they are instead corrupt in that configuration.

If you manually let mdadm guess where to put the devices based on the
stored metadata then it will probably determine the correct order for
you; presuming you haven't already over-written that with invalid
data.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Thu, 4 Mar 2010 17:00:10 -0500 (EST)
"Ken D'Ambrosio" <ken [at] jots.org> wrote:

> On Thu, March 4, 2010 3:50 pm, Neil Brown wrote:
> > On Thu, 4 Mar 2010 11:12:29 -0500 (EST)
> > "Ken D'Ambrosio" <ken [at] jots.org> wrote:
>
> [bad things happened to Ken's RAID-5 here]
>
> >
> > You missed the bit where you provide concrete information rather than
> > vaguaries.
>
> Humble apologies; I'm not well-enough versed with the intricacies of Linux
> RAID to know what's appropriate and not.
>
> > I'm guessing that you created the array over whole-devices, and then
> > partitioned the array - is that correct? If fdisk shows you an
> > unpartitioned array, maybe just the partition table is corrupt. Seems
> > strange.
>
> Actually, no. These were created using /dev/sd[abcd]2 -- I saved off
> space on sd[abcd]1 for swap, /tmp, etc. Done via the Ubuntu installer, if
> that makes a difference. For the record, all the /dev/sd[abcd]1 non-RAID
> partitions look fine.
>
> > To so that I/we don't have to guess, please give exact commands that you
> > run and the exact output so we have access to the same information as you.
>
> Well, I rebooted, and was surprised that nothing RAID-esque came up.
> Since my OS is on one of the afore-mentioned non-RAIDed partitions, the
> OS, itself, booted, but none of the RAID partitions mounted. I tried to
> mount, and failed. That's when I checked the RAID device, /dev/md0.
> fdisk showed it lacking any partitions, but the mdadm.conf file hadn't
> been touched for a couple of weeks, so I was pretty sure nothing there had
> changed. On the off chance that the SCSI drives had re-ordered
> themselves, I went through all 24 permutations of
> mdadm --assemble /dev/md0 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sdd2
> since I wasn't sure if the drive order was significant. All of them
> "worked," inasmuch as they created /dev/md0, but in all cases it was
> partitionless. I also tried mdadm --detail --scan, to verify that it
> matched UUIDs with those in the /etc/mdadm.conf file, and it did (the
> array line looks thusly:
> ARRAY /dev/md0 level=raid5 num-devices=4
> UUID=1e89645a:7e24dcef:9e77d54f:077a6a6f )
>
>
> > Too much data is much much better than not enough.
>
> Granted... but, sometimes -- especially when learning -- there can be a
> bad signal:noise ratio. Tends to make me hesitant when I'm a (relative)
> newbie to a given topic. That being said, I *think* I've figured out what
> I should be doing, but I also think I did it. Did I miss something?

Which part of "please give exact commands that you
run and the exact output"
Did you have trouble with?

Try this:

cat /etc/fstab
cat /etc/mdadm.conf
mdadm -Esvv
mdadm -Asv
blkid -p /dev/md*

And include all the output.

(sorry if I seem grumpy, but I'm a bit tired which makes it harder to appear
polite).

NeilBrown
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

------=_20100305102454_39885
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: 8bit

On Thu, March 4, 2010 8:30 pm, Neil Brown wrote:
> Try this:
>
>
> cat /etc/fstab cat /etc/mdadm.conf mdadm -Esvv mdadm -Asv blkid -p /dev/md*
>
> And include all the output.

See attached. Note that fstab is essentially irrelevant; I'd mounted by
hand, and was going to add the entry after I got the server back in the
server room.

> (sorry if I seem grumpy, but I'm a bit tired which makes it harder to
> appear polite).

Know what you mean; got four hours', myself, due to a virtualization
migration.

For the record, I've got my backup tapes ordered (again) -- if you can
tell me what I did wrong, that would be almost as satisfying as getting
the data back.

Thanks,

-Ken
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


------=_20100305102454_39885
Content-Type: text/plain; name="fdisk.txt"
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment; filename="fdisk.txt"

Disk /dev/md0 doesn't contain a valid partition table

Disk /dev/md0: 4410.9 GB, 4410904805376 bytes
2 heads, 4 sectors/track, 1076881056 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000

------=_20100305102454_39885
Content-Type: application/octet-stream; name="mdadm.conf"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="mdadm.conf"

IyBtZGFkbS5jb25mCiMKIyBQbGVhc2UgcmVmZXIgdG8gbWRhZG0uY29uZig1
KSBmb3IgaW5mb3JtYXRpb24gYWJvdXQgdGhpcyBmaWxlLgojCgojIGJ5IGRl
ZmF1bHQsIHNjYW4gYWxsIHBhcnRpdGlvbnMgKC9wcm9jL3BhcnRpdGlvbnMp
IGZvciBNRCBzdXBlcmJsb2Nrcy4KIyBhbHRlcm5hdGl2ZWx5LCBzcGVjaWZ5
IGRldmljZXMgdG8gc2NhbiwgdXNpbmcgd2lsZGNhcmRzIGlmIGRlc2lyZWQu
CkRFVklDRSBwYXJ0aXRpb25zCgojIGF1dG8tY3JlYXRlIGRldmljZXMgd2l0
aCBEZWJpYW4gc3RhbmRhcmQgcGVybWlzc2lvbnMKQ1JFQVRFIG93bmVyPXJv
b3QgZ3JvdXA9ZGlzayBtb2RlPTA2NjAgYXV0bz15ZXMKCiMgYXV0b21hdGlj
YWxseSB0YWcgbmV3IGFycmF5cyBhcyBiZWxvbmdpbmcgdG8gdGhlIGxvY2Fs
IHN5c3RlbQpIT01FSE9TVCA8c3lzdGVtPgoKIyBpbnN0cnVjdCB0aGUgbW9u
aXRvcmluZyBkYWVtb24gd2hlcmUgdG8gc2VuZCBtYWlsIGFsZXJ0cwpNQUlM
QUREUiByb290CgojIGRlZmluaXRpb25zIG9mIGV4aXN0aW5nIE1EIGFycmF5
cwoKQVJSQVkgL2Rldi9tZDAgbGV2ZWw9cmFpZDUgbnVtLWRldmljZXM9NCBV
VUlEPTFlODk2NDVhOjdlMjRkY2VmOjllNzdkNTRmOjA3N2E2YTZmCgojQVJS
QVkgL2Rldi9tZDAgbGV2ZWw9cmFpZDUgbnVtLWRldmljZXM9NCBkZXZpY2Vz
PXNkYTIsc2RiMixzZGQyLHNkYzIKCiMgVGhpcyBmaWxlIHdhcyBhdXRvLWdl
bmVyYXRlZCBvbiBUaHUsIDExIEZlYiAyMDEwIDIxOjU2OjE2ICswMDAwCiMg
YnkgbWtjb25mICRJZCQK

------=_20100305102454_39885
Content-Type: text/plain; name="mdadm-Asv.txt"
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment; filename="mdadm-Asv.txt"

mdadm: looking for devices for /dev/md0
mdadm: cannot open device /dev/sde1: Device or resource busy
mdadm: /dev/sde1 has wrong uuid.
mdadm: cannot open device /dev/sde: Device or resource busy
mdadm: /dev/sde has wrong uuid.
mdadm: no RAID superblock on /dev/sdd1
mdadm: /dev/sdd1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdd
mdadm: /dev/sdd has wrong uuid.
mdadm: cannot open device /dev/sdc1: Device or resource busy
mdadm: /dev/sdc1 has wrong uuid.
mdadm: cannot open device /dev/sdc: Device or resource busy
mdadm: /dev/sdc has wrong uuid.
mdadm: cannot open device /dev/sdb1: Device or resource busy
mdadm: /dev/sdb1 has wrong uuid.
mdadm: cannot open device /dev/sdb: Device or resource busy
mdadm: /dev/sdb has wrong uuid.
mdadm: cannot open device /dev/sda1: Device or resource busy
mdadm: /dev/sda1 has wrong uuid.
mdadm: cannot open device /dev/sda: Device or resource busy
mdadm: /dev/sda has wrong uuid.
mdadm: cannot open device /dev/loop1: Device or resource busy
mdadm: /dev/loop1 has wrong uuid.
mdadm: cannot open device /dev/loop0: Device or resource busy
mdadm: /dev/loop0 has wrong uuid.
mdadm: /dev/sdd2 is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sdc2 is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sdb2 is identified as a member of /dev/md0, slot 0.
mdadm: /dev/sda2 is identified as a member of /dev/md0, slot 2.
mdadm: added /dev/sdc2 to /dev/md0 as 1
mdadm: added /dev/sda2 to /dev/md0 as 2
mdadm: added /dev/sdd2 to /dev/md0 as 3
mdadm: added /dev/sdb2 to /dev/md0 as 0
mdadm: /dev/md0 has been started with 4 drives.
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/block/9:0
mdadm: cannot open device /dev/sde1: Device or resource busy
mdadm: cannot open device /dev/sde: Device or resource busy
mdadm: cannot open device /dev/sdd2: Device or resource busy
mdadm: no recogniseable superblock on /dev/sdd1
mdadm: cannot open device /dev/sdd: Device or resource busy
mdadm: cannot open device /dev/sdc2: Device or resource busy
mdadm: cannot open device /dev/sdc1: Device or resource busy
mdadm: cannot open device /dev/sdc: Device or resource busy
mdadm: cannot open device /dev/sdb2: Device or resource busy
mdadm: cannot open device /dev/sdb1: Device or resource busy
mdadm: cannot open device /dev/sdb: Device or resource busy
mdadm: cannot open device /dev/sda2: Device or resource busy
mdadm: cannot open device /dev/sda1: Device or resource busy
mdadm: cannot open device /dev/sda: Device or resource busy
mdadm: cannot open device /dev/loop1: Device or resource busy
mdadm: cannot open device /dev/loop0: Device or resource busy
------=_20100305102454_39885
Content-Type: text/plain; name="mdadm-Esvv.txt"
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment; filename="mdadm-Esvv.txt"

mdadm: No md superblock detected on /dev/block/9:0.
mdadm: No md superblock detected on /dev/sde1.
mdadm: No md superblock detected on /dev/sde.
mdadm: No md superblock detected on /dev/sdd1.
mdadm: No md superblock detected on /dev/sdd.
mdadm: No md superblock detected on /dev/sdc1.
mdadm: No md superblock detected on /dev/sdc.
mdadm: No md superblock detected on /dev/sdb1.
mdadm: No md superblock detected on /dev/sdb.
mdadm: No md superblock detected on /dev/sda1.
mdadm: No md superblock detected on /dev/sda.
mdadm: No md superblock detected on /dev/loop1.
mdadm: No md superblock detected on /dev/loop0.
/dev/sdd2:
Magic : a92b4efc
Version : 00.90.00
UUID : 1e89645a:7e24dcef:9e77d54f:077a6a6f
Creation Time : Thu Feb 4 20:54:58 2010
Raid Level : raid5
Used Dev Size : 1435841408 (1369.33 GiB 1470.30 GB)
Array Size : 4307524224 (4107.98 GiB 4410.90 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0

Update Time : Fri Mar 5 07:04:06 2010
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Checksum : d85ef7b3 - correct
Events : 35041

Layout : left-symmetric
Chunk Size : 64K

Number Major Minor RaidDevice State
this 3 8 50 3 active sync /dev/sdd2

0 0 8 18 0 active sync /dev/sdb2
1 1 8 34 1 active sync /dev/sdc2
2 2 8 2 2 active sync /dev/sda2
3 3 8 50 3 active sync /dev/sdd2
/dev/sdc2:
Magic : a92b4efc
Version : 00.90.00
UUID : 1e89645a:7e24dcef:9e77d54f:077a6a6f
Creation Time : Thu Feb 4 20:54:58 2010
Raid Level : raid5
Used Dev Size : 1435841408 (1369.33 GiB 1470.30 GB)
Array Size : 4307524224 (4107.98 GiB 4410.90 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0

Update Time : Fri Mar 5 07:04:06 2010
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Checksum : d85ef79f - correct
Events : 35041

Layout : left-symmetric
Chunk Size : 64K

Number Major Minor RaidDevice State
this 1 8 34 1 active sync /dev/sdc2

0 0 8 18 0 active sync /dev/sdb2
1 1 8 34 1 active sync /dev/sdc2
2 2 8 2 2 active sync /dev/sda2
3 3 8 50 3 active sync /dev/sdd2
/dev/sdb2:
Magic : a92b4efc
Version : 00.90.00
UUID : 1e89645a:7e24dcef:9e77d54f:077a6a6f
Creation Time : Thu Feb 4 20:54:58 2010
Raid Level : raid5
Used Dev Size : 1435841408 (1369.33 GiB 1470.30 GB)
Array Size : 4307524224 (4107.98 GiB 4410.90 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0

Update Time : Fri Mar 5 07:04:06 2010
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Checksum : d85ef78d - correct
Events : 35041

Layout : left-symmetric
Chunk Size : 64K

Number Major Minor RaidDevice State
this 0 8 18 0 active sync /dev/sdb2

0 0 8 18 0 active sync /dev/sdb2
1 1 8 34 1 active sync /dev/sdc2
2 2 8 2 2 active sync /dev/sda2
3 3 8 50 3 active sync /dev/sdd2
/dev/sda2:
Magic : a92b4efc
Version : 00.90.00
UUID : 1e89645a:7e24dcef:9e77d54f:077a6a6f
Creation Time : Thu Feb 4 20:54:58 2010
Raid Level : raid5
Used Dev Size : 1435841408 (1369.33 GiB 1470.30 GB)
Array Size : 4307524224 (4107.98 GiB 4410.90 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0

Update Time : Fri Mar 5 07:04:06 2010
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Checksum : d85ef781 - correct
Events : 35041

Layout : left-symmetric
Chunk Size : 64K

Number Major Minor RaidDevice State
this 2 8 2 2 active sync /dev/sda2

0 0 8 18 0 active sync /dev/sdb2
1 1 8 34 1 active sync /dev/sdc2
2 2 8 2 2 active sync /dev/sda2
3 3 8 50 3 active sync /dev/sdd2
------=_20100305102454_39885
Content-Type: text/plain; name="blkid.txt"
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment; filename="blkid.txt"

/dev/md0: UUID="d6158445-e38e-4942-9395-e1bd29935d22" TYPE="xfs" USAGE="filesystem"
------=_20100305102454_39885--


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Fri, Mar 5, 2010 at 7:24 AM, Ken D'Ambrosio <ken [at] jots.org> wrote:
> On Thu, March 4, 2010 8:30 pm, Neil Brown wrote:
>> Try this:
>>
>>
>> cat /etc/fstab cat /etc/mdadm.conf mdadm -Esvv mdadm -Asv blkid -p /=
dev/md*
>>
>> And include all the output.
>
> See attached. =A0Note that fstab is essentially irrelevant; I'd mount=
ed by
> hand, and was going to add the entry after I got the server back in t=
he
> server room.
>
>> (sorry if I seem grumpy, but I'm a bit tired which makes it harder t=
o
>> appear polite).
>
> Know what you mean; got four hours', myself, due to a virtualization
> migration.
>
> For the record, I've got my backup tapes ordered (again) -- if you ca=
n
> tell me what I did wrong, that would be almost as satisfying as getti=
ng
> the data back.
>
> Thanks,
>
> -Ken
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
>
>

Here's a stab at it. I've never tried to do it like this and haven't
glanced at the code so I can't be sure, but this seems to be the most
likely suspect:

#ARRAY /dev/md0 level=3Draid5 num-devices=3D4 devices=3Dsda2,sdb2,sdd2,=
sdc2

Your commented out version seems to specify this mapping
0,sda2
1,sdb2
2,sdd2
3,sdc2

However the array is detected like this by uuid alone (I re-ordered
the entries so they're easier to read):

mdadm: /dev/sdb2 is identified as a member of /dev/md0, slot 0.
mdadm: /dev/sdc2 is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sda2 is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdd2 is identified as a member of /dev/md0, slot 3.
mdadm: added /dev/sdb2 to /dev/md0 as 0
mdadm: added /dev/sdc2 to /dev/md0 as 1
mdadm: added /dev/sda2 to /dev/md0 as 2
mdadm: added /dev/sdd2 to /dev/md0 as 3
mdadm: /dev/md0 has been started with 4 drives.

This corresponds to:

0,sdb2
1,sdc2
2,sda2
3,sdd2

Note how the drives are out of order? Thus the reason for only using
UUID and NOT explicitly specifying an ordering?
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Fri, 5 Mar 2010 10:24:55 -0500 (EST)
"Ken D'Ambrosio" <ken [at] jots.org> wrote:

> On Thu, March 4, 2010 8:30 pm, Neil Brown wrote:
> > Try this:
> >
> >
> > cat /etc/fstab cat /etc/mdadm.conf mdadm -Esvv mdadm -Asv blkid -p /dev/md*
> >
> > And include all the output.
>
> See attached. Note that fstab is essentially irrelevant; I'd mounted by
> hand, and was going to add the entry after I got the server back in the
> server room.

So fdisk says there is no partition table on /dev/md0, but
but blkid said there is an XFS filesystem there.
So maybe you never partitioned /dev/md0?
What happens if you

mount /dev/md0 /somewhere

??

NeilBrown
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Fri, 5 Mar 2010 12:05:32 -0800
Michael Evans <mjevans1983 [at] gmail.com> wrote:


> Here's a stab at it. I've never tried to do it like this and haven't
> glanced at the code so I can't be sure, but this seems to be the most
> likely suspect:
>
> #ARRAY /dev/md0 level=raid5 num-devices=4 devices=sda2,sdb2,sdd2,sdc2
>
> Your commented out version seems to specify this mapping
> 0,sda2
> 1,sdb2
> 2,sdd2
> 3,sdc2
>

No it doesn't. The 'devices=' lists is an unordered list (as set?). It just
says "only try to include these devices in the array". They still have to
have valid metadata, and it is the content of the metadata the defines the
role of the device in the array.

NeilBrown
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Fri, Mar 5, 2010 at 12:38 PM, Neil Brown <neilb [at] suse.de> wrote:
> On Fri, 5 Mar 2010 12:05:32 -0800
> Michael Evans <mjevans1983 [at] gmail.com> wrote:
>
>
>> Here's a stab at it. I've never tried to do it like this and haven't
>> glanced at the code so I can't be sure, but this seems to be the mos=
t
>> likely suspect:
>>
>> #ARRAY /dev/md0 level=3Draid5 num-devices=3D4 devices=3Dsda2,sdb2,sd=
d2,sdc2
>>
>> Your commented out version seems to specify this mapping
>> 0,sda2
>> 1,sdb2
>> 2,sdd2
>> 3,sdc2
>>
>
> No it doesn't. =A0The 'devices=3D' lists is an unordered list (as set=
?). =A0It just
> says "only try to include these devices in the array". =A0They still =
have to
> have valid metadata, and it is the content of the metadata the define=
s the
> role of the device in the array.
>
> NeilBrown

Then if mounting the filesystem used to work but now doesn't I have no
idea; based on the currently provided information I saw no other
suspect.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Fri, March 5, 2010 3:37 pm, Neil Brown wrote:
> mount /dev/md0 /somewhere
>
> ??

Sorry this took so long; I've been busy dying from Martian Death Flu.
So... I tried what you suggested.

And it worked.

WTF?!

How can you mount something that doesn't show a partition in fdisk? I'm
-very-, -very- confused here. Grateful, yes, but confused. Is this
something I'm just not "getting?"

Thanks much, regardless, for the pointer -- don't know if I would ever
have tried mounting it without a partition showing.

*confused but happy*

-Ken


--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On 09/03/2010 14:43, Ken D'Ambrosio wrote:
> On Fri, March 5, 2010 3:37 pm, Neil Brown wrote:
>> mount /dev/md0 /somewhere
>>
>> ??
>
> Sorry this took so long; I've been busy dying from Martian Death Flu.
> So... I tried what you suggested.
>
> And it worked.
>
> WTF?!
>
> How can you mount something that doesn't show a partition in fdisk? I'm
> -very-, -very- confused here. Grateful, yes, but confused. Is this
> something I'm just not "getting?"

Yes, I think so. You can potentially make and mount a filesystem on any
block device. That could be a whole disc or a partition, or it could be
something else. There are various tools for making new block devices
over the top of others, for example you can use LVM, md or dm. Then
there are yet more tools for making block devices out of other stuff,
like loopback and nbd. All these new block devices may in turn be
partitioned, but they don't have to be. In fact partitioning is just a
simple way of splitting a block device up into other smaller block devices.

Cheers,

John.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Tue, 9 Mar 2010 09:43:48 -0500 (EST)
"Ken D'Ambrosio" <ken [at] jots.org> wrote:

> On Fri, March 5, 2010 3:37 pm, Neil Brown wrote:
> > mount /dev/md0 /somewhere
> >
> > ??
>
> Sorry this took so long; I've been busy dying from Martian Death Flu.
> So... I tried what you suggested.
>
> And it worked.
>
> WTF?!
>
> How can you mount something that doesn't show a partition in fdisk? I'm
> -very-, -very- confused here. Grateful, yes, but confused. Is this
> something I'm just not "getting?"

Yes, this something you are just "not getting", but it is very simple so
won't take a moment to explain.

A partition table is simply a description of how to divide a disk drive up
into smaller units. In stead of 1 big device, you appear to have several
small devices.
You can create a filesystem on any of these things - the big device, or any
of the smaller devices.

You can use fdisk to divide /dev/sda up into /dev/sda1, /dev/sda2, etc, but
you don't have to. You can "mkfs /dev/sda" if you like.
Similarly, you can devide /dev/md0 up into /dev/md0p1, /dev/md0p2, with
fdisk, but you don't have to.
Commonly, people use /dev/sda1 rather than /dev/sda, and
/dev/md0 rather than /dev/md0p1.
But this is just common practice, not enforced (that it wasn't long ago
when /dev/md0p1 could not be created, but that isn't important here).

A partition table does add one byte of extra information - a partition type.
This is a hint as to what sort of thing is stored in the partition. However
that hint is not widely used in Linux. I think some installers use it to
automagically find 'swap' and maybe 'root', but that is about it. For many
purposes the partition type can be ignored.

When you created your filesystem, you must have created it on /dev/md0, not
on a partition.

Hope it is all clear now.

NeilBrown


>
> Thanks much, regardless, for the pointer -- don't know if I would ever
> have tried mounting it without a partition showing.
>
> *confused but happy*
>
> -Ken
>
>

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html