Hi all

I use a 3ware 9500-12 port sata card (JBOD) which will not work without=
a
128mb sodimm. The sodimm socket is flakey and the result is that the
machine occasionally crashes. Yesterday I finally gave in and put
together another
machine so that I can rsync between them. When I turned the machine
on today to set up rync, the RAID array was not gone, but corrupted.
Typical...

I built the array in Aug 2010 using the following command:

mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
--raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64

Using LVM, I did the following:
pvscan
pvcreate -M2 /dev/md0
vgcreate lvm-raid /dev/md0
vgdisplay lvm-raid
vgscan
lvscan
lvcreate -v -l 100%VG -n RAID lvm-raid
lvdisplay /dev/lvm-raid/lvm0

I then formatted using:
mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144
/dev/lvm-raid/RAID

This worked perfectly since I created the array. Now mdadm is coming u=
p
with

proxmox:/dev/md# mdadm --assemble --scan --verbose
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/md/ubuntu:0
mdadm: cannot open device /dev/dm-2: Device or resource busy
mdadm: cannot open device /dev/dm-1: Device or resource busy
mdadm: cannot open device /dev/dm-0: Device or resource busy
mdadm: cannot open device /dev/sdm1: Device or resource busy
mdadm: cannot open device /dev/sdm: Device or resource busy
mdadm: cannot open device /dev/sdl1: Device or resource busy
mdadm: cannot open device /dev/sdl: Device or resource busy
mdadm: cannot open device /dev/sdk1: Device or resource busy
mdadm: cannot open device /dev/sdk: Device or resource busy
mdadm: cannot open device /dev/sdj1: Device or resource busy
mdadm: cannot open device /dev/sdj: Device or resource busy
mdadm: cannot open device /dev/sdh1: Device or resource busy
mdadm: cannot open device /dev/sdh: Device or resource busy
mdadm: cannot open device /dev/sdi1: Device or resource busy
mdadm: cannot open device /dev/sdi: Device or resource busy
mdadm: cannot open device /dev/sdg1: Device or resource busy
mdadm: cannot open device /dev/sdg: Device or resource busy
mdadm: cannot open device /dev/sdf1: Device or resource busy
mdadm: cannot open device /dev/sdf: Device or resource busy
mdadm: cannot open device /dev/sde1: Device or resource busy
mdadm: cannot open device /dev/sde: Device or resource busy
mdadm: no RAID superblock on /dev/sdd
mdadm: cannot open device /dev/sda2: Device or resource busy
mdadm: cannot open device /dev/sda1: Device or resource busy
mdadm: cannot open device /dev/sda: Device or resource busy
mdadm: /dev/sdd1 is identified as a member of /dev/md/pro=EF=BF=BDlox:0=
, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md/pro=EF=BF=BDlox:0
mdadm: no uptodate device for slot 2 of /dev/md/pro=EF=BF=BDlox:0
mdadm: no uptodate device for slot 3 of /dev/md/pro=EF=BF=BDlox:0
mdadm: no uptodate device for slot 4 of /dev/md/pro=EF=BF=BDlox:0
mdadm: no uptodate device for slot 5 of /dev/md/pro=EF=BF=BDlox:0
mdadm: no uptodate device for slot 6 of /dev/md/pro=EF=BF=BDlox:0
mdadm: no uptodate device for slot 7 of /dev/md/pro=EF=BF=BDlox:0
mdadm: no uptodate device for slot 8 of /dev/md/pro=EF=BF=BDlox:0
mdadm: no uptodate device for slot 9 of /dev/md/pro=EF=BF=BDlox:0
mdadm: failed to add /dev/sdd1 to /dev/md/pro=EF=BF=BDlox:0: Invalid ar=
gument
mdadm: /dev/md/pro=EF=BF=BDlox:0 assembled from 0 drives - not enough t=
o start
the array.
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/sdd
mdadm: No arrays found in config file or automatically

pvscan and vgscan show nothing.

So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1.1
--level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sdg1
/dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=3D6=
4

as it seemed that /dev/sdd1 failed to be added to the array. This did
nothing.

dmesg contains:

md: invalid superblock checksum on sdd1
md: sdd1 does not have a valid v1.1 superblock, not importing!
md: md_import_device returned -22

The output of mdadm -E is as follows:

proxmox:~# mdadm -E /dev/sd[d-m]1
/dev/sdd1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : c4f62f32:4244a1db:b7746203:f10b5227
Name : pro=C3=B8lox:0
Creation Time : Sat Aug 21 19:16:38 2010
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : b0ffd0a1:866676af:3ae4c03c:d219676e

Update Time : Mon Feb 7 21:08:29 2011
Checksum : 13aa9685 - expected 93aa9672
Events : 60802

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 0
Array State : AAAAAAAAAA ('A' =3D=3D active, '.' =3D=3D missing)
/dev/sde1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 4121fa31:7543218c:f42a937d:fddf04e8

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 5f965c84 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 1
Array State : .AAAAAAAAA ('A' =3D=3D active, '.' =3D=3D missing)
/dev/sdf1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : e70da8ed:c80e9533:7d8e200e:dc285255

Update Time : Thu Feb 10 12:17:59 2011
Checksum : c092c0e5 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 2
Array State : .AAAAAAAAA ('A' =3D=3D active, '.' =3D=3D missing)
/dev/sdg1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 5878fdbb:d0c7c892:40c8fa6c:0c5e257b

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 72c95353 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 3
Array State : .AAAAAAAAA ('A' =3D=3D active, '.' =3D=3D missing)
/dev/sdh1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 7bced3d0:5ebac414:02effc8b:ac1ad69e

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 4c4e7f67 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 4
Array State : .AAAAAAAAA ('A' =3D=3D active, '.' =3D=3D missing)
/dev/sdi1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 56513558:a25025bb:11e8b563:ae42f8d4

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 87ebb998 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 5
Array State : .AAAAAAAAA ('A' =3D=3D active, '.' =3D=3D missing)
/dev/sdj1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 90621937:23cff36c:c55b1581:ed28b433

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 94b9a346 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 6
Array State : .AAAAAAAAA ('A' =3D=3D active, '.' =3D=3D missing)
/dev/sdk1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 72d5ce06:855aa34c:3a354abc:0ddb6e80

Update Time : Thu Feb 10 12:17:59 2011
Checksum : cc0e2d20 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 7
Array State : .AAAAAAAAA ('A' =3D=3D active, '.' =3D=3D missing)
/dev/sdl1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 8792e04a:ee298ace:fec79850:ddd3167b

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 20fd4534 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 8
Array State : .AAAAAAAAA ('A' =3D=3D active, '.' =3D=3D missing)
/dev/sdm1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 8916d09c:448a5791:c2923182:ab99eb92

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 7f27ba1f - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 9
Array State : .AAAAAAAAA ('A' =3D=3D active, '.' =3D=3D missing)

I'm confident the data is on there, I just need to get a backup of it o=
ff !

Yours desperately.

regards
Simon McNair


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Simon,

On 02/10/2011 11:16 AM, Simon McNair wrote:
>
[trim /]
>
> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1.1
> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sdg1
> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=3D=
64

Uh-Oh. You told mdadm to create an array with a missing member, but yo=
u didn't tell it to assume-clean.

> as it seemed that /dev/sdd1 failed to be added to the array. This di=
d nothing.

According to your mdadm -E output below, it did what you asked: it cre=
ated a new array from the devices given (note the new array UUID on the=
nine devices). Since you have one device missing, it can't rebuild, s=
o you might not have lost your data yet.

That mdadm -E /dev/sdd1 reports its array assembled and clean is odd.

Please show us the output of /proc/mdstat and then "mdadm -D /dev/md*" =
or "mdadm -D /dev/md/*"

Phil

>
> dmesg contains:
>
> md: invalid superblock checksum on sdd1
> md: sdd1 does not have a valid v1.1 superblock, not importing!
> md: md_import_device returned -22
>
> The output of mdadm -E is as follows:
>
> proxmox:~# mdadm -E /dev/sd[d-m]1
> /dev/sdd1:
> Magic : a92b4efc
> Version : 1.1
> Feature Map : 0x0
> Array UUID : c4f62f32:4244a1db:b7746203:f10b5227
> Name : pro=C3=B8lox:0
> Creation Time : Sat Aug 21 19:16:38 2010
> Raid Level : raid5
> Raid Devices : 10
>
> Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
> Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
> Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
> Data Offset : 264 sectors
> Super Offset : 0 sectors
> State : clean
> Device UUID : b0ffd0a1:866676af:3ae4c03c:d219676e
>
> Update Time : Mon Feb 7 21:08:29 2011
> Checksum : 13aa9685 - expected 93aa9672
> Events : 60802
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Device Role : Active device 0
> Array State : AAAAAAAAAA ('A' =3D=3D active, '.' =3D=3D missing)
> /dev/sde1:
> Magic : a92b4efc
> Version : 1.1
> Feature Map : 0x0
> Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
> Name : ubuntu:0
> Creation Time : Thu Feb 10 11:59:48 2011
> Raid Level : raid5
> Raid Devices : 10
>
> Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
> Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
> Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
> Data Offset : 264 sectors
> Super Offset : 0 sectors
> State : clean
> Device UUID : 4121fa31:7543218c:f42a937d:fddf04e8
>
> Update Time : Thu Feb 10 12:17:59 2011
> Checksum : 5f965c84 - correct
> Events : 4
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Device Role : Active device 1
> Array State : .AAAAAAAAA ('A' =3D=3D active, '.' =3D=3D missing)
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Thu, 10 Feb 2011 16:16:44 +0000 Simon McNair <simonmcnair [at] gmail.com>=
wrote:

>
> Hi all
>
> I use a 3ware 9500-12 port sata card (JBOD) which will not work witho=
ut a
> 128mb sodimm. The sodimm socket is flakey and the result is that the
> machine occasionally crashes. Yesterday I finally gave in and put
> together another
> machine so that I can rsync between them. When I turned the machine
> on today to set up rync, the RAID array was not gone, but corrupted.
> Typical...

Presumably the old machine was called 'ubuntu' and the new machine 'pro=
=C3=B8lox'


>
> I built the array in Aug 2010 using the following command:
>
> mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
> --raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64
>
> Using LVM, I did the following:
> pvscan
> pvcreate -M2 /dev/md0
> vgcreate lvm-raid /dev/md0
> vgdisplay lvm-raid
> vgscan
> lvscan
> lvcreate -v -l 100%VG -n RAID lvm-raid
> lvdisplay /dev/lvm-raid/lvm0
>
> I then formatted using:
> mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144
> /dev/lvm-raid/RAID
>
> This worked perfectly since I created the array. Now mdadm is coming=
up
> with
>
> proxmox:/dev/md# mdadm --assemble --scan --verbose
> mdadm: looking for devices for further assembly
> mdadm: no recogniseable superblock on /dev/md/ubuntu:0

And it seems that ubuntu:0 have been successfully assembled.
It is missing one device for some reason (sdd1) but RAID can cope with =
that.



> mdadm: cannot open device /dev/dm-2: Device or resource busy
> mdadm: cannot open device /dev/dm-1: Device or resource busy
> mdadm: cannot open device /dev/dm-0: Device or resource busy
> mdadm: cannot open device /dev/sdm1: Device or resource busy
> mdadm: cannot open device /dev/sdm: Device or resource busy
> mdadm: cannot open device /dev/sdl1: Device or resource busy
> mdadm: cannot open device /dev/sdl: Device or resource busy
> mdadm: cannot open device /dev/sdk1: Device or resource busy
> mdadm: cannot open device /dev/sdk: Device or resource busy
> mdadm: cannot open device /dev/sdj1: Device or resource busy
> mdadm: cannot open device /dev/sdj: Device or resource busy
> mdadm: cannot open device /dev/sdh1: Device or resource busy
> mdadm: cannot open device /dev/sdh: Device or resource busy
> mdadm: cannot open device /dev/sdi1: Device or resource busy
> mdadm: cannot open device /dev/sdi: Device or resource busy
> mdadm: cannot open device /dev/sdg1: Device or resource busy
> mdadm: cannot open device /dev/sdg: Device or resource busy
> mdadm: cannot open device /dev/sdf1: Device or resource busy
> mdadm: cannot open device /dev/sdf: Device or resource busy
> mdadm: cannot open device /dev/sde1: Device or resource busy
> mdadm: cannot open device /dev/sde: Device or resource busy
> mdadm: no RAID superblock on /dev/sdd
> mdadm: cannot open device /dev/sda2: Device or resource busy
> mdadm: cannot open device /dev/sda1: Device or resource busy
> mdadm: cannot open device /dev/sda: Device or resource busy
> mdadm: /dev/sdd1 is identified as a member of /dev/md/pro=EF=BF=BDlox=
:0, slot 0.
> mdadm: no uptodate device for slot 1 of /dev/md/pro=EF=BF=BDlox:0
> mdadm: no uptodate device for slot 2 of /dev/md/pro=EF=BF=BDlox:0
> mdadm: no uptodate device for slot 3 of /dev/md/pro=EF=BF=BDlox:0
> mdadm: no uptodate device for slot 4 of /dev/md/pro=EF=BF=BDlox:0
> mdadm: no uptodate device for slot 5 of /dev/md/pro=EF=BF=BDlox:0
> mdadm: no uptodate device for slot 6 of /dev/md/pro=EF=BF=BDlox:0
> mdadm: no uptodate device for slot 7 of /dev/md/pro=EF=BF=BDlox:0
> mdadm: no uptodate device for slot 8 of /dev/md/pro=EF=BF=BDlox:0
> mdadm: no uptodate device for slot 9 of /dev/md/pro=EF=BF=BDlox:0
> mdadm: failed to add /dev/sdd1 to /dev/md/pro=EF=BF=BDlox:0: Invalid =
argument
> mdadm: /dev/md/pro=EF=BF=BDlox:0 assembled from 0 drives - not enough=
to start
> the array.

This looks like it is *after* to trying the --create command you give
below.. It is best to report things in the order they happen, else you=
can
confuse people (or get caught out!).


> mdadm: looking for devices for further assembly
> mdadm: no recogniseable superblock on /dev/sdd
> mdadm: No arrays found in config file or automatically
>
> pvscan and vgscan show nothing.
>
> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1.1
> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sdg1
> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=3D=
64
>
> as it seemed that /dev/sdd1 failed to be added to the array. This di=
d
> nothing.

It did not to nothing. It wrote a superblock to /dev/sdd1 and complain=
ed
that it couldn't write to all the others --- didn't it?


>
> dmesg contains:
>
> md: invalid superblock checksum on sdd1

I guess that is why sdd1 was missing from 'ubuntu:0'. Though as I cann=
ot
tell if this happened before or after any of the various things reporte=
d
above, it is hard to be sure.


The real mystery is why 'pvscan' reports nothing.

What about
pvscan --verbose

or

blkid -p /dev/md/ubuntu:0

or even

dd of=3D/dev/md/ubuntu:0 count=3D8 | od -c

??

NeilBrown


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Neal,
Thanks for the response. Phil Turmel has been giving me some kind
assistance. After some investigation we have found that the 3ware card=

I'm using as a controller (as JBOD) is making the whole thing far too
flakey and providing some data corruption (hopefully not for too long).=

Thankfully this is mainly a data archive so the data changes very
infrequently. I have ordered a Supermicro 8 port sata controller and
intend to use that, in conjunction with the 6 onboard sata ports to try=

and get the array back in shape. Some seriously bad stuff has happened=
,
so I'm hoping the data is still retrievable.

I'll cc the latest update to the list. Any input you can provide would=

be seriously welcome.

regards
Simon

On 15/02/2011 04:53, NeilBrown wrote:
> On Thu, 10 Feb 2011 16:16:44 +0000 Simon McNair<simonmcnair [at] gmail.com=
> wrote:
>
>> Hi all
>>
>> I use a 3ware 9500-12 port sata card (JBOD) which will not work with=
out a
>> 128mb sodimm. The sodimm socket is flakey and the result is that th=
e
>> machine occasionally crashes. Yesterday I finally gave in and put
>> together another
>> machine so that I can rsync between them. When I turned the machine
>> on today to set up rync, the RAID array was not gone, but corrupted.
>> Typical...
> Presumably the old machine was called 'ubuntu' and the new machine 'p=
ro=C3=B8lox'
>
>
>> I built the array in Aug 2010 using the following command:
>>
>> mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
>> --raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64
>>
>> Using LVM, I did the following:
>> pvscan
>> pvcreate -M2 /dev/md0
>> vgcreate lvm-raid /dev/md0
>> vgdisplay lvm-raid
>> vgscan
>> lvscan
>> lvcreate -v -l 100%VG -n RAID lvm-raid
>> lvdisplay /dev/lvm-raid/lvm0
>>
>> I then formatted using:
>> mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144
>> /dev/lvm-raid/RAID
>>
>> This worked perfectly since I created the array. Now mdadm is comin=
g up
>> with
>>
>> proxmox:/dev/md# mdadm --assemble --scan --verbose
>> mdadm: looking for devices for further assembly
>> mdadm: no recogniseable superblock on /dev/md/ubuntu:0
> And it seems that ubuntu:0 have been successfully assembled.
> It is missing one device for some reason (sdd1) but RAID can cope wit=
h that.
>
>
>
>> mdadm: cannot open device /dev/dm-2: Device or resource busy
>> mdadm: cannot open device /dev/dm-1: Device or resource busy
>> mdadm: cannot open device /dev/dm-0: Device or resource busy
>> mdadm: cannot open device /dev/sdm1: Device or resource busy
>> mdadm: cannot open device /dev/sdm: Device or resource busy
>> mdadm: cannot open device /dev/sdl1: Device or resource busy
>> mdadm: cannot open device /dev/sdl: Device or resource busy
>> mdadm: cannot open device /dev/sdk1: Device or resource busy
>> mdadm: cannot open device /dev/sdk: Device or resource busy
>> mdadm: cannot open device /dev/sdj1: Device or resource busy
>> mdadm: cannot open device /dev/sdj: Device or resource busy
>> mdadm: cannot open device /dev/sdh1: Device or resource busy
>> mdadm: cannot open device /dev/sdh: Device or resource busy
>> mdadm: cannot open device /dev/sdi1: Device or resource busy
>> mdadm: cannot open device /dev/sdi: Device or resource busy
>> mdadm: cannot open device /dev/sdg1: Device or resource busy
>> mdadm: cannot open device /dev/sdg: Device or resource busy
>> mdadm: cannot open device /dev/sdf1: Device or resource busy
>> mdadm: cannot open device /dev/sdf: Device or resource busy
>> mdadm: cannot open device /dev/sde1: Device or resource busy
>> mdadm: cannot open device /dev/sde: Device or resource busy
>> mdadm: no RAID superblock on /dev/sdd
>> mdadm: cannot open device /dev/sda2: Device or resource busy
>> mdadm: cannot open device /dev/sda1: Device or resource busy
>> mdadm: cannot open device /dev/sda: Device or resource busy
>> mdadm: /dev/sdd1 is identified as a member of /dev/md/pro=EF=BF=BDlo=
x:0, slot 0.
>> mdadm: no uptodate device for slot 1 of /dev/md/pro=EF=BF=BDlox:0
>> mdadm: no uptodate device for slot 2 of /dev/md/pro=EF=BF=BDlox:0
>> mdadm: no uptodate device for slot 3 of /dev/md/pro=EF=BF=BDlox:0
>> mdadm: no uptodate device for slot 4 of /dev/md/pro=EF=BF=BDlox:0
>> mdadm: no uptodate device for slot 5 of /dev/md/pro=EF=BF=BDlox:0
>> mdadm: no uptodate device for slot 6 of /dev/md/pro=EF=BF=BDlox:0
>> mdadm: no uptodate device for slot 7 of /dev/md/pro=EF=BF=BDlox:0
>> mdadm: no uptodate device for slot 8 of /dev/md/pro=EF=BF=BDlox:0
>> mdadm: no uptodate device for slot 9 of /dev/md/pro=EF=BF=BDlox:0
>> mdadm: failed to add /dev/sdd1 to /dev/md/pro=EF=BF=BDlox:0: Invalid=
argument
>> mdadm: /dev/md/pro=EF=BF=BDlox:0 assembled from 0 drives - not enoug=
h to start
>> the array.
> This looks like it is *after* to trying the --create command you give
> below.. It is best to report things in the order they happen, else y=
ou can
> confuse people (or get caught out!).
>
>
>> mdadm: looking for devices for further assembly
>> mdadm: no recogniseable superblock on /dev/sdd
>> mdadm: No arrays found in config file or automatically
>>
>> pvscan and vgscan show nothing.
>>
>> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1.=
1
>> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sdg=
1
>> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=3D=
64
>>
>> as it seemed that /dev/sdd1 failed to be added to the array. This d=
id
>> nothing.
> It did not to nothing. It wrote a superblock to /dev/sdd1 and compla=
ined
> that it couldn't write to all the others --- didn't it?
>
>
>> dmesg contains:
>>
>> md: invalid superblock checksum on sdd1
> I guess that is why sdd1 was missing from 'ubuntu:0'. Though as I ca=
nnot
> tell if this happened before or after any of the various things repor=
ted
> above, it is hard to be sure.
>
>
> The real mystery is why 'pvscan' reports nothing.
>
> What about
> pvscan --verbose
>
> or
>
> blkid -p /dev/md/ubuntu:0
>
> or even
>
> dd of=3D/dev/md/ubuntu:0 count=3D8 | od -c
>
> ??
>
> NeilBrown
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Neil,

Since Simon has responded, let me summarize the assistance I provided p=
er his off-list request:

On 02/14/2011 11:53 PM, NeilBrown wrote:
> On Thu, 10 Feb 2011 16:16:44 +0000 Simon McNair <simonmcnair [at] gmail.co=
m> wrote:
>
>>
>> Hi all
>>
>> I use a 3ware 9500-12 port sata card (JBOD) which will not work with=
out a
>> 128mb sodimm. The sodimm socket is flakey and the result is that th=
e
>> machine occasionally crashes. Yesterday I finally gave in and put
>> together another
>> machine so that I can rsync between them. When I turned the machine
>> on today to set up rync, the RAID array was not gone, but corrupted.=

>> Typical...
>
> Presumably the old machine was called 'ubuntu' and the new machine 'p=
ro=C3=B8lox'
>
>
>>
>> I built the array in Aug 2010 using the following command:
>>
>> mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
>> --raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64
>>
>> Using LVM, I did the following:
>> pvscan
>> pvcreate -M2 /dev/md0
>> vgcreate lvm-raid /dev/md0
>> vgdisplay lvm-raid
>> vgscan
>> lvscan
>> lvcreate -v -l 100%VG -n RAID lvm-raid
>> lvdisplay /dev/lvm-raid/lvm0
>>
>> I then formatted using:
>> mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144
>> /dev/lvm-raid/RAID
>>
>> This worked perfectly since I created the array. Now mdadm is comin=
g up
>> with
>>
>> proxmox:/dev/md# mdadm --assemble --scan --verbose
>> mdadm: looking for devices for further assembly
>> mdadm: no recogniseable superblock on /dev/md/ubuntu:0
>
> And it seems that ubuntu:0 have been successfully assembled.
> It is missing one device for some reason (sdd1) but RAID can cope wit=
h that.

3ware card is compromised, with a loose buffer memory dimm. Some of it=
s ECC errors were caught and reported in dmesg. Its likely, based on t=
he loose memory socket, that many multiple-bit errors got through.

[trim /]

>> mdadm: no uptodate device for slot 8 of /dev/md/pro=EF=BF=BDlox:0
>> mdadm: no uptodate device for slot 9 of /dev/md/pro=EF=BF=BDlox:0
>> mdadm: failed to add /dev/sdd1 to /dev/md/pro=EF=BF=BDlox:0: Invalid=
argument
>> mdadm: /dev/md/pro=EF=BF=BDlox:0 assembled from 0 drives - not enoug=
h to start
>> the array.
>
> This looks like it is *after* to trying the --create command you give
> below.. It is best to report things in the order they happen, else y=
ou can
> confuse people (or get caught out!).

Yes, this was after.

>> mdadm: looking for devices for further assembly
>> mdadm: no recogniseable superblock on /dev/sdd
>> mdadm: No arrays found in config file or automatically
>>
>> pvscan and vgscan show nothing.
>>
>> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1.=
1
>> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sdg=
1
>> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=3D=
64
>>
>> as it seemed that /dev/sdd1 failed to be added to the array. This d=
id
>> nothing.
>
> It did not to nothing. It wrote a superblock to /dev/sdd1 and compla=
ined
> that it couldn't write to all the others --- didn't it?

There were multiple attempts to create. One wrote to just sdd1, anothe=
r succeeded with all but sdd1.

>> dmesg contains:
>>
>> md: invalid superblock checksum on sdd1
>
> I guess that is why sdd1 was missing from 'ubuntu:0'. Though as I ca=
nnot
> tell if this happened before or after any of the various things repor=
ted
> above, it is hard to be sure.
>
>
> The real mystery is why 'pvscan' reports nothing.

The original array was created with mdadm v2.6.7, and had a data offset=
of 264 sectors. After Simon's various attempts to --create, he ended =
up with data offset of 2048, using mdadm v3.1.4. The mdadm -E reports =
he posted to the list showed the 264 offset. We didn't realize the off=
set had been updated until somewhat later in our troubleshooting effort=
s.

In any case, pvscan couldn't see the LVM signature because it wasn't th=
ere (at offset 2048).

> What about
> pvscan --verbose
>
> or
>
> blkid -p /dev/md/ubuntu:0
>
> or even
>
> dd of=3D/dev/md/ubuntu:0 count=3D8 | od -c

=46ortunately, Simon did have a copy of his LVM configuration. With th=
e help of dd, strings, and grep, we did locate his LVM sig at the corre=
ct location on sdd1 (for data offset 264). After a number of attempts =
to bypass LVM and access his single LV with dmsetup (based on his backe=
d up configuration, on the assembled new array less sdd1), I realized t=
hat the data offset was wrong on the recreated array, and went looking =
for the cause. I found your git commit that changed that logic last sp=
ring, and recommended that Simon revert to the default package for his =
ubuntu install, which is v2.6.7.

Simon has now attempted to recreate the array with v2.6.7, but the cont=
roller is throwing too many errors to succeed, and I suggested it was t=
oo flakey to trust any further. Based on the existence of the LVM sig =
on sdd1, I believe Simon's data is (mostly) intact, and only needs a su=
ccessful create operation with a properly functioning controller. (He =
might also need to perform an lvm vgcfgrestore, but he has the necessar=
y backup file.)

A new controller is on order.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Phil,
Thanks for filling in the gaps, I had forgotton quite how much help and=

assistance you had provided up to now. You're a real godsend.

To fill in some of the other pieces of info:
The original machine was called proxmox (KVM virtualisation machine) an=
d
I used an Ubuntu live cd to see if it was the software which was
preventing progress or not. It does seem like there is data corruption=

in the machine name for some reason.

The original company I ordered the controller through sent me an email
at 4pm saying that they could not fulfill the order, 4pm was too late
for them to pick & pack so there was another 24hr turn around delay.
The supermicro card should arrive here tomorrow and hopefully I'll be
able to get a dd of each of the drives prior to Phil coming online.

=46or some reason blkid doesn't exist on my machine even though I have
e2fs-utils installed. V weird.

Simon


On 15/02/2011 14:51, Phil Turmel wrote:
> Hi Neil,
>
> Since Simon has responded, let me summarize the assistance I provided=
per his off-list request:
>
> On 02/14/2011 11:53 PM, NeilBrown wrote:
>> On Thu, 10 Feb 2011 16:16:44 +0000 Simon McNair<simonmcnair [at] gmail.co=
m> wrote:
>>
>>> Hi all
>>>
>>> I use a 3ware 9500-12 port sata card (JBOD) which will not work wit=
hout a
>>> 128mb sodimm. The sodimm socket is flakey and the result is that t=
he
>>> machine occasionally crashes. Yesterday I finally gave in and put
>>> together another
>>> machine so that I can rsync between them. When I turned the machin=
e
>>> on today to set up rync, the RAID array was not gone, but corrupted=
=2E
>>> Typical...
>> Presumably the old machine was called 'ubuntu' and the new machine '=
pro=C3=B8lox'
>>
>>
>>> I built the array in Aug 2010 using the following command:
>>>
>>> mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
>>> --raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64
>>>
>>> Using LVM, I did the following:
>>> pvscan
>>> pvcreate -M2 /dev/md0
>>> vgcreate lvm-raid /dev/md0
>>> vgdisplay lvm-raid
>>> vgscan
>>> lvscan
>>> lvcreate -v -l 100%VG -n RAID lvm-raid
>>> lvdisplay /dev/lvm-raid/lvm0
>>>
>>> I then formatted using:
>>> mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144
>>> /dev/lvm-raid/RAID
>>>
>>> This worked perfectly since I created the array. Now mdadm is comi=
ng up
>>> with
>>>
>>> proxmox:/dev/md# mdadm --assemble --scan --verbose
>>> mdadm: looking for devices for further assembly
>>> mdadm: no recogniseable superblock on /dev/md/ubuntu:0
>> And it seems that ubuntu:0 have been successfully assembled.
>> It is missing one device for some reason (sdd1) but RAID can cope wi=
th that.
> 3ware card is compromised, with a loose buffer memory dimm. Some of =
its ECC errors were caught and reported in dmesg. Its likely, based on=
the loose memory socket, that many multiple-bit errors got through.
>
> [trim /]
>
>>> mdadm: no uptodate device for slot 8 of /dev/md/pro=EF=BF=BDlox:0
>>> mdadm: no uptodate device for slot 9 of /dev/md/pro=EF=BF=BDlox:0
>>> mdadm: failed to add /dev/sdd1 to /dev/md/pro=EF=BF=BDlox:0: Invali=
d argument
>>> mdadm: /dev/md/pro=EF=BF=BDlox:0 assembled from 0 drives - not enou=
gh to start
>>> the array.
>> This looks like it is *after* to trying the --create command you giv=
e
>> below.. It is best to report things in the order they happen, else =
you can
>> confuse people (or get caught out!).
> Yes, this was after.
>
>>> mdadm: looking for devices for further assembly
>>> mdadm: no recogniseable superblock on /dev/sdd
>>> mdadm: No arrays found in config file or automatically
>>>
>>> pvscan and vgscan show nothing.
>>>
>>> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1=
=2E1
>>> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sd=
g1
>>> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=
=3D64
>>>
>>> as it seemed that /dev/sdd1 failed to be added to the array. This =
did
>>> nothing.
>> It did not to nothing. It wrote a superblock to /dev/sdd1 and compl=
ained
>> that it couldn't write to all the others --- didn't it?
> There were multiple attempts to create. One wrote to just sdd1, anot=
her succeeded with all but sdd1.
>
>>> dmesg contains:
>>>
>>> md: invalid superblock checksum on sdd1
>> I guess that is why sdd1 was missing from 'ubuntu:0'. Though as I c=
annot
>> tell if this happened before or after any of the various things repo=
rted
>> above, it is hard to be sure.
>>
>>
>> The real mystery is why 'pvscan' reports nothing.
> The original array was created with mdadm v2.6.7, and had a data offs=
et of 264 sectors. After Simon's various attempts to --create, he ende=
d up with data offset of 2048, using mdadm v3.1.4. The mdadm -E report=
s he posted to the list showed the 264 offset. We didn't realize the o=
ffset had been updated until somewhat later in our troubleshooting effo=
rts.
>
> In any case, pvscan couldn't see the LVM signature because it wasn't =
there (at offset 2048).
>
>> What about
>> pvscan --verbose
>>
>> or
>>
>> blkid -p /dev/md/ubuntu:0
>>
>> or even
>>
>> dd of=3D/dev/md/ubuntu:0 count=3D8 | od -c
> Fortunately, Simon did have a copy of his LVM configuration. With th=
e help of dd, strings, and grep, we did locate his LVM sig at the corre=
ct location on sdd1 (for data offset 264). After a number of attempts =
to bypass LVM and access his single LV with dmsetup (based on his backe=
d up configuration, on the assembled new array less sdd1), I realized t=
hat the data offset was wrong on the recreated array, and went looking =
for the cause. I found your git commit that changed that logic last sp=
ring, and recommended that Simon revert to the default package for his =
ubuntu install, which is v2.6.7.
>
> Simon has now attempted to recreate the array with v2.6.7, but the co=
ntroller is throwing too many errors to succeed, and I suggested it was=
too flakey to trust any further. Based on the existence of the LVM si=
g on sdd1, I believe Simon's data is (mostly) intact, and only needs a =
successful create operation with a properly functioning controller. (H=
e might also need to perform an lvm vgcfgrestore, but he has the necess=
ary backup file.)
>
> A new controller is on order.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On 02/15/2011 02:04 PM, Simon McNair wrote:
> Phil,
> Thanks for filling in the gaps, I had forgotton quite how much help and assistance you had provided up to now. You're a real godsend.
>
> To fill in some of the other pieces of info:
> The original machine was called proxmox (KVM virtualisation machine) and I used an Ubuntu live cd to see if it was the software which was preventing progress or not. It does seem like there is data corruption in the machine name for some reason.
>
> The original company I ordered the controller through sent me an email at 4pm saying that they could not fulfill the order, 4pm was too late for them to pick & pack so there was another 24hr turn around delay. The supermicro card should arrive here tomorrow and hopefully I'll be able to get a dd of each of the drives prior to Phil coming online.
>
> For some reason blkid doesn't exist on my machine even though I have e2fs-utils installed. V weird.

FWIW, on my VPS running Ubuntu Server 10.10, the blkid executable is part of util-linux, and installed in /sbin. You might need to re-install it.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

--Sig_/fUQ2CcW+MQ_xtLnDIIS+KHg
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

On Tue, 15 Feb 2011 14:37:55 -0500
Phil Turmel <philip [at] turmel.org> wrote:

> > For some reason blkid doesn't exist on my machine even though I have
> > e2fs-utils installed. V weird.

Weird is that you think it is somehow a "e2fs-utils" program.

dpkg -S blkid
....
util-linux: /sbin/blkid
....

>
> FWIW, on my VPS running Ubuntu Server 10.10, the blkid executable is part=
of
> util-linux, and installed in /sbin. You might need to re-install it.

--
With respect,
Roman

--Sig_/fUQ2CcW+MQ_xtLnDIIS+KHg
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAk1a19QACgkQTLKSvz+PZwgzRgCggc5PkkLeGXlGvKlLiAL/ iXvc
JRYAnjvEhUbHENECZNV0QHpsZV+MOR+o
=WXj6
-----END PGP SIGNATURE-----

--Sig_/fUQ2CcW+MQ_xtLnDIIS+KHg--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

I got my info from:

http://linux.die.net/man/8/blkid

"*blkid* is part the e2fsprogs package since version 1.26 and is
available from http://e2fsprogs.sourceforge.net
<http://e2fsprogs.sourceforge.net/>. "

I also did apt-file search blkid and it showed up as e2fsprogs or was it
e2fs-utils (I forget). The system I was/am running was Debian Lenny x64.

This just shows how little I know about Linux (I've only been dabbling
for 6 months or so). I'll install util-linux next time the machine
start up. Thanks for the pointer.

regards
Simon

On 15/02/2011 19:45, Roman Mamedov wrote:
> On Tue, 15 Feb 2011 14:37:55 -0500
> Phil Turmel<philip [at] turmel.org> wrote:
>
>>> For some reason blkid doesn't exist on my machine even though I have
>>> e2fs-utils installed. V weird.
> Weird is that you think it is somehow a "e2fs-utils" program.
>
> dpkg -S blkid
> ...
> util-linux: /sbin/blkid
> ...
>
>> FWIW, on my VPS running Ubuntu Server 10.10, the blkid executable is part of
>> util-linux, and installed in /sbin. You might need to re-install it.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

latest update. A bit long I'm afraid...

hi, card installed and devices plugged in. Now have
sda sda2 sdb1 sdc1 sde sdf1 sdg1 sdi sdj1 sdk1 sdl1 sdm1
sda1 sdb sdc sdd sdf sdg sdh sdj sdk sdl sdm


proxmox:/home/simon# ./lsdrv.sh
Controller device [at] pci0000:00/0000:00:01.0/0000:01:00.0 [mvsas]
SCSI storage controller: Marvell Technology Group Ltd.
MV64460/64461/64462 Sys tem
Controller, Revision B (rev 01)
host4: /dev/sdf ATA Hitachi HDS72101 {SN: GTA000PAGABXRA}
host4: /dev/sdg ATA Hitachi HDS72101 {SN: GTA000PAGAA5DA}
host4: /dev/sdh ATA Hitachi HDS72101 {SN: GTA000PAG9NL9A}
host4: /dev/sdi ATA Hitachi HDS72101 {SN: GTA000PAGA8V4A}
host4: /dev/sdj ATA Hitachi HDS72101 {SN: GTD000PAGMT9GD}
host4: /dev/sdk ATA Hitachi HDS72101 {SN: GTG000PAG18BJC}
host4: /dev/sdl ATA Hitachi HDS72101 {SN: GTG000PAG1DPLC}
host4: /dev/sdm ATA Hitachi HDS72101 {SN: GTA000PAG7WMEA}
Controller device [at]
pci0000:00/0000:00:1a.7/usb1/1-4/1-4.1/1-4.1.1/1-4.1.1:1.0=2 0
[ usb-storage]
Bus 001 Device 007: ID 0424:2228 Standard Microsystems Corp. 9-in-2
Card Reade r {SN: 08050920003A}
host9: /dev/sdd Generic Flash HS-CF
host9: /dev/sde Generic Flash HS-COMBO
Controller device [at] pci0000:00/0000:00:1c.4/0000:04:00.0 [ahci]
SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA
Controller (rev 03)
host7: [Empty]
host8: [Empty]
Controller device [at] pci0000:00/0000:00:1c.4/0000:04:00.1 [pata_jmicron]
IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA
Controller (r ev 03)
host5: [Empty]
host6: [Empty]
Controller device [at] pci0000:00/0000:00:1f.2 [ata_piix]
IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA
IDE Contro ller #1
host0: /dev/sda ATA STM3500418AS {SN: 9VM3QJ5C}
host1: /dev/sr0 Optiarc DVD RW AD-5240S
Controller device [at] pci0000:00/0000:00:1f.5 [ata_piix]
IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA
IDE Contro ller #2
host2: /dev/sdb ATA Hitachi HDS72101 {SN: GTD000PAGMT8DD}
host3: /dev/sdc ATA Hitachi HDS72101 {SN: GTG000PAG04V0C}


proxmox:/home/simon# parted -l
Model: ATA STM3500418AS (scsi)
Disk /dev/sda: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.8kB 537MB 537MB primary ext3 boot
2 537MB 500GB 500GB primary lvm


Error: The backup GPT table is not at the end of the disk, as it should=

be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the
end (and removing the old backup)?
=46ix/Cancel? c

Error: The backup GPT table is not at the end of the disk, as it should=

be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the
end (and removing the old backup)?
=46ix/Cancel? c

Error: The backup GPT table is not at the end of the disk, as it should=

be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the
end (and removing the old backup)?
=46ix/Cancel? c

Model: Linux device-mapper (linear) (dm)
Disk /dev/mapper/pve-data: 380GB
Sector size (logical/physical): 512B/512B
Partition Table: loop

Number Start End Size File system Flags
1 0.00B 380GB 380GB ext3


cModel: Linux device-mapper (linear) (dm)
Disk /dev/mapper/pve-root: 103GB
Sector size (logical/physical): 512B/512B
Partition Table: loop

Number Start End Size File system Flags
1 0.00B 103GB 103GB ext3


Model: Linux device-mapper (linear) (dm)
Disk /dev/mapper/pve-swap: 11.8GB
Sector size (logical/physical): 512B/512B
Partition Table: loop

Number Start End Size File system Flags
1 0.00B 11.8GB 11.8GB linux-swap


Error: The backup GPT table is not at the end of the disk, as it should=

be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the
end (and removing the old backup)?
=46ix/Cancel? c

Error: /dev/sdh: unrecognised disk label

Error: /dev/sdi: unrecognised disk label

Error: The backup GPT table is not at the end of the disk, as it should=

be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the
end (and removing the old backup)?
=46ix/Cancel? c

Error: The backup GPT table is not at the end of the disk, as it should=

be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the
end (and removing the old backup)?
=46ix/Cancel? c

Error: The backup GPT table is not at the end of the disk, as it should=

be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the
end (and removing the old backup)?
=46ix/Cancel? c

Error: The backup GPT table is not at the end of the disk, as it should=

be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the
end (and removing the old backup)?
=46ix/Cancel? c

Error: /dev/md0: unrecognised disk label


proxmox:/home/simon# mdadm --assemble --scan --verbose
mdadm: looking for devices for /dev/md/0
mdadm: cannot open device /dev/dm-2: Device or resource busy
mdadm: /dev/dm-2 has wrong uuid.
mdadm: cannot open device /dev/dm-1: Device or resource busy
mdadm: /dev/dm-1 has wrong uuid.
mdadm: cannot open device /dev/dm-0: Device or resource busy
mdadm: /dev/dm-0 has wrong uuid.
mdadm: no RAID superblock on /dev/sdf1
mdadm: /dev/sdf1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdf
mdadm: /dev/sdf has wrong uuid.
mdadm: no RAID superblock on /dev/sdm1
mdadm: /dev/sdm1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdm
mdadm: /dev/sdm has wrong uuid.
mdadm: no RAID superblock on /dev/sdl1
mdadm: /dev/sdl1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdl
mdadm: /dev/sdl has wrong uuid.
mdadm: no RAID superblock on /dev/sdk1
mdadm: /dev/sdk1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdk
mdadm: /dev/sdk has wrong uuid.
mdadm: no RAID superblock on /dev/sdj1
mdadm: /dev/sdj1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdj
mdadm: /dev/sdj has wrong uuid.
mdadm: /dev/sdi has wrong uuid.
mdadm: /dev/sdh has wrong uuid.
mdadm: no RAID superblock on /dev/sdg1
mdadm: /dev/sdg1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdg
mdadm: /dev/sdg has wrong uuid.
mdadm: no RAID superblock on /dev/sdc1
mdadm: /dev/sdc1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdc
mdadm: /dev/sdc has wrong uuid.
mdadm: no RAID superblock on /dev/sdb1
mdadm: /dev/sdb1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdb
mdadm: /dev/sdb has wrong uuid.
mdadm: cannot open device /dev/sda2: Device or resource busy
mdadm: /dev/sda2 has wrong uuid.
mdadm: cannot open device /dev/sda1: Device or resource busy
mdadm: /dev/sda1 has wrong uuid.
mdadm: cannot open device /dev/sda: Device or resource busy
mdadm: /dev/sda has wrong uuid.
mdadm: no devices found for /dev/md/0
mdadm: looking for devices for further assembly
mdadm: cannot open device /dev/dm-2: Device or resource busy
mdadm: cannot open device /dev/dm-1: Device or resource busy
mdadm: cannot open device /dev/dm-0: Device or resource busy
mdadm: no recogniseable superblock on /dev/sdf1
mdadm: no recogniseable superblock on /dev/sdf
mdadm: no recogniseable superblock on /dev/sdm1
mdadm: no recogniseable superblock on /dev/sdm
mdadm: no recogniseable superblock on /dev/sdl1
mdadm: no recogniseable superblock on /dev/sdl
mdadm: no recogniseable superblock on /dev/sdk1
mdadm: no recogniseable superblock on /dev/sdk
mdadm: no recogniseable superblock on /dev/sdj1
mdadm: no recogniseable superblock on /dev/sdj
mdadm: /dev/sdi is not built for host proxmox.
mdadm: /dev/sdh is not built for host proxmox.
mdadm: no recogniseable superblock on /dev/sdg1
mdadm: no recogniseable superblock on /dev/sdg
mdadm: no recogniseable superblock on /dev/sdc1
mdadm: no recogniseable superblock on /dev/sdc
mdadm: no recogniseable superblock on /dev/sdb1
mdadm: no recogniseable superblock on /dev/sdb
mdadm: cannot open device /dev/sda2: Device or resource busy
mdadm: cannot open device /dev/sda1: Device or resource busy
mdadm: cannot open device /dev/sda: Device or resource busy

proxmox:/home/simon# apt-show-versions -a mdadm
mdadm 2.6.7.2-3 install ok installed
mdadm 2.6.7.2-3 lenny ftp.uk.debian.org
No stable version
No testing version
mdadm 3.1.4-1+8efb9d1 sid ftp.uk.debian.org
mdadm/lenny uptodate 2.6.7.2-3

anything else you want ?

Simon

On 15/02/2011 14:51, Phil Turmel wrote:
> Hi Neil,
>
> Since Simon has responded, let me summarize the assistance I provided=
per his off-list request:
>
> On 02/14/2011 11:53 PM, NeilBrown wrote:
>> On Thu, 10 Feb 2011 16:16:44 +0000 Simon McNair<simonmcnair [at] gmail.co=
m> wrote:
>>
>>> Hi all
>>>
>>> I use a 3ware 9500-12 port sata card (JBOD) which will not work wit=
hout a
>>> 128mb sodimm. The sodimm socket is flakey and the result is that t=
he
>>> machine occasionally crashes. Yesterday I finally gave in and put
>>> together another
>>> machine so that I can rsync between them. When I turned the machin=
e
>>> on today to set up rync, the RAID array was not gone, but corrupted=
=2E
>>> Typical...
>> Presumably the old machine was called 'ubuntu' and the new machine '=
pro=C3=B8lox'
>>
>>
>>> I built the array in Aug 2010 using the following command:
>>>
>>> mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
>>> --raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64
>>>
>>> Using LVM, I did the following:
>>> pvscan
>>> pvcreate -M2 /dev/md0
>>> vgcreate lvm-raid /dev/md0
>>> vgdisplay lvm-raid
>>> vgscan
>>> lvscan
>>> lvcreate -v -l 100%VG -n RAID lvm-raid
>>> lvdisplay /dev/lvm-raid/lvm0
>>>
>>> I then formatted using:
>>> mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144
>>> /dev/lvm-raid/RAID
>>>
>>> This worked perfectly since I created the array. Now mdadm is comi=
ng up
>>> with
>>>
>>> proxmox:/dev/md# mdadm --assemble --scan --verbose
>>> mdadm: looking for devices for further assembly
>>> mdadm: no recogniseable superblock on /dev/md/ubuntu:0
>> And it seems that ubuntu:0 have been successfully assembled.
>> It is missing one device for some reason (sdd1) but RAID can cope wi=
th that.
> 3ware card is compromised, with a loose buffer memory dimm. Some of =
its ECC errors were caught and reported in dmesg. Its likely, based on=
the loose memory socket, that many multiple-bit errors got through.
>
> [trim /]
>
>>> mdadm: no uptodate device for slot 8 of /dev/md/pro=EF=BF=BDlox:0
>>> mdadm: no uptodate device for slot 9 of /dev/md/pro=EF=BF=BDlox:0
>>> mdadm: failed to add /dev/sdd1 to /dev/md/pro=EF=BF=BDlox:0: Invali=
d argument
>>> mdadm: /dev/md/pro=EF=BF=BDlox:0 assembled from 0 drives - not enou=
gh to start
>>> the array.
>> This looks like it is *after* to trying the --create command you giv=
e
>> below.. It is best to report things in the order they happen, else =
you can
>> confuse people (or get caught out!).
> Yes, this was after.
>
>>> mdadm: looking for devices for further assembly
>>> mdadm: no recogniseable superblock on /dev/sdd
>>> mdadm: No arrays found in config file or automatically
>>>
>>> pvscan and vgscan show nothing.
>>>
>>> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1=
=2E1
>>> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sd=
g1
>>> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=
=3D64
>>>
>>> as it seemed that /dev/sdd1 failed to be added to the array. This =
did
>>> nothing.
>> It did not to nothing. It wrote a superblock to /dev/sdd1 and compl=
ained
>> that it couldn't write to all the others --- didn't it?
> There were multiple attempts to create. One wrote to just sdd1, anot=
her succeeded with all but sdd1.
>
>>> dmesg contains:
>>>
>>> md: invalid superblock checksum on sdd1
>> I guess that is why sdd1 was missing from 'ubuntu:0'. Though as I c=
annot
>> tell if this happened before or after any of the various things repo=
rted
>> above, it is hard to be sure.
>>
>>
>> The real mystery is why 'pvscan' reports nothing.
> The original array was created with mdadm v2.6.7, and had a data offs=
et of 264 sectors. After Simon's various attempts to --create, he ende=
d up with data offset of 2048, using mdadm v3.1.4. The mdadm -E report=
s he posted to the list showed the 264 offset. We didn't realize the o=
ffset had been updated until somewhat later in our troubleshooting effo=
rts.
>
> In any case, pvscan couldn't see the LVM signature because it wasn't =
there (at offset 2048).
>
>> What about
>> pvscan --verbose
>>
>> or
>>
>> blkid -p /dev/md/ubuntu:0
>>
>> or even
>>
>> dd of=3D/dev/md/ubuntu:0 count=3D8 | od -c
> Fortunately, Simon did have a copy of his LVM configuration. With th=
e help of dd, strings, and grep, we did locate his LVM sig at the corre=
ct location on sdd1 (for data offset 264). After a number of attempts =
to bypass LVM and access his single LV with dmsetup (based on his backe=
d up configuration, on the assembled new array less sdd1), I realized t=
hat the data offset was wrong on the recreated array, and went looking =
for the cause. I found your git commit that changed that logic last sp=
ring, and recommended that Simon revert to the default package for his =
ubuntu install, which is v2.6.7.
>
> Simon has now attempted to recreate the array with v2.6.7, but the co=
ntroller is throwing too many errors to succeed, and I suggested it was=
too flakey to trust any further. Based on the existence of the LVM si=
g on sdd1, I believe Simon's data is (mostly) intact, and only needs a =
successful create operation with a properly functioning controller. (H=
e might also need to perform an lvm vgcfgrestore, but he has the necess=
ary backup file.)
>
> A new controller is on order.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

one other snippet:

proxmox:/home/simon# for x in /dev/sd{d..m} ; do echo $x ; dd if=3D$x
skip=3D2312 count=3D128 2>/dev/null |strings |grep
9fAJEz-HcaP-RQ51-fV8b-nxrN-Uqwb-PPnOLJ ; done
/dev/sdd
/dev/sde
/dev/sdf
/dev/sdg
/dev/sdh
/dev/sdi
id =3D "9fAJEz-HcaP-RQ51-fV8b-nxrN-Uqwb-PPnOLJ"
id =3D "9fAJEz-HcaP-RQ51-fV8b-nxrN-Uqwb-PPnOLJ"
/dev/sdj
/dev/sdk
/dev/sdl
/dev/sdm



On 15/02/2011 14:51, Phil Turmel wrote:
> Hi Neil,
>
> Since Simon has responded, let me summarize the assistance I provided=
per his off-list request:
>
> On 02/14/2011 11:53 PM, NeilBrown wrote:
>> On Thu, 10 Feb 2011 16:16:44 +0000 Simon McNair<simonmcnair [at] gmail.co=
m> wrote:
>>
>>> Hi all
>>>
>>> I use a 3ware 9500-12 port sata card (JBOD) which will not work wit=
hout a
>>> 128mb sodimm. The sodimm socket is flakey and the result is that t=
he
>>> machine occasionally crashes. Yesterday I finally gave in and put
>>> together another
>>> machine so that I can rsync between them. When I turned the machin=
e
>>> on today to set up rync, the RAID array was not gone, but corrupted=
=2E
>>> Typical...
>> Presumably the old machine was called 'ubuntu' and the new machine '=
pro=C3=B8lox'
>>
>>
>>> I built the array in Aug 2010 using the following command:
>>>
>>> mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
>>> --raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64
>>>
>>> Using LVM, I did the following:
>>> pvscan
>>> pvcreate -M2 /dev/md0
>>> vgcreate lvm-raid /dev/md0
>>> vgdisplay lvm-raid
>>> vgscan
>>> lvscan
>>> lvcreate -v -l 100%VG -n RAID lvm-raid
>>> lvdisplay /dev/lvm-raid/lvm0
>>>
>>> I then formatted using:
>>> mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144
>>> /dev/lvm-raid/RAID
>>>
>>> This worked perfectly since I created the array. Now mdadm is comi=
ng up
>>> with
>>>
>>> proxmox:/dev/md# mdadm --assemble --scan --verbose
>>> mdadm: looking for devices for further assembly
>>> mdadm: no recogniseable superblock on /dev/md/ubuntu:0
>> And it seems that ubuntu:0 have been successfully assembled.
>> It is missing one device for some reason (sdd1) but RAID can cope wi=
th that.
> 3ware card is compromised, with a loose buffer memory dimm. Some of =
its ECC errors were caught and reported in dmesg. Its likely, based on=
the loose memory socket, that many multiple-bit errors got through.
>
> [trim /]
>
>>> mdadm: no uptodate device for slot 8 of /dev/md/pro=EF=BF=BDlox:0
>>> mdadm: no uptodate device for slot 9 of /dev/md/pro=EF=BF=BDlox:0
>>> mdadm: failed to add /dev/sdd1 to /dev/md/pro=EF=BF=BDlox:0: Invali=
d argument
>>> mdadm: /dev/md/pro=EF=BF=BDlox:0 assembled from 0 drives - not enou=
gh to start
>>> the array.
>> This looks like it is *after* to trying the --create command you giv=
e
>> below.. It is best to report things in the order they happen, else =
you can
>> confuse people (or get caught out!).
> Yes, this was after.
>
>>> mdadm: looking for devices for further assembly
>>> mdadm: no recogniseable superblock on /dev/sdd
>>> mdadm: No arrays found in config file or automatically
>>>
>>> pvscan and vgscan show nothing.
>>>
>>> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1=
=2E1
>>> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sd=
g1
>>> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=
=3D64
>>>
>>> as it seemed that /dev/sdd1 failed to be added to the array. This =
did
>>> nothing.
>> It did not to nothing. It wrote a superblock to /dev/sdd1 and compl=
ained
>> that it couldn't write to all the others --- didn't it?
> There were multiple attempts to create. One wrote to just sdd1, anot=
her succeeded with all but sdd1.
>
>>> dmesg contains:
>>>
>>> md: invalid superblock checksum on sdd1
>> I guess that is why sdd1 was missing from 'ubuntu:0'. Though as I c=
annot
>> tell if this happened before or after any of the various things repo=
rted
>> above, it is hard to be sure.
>>
>>
>> The real mystery is why 'pvscan' reports nothing.
> The original array was created with mdadm v2.6.7, and had a data offs=
et of 264 sectors. After Simon's various attempts to --create, he ende=
d up with data offset of 2048, using mdadm v3.1.4. The mdadm -E report=
s he posted to the list showed the 264 offset. We didn't realize the o=
ffset had been updated until somewhat later in our troubleshooting effo=
rts.
>
> In any case, pvscan couldn't see the LVM signature because it wasn't =
there (at offset 2048).
>
>> What about
>> pvscan --verbose
>>
>> or
>>
>> blkid -p /dev/md/ubuntu:0
>>
>> or even
>>
>> dd of=3D/dev/md/ubuntu:0 count=3D8 | od -c
> Fortunately, Simon did have a copy of his LVM configuration. With th=
e help of dd, strings, and grep, we did locate his LVM sig at the corre=
ct location on sdd1 (for data offset 264). After a number of attempts =
to bypass LVM and access his single LV with dmsetup (based on his backe=
d up configuration, on the assembled new array less sdd1), I realized t=
hat the data offset was wrong on the recreated array, and went looking =
for the cause. I found your git commit that changed that logic last sp=
ring, and recommended that Simon revert to the default package for his =
ubuntu install, which is v2.6.7.
>
> Simon has now attempted to recreate the array with v2.6.7, but the co=
ntroller is throwing too many errors to succeed, and I suggested it was=
too flakey to trust any further. Based on the existence of the LVM si=
g on sdd1, I believe Simon's data is (mostly) intact, and only needs a =
successful create operation with a properly functioning controller. (H=
e might also need to perform an lvm vgcfgrestore, but he has the necess=
ary backup file.)
>
> A new controller is on order.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Good morning, Simon,

On 02/16/2011 08:51 AM, Simon McNair wrote:
> latest update. A bit long I'm afraid...
>
> hi, card installed and devices plugged in. Now have
> sda sda2 sdb1 sdc1 sde sdf1 sdg1 sdi sdj1 sdk1 sdl1 sdm1
> sda1 sdb sdc sdd sdf sdg sdh sdj sdk sdl sdm

Hmmm.

> proxmox:/home/simon# ./lsdrv.sh
> Controller device [at] pci0000:00/0000:00:01.0/0000:01:00.0 [mvsas]
> SCSI storage controller: Marvell Technology Group Ltd. MV64460/64461/64462 Sys tem Controller, Revision B (rev 01)
> host4: /dev/sdf ATA Hitachi HDS72101 {SN: GTA000PAGABXRA}
> host4: /dev/sdg ATA Hitachi HDS72101 {SN: GTA000PAGAA5DA}
> host4: /dev/sdh ATA Hitachi HDS72101 {SN: GTA000PAG9NL9A}
> host4: /dev/sdi ATA Hitachi HDS72101 {SN: GTA000PAGA8V4A}
> host4: /dev/sdj ATA Hitachi HDS72101 {SN: GTD000PAGMT9GD}
> host4: /dev/sdk ATA Hitachi HDS72101 {SN: GTG000PAG18BJC}
> host4: /dev/sdl ATA Hitachi HDS72101 {SN: GTG000PAG1DPLC}
> host4: /dev/sdm ATA Hitachi HDS72101 {SN: GTA000PAG7WMEA}
> Controller device [at] pci0000:00/0000:00:1a.7/usb1/1-4/1-4.1/1-4.1.1/1-4.1.1:1.0 [ usb-storage]
> Bus 001 Device 007: ID 0424:2228 Standard Microsystems Corp. 9-in-2 Card Reade r {SN: 08050920003A}
> host9: /dev/sdd Generic Flash HS-CF
> host9: /dev/sde Generic Flash HS-COMBO
> Controller device [at] pci0000:00/0000:00:1c.4/0000:04:00.0 [ahci]
> SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
> host7: [Empty]
> host8: [Empty]
> Controller device [at] pci0000:00/0000:00:1c.4/0000:04:00.1 [pata_jmicron]
> IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (r ev 03)
> host5: [Empty]
> host6: [Empty]
> Controller device [at] pci0000:00/0000:00:1f.2 [ata_piix]
> IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Contro ller #1
> host0: /dev/sda ATA STM3500418AS {SN: 9VM3QJ5C}
> host1: /dev/sr0 Optiarc DVD RW AD-5240S
> Controller device [at] pci0000:00/0000:00:1f.5 [ata_piix]
> IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Contro ller #2
> host2: /dev/sdb ATA Hitachi HDS72101 {SN: GTD000PAGMT8DD}
> host3: /dev/sdc ATA Hitachi HDS72101 {SN: GTG000PAG04V0C}

Good thing we recorded the serial numbers. From an earlier run of "lsdrv":

> Phil,
> sg3-utils did the job :-)
> Sorry for doubting you.
>
> host6: /dev/sdd AMCC 9500S-12 DISK {SN: PAGA8V4A3B8378002254}
> host6: /dev/sde AMCC 9500S-12 DISK {SN: PAG9NL9A3B8387004C88}
> host6: /dev/sdf AMCC 9500S-12 DISK {SN: PAGAA5DA3B8396001AE0}
> host6: /dev/sdg AMCC 9500S-12 DISK {SN: PAGABXRA3B83A0005EFB}
> host6: /dev/sdh AMCC 9500S-12 DISK {SN: PAG7WMEA3B83AA003E4F}
> host6: /dev/sdi AMCC 9500S-12 DISK {SN: PAG1DPLC3B83B40026E6}
> host6: /dev/sdj AMCC 9500S-12 DISK {SN: PAG18BJC3B83C3004760}
> host6: /dev/sdk AMCC 9500S-12 DISK {SN: PAGMT9GD3B83CD004B18}
> host6: /dev/sdl AMCC 9500S-12 DISK {SN: PAGMT8DD3B83D70021FF}
> host6: /dev/sdm AMCC 9500S-12 DISK {SN: PAG04V0C3B83E100ACA0}
>
> Simon

I don't know why the serial numbers are formatted differently, but we can still tell them apart (the eight characters starting with "PAG").

So, our device order in your new setup is: [ihgfmlkjbc], where /dev/sdi corresponds to the original report's /dev/sdd, which matches the sig grep in your other note.

Another note: The controller for sd[abc] is still showing ata_piix as its controller. That means you cannot hot-plug those ports. If you change your BIOS to AHCI mode instead of "Compatibility" or "Emulation", the full-featured ahci driver will run those ports. Not urgent, but I highly recommend it.

> proxmox:/home/simon# parted -l
> Model: ATA STM3500418AS (scsi)
> Disk /dev/sda: 500GB
> Sector size (logical/physical): 512B/512B
> Partition Table: msdos
>
> Number Start End Size Type File system Flags
> 1 32.8kB 537MB 537MB primary ext3 boot
> 2 537MB 500GB 500GB primary lvm
>
>
> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
> Fix/Cancel? c

The 3ware controller must have reserved some space at the end of each drive for its own use. Didn't know it'd do that. You will have to fix that.

[trim /]

> Error: /dev/sdh: unrecognised disk label
>
> Error: /dev/sdi: unrecognised disk label

It seems the flaky controller took out these partition tables. Hope that's all it got. They'll have to be re-created with parted.

Please run parted on each of the ten drives, and make sure they end up like so:

> proxmox:/home/simon# for x in sd{d..m} ; do parted -s /dev/$x unit s
> print ; done
> Model: AMCC 9500S-12 DISK (scsi)
> Disk /dev/sdd: 1953103872s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid

Make sure you request "unit s" before your other commands so we can make sure it matches.

When you think you are done, check them all:

for x in /dev/sd{i,h,g,f,m,l,k,j,b,c} ; do parted -s $x unit s print ; done

After that, create:

mdadm --create --verbose --assume-clean /dev/md0 --metadata=1.1 --level=5 --raid-devices=10 /dev/sd{i,h,g,f,m,l,k,j,b,c}1 --chunk=64

And finally:

pvscan --verbose

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Phil,
A couple of questions please.

On 16/02/2011 14:37, Phil Turmel wrote:
> Good morning, Simon,
>
> On 02/16/2011 08:51 AM, Simon McNair wrote:
>> latest update. A bit long I'm afraid...
>>
>> hi, card installed and devices plugged in. Now have
>> sda sda2 sdb1 sdc1 sde sdf1 sdg1 sdi sdj1 sdk1 sdl1 sdm1
>> sda1 sdb sdc sdd sdf sdg sdh sdj sdk sdl sdm
> Hmmm.
>
>> proxmox:/home/simon# ./lsdrv.sh
>> Controller device [at] pci0000:00/0000:00:01.0/0000:01:00.0 [mvsas]
>> SCSI storage controller: Marvell Technology Group Ltd. MV64460/64461/64462 Sys tem Controller, Revision B (rev 01)
>> host4: /dev/sdf ATA Hitachi HDS72101 {SN: GTA000PAGABXRA}
>> host4: /dev/sdg ATA Hitachi HDS72101 {SN: GTA000PAGAA5DA}
>> host4: /dev/sdh ATA Hitachi HDS72101 {SN: GTA000PAG9NL9A}
>> host4: /dev/sdi ATA Hitachi HDS72101 {SN: GTA000PAGA8V4A}
>> host4: /dev/sdj ATA Hitachi HDS72101 {SN: GTD000PAGMT9GD}
>> host4: /dev/sdk ATA Hitachi HDS72101 {SN: GTG000PAG18BJC}
>> host4: /dev/sdl ATA Hitachi HDS72101 {SN: GTG000PAG1DPLC}
>> host4: /dev/sdm ATA Hitachi HDS72101 {SN: GTA000PAG7WMEA}
>> Controller device [at] pci0000:00/0000:00:1a.7/usb1/1-4/1-4.1/1-4.1.1/1-4.1.1:1.0 [ usb-storage]
>> Bus 001 Device 007: ID 0424:2228 Standard Microsystems Corp. 9-in-2 Card Reade r {SN: 08050920003A}
>> host9: /dev/sdd Generic Flash HS-CF
>> host9: /dev/sde Generic Flash HS-COMBO
>> Controller device [at] pci0000:00/0000:00:1c.4/0000:04:00.0 [ahci]
>> SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
>> host7: [Empty]
>> host8: [Empty]
>> Controller device [at] pci0000:00/0000:00:1c.4/0000:04:00.1 [pata_jmicron]
>> IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (r ev 03)
>> host5: [Empty]
>> host6: [Empty]
>> Controller device [at] pci0000:00/0000:00:1f.2 [ata_piix]
>> IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Contro ller #1
>> host0: /dev/sda ATA STM3500418AS {SN: 9VM3QJ5C}
>> host1: /dev/sr0 Optiarc DVD RW AD-5240S
>> Controller device [at] pci0000:00/0000:00:1f.5 [ata_piix]
>> IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Contro ller #2
>> host2: /dev/sdb ATA Hitachi HDS72101 {SN: GTD000PAGMT8DD}
>> host3: /dev/sdc ATA Hitachi HDS72101 {SN: GTG000PAG04V0C}
> Good thing we recorded the serial numbers. From an earlier run of "lsdrv":
>
>> Phil,
>> sg3-utils did the job :-)
>> Sorry for doubting you.
>>
>> host6: /dev/sdd AMCC 9500S-12 DISK {SN: PAGA8V4A3B8378002254}
>> host6: /dev/sde AMCC 9500S-12 DISK {SN: PAG9NL9A3B8387004C88}
>> host6: /dev/sdf AMCC 9500S-12 DISK {SN: PAGAA5DA3B8396001AE0}
>> host6: /dev/sdg AMCC 9500S-12 DISK {SN: PAGABXRA3B83A0005EFB}
>> host6: /dev/sdh AMCC 9500S-12 DISK {SN: PAG7WMEA3B83AA003E4F}
>> host6: /dev/sdi AMCC 9500S-12 DISK {SN: PAG1DPLC3B83B40026E6}
>> host6: /dev/sdj AMCC 9500S-12 DISK {SN: PAG18BJC3B83C3004760}
>> host6: /dev/sdk AMCC 9500S-12 DISK {SN: PAGMT9GD3B83CD004B18}
>> host6: /dev/sdl AMCC 9500S-12 DISK {SN: PAGMT8DD3B83D70021FF}
>> host6: /dev/sdm AMCC 9500S-12 DISK {SN: PAG04V0C3B83E100ACA0}
>>
>> Simon
> I don't know why the serial numbers are formatted differently, but we can still tell them apart (the eight characters starting with "PAG").
>
> So, our device order in your new setup is: [ihgfmlkjbc], where /dev/sdi corresponds to the original report's /dev/sdd, which matches the sig grep in your other note.
>
> Another note: The controller for sd[abc] is still showing ata_piix as its controller. That means you cannot hot-plug those ports. If you change your BIOS to AHCI mode instead of "Compatibility" or "Emulation", the full-featured ahci driver will run those ports. Not urgent, but I highly recommend it.
>
Will do that now, before I forget

>> proxmox:/home/simon# parted -l
>> Model: ATA STM3500418AS (scsi)
>> Disk /dev/sda: 500GB
>> Sector size (logical/physical): 512B/512B
>> Partition Table: msdos
>>
>> Number Start End Size Type File system Flags
>> 1 32.8kB 537MB 537MB primary ext3 boot
>> 2 537MB 500GB 500GB primary lvm
>>
>>
>> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
>> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
>> Fix/Cancel? c
> The 3ware controller must have reserved some space at the end of each drive for its own use. Didn't know it'd do that. You will have to fix that.
>
> [trim /]
>
Do you have any suggestions on how I can fix that ? I don't have a clue
>> Error: /dev/sdh: unrecognised disk label
>>
>> Error: /dev/sdi: unrecognised disk label
> It seems the flaky controller took out these partition tables. Hope that's all it got. They'll have to be re-created with parted.
>
> Please run parted on each of the ten drives, and make sure they end up like so:
>
>> proxmox:/home/simon# for x in sd{d..m} ; do parted -s /dev/$x unit s
>> print ; done
>> Model: AMCC 9500S-12 DISK (scsi)
>> Disk /dev/sdd: 1953103872s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
> Make sure you request "unit s" before your other commands so we can make sure it matches.
>
> When you think you are done, check them all:
>

when I was trying to figure out the command for this using 'man parted'
I came across this:
" rescue start end
Rescue a lost partition that was located
somewhere between start and end. If a partition is
found, parted will ask if you want to create an
entry for it in the partition table."
Is it worth trying ?

I originally created the partitions like so:
parted -s /dev/sdb rm 1
parted -s /dev/sdb mklabel gpt
parted -s --align optimal /dev/sdb mkpart primary ext4 .512 100%
parted -s /dev/sdb set 1 raid on
parted -s /dev/sdb align-check optimal 1

so to recreate the above I would do:
parted -s /dev/sdb mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdc mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdf mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdg mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdh mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdi mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdj mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdk mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdl mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdm mkpart primary ext4 2048s 1953101823s

I'm guessing the backups that I want to do can wait until any potential
fsck ?

sorry if the questions are dumb but I'm not sure what I'm doing and I'd
rather ask more questions than fewer and understand the implications of
what I'm doing.

thanks
Simon

> for x in /dev/sd{i,h,g,f,m,l,k,j,b,c} ; do parted -s $x unit s print ; done
>
> After that, create:
>
> mdadm --create --verbose --assume-clean /dev/md0 --metadata=1.1 --level=5 --raid-devices=10 /dev/sd{i,h,g,f,m,l,k,j,b,c}1 --chunk=64
>
> And finally:
>
> pvscan --verbose
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On 02/16/2011 12:49 PM, Simon McNair wrote:
> Hi Phil,
> A couple of questions please.

[trim /]

>>> Simon
>> I don't know why the serial numbers are formatted differently, but we can still tell them apart (the eight characters starting with "PAG").
>>
>> So, our device order in your new setup is: [ihgfmlkjbc], where /dev/sdi corresponds to the original report's /dev/sdd, which matches the sig grep in your other note.
>>
>> Another note: The controller for sd[abc] is still showing ata_piix as its controller. That means you cannot hot-plug those ports. If you change your BIOS to AHCI mode instead of "Compatibility" or "Emulation", the full-featured ahci driver will run those ports. Not urgent, but I highly recommend it.
>>
> Will do that now, before I forget

Hot-pluggability with suitable trays is very handy! :)

[trim /]

>>> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
>>> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
>>> Fix/Cancel? c
>> The 3ware controller must have reserved some space at the end of each drive for its own use. Didn't know it'd do that. You will have to fix that.
>>
>> [trim /]
>>
> Do you have any suggestions on how I can fix that ? I don't have a clue

Just do 'parted /dev/sd?' and on the ones it offers to fix, say yes. Then request 'unit s' and 'print' to verify that it is correct.

[trim /]

> when I was trying to figure out the command for this using 'man parted' I came across this:
> " rescue start end
> Rescue a lost partition that was located somewhere between start and end. If a partition is
> found, parted will ask if you want to create an entry for it in the partition table."
> Is it worth trying ?

Nah. That's for when you don't know exactly where the partition is. We know.

> I originally created the partitions like so:
> parted -s /dev/sdb rm 1
> parted -s /dev/sdb mklabel gpt
> parted -s --align optimal /dev/sdb mkpart primary ext4 .512 100%
> parted -s /dev/sdb set 1 raid on
> parted -s /dev/sdb align-check optimal 1
>
> so to recreate the above I would do:
> parted -s /dev/sdb mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdc mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdf mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdg mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdh mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdi mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdj mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdk mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdl mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdm mkpart primary ext4 2048s 1953101823s

Only recreate the partition tables where you have to, i.e., the 'Fix' option above didn't work. And don't specify a filesystem.

Probably just /dev/sdh and /dev/sdi. Like so, though:

parted -s /dev/sdh mklabel gpt mkpart primary 2048s 1953101823s set 1 raid on
parted -s /dev/sdi mklabel gpt mkpart primary 2048s 1953101823s set 1 raid on

> I'm guessing the backups that I want to do can wait until any potential fsck ?

Do an 'fsck -N' first, and if it passes, or has few errors, mount the filesystem readonly and grab your backup. Then let fsck have at it for real. If anything gets fixed, compare your backup from the read-only fs to the fixed fs.

Given your flaky old controller, I expect there'll be *some* problems.

> sorry if the questions are dumb but I'm not sure what I'm doing and I'd rather ask more questions than fewer and understand the implications of what I'm doing.

Oh, no. You are right to be paranoid. If anything looks funny, stop.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

parted -l gives

Error: The backup GPT table is not at the end of the disk, as it should
be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the
end (and removing the old backup)?
Fix/Cancel? fix
Warning: Not all of the space available to /dev/sdb appears to be used,
you can fix the GPT to use all of the space (an
extra 421296 blocks) or continue with the current setting?
Fix/Ignore?

I'm guessing ignore ?

Simon

On 16/02/2011 18:14, Phil Turmel wrote:
> On 02/16/2011 12:49 PM, Simon McNair wrote:
>> Hi Phil,
>> A couple of questions please.
> [trim /]
>
>>>> Simon
>>> I don't know why the serial numbers are formatted differently, but we can still tell them apart (the eight characters starting with "PAG").
>>>
>>> So, our device order in your new setup is: [ihgfmlkjbc], where /dev/sdi corresponds to the original report's /dev/sdd, which matches the sig grep in your other note.
>>>
>>> Another note: The controller for sd[abc] is still showing ata_piix as its controller. That means you cannot hot-plug those ports. If you change your BIOS to AHCI mode instead of "Compatibility" or "Emulation", the full-featured ahci driver will run those ports. Not urgent, but I highly recommend it.
>>>
>> Will do that now, before I forget
> Hot-pluggability with suitable trays is very handy! :)
>
> [trim /]
>
>>>> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
>>>> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
>>>> Fix/Cancel? c
>>> The 3ware controller must have reserved some space at the end of each drive for its own use. Didn't know it'd do that. You will have to fix that.
>>>
>>> [trim /]
>>>
>> Do you have any suggestions on how I can fix that ? I don't have a clue
> Just do 'parted /dev/sd?' and on the ones it offers to fix, say yes. Then request 'unit s' and 'print' to verify that it is correct.
>
> [trim /]
>
>> when I was trying to figure out the command for this using 'man parted' I came across this:
>> " rescue start end
>> Rescue a lost partition that was located somewhere between start and end. If a partition is
>> found, parted will ask if you want to create an entry for it in the partition table."
>> Is it worth trying ?
> Nah. That's for when you don't know exactly where the partition is. We know.
>
>> I originally created the partitions like so:
>> parted -s /dev/sdb rm 1
>> parted -s /dev/sdb mklabel gpt
>> parted -s --align optimal /dev/sdb mkpart primary ext4 .512 100%
>> parted -s /dev/sdb set 1 raid on
>> parted -s /dev/sdb align-check optimal 1
>>
>> so to recreate the above I would do:
>> parted -s /dev/sdb mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdc mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdf mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdg mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdh mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdi mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdj mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdk mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdl mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdm mkpart primary ext4 2048s 1953101823s
> Only recreate the partition tables where you have to, i.e., the 'Fix' option above didn't work. And don't specify a filesystem.
>
> Probably just /dev/sdh and /dev/sdi. Like so, though:
>
> parted -s /dev/sdh mklabel gpt mkpart primary 2048s 1953101823s set 1 raid on
> parted -s /dev/sdi mklabel gpt mkpart primary 2048s 1953101823s set 1 raid on
>
>> I'm guessing the backups that I want to do can wait until any potential fsck ?
> Do an 'fsck -N' first, and if it passes, or has few errors, mount the filesystem readonly and grab your backup. Then let fsck have at it for real. If anything gets fixed, compare your backup from the read-only fs to the fixed fs.
>
> Given your flaky old controller, I expect there'll be *some* problems.
>
>> sorry if the questions are dumb but I'm not sure what I'm doing and I'd rather ask more questions than fewer and understand the implications of what I'm doing.
> Oh, no. You are right to be paranoid. If anything looks funny, stop.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On 02/16/2011 01:18 PM, Simon McNair wrote:
> parted -l gives
>
> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
> Fix/Cancel? fix
> Warning: Not all of the space available to /dev/sdb appears to be used, you can fix the GPT to use all of the space (an
> extra 421296 blocks) or continue with the current setting?
> Fix/Ignore?
>
> I'm guessing ignore ?

Either should work. If you 'Fix' for all of the drives, you could later 'grow' your array to include the extra space.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On 02/16/2011 01:22 PM, Phil Turmel wrote:
> On 02/16/2011 01:18 PM, Simon McNair wrote:
>> parted -l gives
>>
>> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
>> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
>> Fix/Cancel? fix
>> Warning: Not all of the space available to /dev/sdb appears to be used, you can fix the GPT to use all of the space (an
>> extra 421296 blocks) or continue with the current setting?
>> Fix/Ignore?
>>
>> I'm guessing ignore ?
>
> Either should work. If you 'Fix' for all of the drives, you could later 'grow' your array to include the extra space.

It's vital that the start sector be 2048, though. Please do recheck after all are done.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

have done this.... Results are:

proxmox:/home/simon# ./lsdrv.sh
Controller device [at] pci0000:00/0000:00:01.0/0000:01:00.0 [mvsas]
SCSI storage controller: Marvell Technology Group Ltd.
MV64460/64461/64462 System Controller, Revision B (rev 01)
host0: /dev/sdf ATA Hitachi HDS72101 {SN: GTA000PAGABXRA}
host0: /dev/sdg ATA Hitachi HDS72101 {SN: GTA000PAGAA5DA}
host0: /dev/sdh ATA Hitachi HDS72101 {SN: GTA000PAG9NL9A}
host0: /dev/sdi ATA Hitachi HDS72101 {SN: GTA000PAGA8V4A}
host0: /dev/sdj ATA Hitachi HDS72101 {SN: GTD000PAGMT9GD}
host0: /dev/sdk ATA Hitachi HDS72101 {SN: GTG000PAG18BJC}
host0: /dev/sdl ATA Hitachi HDS72101 {SN: GTG000PAG1DPLC}
host0: /dev/sdm ATA Hitachi HDS72101 {SN: GTA000PAG7WMEA}
Controller device [at]
pci0000:00/0000:00:1a.7/usb1/1-4/1-4.1/1-4.1.1/1-4.1.1:1.0 [usb-storage]
Bus 001 Device 007: ID 0424:2228 Standard Microsystems Corp. 9-in-2
Card Reader {SN: 08050920003A}
host11: /dev/sdb Generic Flash HS-CF
host11: /dev/sdc Generic Flash HS-COMBO
Controller device [at] pci0000:00/0000:00:1c.4/0000:04:00.0 [ahci]
SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA
Controller (rev 03)
host9: [Empty]
host10: [Empty]
Controller device [at] pci0000:00/0000:00:1c.4/0000:04:00.1 [pata_jmicron]
IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA
Controller (rev 03)
host1: [Empty]
host2: [Empty]
Controller device [at] pci0000:00/0000:00:1f.2 [ahci]
SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI
Controller
host3: /dev/sda ATA STM3500418AS {SN: 9VM3QJ5C}
host4: /dev/sr0 Optiarc DVD RW AD-5240S
host5: [Empty]
host6: [Empty]
host7: /dev/sdd ATA Hitachi HDS72101 {SN: GTD000PAGMT8DD}
host8: /dev/sde ATA Hitachi HDS72101 {SN: GTG000PAG04V0C}

enabling achi has switched the drive letters around for a couple of
drives again

parted list is:

proxmox:/home/simon# for x in /dev/sd{i,h,g,f,m,l,k,j,d,e} ; do parted
-s $x unit s print ; done
Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdi: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdh: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdg: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdf: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdm: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdl: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdk: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdj: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdd: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sde: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

is it now:

mdadm --create --verbose --assume-clean /dev/md0 --metadata=1.1
--level=5 --raid-devices=10 /dev/sd{i,h,g,f,m,l,k,j,d,e}1 --chunk=64

?
regards
Simon



On 16/02/2011 18:25, Phil Turmel wrote:
> On 02/16/2011 01:22 PM, Phil Turmel wrote:
>> On 02/16/2011 01:18 PM, Simon McNair wrote:
>>> parted -l gives
>>>
>>> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
>>> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
>>> Fix/Cancel? fix
>>> Warning: Not all of the space available to /dev/sdb appears to be used, you can fix the GPT to use all of the space (an
>>> extra 421296 blocks) or continue with the current setting?
>>> Fix/Ignore?
>>>
>>> I'm guessing ignore ?
>> Either should work. If you 'Fix' for all of the drives, you could later 'grow' your array to include the extra space.
> It's vital that the start sector be 2048, though. Please do recheck after all are done.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On 02/16/2011 01:52 PM, Simon McNair wrote:
> have done this.... Results are:
>
> proxmox:/home/simon# ./lsdrv.sh
> Controller device [at] pci0000:00/0000:00:01.0/0000:01:00.0 [mvsas]
> SCSI storage controller: Marvell Technology Group Ltd. MV64460/64461/64462 System Controller, Revision B (rev 01)
> host0: /dev/sdf ATA Hitachi HDS72101 {SN: GTA000PAGABXRA}
> host0: /dev/sdg ATA Hitachi HDS72101 {SN: GTA000PAGAA5DA}
> host0: /dev/sdh ATA Hitachi HDS72101 {SN: GTA000PAG9NL9A}
> host0: /dev/sdi ATA Hitachi HDS72101 {SN: GTA000PAGA8V4A}
> host0: /dev/sdj ATA Hitachi HDS72101 {SN: GTD000PAGMT9GD}
> host0: /dev/sdk ATA Hitachi HDS72101 {SN: GTG000PAG18BJC}
> host0: /dev/sdl ATA Hitachi HDS72101 {SN: GTG000PAG1DPLC}
> host0: /dev/sdm ATA Hitachi HDS72101 {SN: GTA000PAG7WMEA}
> Controller device [at] pci0000:00/0000:00:1a.7/usb1/1-4/1-4.1/1-4.1.1/1-4.1.1:1.0 [usb-storage]
> Bus 001 Device 007: ID 0424:2228 Standard Microsystems Corp. 9-in-2 Card Reader {SN: 08050920003A}
> host11: /dev/sdb Generic Flash HS-CF
> host11: /dev/sdc Generic Flash HS-COMBO
> Controller device [at] pci0000:00/0000:00:1c.4/0000:04:00.0 [ahci]
> SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
> host9: [Empty]
> host10: [Empty]
> Controller device [at] pci0000:00/0000:00:1c.4/0000:04:00.1 [pata_jmicron]
> IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
> host1: [Empty]
> host2: [Empty]
> Controller device [at] pci0000:00/0000:00:1f.2 [ahci]
> SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
> host3: /dev/sda ATA STM3500418AS {SN: 9VM3QJ5C}
> host4: /dev/sr0 Optiarc DVD RW AD-5240S
> host5: [Empty]
> host6: [Empty]
> host7: /dev/sdd ATA Hitachi HDS72101 {SN: GTD000PAGMT8DD}
> host8: /dev/sde ATA Hitachi HDS72101 {SN: GTG000PAG04V0C}
>
> enabling achi has switched the drive letters around for a couple of drives again
>
> parted list is:
>
> proxmox:/home/simon# for x in /dev/sd{i,h,g,f,m,l,k,j,d,e} ; do parted -s $x unit s print ; done
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdi: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdh: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdg: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdf: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdm: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdl: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdk: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdj: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdd: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sde: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid

All looks good.

> is it now:
>
> mdadm --create --verbose --assume-clean /dev/md0 --metadata=1.1 --level=5 --raid-devices=10 /dev/sd{i,h,g,f,m,l,k,j,d,e}1 --chunk=64

Yes.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

pvscan is:
proxmox:/home/simon# pvscan --verbose
Wiping cache of LVM-capable devices
Wiping internal VG cache
Walking through all physical volumes
PV /dev/sda2 VG pve lvm2 [465.26 GB / 4.00 GB free]
PV /dev/md0 VG lvm-raid lvm2 [8.19 TB / 0 free]
Total: 2 [655.05 GB] / in use: 2 [655.05 GB] / in no VG: 0 [0 ]

lvm-raid is there ....

proxmox:/home/simon# fsck.ext4 -n /dev/md0
e2fsck 1.41.3 (12-Oct-2008)
fsck.ext4: Superblock invalid, trying backup blocks...
Superblock has an invalid ext3 journal (inode 8).
Clear? no

fsck.ext4: Illegal inode number while checking ext3 journal for /dev/md0

I thought I needed to run fsck against the lvm group ? I used to do...
fsck.ext4 /dev/lvm-raid/RAID so do I need to mount it or something ?

cheers
Simon


On 16/02/2011 18:57, Phil Turmel wrote:
> On 02/16/2011 01:52 PM, Simon McNair wrote:
>> have done this.... Results are:
>>
>> proxmox:/home/simon# ./lsdrv.sh
>> Controller device [at] pci0000:00/0000:00:01.0/0000:01:00.0 [mvsas]
>> SCSI storage controller: Marvell Technology Group Ltd. MV64460/64461/64462 System Controller, Revision B (rev 01)
>> host0: /dev/sdf ATA Hitachi HDS72101 {SN: GTA000PAGABXRA}
>> host0: /dev/sdg ATA Hitachi HDS72101 {SN: GTA000PAGAA5DA}
>> host0: /dev/sdh ATA Hitachi HDS72101 {SN: GTA000PAG9NL9A}
>> host0: /dev/sdi ATA Hitachi HDS72101 {SN: GTA000PAGA8V4A}
>> host0: /dev/sdj ATA Hitachi HDS72101 {SN: GTD000PAGMT9GD}
>> host0: /dev/sdk ATA Hitachi HDS72101 {SN: GTG000PAG18BJC}
>> host0: /dev/sdl ATA Hitachi HDS72101 {SN: GTG000PAG1DPLC}
>> host0: /dev/sdm ATA Hitachi HDS72101 {SN: GTA000PAG7WMEA}
>> Controller device [at] pci0000:00/0000:00:1a.7/usb1/1-4/1-4.1/1-4.1.1/1-4.1.1:1.0 [usb-storage]
>> Bus 001 Device 007: ID 0424:2228 Standard Microsystems Corp. 9-in-2 Card Reader {SN: 08050920003A}
>> host11: /dev/sdb Generic Flash HS-CF
>> host11: /dev/sdc Generic Flash HS-COMBO
>> Controller device [at] pci0000:00/0000:00:1c.4/0000:04:00.0 [ahci]
>> SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
>> host9: [Empty]
>> host10: [Empty]
>> Controller device [at] pci0000:00/0000:00:1c.4/0000:04:00.1 [pata_jmicron]
>> IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
>> host1: [Empty]
>> host2: [Empty]
>> Controller device [at] pci0000:00/0000:00:1f.2 [ahci]
>> SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
>> host3: /dev/sda ATA STM3500418AS {SN: 9VM3QJ5C}
>> host4: /dev/sr0 Optiarc DVD RW AD-5240S
>> host5: [Empty]
>> host6: [Empty]
>> host7: /dev/sdd ATA Hitachi HDS72101 {SN: GTD000PAGMT8DD}
>> host8: /dev/sde ATA Hitachi HDS72101 {SN: GTG000PAG04V0C}
>>
>> enabling achi has switched the drive letters around for a couple of drives again
>>
>> parted list is:
>>
>> proxmox:/home/simon# for x in /dev/sd{i,h,g,f,m,l,k,j,d,e} ; do parted -s $x unit s print ; done
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdi: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdh: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdg: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdf: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdm: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdl: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdk: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdj: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdd: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sde: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
> All looks good.
>
>> is it now:
>>
>> mdadm --create --verbose --assume-clean /dev/md0 --metadata=1.1 --level=5 --raid-devices=10 /dev/sd{i,h,g,f,m,l,k,j,d,e}1 --chunk=64
> Yes.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On 02/16/2011 02:07 PM, Simon McNair wrote:
> pvscan is:
> proxmox:/home/simon# pvscan --verbose
> Wiping cache of LVM-capable devices
> Wiping internal VG cache
> Walking through all physical volumes
> PV /dev/sda2 VG pve lvm2 [465.26 GB / 4.00 GB free]
> PV /dev/md0 VG lvm-raid lvm2 [8.19 TB / 0 free]
> Total: 2 [655.05 GB] / in use: 2 [655.05 GB] / in no VG: 0 [0 ]
>
> lvm-raid is there ....

Very good.

> proxmox:/home/simon# fsck.ext4 -n /dev/md0

Uh, no.

> e2fsck 1.41.3 (12-Oct-2008)
> fsck.ext4: Superblock invalid, trying backup blocks...
> Superblock has an invalid ext3 journal (inode 8).
> Clear? no
>
> fsck.ext4: Illegal inode number while checking ext3 journal for /dev/md0


> I thought I needed to run fsck against the lvm group ? I used to do... fsck.ext4 /dev/lvm-raid/RAID so do I need to mount it or something ?

vgscan --verbose

lvscan --verbose


Then either:

fsck -N /dev/mapper/lvm-raid-RAID

or:

fsck -N /dev/lvm-raid/RAID

depends on what udev is doing.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

proxmox:/home/simon# vgscan --verbose
Wiping cache of LVM-capable devices
Wiping internal VG cache
Reading all physical volumes. This may take a while...
Finding all volume groups
Finding volume group "pve"
Found volume group "pve" using metadata type lvm2
Finding volume group "lvm-raid"
Found volume group "lvm-raid" using metadata type lvm2
proxmox:/home/simon#
proxmox:/home/simon# lvscan --verbose
Finding all logical volumes
ACTIVE '/dev/pve/swap' [11.00 GB] inherit
ACTIVE '/dev/pve/root' [96.00 GB] inherit
ACTIVE '/dev/pve/data' [354.26 GB] inherit
inactive '/dev/lvm-raid/RAID' [8.19 TB] inherit

proxmox:/home/simon# vgchange -ay
3 logical volume(s) in volume group "pve" now active
1 logical volume(s) in volume group "lvm-raid" now active

proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm-raid-RAID
e2fsck 1.41.3 (12-Oct-2008)
fsck.ext4: No such file or directory while trying to open
/dev/mapper/lvm-raid-RAID

The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>

proxmox:/home/simon# fsck.ext4 -n /dev/mapper/
control lvm--raid-RAID pve-data pve-root pve-swap
proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm--raid-RAID
e2fsck 1.41.3 (12-Oct-2008)
/dev/mapper/lvm--raid-RAID has unsupported feature(s): FEATURE_I31
e2fsck: Get a newer version of e2fsck!

my version of e2fsck always worked before ?

Simon

On 16/02/2011 19:10, Phil Turmel wrote:
> On 02/16/2011 02:07 PM, Simon McNair wrote:
>> pvscan is:
>> proxmox:/home/simon# pvscan --verbose
>> Wiping cache of LVM-capable devices
>> Wiping internal VG cache
>> Walking through all physical volumes
>> PV /dev/sda2 VG pve lvm2 [465.26 GB / 4.00 GB free]
>> PV /dev/md0 VG lvm-raid lvm2 [8.19 TB / 0 free]
>> Total: 2 [655.05 GB] / in use: 2 [655.05 GB] / in no VG: 0 [0 ]
>>
>> lvm-raid is there ....
> Very good.
>
>> proxmox:/home/simon# fsck.ext4 -n /dev/md0
> Uh, no.
>
>> e2fsck 1.41.3 (12-Oct-2008)
>> fsck.ext4: Superblock invalid, trying backup blocks...
>> Superblock has an invalid ext3 journal (inode 8).
>> Clear? no
>>
>> fsck.ext4: Illegal inode number while checking ext3 journal for /dev/md0
>
>> I thought I needed to run fsck against the lvm group ? I used to do... fsck.ext4 /dev/lvm-raid/RAID so do I need to mount it or something ?
> vgscan --verbose
>
> lvscan --verbose
>
>
> Then either:
>
> fsck -N /dev/mapper/lvm-raid-RAID
>
> or:
>
> fsck -N /dev/lvm-raid/RAID
>
> depends on what udev is doing.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On 02/16/2011 02:15 PM, Simon McNair wrote:
> proxmox:/home/simon# vgscan --verbose
> Wiping cache of LVM-capable devices
> Wiping internal VG cache
> Reading all physical volumes. This may take a while...
> Finding all volume groups
> Finding volume group "pve"
> Found volume group "pve" using metadata type lvm2
> Finding volume group "lvm-raid"
> Found volume group "lvm-raid" using metadata type lvm2
> proxmox:/home/simon#
> proxmox:/home/simon# lvscan --verbose
> Finding all logical volumes
> ACTIVE '/dev/pve/swap' [11.00 GB] inherit
> ACTIVE '/dev/pve/root' [96.00 GB] inherit
> ACTIVE '/dev/pve/data' [354.26 GB] inherit
> inactive '/dev/lvm-raid/RAID' [8.19 TB] inherit
>
> proxmox:/home/simon# vgchange -ay
> 3 logical volume(s) in volume group "pve" now active
> 1 logical volume(s) in volume group "lvm-raid" now active

Heh. Figures.

> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm-raid-RAID

Actually, I wanted you to try with a capital N. Lower case 'n' is similar, but not quite the same.

> e2fsck 1.41.3 (12-Oct-2008)
> fsck.ext4: No such file or directory while trying to open /dev/mapper/lvm-raid-RAID
>
> The superblock could not be read or does not describe a correct ext2
> filesystem. If the device is valid and it really contains an ext2
> filesystem (and not swap or ufs or something else), then the superblock
> is corrupt, and you might try running e2fsck with an alternate superblock:
> e2fsck -b 8193 <device>
>
> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/
> control lvm--raid-RAID pve-data pve-root pve-swap

Strange. I guess it does that to distinguish dashes in the VG name from dashes between VG and LV names.

> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm--raid-RAID
> e2fsck 1.41.3 (12-Oct-2008)
> /dev/mapper/lvm--raid-RAID has unsupported feature(s): FEATURE_I31
> e2fsck: Get a newer version of e2fsck!
>
> my version of e2fsck always worked before ?

v1.41.14 was release 7 weeks ago. But, I suspect there's corruption in the superblock. Do you still have your disk images tucked away somewhere safe?

If so, try:

1) The '-b' option to e2fsck. We need to experiment with '-n -b offset' to find the alternate superblock. Trying 'offset' = to 8193, 16384, and 32768, per the man-page.

2) A newer e2fsprogs.

Finally,
3) mount -r /dev/lvm-raid/RAID /mnt/whatever

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Phil,
Jeez I'm having a bad week (my windows 7x64 machine has just started
randomly crashing, my Thecus n5200 is playing up and the weather has
been dire so I've not been able to put my new shed up...oh and I have
ongoing 'other' issues as you're well aware ;-)) The thecus n5200 has
5x2TB hdd's. I wiped out the existing raid5 array to create a jbod span
of 10tb in order to hold my 9tb of backups. The Thecus has had a hissy
fit and I've had to set the process off again, so you can bet it'll be a
day or two before it get's the drives formatted (it's not a very
powerful nas), then I'll do the backups, then I'll try as you suggested.

Thanks for the ongoing assistance.

Simon

On 16/02/2011 19:36, Phil Turmel wrote:
> On 02/16/2011 02:15 PM, Simon McNair wrote:
>> proxmox:/home/simon# vgscan --verbose
>> Wiping cache of LVM-capable devices
>> Wiping internal VG cache
>> Reading all physical volumes. This may take a while...
>> Finding all volume groups
>> Finding volume group "pve"
>> Found volume group "pve" using metadata type lvm2
>> Finding volume group "lvm-raid"
>> Found volume group "lvm-raid" using metadata type lvm2
>> proxmox:/home/simon#
>> proxmox:/home/simon# lvscan --verbose
>> Finding all logical volumes
>> ACTIVE '/dev/pve/swap' [11.00 GB] inherit
>> ACTIVE '/dev/pve/root' [96.00 GB] inherit
>> ACTIVE '/dev/pve/data' [354.26 GB] inherit
>> inactive '/dev/lvm-raid/RAID' [8.19 TB] inherit
>>
>> proxmox:/home/simon# vgchange -ay
>> 3 logical volume(s) in volume group "pve" now active
>> 1 logical volume(s) in volume group "lvm-raid" now active
> Heh. Figures.
>
>> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm-raid-RAID
> Actually, I wanted you to try with a capital N. Lower case 'n' is similar, but not quite the same.
>
>> e2fsck 1.41.3 (12-Oct-2008)
>> fsck.ext4: No such file or directory while trying to open /dev/mapper/lvm-raid-RAID
>>
>> The superblock could not be read or does not describe a correct ext2
>> filesystem. If the device is valid and it really contains an ext2
>> filesystem (and not swap or ufs or something else), then the superblock
>> is corrupt, and you might try running e2fsck with an alternate superblock:
>> e2fsck -b 8193<device>
>>
>> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/
>> control lvm--raid-RAID pve-data pve-root pve-swap
> Strange. I guess it does that to distinguish dashes in the VG name from dashes between VG and LV names.
>
>> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm--raid-RAID
>> e2fsck 1.41.3 (12-Oct-2008)
>> /dev/mapper/lvm--raid-RAID has unsupported feature(s): FEATURE_I31
>> e2fsck: Get a newer version of e2fsck!
>>
>> my version of e2fsck always worked before ?
> v1.41.14 was release 7 weeks ago. But, I suspect there's corruption in the superblock. Do you still have your disk images tucked away somewhere safe?
>
> If so, try:
>
> 1) The '-b' option to e2fsck. We need to experiment with '-n -b offset' to find the alternate superblock. Trying 'offset' = to 8193, 16384, and 32768, per the man-page.
>
> 2) A newer e2fsprogs.
>
> Finally,
> 3) mount -r /dev/lvm-raid/RAID /mnt/whatever
>
> Phil
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo [at] vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On 02/16/2011 04:28 PM, Simon McNair wrote:
> Phil,
> Jeez I'm having a bad week (my windows 7x64 machine has just started randomly crashing, my Thecus n5200 is playing up and the weather has been dire so I've not been able to put my new shed up...oh and I have ongoing 'other' issues as you're well aware ;-)) The thecus n5200 has 5x2TB hdd's. I wiped out the existing raid5 array to create a jbod span of 10tb in order to hold my 9tb of backups. The Thecus has had a hissy fit and I've had to set the process off again, so you can bet it'll be a day or two before it get's the drives formatted (it's not a very powerful nas), then I'll do the backups, then I'll try as you suggested.

That's fine.
>
> Thanks for the ongoing assistance.

No problem.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

I just went to bed and one last question popped in to my mind. Since
there is a fair timezone gap I thought I'd be presumptuous and ask it
in the hope I can turn it around a bit quicker in the morning.

My suspicion is that once I have 5 formatted 2tb drives I may be lucky
to get 10x 1tb dd images on to it. Can I feed the dd process in to
tar, bzip2, zip or something else which will give me enough space to
fit the images on ?

Will I get more usable space from 5x2tb partitions or from 1xspanned
volume ? (the thecus pretty much only allows you to create raid
volumes so a jbod needs to be 5x1tb arrays or 1 spanned volume or
stripe).

TIA
Simon

On 16 Feb 2011, at 21:30, Phil Turmel <philip [at] turmel.org> wrote:

> On 02/16/2011 04:28 PM, Simon McNair wrote:
>> Phil,
>> Jeez I'm having a bad week (my windows 7x64 machine has just started randomly crashing, my Thecus n5200 is playing up and the weather has been dire so I've not been able to put my new shed up...oh and I have ongoing 'other' issues as you're well aware ;-)) The thecus n5200 has 5x2TB hdd's. I wiped out the existing raid5 array to create a jbod span of 10tb in order to hold my 9tb of backups. The Thecus has had a hissy fit and I've had to set the process off again, so you can bet it'll be a day or two before it get's the drives formatted (it's not a very powerful nas), then I'll do the backups, then I'll try as you suggested.
>
> That's fine.
>>
>> Thanks for the ongoing assistance.
>
> No problem.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On 02/16/2011 05:44 PM, Simon Mcnair wrote:
> I just went to bed and one last question popped in to my mind. Since
> there is a fair timezone gap I thought I'd be presumptuous and ask it
> in the hope I can turn it around a bit quicker in the morning.
>
> My suspicion is that once I have 5 formatted 2tb drives I may be lucky
> to get 10x 1tb dd images on to it. Can I feed the dd process in to
> tar, bzip2, zip or something else which will give me enough space to
> fit the images on ?
>
> Will I get more usable space from 5x2tb partitions or from 1xspanned
> volume ? (the thecus pretty much only allows you to create raid
> volumes so a jbod needs to be 5x1tb arrays or 1 spanned volume or
> stripe).

I'd use one spanned volume, and gzip. I'd simultaneously generate an md5sum while streaming to your thecus. A script like so:

#! /bin/bash
#
function usage() {
printf "Usage:\n\t%s devname\n\n" "`basename \"$0\"`"
printf "'devname' must be a relative path in /dev/ to the desired block device.\n"
exit 1
}

# Verify the supplied name is a device
test -b "/dev/$1" || usage

# Convert path separators and spaces into dashes
outfile="`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"

# Create a side-stream for computing the MD5 of the data read
fifo=`mktemp -u`
mkfifo $fifo || exit
md5sum -b <$fifo >/mnt/thecus/$outfile.md5 &

# Read the device and compress it
dd if="/dev/$1" bs=1M | tee 2>$fifo | gzip >/mnt/thecus/$outfile.gz

# Wait for the background task to close
wait
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Phil,
After a couple of attempts I realised that all I needed to do was
=2E/backup.sh sdi rather than specifying ./backup.sh /dev/sdi schoolbo=
y
error huh, rtfm.

I tried modifying the code, so I could kick them all off at once, but
I wanted to check that this would work from a multithreaded/sane
perspective (yes I know it's a bit of IO, but iotop seems to infer I'm
only getting 10M/s throughput from the disk anyway).

is this code kosha ?

#! /bin/bash
#
function usage() {
printf "Usage:\n\t%s devname\n\n" "`basename \"$0\"`"
printf "'devname' must be a relative path in /dev/ to the
desired block device.\n"
exit 1
}

# Verify the supplied name is a device
test -b "/dev/$1" || usage

# Convert path separators and spaces into dashes
outfile=3D"`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"

# Create a side-stream for computing the MD5 of the data read
fifo=3D`mktemp -u`
mkfifo $fifo || exit
md5sum -b <$fifo >$2/$outfile.md5 &

# Read the device and compress it
dd if=3D"/dev/$1" bs=3D1M | tee 2>$fifo | gzip >$2/$outfile.gz

# Wait for the background task to close
wait

I was a little concerned as I didn't see the hdd drive LED light up
for my attempt even though the file was growing nicely on my CIFS
share. I'm dumping it on the 5x2TB disks (JBOD) in my windows 7 box
as my Thecus is not a happy chappy and I need time to mount the DOM
module in another machine to find out why.

cheers
Simon


On 16 February 2011 23:39, Phil Turmel <philip [at] turmel.org> wrote:
> On 02/16/2011 05:44 PM, Simon Mcnair wrote:
>> I just went to bed and one last question popped in to my mind. Since
>> there is a fair timezone gap I thought I'd be presumptuous and ask i=
t
>> in the hope I can turn it around a bit quicker in the morning.
>>
>> My suspicion is that once I have 5 formatted 2tb drives I may be luc=
ky
>> to get 10x 1tb dd images on to it. Can I feed the dd process in to
>> tar, bzip2, zip or something else which will give me enough space to
>> fit the images on ?
>>
>> Will I get more usable space from 5x2tb partitions or from 1xspanned
>> volume ? (the thecus pretty much only allows you to create raid
>> volumes so a jbod needs to be 5x1tb arrays or 1 spanned volume or
>> stripe).
>
> I'd use one spanned volume, and gzip. =A0I'd simultaneously generate =
an md5sum while streaming to your thecus. =A0A script like so:
>
> #! /bin/bash
> #
> function usage() {
> =A0 =A0 =A0 =A0printf "Usage:\n\t%s devname\n\n" "`basename \"$0\"`"
> =A0 =A0 =A0 =A0printf "'devname' must be a relative path in /dev/ to =
the desired block device.\n"
> =A0 =A0 =A0 =A0exit 1
> }
>
> # Verify the supplied name is a device
> test -b "/dev/$1" || usage
>
> # Convert path separators and spaces into dashes
> outfile=3D"`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"
>
> # Create a side-stream for computing the MD5 of the data read
> fifo=3D`mktemp -u`
> mkfifo $fifo || exit
> md5sum -b <$fifo >/mnt/thecus/$outfile.md5 &
>
> # Read the device and compress it
> dd if=3D"/dev/$1" bs=3D1M | tee 2>$fifo | gzip >/mnt/thecus/$outfile.=
gz
>
> # Wait for the background task to close
> wait
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

This is a multi-part message in MIME format.
--------------060200030107030205010800
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On 02/17/2011 08:26 AM, Simon Mcnair wrote:
> Phil,
> After a couple of attempts I realised that all I needed to do was
> ./backup.sh sdi rather than specifying ./backup.sh /dev/sdi schoolboy
> error huh, rtfm.
>
> I tried modifying the code, so I could kick them all off at once, but
> I wanted to check that this would work from a multithreaded/sane
> perspective (yes I know it's a bit of IO, but iotop seems to infer I'm
> only getting 10M/s throughput from the disk anyway).

That sounds suspiciously like a 100 meg ethernet bottleneck, so parallel operation won't help. If so, this'll take a very long time.

> is this code kosha ?

Looks pretty good. I've attached a slight update with your changes and a bugfix.

> #! /bin/bash
> #
> function usage() {
> printf "Usage:\n\t%s devname\n\n" "`basename \"$0\"`"
> printf "'devname' must be a relative path in /dev/ to the
> desired block device.\n"
> exit 1
> }
>
> # Verify the supplied name is a device
> test -b "/dev/$1" || usage
>
> # Convert path separators and spaces into dashes
> outfile="`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"
>
> # Create a side-stream for computing the MD5 of the data read
> fifo=`mktemp -u`
> mkfifo $fifo || exit
> md5sum -b <$fifo >$2/$outfile.md5 &
>
> # Read the device and compress it
> dd if="/dev/$1" bs=1M | tee 2>$fifo | gzip >$2/$outfile.gz
>
> # Wait for the background task to close
> wait
>
> I was a little concerned as I didn't see the hdd drive LED light up
> for my attempt even though the file was growing nicely on my CIFS
> share. I'm dumping it on the 5x2TB disks (JBOD) in my windows 7 box
> as my Thecus is not a happy chappy and I need time to mount the DOM
> module in another machine to find out why.

If you're willing to dismantle the thecus, directly connecting its drives to your crippled system will make things go much faster. You've got four empty motherboard SATA ports. You just have to watch out for the power load.

Phil

--------------060200030107030205010800
Content-Type: application/x-sh;
name="block2gz.sh"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="block2gz.sh"

#! /bin/bash
#
function usage() {
printf "Usage:\n\t%s devname foldername\n\n" "`basename \"$0\"`"
printf "'devname' must be a relative path in /dev/ to the desired block device.\n"
printf "'foldername' is the directory that will receive the image and md5 files.\n"
exit 1
}

# Verify the supplied name is a device
test -b "/dev/$1" || usage

# Verify the supplied folder is a directory
test -d "$2" || usage

# Convert path separators and spaces into dashes
outfile="`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"

# Create a side-stream for computing the MD5 of the data read
fifo=`mktemp -u`
mkfifo $fifo || exit
md5sum -b <$fifo >$2/$outfile.md5 &

# Read the device and compress it
dd if="/dev/$1" bs=1M | tee 2>$fifo | gzip >$2/$outfile.gz

# Wait for the background task to close
wait
rm -f "$fifo"

--------------060200030107030205010800--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Phil, I think I need to connect them to the crippled machine directly,
I was just trying to leave it alone, I know that doesn't make much
sense but I'm more familiar with windows than Linux. The cards that I
have are all connected at gigabit and bonded. That machine has two
ports, so does the other. I'm running two asus p6ts (1 deluxe and 1
se) with i7 920's

I cant understand why it's so slow though I have seen some reports
that the supermicro card is quite slow.
Simon
Ps thanks for the bugfixes and updates
Simon

On 17 Feb 2011, at 13:48, Phil Turmel <philip [at] turmel.org> wrote:

> On 02/17/2011 08:26 AM, Simon Mcnair wrote:
>> Phil,
>> After a couple of attempts I realised that all I needed to do was
>> ./backup.sh sdi rather than specifying ./backup.sh /dev/sdi schoolboy
>> error huh, rtfm.
>>
>> I tried modifying the code, so I could kick them all off at once, but
>> I wanted to check that this would work from a multithreaded/sane
>> perspective (yes I know it's a bit of IO, but iotop seems to infer I'm
>> only getting 10M/s throughput from the disk anyway).
>
> That sounds suspiciously like a 100 meg ethernet bottleneck, so parallel operation won't help. If so, this'll take a very long time.
>
>> is this code kosha ?
>
> Looks pretty good. I've attached a slight update with your changes and a bugfix.
>
>> #! /bin/bash
>> #
>> function usage() {
>> printf "Usage:\n\t%s devname\n\n" "`basename \"$0\"`"
>> printf "'devname' must be a relative path in /dev/ to the
>> desired block device.\n"
>> exit 1
>> }
>>
>> # Verify the supplied name is a device
>> test -b "/dev/$1" || usage
>>
>> # Convert path separators and spaces into dashes
>> outfile="`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"
>>
>> # Create a side-stream for computing the MD5 of the data read
>> fifo=`mktemp -u`
>> mkfifo $fifo || exit
>> md5sum -b <$fifo >$2/$outfile.md5 &
>>
>> # Read the device and compress it
>> dd if="/dev/$1" bs=1M | tee 2>$fifo | gzip >$2/$outfile.gz
>>
>> # Wait for the background task to close
>> wait
>>
>> I was a little concerned as I didn't see the hdd drive LED light up
>> for my attempt even though the file was growing nicely on my CIFS
>> share. I'm dumping it on the 5x2TB disks (JBOD) in my windows 7 box
>> as my Thecus is not a happy chappy and I need time to mount the DOM
>> module in another machine to find out why.
>
> If you're willing to dismantle the thecus, directly connecting its drives to your crippled system will make things go much faster. You've got four empty motherboard SATA ports. You just have to watch out for the power load.
>
> Phil
> <block2gz.sh>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

19390 root 17.49 M/s 0 B/s 0.00 % 21.36 % dd if /dev/sde b=
s 1M
19333 root 16.52 M/s 0 B/s 0.00 % 17.07 % dd if /dev/sdd b=
s 1M
19503 root 15.79 M/s 0 B/s 0.00 % 13.91 % dd if /dev/sdf b=
s 1M
18896 root 0 B/s 16.10 M/s 0.00 % 0.00 % ntfs-3g
/dev/sdb1 /media/ntfs3g/sdb
18909 root 0 B/s 13.49 M/s 0.00 % 0.00 % ntfs-3g
/dev/sdc1 /media/ntfs3g/sdc
18920 root 0 B/s 15.73 M/s 0.00 % 0.00 % ntfs-3g
/dev/sdn1 /media/ntfs3g/sdn

this will certainly be quicker :-)

On 17 February 2011 13:56, Simon Mcnair <simonmcnair [at] gmail.com> wrote:
> Phil, I think I need to connect them to the crippled machine directly=
,
> I was just trying to leave it alone, I know that doesn't make much
> sense but I'm more familiar with windows than Linux. The cards that I
> have are all connected at gigabit and bonded. That machine has two
> ports, so does the other. I'm running two asus p6ts (1 deluxe and 1
> se) with i7 920's
>
> I cant understand why it's so slow though I have seen some reports
> that the supermicro card is quite slow.
> Simon
> Ps thanks for the bugfixes and updates
> Simon
>
> On 17 Feb 2011, at 13:48, Phil Turmel <philip [at] turmel.org> wrote:
>
>> On 02/17/2011 08:26 AM, Simon Mcnair wrote:
>>> Phil,
>>> After a couple of attempts I realised that all I needed to do was
>>> ./backup.sh sdi rather than specifying ./backup.sh /dev/sdi =A0scho=
olboy
>>> error huh, rtfm.
>>>
>>> I tried modifying the code, so I could kick them all off at once, b=
ut
>>> I wanted to check that this would work from a multithreaded/sane
>>> perspective (yes I know it's a bit of IO, but iotop seems to infer =
I'm
>>> only getting 10M/s throughput from the disk anyway).
>>
>> That sounds suspiciously like a 100 meg ethernet bottleneck, so para=
llel operation won't help. =A0If so, this'll take a very long time.
>>
>>> is this code kosha ?
>>
>> Looks pretty good. =A0I've attached a slight update with your change=
s and a bugfix.
>>
>>> #! /bin/bash
>>> #
>>> function usage() {
>>> =A0 =A0 =A0 printf "Usage:\n\t%s devname\n\n" "`basename \"$0\"`"
>>> =A0 =A0 =A0 printf "'devname' must be a relative path in /dev/ to t=
he
>>> desired block device.\n"
>>> =A0 =A0 =A0 exit 1
>>> }
>>>
>>> # Verify the supplied name is a device
>>> test -b "/dev/$1" || usage
>>>
>>> # Convert path separators and spaces into dashes
>>> outfile=3D"`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"
>>>
>>> # Create a side-stream for computing the MD5 of the data read
>>> fifo=3D`mktemp -u`
>>> mkfifo $fifo || exit
>>> md5sum -b <$fifo >$2/$outfile.md5 &
>>>
>>> # Read the device and compress it
>>> dd if=3D"/dev/$1" bs=3D1M | tee 2>$fifo | gzip >$2/$outfile.gz
>>>
>>> # Wait for the background task to close
>>> wait
>>>
>>> I was a little concerned as I didn't see the hdd drive LED light up
>>> for my attempt even though the file was growing nicely on my CIFS
>>> share. =A0I'm dumping it on the 5x2TB disks (JBOD) in my windows 7 =
box
>>> as my Thecus is not a happy chappy and I need time to mount the DOM
>>> module in another machine to find out why.
>>
>> If you're willing to dismantle the thecus, directly connecting its d=
rives to your crippled system will make things go much faster. =A0You'v=
e got four empty motherboard SATA ports. =A0You just have to watch out =
for the power load.
>>
>> Phil
>> <block2gz.sh>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

proxmox:~# dpkg -S blkid
libblkid1: /lib/libblkid.so.1.0
libblkid1: /lib/libblkid.so.1
libblkid1: /usr/share/doc/libblkid1/changelog.Debian.gz
libblkid1: /usr/share/doc/libblkid1/copyright
libblkid1: /usr/share/doc/libblkid1
e2fsprogs-dbg: /usr/lib/debug/sbin/blkid

proxmox:~# apt-get install e2fsprogs-dbg
Reading package lists... Done
Building dependency tree
Reading state information... Done
e2fsprogs-dbg is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.


I still can't blkid installed, even though I have e2fsprogs-dbg install=
ed...

On 15 February 2011 19:45, Roman Mamedov <rm [at] romanrm.ru> wrote:
> On Tue, 15 Feb 2011 14:37:55 -0500
> Phil Turmel <philip [at] turmel.org> wrote:
>
>> > For some reason blkid doesn't exist on my machine even though I ha=
ve
>> > e2fs-utils installed. =A0V weird.
>
> Weird is that you think it is somehow a "e2fs-utils" program.
>
> dpkg -S blkid
> ...
> util-linux: /sbin/blkid
> ...
>
>>
>> FWIW, on my VPS running Ubuntu Server 10.10, the blkid executable is=
part of
>> util-linux, and installed in /sbin. =A0You might need to re-install =
it.
>
> --
> With respect,
> Roman
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

--Sig_/_+hBX8RuWkGbV/G3w3j_Tac
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

On Thu, 17 Feb 2011 15:10:32 +0000
Simon Mcnair <simonmcnair [at] gmail.com> wrote:

> proxmox:~# dpkg -S blkid
> libblkid1: /lib/libblkid.so.1.0
> libblkid1: /lib/libblkid.so.1
> libblkid1: /usr/share/doc/libblkid1/changelog.Debian.gz
> libblkid1: /usr/share/doc/libblkid1/copyright
> libblkid1: /usr/share/doc/libblkid1
> e2fsprogs-dbg: /usr/lib/debug/sbin/blkid
>
> proxmox:~# apt-get install e2fsprogs-dbg
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> e2fsprogs-dbg is already the newest version.
> 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
>
>
> I still can't blkid installed, even though I have e2fsprogs-dbg installed=
....

/usr/lib/debug/sbin/ is not in system 'path', so you won't be able to run t=
he
program just by entering 'blkid' into the shell prompt.
Either use the complete file name and path, e.g.

# /usr/lib/debug/sbin/blkid --help

or symlink /usr/lib/debug/sbin/blkid to something like /usr/local/sbin/blki=
d.

--
With respect,
Roman

--Sig_/_+hBX8RuWkGbV/G3w3j_Tac
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAk1dQd0ACgkQTLKSvz+PZwi34ACferMv2HLit5FOWLHVKKBh rIMj
ZFYAnRRdNLho7QsjqbIXtEjwArdb6fi5
=WojE
-----END PGP SIGNATURE-----

--Sig_/_+hBX8RuWkGbV/G3w3j_Tac--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On 02/17/2011 09:34 AM, Simon Mcnair wrote:
> 19390 root 17.49 M/s 0 B/s 0.00 % 21.36 % dd if /dev/sde bs 1M
> 19333 root 16.52 M/s 0 B/s 0.00 % 17.07 % dd if /dev/sdd bs 1M
> 19503 root 15.79 M/s 0 B/s 0.00 % 13.91 % dd if /dev/sdf bs 1M
> 18896 root 0 B/s 16.10 M/s 0.00 % 0.00 % ntfs-3g
> /dev/sdb1 /media/ntfs3g/sdb
> 18909 root 0 B/s 13.49 M/s 0.00 % 0.00 % ntfs-3g
> /dev/sdc1 /media/ntfs3g/sdc
> 18920 root 0 B/s 15.73 M/s 0.00 % 0.00 % ntfs-3g
> /dev/sdn1 /media/ntfs3g/sdn
>
> this will certainly be quicker :-)

But still a long time...

Let me know.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Roman,
Sorry to be so persistent on something that is not Linux raid specific,
but I still can't get blkid to run, the message is:

proxmox:~# ls -la /usr/lib/debug/sbin/blkid
-rwx------ 1 root root 19887 2008-10-13 04:54 /usr/lib/debug/sbin/blkid

proxmox:~# /usr/lib/debug/sbin/blkid --help
-bash: /usr/lib/debug/sbin/blkid: cannot execute binary file

If I can't run it directly I'm pretty sure symlinking it will not make
any difference. It's weird, normally when I apt-get things it modifies
the path etc and does everything required to make it work, I would have
thought if I installed a debug version of something it should add the
debug path as well ?

blkid always works on my Ubuntu boxes, I don't know why this is any
different (apart from being a different distribution, it's still a
mature program, package and platform).

:-)

Simon

On 17/02/2011 15:42, Roman Mamedov wrote:
> On Thu, 17 Feb 2011 15:10:32 +0000
> Simon Mcnair<simonmcnair [at] gmail.com> wrote:
>
>> proxmox:~# dpkg -S blkid
>> libblkid1: /lib/libblkid.so.1.0
>> libblkid1: /lib/libblkid.so.1
>> libblkid1: /usr/share/doc/libblkid1/changelog.Debian.gz
>> libblkid1: /usr/share/doc/libblkid1/copyright
>> libblkid1: /usr/share/doc/libblkid1
>> e2fsprogs-dbg: /usr/lib/debug/sbin/blkid
>>
>> proxmox:~# apt-get install e2fsprogs-dbg
>> Reading package lists... Done
>> Building dependency tree
>> Reading state information... Done
>> e2fsprogs-dbg is already the newest version.
>> 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
>>
>>
>> I still can't blkid installed, even though I have e2fsprogs-dbg installed...
> /usr/lib/debug/sbin/ is not in system 'path', so you won't be able to run the
> program just by entering 'blkid' into the shell prompt.
> Either use the complete file name and path, e.g.
>
> # /usr/lib/debug/sbin/blkid --help
>
> or symlink /usr/lib/debug/sbin/blkid to something like /usr/local/sbin/blkid.
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

time passes. You are eaten by a Grue. Sheesh this is taking a long ti=
me.

simon [at] proxmox:~$ top

top - 09:26:19 up 20:36, 11 users, load average: 2.46, 2.49, 2.39
Tasks: 334 total, 9 running, 325 sleeping, 0 stopped, 0 zombie
Cpu(s): 52.8%us, 9.4%sy, 0.0%ni, 35.2%id, 2.5%wa, 0.0%hi, 0.2%si, =
0.0%st
Mem: 12299244k total, 12227584k used, 71660k free, 11042868k buffer=
s
Swap: 11534332k total, 0k used, 11534332k free, 208204k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
18896 root 20 0 12288 1072 460 S 63 0.0 431:16.55 ntfs-3g
18909 root 20 0 12288 1064 460 R 62 0.0 419:17.11 ntfs-3g
18920 root 20 0 12288 1100 492 R 62 0.0 428:17.33 ntfs-3g
9210 root 20 0 4068 520 328 S 54 0.0 661:59.92 gzip
9138 root 20 0 4068 524 328 R 53 0.0 647:29.07 gzip
9247 root 20 0 4068 524 328 S 53 0.0 651:28.36 gzip
25678 root 20 0 4068 524 328 R 52 0.0 439:49.20 gzip
24957 root 20 0 4068 524 328 R 51 0.0 437:44.01 gzip
25792 root 20 0 4068 524 328 S 48 0.0 433:28.14 gzip

This is hardly touching the CPU on the box (in my opinion), any advice
on using renice ? I've never used it before, but now seems like a
good time ?

tia
Simon

On 16 February 2011 19:36, Phil Turmel <philip [at] turmel.org> wrote:
> On 02/16/2011 02:15 PM, Simon McNair wrote:
>> proxmox:/home/simon# vgscan --verbose
>> =A0 =A0 Wiping cache of LVM-capable devices
>> =A0 =A0 Wiping internal VG cache
>> =A0 Reading all physical volumes. =A0This may take a while...
>> =A0 =A0 Finding all volume groups
>> =A0 =A0 Finding volume group "pve"
>> =A0 Found volume group "pve" using metadata type lvm2
>> =A0 =A0 Finding volume group "lvm-raid"
>> =A0 Found volume group "lvm-raid" using metadata type lvm2
>> proxmox:/home/simon#
>> proxmox:/home/simon# lvscan --verbose
>> =A0 =A0 Finding all logical volumes
>> =A0 ACTIVE =A0 =A0 =A0 =A0 =A0 =A0'/dev/pve/swap' [11.00 GB] inherit
>> =A0 ACTIVE =A0 =A0 =A0 =A0 =A0 =A0'/dev/pve/root' [96.00 GB] inherit
>> =A0 ACTIVE =A0 =A0 =A0 =A0 =A0 =A0'/dev/pve/data' [354.26 GB] inheri=
t
>> =A0 inactive =A0 =A0 =A0 =A0 =A0'/dev/lvm-raid/RAID' [8.19 TB] inher=
it
>>
>> proxmox:/home/simon# vgchange -ay
>> =A0 3 logical volume(s) in volume group "pve" now active
>> =A0 1 logical volume(s) in volume group "lvm-raid" now active
>
> Heh. =A0Figures.
>
>> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm-raid-RAID
>
> Actually, I wanted you to try with a capital N. =A0Lower case 'n' is =
similar, but not quite the same.
>
>> e2fsck 1.41.3 (12-Oct-2008)
>> fsck.ext4: No such file or directory while trying to open /dev/mappe=
r/lvm-raid-RAID
>>
>> The superblock could not be read or does not describe a correct ext2
>> filesystem. =A0If the device is valid and it really contains an ext2
>> filesystem (and not swap or ufs or something else), then the superbl=
ock
>> is corrupt, and you might try running e2fsck with an alternate super=
block:
>> =A0 =A0 e2fsck -b 8193 <device>
>>
>> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/
>> control =A0 =A0 =A0 =A0 lvm--raid-RAID =A0pve-data =A0 =A0 =A0 =A0pv=
e-root =A0 =A0 =A0 =A0pve-swap
>
> Strange. =A0I guess it does that to distinguish dashes in the VG name=
from dashes between VG and LV names.
>
>> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm--raid-RAID
>> e2fsck 1.41.3 (12-Oct-2008)
>> /dev/mapper/lvm--raid-RAID has unsupported feature(s): FEATURE_I31
>> e2fsck: Get a newer version of e2fsck!
>>
>> my version of e2fsck always worked before ?
>
> v1.41.14 was release 7 weeks ago. =A0But, I suspect there's corruptio=
n in the superblock. =A0Do you still have your disk images tucked away =
somewhere safe?
>
> If so, try:
>
> 1) The '-b' option to e2fsck. =A0We need to experiment with '-n -b of=
fset' to find the alternate superblock. =A0Trying 'offset' =3D to 8193,=
16384, and 32768, per the man-page.
>
> 2) A newer e2fsprogs.
>
> Finally,
> 3) mount -r /dev/lvm-raid/RAID /mnt/whatever
>
> Phil
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

--C7zPtVaVf+AK4Oqc
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri Feb 18, 2011 at 09:13:33AM +0000, Simon McNair wrote:

> Hi Roman,
> Sorry to be so persistent on something that is not Linux raid specific,=

> but I still can't get blkid to run, the message is:
>
> proxmox:~# ls -la /usr/lib/debug/sbin/blkid
> -rwx------ 1 root root 19887 2008-10-13 04:54 /usr/lib/debug/sbin/blkid
>
> proxmox:~# /usr/lib/debug/sbin/blkid --help
> -bash: /usr/lib/debug/sbin/blkid: cannot execute binary file
>
> If I can't run it directly I'm pretty sure symlinking it will not make
> any difference. It's weird, normally when I apt-get things it modifies=

> the path etc and does everything required to make it work, I would have=

> thought if I installed a debug version of something it should add the
> debug path as well ?
>
I suspect that those are just the debug symbols - they're split off from
the binary so they can be automatically pulled in when needed, but don't
bloat the application itself. From the package naming, the actual blkid
binary should be in the e2fsprogs package - do you have that installed?

Cheers,
Robin
--
___
( ' } | Robin Hill <robin [at] robinhill.me.uk> |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |

--C7zPtVaVf+AK4Oqc
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (GNU/Linux)

iEYEARECAAYFAk1ePfkACgkQShxCyD40xBKB0gCg4FWbYa1tLrwhvWwc6vCu nW1O
UvsAnj9K7vBAfri062hPqIKu7j1t2zAG
=Rwwt
-----END PGP SIGNATURE-----

--C7zPtVaVf+AK4Oqc--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Robin,

proxmox:~# apt-get install e2fsprogs
Reading package lists... Done
Building dependency tree
Reading state information... Done
e2fsprogs is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.

Yeah, It's installed :-)

Simon

On 18 February 2011 09:38, Robin Hill <robin [at] robinhill.me.uk> wrote:
> On Fri Feb 18, 2011 at 09:13:33AM +0000, Simon McNair wrote:
>
>> Hi Roman,
>> Sorry to be so persistent on something that is not Linux raid specif=
ic,
>> but I still can't get blkid to run, the message is:
>>
>> proxmox:~# ls -la /usr/lib/debug/sbin/blkid
>> -rwx------ 1 root root 19887 2008-10-13 04:54 /usr/lib/debug/sbin/bl=
kid
>>
>> proxmox:~# /usr/lib/debug/sbin/blkid --help
>> -bash: /usr/lib/debug/sbin/blkid: cannot execute binary file
>>
>> If I can't run it directly I'm pretty sure symlinking it will not ma=
ke
>> any difference. =A0It's weird, normally when I apt-get things it mod=
ifies
>> the path etc and does everything required to make it work, I would h=
ave
>> thought if I installed a debug version of something it should add th=
e
>> debug path as well ?
>>
> I suspect that those are just the debug symbols - they're split off f=
rom
> the binary so they can be automatically pulled in when needed, but do=
n't
> bloat the application itself. =A0From the package naming, the actual =
blkid
> binary should be in the e2fsprogs package - do you have that installe=
d?
>
> Cheers,
> =A0 =A0Robin
> --
> =A0 =A0 ___
> =A0 =A0( ' } =A0 =A0 | =A0 =A0 =A0 Robin Hill =A0 =A0 =A0 =A0<robin [at] r=
obinhill.me.uk> |
> =A0 / / ) =A0 =A0 =A0| Little Jim says .... =A0 =A0 =A0 =A0 =A0 =A0 =A0=
=A0 =A0 =A0 =A0 =A0 =A0 =A0|
> =A0// !! =A0 =A0 =A0 | =A0 =A0 =A0"He fallen in de water !!" =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 |
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On 02/18/2011 04:31 AM, Simon Mcnair wrote:
> time passes. You are eaten by a Grue. Sheesh this is taking a long time.
>
> simon [at] proxmox:~$ top
>
> top - 09:26:19 up 20:36, 11 users, load average: 2.46, 2.49, 2.39
> Tasks: 334 total, 9 running, 325 sleeping, 0 stopped, 0 zombie
> Cpu(s): 52.8%us, 9.4%sy, 0.0%ni, 35.2%id, 2.5%wa, 0.0%hi, 0.2%si, 0.0%st
> Mem: 12299244k total, 12227584k used, 71660k free, 11042868k buffers
> Swap: 11534332k total, 0k used, 11534332k free, 208204k cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 18896 root 20 0 12288 1072 460 S 63 0.0 431:16.55 ntfs-3g
> 18909 root 20 0 12288 1064 460 R 62 0.0 419:17.11 ntfs-3g
> 18920 root 20 0 12288 1100 492 R 62 0.0 428:17.33 ntfs-3g
> 9210 root 20 0 4068 520 328 S 54 0.0 661:59.92 gzip
> 9138 root 20 0 4068 524 328 R 53 0.0 647:29.07 gzip
> 9247 root 20 0 4068 524 328 S 53 0.0 651:28.36 gzip
> 25678 root 20 0 4068 524 328 R 52 0.0 439:49.20 gzip
> 24957 root 20 0 4068 524 328 R 51 0.0 437:44.01 gzip
> 25792 root 20 0 4068 524 328 S 48 0.0 433:28.14 gzip
>
> This is hardly touching the CPU on the box (in my opinion), any advice
> on using renice ? I've never used it before, but now seems like a
> good time ?

Probably wouldn't help.

You have a total usage of 65%. I suspect that each of the processors running gzip are nearly pegged, and everything else is loafing along. Hit '1' in top to show the CPU usage per-cpu. Single-threaded gzip is holding you back.

If you had enough space for saving the partitions uncompressed, it would be much faster. The PCIe x4 interface on the SuperMicro has a theoretical performance of 1GByte/s, which would be ~ 125MB/s per drive. From what I've read, that card actually delivers ~ 75MB/s per drive when they're all busy.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

pci express x4 =3D 2.5*4gbps =3D 10gbps not?!

2011/2/18 Phil Turmel <philip [at] turmel.org>:
> On 02/18/2011 04:31 AM, Simon Mcnair wrote:
>> time passes. =A0You are eaten by a Grue. =A0Sheesh this is taking a =
long time.
>>
>> simon [at] proxmox:~$ top
>>
>> top - 09:26:19 up 20:36, 11 users, =A0load average: 2.46, 2.49, 2.39
>> Tasks: 334 total, =A0 9 running, 325 sleeping, =A0 0 stopped, =A0 0 =
zombie
>> Cpu(s): 52.8%us, =A09.4%sy, =A00.0%ni, 35.2%id, =A02.5%wa, =A00.0%hi=
, =A00.2%si, =A00.0%st
>> Mem: =A012299244k total, 12227584k used, =A0 =A071660k free, 1104286=
8k buffers
>> Swap: 11534332k total, =A0 =A0 =A0 =A00k used, 11534332k free, =A0 2=
08204k cached
>>
>> =A0 PID USER =A0 =A0 =A0PR =A0NI =A0VIRT =A0RES =A0SHR S %CPU %MEM =A0=
=A0TIME+ =A0COMMAND
>> 18896 root =A0 =A0 =A020 =A0 0 12288 1072 =A0460 S =A0 63 =A00.0 431=
:16.55 ntfs-3g
>> 18909 root =A0 =A0 =A020 =A0 0 12288 1064 =A0460 R =A0 62 =A00.0 419=
:17.11 ntfs-3g
>> 18920 root =A0 =A0 =A020 =A0 0 12288 1100 =A0492 R =A0 62 =A00.0 428=
:17.33 ntfs-3g
>> =A09210 root =A0 =A0 =A020 =A0 0 =A04068 =A0520 =A0328 S =A0 54 =A00=
=2E0 661:59.92 gzip
>> =A09138 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 R =A0 53 =A00=
=2E0 647:29.07 gzip
>> =A09247 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 S =A0 53 =A00=
=2E0 651:28.36 gzip
>> 25678 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 R =A0 52 =A00.0=
439:49.20 gzip
>> 24957 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 R =A0 51 =A00.0=
437:44.01 gzip
>> 25792 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 S =A0 48 =A00.0=
433:28.14 gzip
>>
>> This is hardly touching the CPU on the box (in my opinion), any advi=
ce
>> on using renice ? =A0I've never used it before, but now seems like a
>> good time ?
>
> Probably wouldn't help.
>
> You have a total usage of 65%. =A0I suspect that each of the processo=
rs running gzip are nearly pegged, and everything else is loafing along=
=2E =A0Hit '1' in top to show the CPU usage per-cpu. =A0Single-threaded=
gzip is holding you back.
>
> If you had enough space for saving the partitions uncompressed, it wo=
uld be much faster. =A0The PCIe x4 interface on the SuperMicro has a th=
eoretical performance of 1GByte/s, which would be ~ 125MB/s per drive. =
=A0From what I've read, that card actually delivers ~ 75MB/s per drive =
when they're all busy.
>
> Phil
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"=
in
> the body of a message to majordomo [at] vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>



--
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo [at] vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html