External bitmap, questions

Hi,

From the mdadm manual:

"-b, --bitmap=
[...]
Note: external bitmaps are only known to work on ext2 and ext3.
Storing bitmap files on other filesystems may result in serious
problems."

I was planning to store the external bitmap on an ext4 partition. Will
this be a problem, or is the warning there because it hasn't been
tested enough but no problems found?

Thanks,
// Mathias
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: External bitmap, questions

On Sat, 19 Feb 2011 00:47:10 +0000 Mathias Bur=E9n com>
wrote:

> Hi,
>=20
> >From the mdadm manual:
>=20
> "-b, --bitmap=3D
> [...]
> Note: external bitmaps are only known to work on ext2 and ext3.
> Storing bitmap files on other filesystems may result in serious
> problems."
>=20
> I was planning to store the external bitmap on an ext4 partition. Wil=
l
> this be a problem, or is the warning there because it hasn't been
> tested enough but no problems found?

External bitmaps use 'BMAP' to find where the file lives on the device =
and
then writes directly to the device - not through the filesystem.

So as long as there is no tail-packing to block migration happening it =
should
work fine.

I haven't looked inside ext4 but I am fairly confident that external bi=
tmaps
will work properly.

NeilBrown

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: External bitmap, questions

On 19 February 2011 04:49, NeilBrown wrote:
> On Sat, 19 Feb 2011 00:47:10 +0000 Mathias Burén gmail.com>
> wrote:
>
>> Hi,
>>
>> >From the mdadm manual:
>>
>> "-b, --bitmap=3D
>> [...]
>> Note: external bitmaps are only known to work on ext2 and ext3.
>> Storing bitmap files on other filesystems may result in serious
>> problems."
>>
>> I was planning to store the external bitmap on an ext4 partition. Wi=
ll
>> this be a problem, or is the warning there because it hasn't been
>> tested enough but no problems found?
>
> External bitmaps use 'BMAP' to find where the file lives on the devic=
e and
> then writes directly to the device - not through the filesystem.
>
> So as long as there is no tail-packing to block migration happening i=
t should
> work fine.
>
> I haven't looked inside ext4 but I am fairly confident that external =
bitmaps
> will work properly.
>
> NeilBrown
>
>

Thanks, Neil. Are there any differences between using an internal or
external bitmap? Also, does one need a bitmap at all when not planning
to "mess" with the array? (does it provide any other purpose?)

// Mathias
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: External bitmap, questions

On Sat, 19 Feb 2011 04:55:32 +0000 Mathias Bur=E9n com>
wrote:

> On 19 February 2011 04:49, NeilBrown wrote:
> > On Sat, 19 Feb 2011 00:47:10 +0000 Mathias Bur=E9n mail.com>
> > wrote:
> >
> >> Hi,
> >>
> >> >From the mdadm manual:
> >>
> >> "-b, --bitmap=3D
> >> [...]
> >> Note: external bitmaps are only known to work on ext2 and ext3.
> >> Storing bitmap files on other filesystems may result in serious
> >> problems."
> >>
> >> I was planning to store the external bitmap on an ext4 partition. =
Will
> >> this be a problem, or is the warning there because it hasn't been
> >> tested enough but no problems found?
> >
> > External bitmaps use 'BMAP' to find where the file lives on the dev=
ice and
> > then writes directly to the device - not through the filesystem.
> >
> > So as long as there is no tail-packing to block migration happening=
it should
> > work fine.
> >
> > I haven't looked inside ext4 but I am fairly confident that externa=
l bitmaps
> > will work properly.
> >
> > NeilBrown
> >
> >
>=20
> Thanks, Neil. Are there any differences between using an internal or
> external bitmap? Also, does one need a bitmap at all when not plannin=
g
> to "mess" with the array? (does it provide any other purpose?)

The difference between internal and external is simply that one is stor=
ed in
the array - a copy on each device - and the other is stored externally =
to the
array - a single copy in a file.
External are slightly harder to work with as you need somewhere separat=
e to
store the bitmap (it cannot be in the array) and you need to tell mdadm=
where
to find it (so it needs to be listed in mdadm.conf).

I don't know what you mean by "mess" with the array. The main purpose =
of the
bitmap is the accelerate resync after an unclean shutdown. The seconda=
ry
purpose is to accelerate recovery if you remove and then re-add a devic=
e.
Maybe that is what you mean by "mess with".

So if you don't want to "mess" with the array and you are certain that =
your
machine will never crash, then you don't really need a bitmap.

NeilBrown

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: External bitmap, questions

On 19 February 2011 05:30, NeilBrown wrote:
> On Sat, 19 Feb 2011 04:55:32 +0000 Mathias Burén gmail.com>
> wrote:
>
>> On 19 February 2011 04:49, NeilBrown wrote:
>> > On Sat, 19 Feb 2011 00:47:10 +0000 Mathias Burén en@gmail.com>
>> > wrote:
>> >
>> >> Hi,
>> >>
>> >> >From the mdadm manual:
>> >>
>> >> "-b, --bitmap=3D
>> >> [...]
>> >> Note: external bitmaps are only known to work on ext2 and ext3.
>> >> Storing bitmap files on other filesystems may result in serious
>> >> problems."
>> >>
>> >> I was planning to store the external bitmap on an ext4 partition.=
Will
>> >> this be a problem, or is the warning there because it hasn't been
>> >> tested enough but no problems found?
>> >
>> > External bitmaps use 'BMAP' to find where the file lives on the de=
vice and
>> > then writes directly to the device - not through the filesystem.
>> >
>> > So as long as there is no tail-packing to block migration happenin=
g it should
>> > work fine.
>> >
>> > I haven't looked inside ext4 but I am fairly confident that extern=
al bitmaps
>> > will work properly.
>> >
>> > NeilBrown
>> >
>> >
>>
>> Thanks, Neil. Are there any differences between using an internal or
>> external bitmap? Also, does one need a bitmap at all when not planni=
ng
>> to "mess" with the array? (does it provide any other purpose?)
>
> The difference between internal and external is simply that one is st=
ored in
> the array - a copy on each device - and the other is stored externall=
y to the
> array  - a single copy in a file.
> External are slightly harder to work with as you need somewhere separ=
ate to
> store the bitmap (it cannot be in the array) and you need to tell mda=
dm where
> to find it (so it needs to be listed in mdadm.conf).
>
> I don't know what you mean by "mess" with the array.  The main p=
urpose of the
> bitmap is the accelerate resync after an unclean shutdown.  The =
secondary
> purpose is to accelerate recovery if you remove and then re-add a dev=
ice.
> Maybe that is what you mean by "mess with".
>
> So if you don't want to "mess" with the array and you are certain tha=
t your
> machine will never crash, then you don't really need a bitmap.
>
> NeilBrown
>
>

Hi,

Yes, you understood me correctly! Sorry, I should've been more clear.
That answers my questions, thanks a bunch :-)

// Mahtias
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: External bitmap, questions

--Sig_/_4sGrJF=vZ9FaG84/PpBlRX
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Sat, 19 Feb 2011 04:55:32 +0000
Mathias Burén wrote:

> > External bitmaps use 'BMAP' to find where the file lives on the device =
and
> > then writes directly to the device - not through the filesystem.
> >
> > So as long as there is no tail-packing to block migration happening it
> > should work fine.
> >
> > I haven't looked inside ext4 but I am fairly confident that external
> > bitmaps will work properly.
> >
> > NeilBrown
> >
> >
>=20
> Thanks, Neil. Are there any differences between using an internal or
> external bitmap? Also, does one need a bitmap at all when not planning
> to "mess" with the array? (does it provide any other purpose?)

I have successfully stored an external bitmap on XFS for a brief time.
However it proved difficult to ensure that the device storing the bitmap is
mounted, accessible and writable prior to boot scripts attempting to run th=
e md
arrays, so since then I just switched to an internal bitmap. The negative
performance it had can be minimized to almost zero by using a larger bitmap
chunk size.

--=20
With respect,
Roman

--Sig_/_4sGrJF=vZ9FaG84/PpBlRX
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAk1ftC0ACgkQTLKSvz+PZwiZ+wCfU4oC+xYm31Osba5/vZ2Z g2UW
9DgAnRTSKdAXcawOkCvQs0xd7kqHUK6v
=Q+PW
-----END PGP SIGNATURE-----

--Sig_/_4sGrJF=vZ9FaG84/PpBlRX--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: External bitmap, questions

On 19 February 2011 12:14, Roman Mamedov wrote:
> On Sat, 19 Feb 2011 04:55:32 +0000
> Mathias Burén wrote:
>
>> > External bitmaps use 'BMAP' to find where the file lives on the de=
vice and
>> > then writes directly to the device - not through the filesystem.
>> >
>> > So as long as there is no tail-packing to block migration happenin=
g it
>> > should work fine.
>> >
>> > I haven't looked inside ext4 but I am fairly confident that extern=
al
>> > bitmaps will work properly.
>> >
>> > NeilBrown
>> >
>> >
>>
>> Thanks, Neil. Are there any differences between using an internal or
>> external bitmap? Also, does one need a bitmap at all when not planni=
ng
>> to "mess" with the array? (does it provide any other purpose?)
>
> I have successfully stored an external bitmap on XFS for a brief time=

> However it proved difficult to ensure that the device storing the bit=
map is
> mounted, accessible and writable prior to boot scripts attempting to =
run the md
> arrays, so since then I just switched to an internal bitmap. The nega=
tive
> performance it had can be minimized to almost zero by using a larger =
bitmap
> chunk size.
>
> --
> With respect,
> Roman
>

I ended up storing the bitmap on my root filesystem (ext4) for now.
It's currently at the default size but I think I'll recreate it with
64MB size.

// Mathias
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: External bitmap, questions

--Sig_/Cg2i4refBp7yCrXKsAx4aoP
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Sat, 19 Feb 2011 12:20:41 +0000
Mathias Burén wrote:

> I ended up storing the bitmap on my root filesystem (ext4) for now.
> It's currently at the default size but I think I'll recreate it with
> 64MB size.

Thing is, the root filesystem is initially mounted read-only, after that
perhaps fsck is executed if needed, and only a bit later it is remounted
read-write. You need to check at which point in this process your distro's
boot scripts initialize md arrays, and whether or not you will end up trying
to do that with their bitmaps on a read-only FS.

--=20
With respect,
Roman

--Sig_/Cg2i4refBp7yCrXKsAx4aoP
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAk1fuG8ACgkQTLKSvz+PZwjQgACggT2U9qN+J5eFsPFvxSKX m2k0
g9gAnRq5wmuM/CPL8+IxknIjypvNOA2l
=0TtO
-----END PGP SIGNATURE-----

--Sig_/Cg2i4refBp7yCrXKsAx4aoP--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html