The OS Command for pg_hotbackup -- Use lvmsnapshot instead of tarcvzf

--_000_54E5385D47A89542BC678F5426799F3904961EF60FOSTIEX01ost ig_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

I am currently using the pg_hotbackup Perl script to backup my production P=
ostgreSQL database. However, the next production database is going to be c=
lose to a terrabyte in size. We feel that a tar cvzf command is not the co=
rrect OS backup solution to use. Instead we would like to use lvmsnapshot.=
Is anyone using the pg_hotbackup script but instead of the tar cvzf OS co=
mmand are they using lvmsnapshot?

Or, is anyone using lvmsnapshot and doing point-in-time recoveries?

Thanks,

Sandra Arnold
Sr. Database Administrator
DOE/OSTI
Oak Ridge, TN

--_000_54E5385D47A89542BC678F5426799F3904961EF60FOSTIEX01ost ig_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content=3D"text/html; charset=3Dus-ascii" http-equiv=3DContent-Type>
<META name=3DGENERATOR content=3D"MSHTML 8.00.6001.18876"></HEAD>
<BODY>
<DIV><FONT size=3D2 face=3DArial><SPAN class=3D876543715-02032010>I am curr=
ently using
the pg_hotbackup Perl script to backup my production PostgreSQL database.&n=
bsp;
However, the next production database is going to be close to a terrabyte i=
n
size.  We feel that a tar cvzf command is not the correct OS backup
solution to use.  Instead we would like to use lvmsnapshot.  Is a=
nyone
using the pg_hotbackup script but instead of the tar cvzf OS command a=
re
they using lvmsnapshot?</SPAN></FONT></DIV>
<DIV><FONT size=3D2 face=3DArial><SPAN
class=3D876543715-02032010></SPAN></FONT> </DIV>
<DIV><FONT size=3D2 face=3DArial><SPAN class=3D876543715-02032010>Or, is an=
yone using
lvmsnapshot and doing point-in-time recoveries?</SPAN></FONT></DIV>
<DIV><FONT size=3D2 face=3DArial><SPAN
class=3D876543715-02032010></SPAN></FONT> </DIV>
<DIV><FONT size=3D2 face=3DArial><SPAN
class=3D876543715-02032010>Thanks,</SPAN></FONT></DIV><SPAN
class=3D876543715-02032010>
<DIV><BR><FONT size=3D2 face=3DArial>Sandra Arnold</FONT></DIV>
<DIV><SPAN class=3D876543715-02032010><FONT size=3D2 face=3DArial>Sr. Datab=
ase
Administrator</FONT></SPAN></DIV>
<DIV><SPAN class=3D876543715-02032010><FONT size=3D2
face=3DArial>DOE/OSTI</FONT></SPAN></DIV>
<DIV><SPAN class=3D876543715-02032010><FONT size=3D2 face=3DArial>Oak Ridge=
,
TN</FONT></SPAN></SPAN></DIV></BODY></HTML>

--_000_54E5385D47A89542BC678F5426799F3904961EF60FOSTIEX01ost ig_--
ArnoldS [ Di, 02 März 2010 16:43 ] [ ID #2033784 ]

Re: The OS Command for pg_hotbackup -- Use lvmsnapshot insteadof tar cvzf

Arnold, Sandra wrote:
> I am currently using the pg_hotbackup Perl script to backup my
> production PostgreSQL database. However, the next production database
> is going to be close to a terrabyte in size. We feel that a tar cvzf
> command is not the correct OS backup solution to use. Instead we
> would like to use lvmsnapshot. Is anyone using the pg_hotbackup
> script but instead of the tar cvzf OS command are they using lvmsnapshot?
I believe that if you take an LVM snapshot and the change size exceeds
the snapshot size (a configurable value) then the snapshot gets
released, so you might want to be careful with this solution (I don't
know how much data you accumulate, or how fast you accumulate it, or if
that rate changes, etc.). You'll also need to ensure that you snapshot
any and all tablespaces (if you use pg_start_backup() first the
snapshots need not be simultaneous, but they still must be done).

You might consider using rsync instead, it should be a lot faster than
tar, and doesn't have the same downside as using snapshots. You'll also
find that it is fairly widely used in the PostgreSQL community.

--
Chander Ganesan
Open Technology Group, Inc.
One Copley Parkway, Suite 210
Morrisville, NC 27560
919-463-0999/877-258-8987
http://www.otg-nc.com
Expert PostgreSQL, Python, PHP, and other open source training.


--
Sent via pgsql-admin mailing list (pgsql-admin [at] postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
chander [ Di, 02 März 2010 19:09 ] [ ID #2033788 ]

Re: [ADMIN] The OS Command for pg_hotbackup -- Use lvmsnapshot

On Tue, Mar 2, 2010 at 8:43 AM, Arnold, Sandra <ArnoldS [at] osti.gov> wrote:
> I am currently using the pg_hotbackup Perl script to backup my production
> PostgreSQL database.=A0 However, the next production database is going to=
be
> close to a terrabyte in size.=A0 We feel that a tar cvzf command is not t=
he
> correct OS backup solution to use.=A0 Instead we would like to use
> lvmsnapshot.=A0 Is anyone using the pg_hotbackup script but instead of=A0=
the tar
> cvzf OS command are they using lvmsnapshot?
>
> Or, is anyone using lvmsnapshot and doing point-in-time recoveries?

I do not run my databases on LVM because, at least in the past, LVM
did not properly honor fsync / write barrier commands. I don't know
if this has since been fixed. If your transaction rate on LVM is
unnaturally higher than it should / could be, then suspect it is not
fsyncing, and putting your data on it is putting it at risk.

--
Sent via pgsql-general mailing list (pgsql-general [at] postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
Scott Marlowe [ Di, 02 März 2010 20:20 ] [ ID #2033789 ]

Re: The OS Command for pg_hotbackup -- Use lvmsnapshot

FYI, here's the conversations on the subject of LVM and write barriers
from last year about this time:

http://www.redhat.com/archives/linux-lvm/2009-March/msg00025 .html
http://archives.postgresql.org/pgsql-general/2009-03/msg0020 4.php

Note that it's also considered bad form to post the same question to
multiple lists.

--
Sent via pgsql-admin mailing list (pgsql-admin [at] postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
Scott Marlowe [ Di, 02 März 2010 20:24 ] [ ID #2033790 ]

Re: [ADMIN] The OS Command for pg_hotbackup -- Use lvmsnapshot

On Tue, Mar 2, 2010 at 12:20 PM, Scott Marlowe <scott.marlowe [at] gmail.com> wr=
ote:
> On Tue, Mar 2, 2010 at 8:43 AM, Arnold, Sandra <ArnoldS [at] osti.gov> wrote:
>> I am currently using the pg_hotbackup Perl script to backup my production
>> PostgreSQL database.=A0 However, the next production database is going t=
o be
>> close to a terrabyte in size.=A0 We feel that a tar cvzf command is not =
the
>> correct OS backup solution to use.=A0 Instead we would like to use
>> lvmsnapshot.=A0 Is anyone using the pg_hotbackup script but instead of=
=A0the tar
>> cvzf OS command are they using lvmsnapshot?
>>
>> Or, is anyone using lvmsnapshot and doing point-in-time recoveries?
>
> I do not run my databases on LVM because, at least in the past, LVM
> did not properly honor fsync / write barrier commands. =A0I don't know
> if this has since =A0been fixed. =A0 If your transaction rate on LVM is
> unnaturally higher than it should / could be, then suspect it is not
> fsyncing, and putting your data on it is putting it at risk.

=46rom what I've read further on, it seems it's ok as long as it's on a
storage system like a battery backed RAID controller. Still, someone
on the list did some testing and LVM resulted in a reduced throughput
when on large fast RAID arrays, so do some testing before you set off
into production with LVM.

--
Sent via pgsql-general mailing list (pgsql-general [at] postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
Scott Marlowe [ Di, 02 März 2010 20:54 ] [ ID #2033791 ]
Datenbanken » gmane.comp.db.postgresql.admin » The OS Command for pg_hotbackup -- Use lvmsnapshot instead of tarcvzf

Vorheriges Thema: Restart failed with 'service postgresql restart'
Nächstes Thema: LATIN2 encoding and Polish_Poland locale

[ Startseite ] [ Administrator ] [ Suche ] [ Hinweise ] [ Impressum ]

Powered by FudForum