WAL backup

Hi all

I'm not sure I understand

http://www.postgresql.org/docs/8.3/interactive/continuous-ar chiving.html

My problem:

I've huge database ~ 1To (soon 2 To) and I need backup.
I can use pgdump because it's too long to do the backup.
So I like to use =ABcontinous-archiving=BB.

For the continuous archiving I do

Modifying postgresql.conf with :

archive_mode =3D on
archive_command =3D 'cp -i %p /databases/Archives/%f </dev/null'

restart postgresql

after that I make a big backup:

psql -c "select pg_start_backup('complete_backup');"
rsync -av /databases/pgsql /databases/Archives
psql -c "select pg_stop_backup();"

before this moment I use =ABcontinuous-archiving=BB by put in my crontab
something like (every hours)

rsync -av /databases/pgsql /databases/Archives

is that correct ?

And more important : How long I can keep this the rsync ? I mean after ho=
w
long it's good to make a new =ABbig backup=BB ?

Regards.


--
Albert SHIH
SIO batiment 15
Observatoire de Paris Meudon
5 Place Jules Janssen
92195 Meudon Cedex
T=E9l=E9phone : 01 45 07 76 26/06 86 69 95 71
Heure local/Local time:
Mar 28 jul 2009 09:23:23 CEST

--
Sent via pgsql-admin mailing list (pgsql-admin [at] postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
Albert Shih [ Di, 28 Juli 2009 09:46 ] [ ID #2010051 ]

Re: WAL backup

--0016363ba756a9a9ef046fc2e1a9
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Tue, Jul 28, 2009 at 1:46 AM, Albert Shih <Albert.Shih [at] obspm.fr> wrote:

> Hi all
>
> I'm not sure I understand
>
>
> http://www.postgresql.org/docs/8.3/interactive/continuous-ar chiving.html
>
> My problem:
>
> I've huge database ~ 1To (soon 2 To) and I need backup.
> I can use pgdump because it's too long to do the backup.
> So I like to use =ABcontinous-archiving=BB.
>
> *SNIP*
> before this moment I use =ABcontinuous-archiving=BB by put in my c=
rontab
> something like (every hours)
>
> rsync -av /databases/pgsql /databases/Archives
>
> is that correct ?
>
> And more important : How long I can keep this the rsync ? I mean after ho=
w
> long it's good to make a new =ABbig backup=BB ?
>


Up to the rsync in your crontab your configuration looked good. Lose the
cron'd rsync.

The first rsync executed between pg_start_backup() and pg_stop_backup()
creates the baseline / "full" backup of the database. Be sure you get
tablespace directories outside of the default PGDATA! The archive_command
adds the WAL logs as needed after and are the "incrementals".

That's all you need for the backups.

How often do you need to create the baseline? Depends how many WAL logs ar=
e
being created and how quickly you want the recovery to be. I have a 800 GB
database and in one week over 4,000 WAL logs are created. This database
setup takes about 16-24 hours to recover. but at least 8 hours of that is
restoring the "full" and the remainder is applying the WAL logs.

FWIW, I send all backup data (baseline & archived WALs) to a server which i=
s
responsible for backing it up to tape. This allows the database server to
be a database server without busying itself with actual backups. The
standby server, eventually, will be used as a warm standby in case the
primary crashes.

Greg

--0016363ba756a9a9ef046fc2e1a9
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div class=3D"gmail_quote">On Tue, Jul 28, 2009 at 1:46 AM, Albert Shih <sp=
an dir=3D"ltr"><<a href=3D"mailto:Albert.Shih [at] obspm.fr">Albert.Shih [at] obsp=
m.fr</a>></span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"bo=
rder-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding=
-left: 1ex;">
Hi all<br>
<br>
I'm not sure I understand<br>
<br>
=A0 =A0 =A0 =A0<a href=3D"http://www.postgresql.org/docs/8.3/interactive/c=
ontinuous-archiving.html" target=3D"_blank">http://www.postgresql.org/docs/=
8.3/interactive/continuous-archiving.html</a><br>
<br>
My problem:<br>
<br>
=A0 =A0 =A0 =A0I've huge database ~ 1To (soon 2 To) and I need backup.=
<br>
=A0 =A0 =A0 =A0I can use pgdump because it's too long to do the backup=
..<br>
=A0 =A0 =A0 =A0So I like to use =ABcontinous-archiving=BB.<br>
<br>
*SNIP*<br>
=A0 =A0 =A0 =A0before this moment I use =ABcontinuous-archiving=BB by put =
in my crontab<br>
=A0 =A0 =A0 =A0something like (every hours)<br>
<br>
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0rsync -av /databases/pgsql /databases/Archi=
ves<br>
<br>
is that correct ?<br>
<br>
And more important : How long I can keep this the rsync ? I mean after how<=
br>
long it's good to make a new =ABbig backup=BB ?<br></blockquote></div><=
br><br>Up to the rsync in your crontab your configuration looked good.=A0 L=
ose the cron'd rsync.<br><br>The first rsync executed between pg_start_=
backup() and pg_stop_backup() creates the baseline / "full" backu=
p of the database.=A0 Be sure you get tablespace directories outside of the=
default PGDATA!=A0 The archive_command adds the WAL logs as needed after a=
nd are the "incrementals".<br>
<br>That's all you need for the backups.<br><br>How often do you need t=
o create the baseline?=A0 Depends how many WAL logs are being created and h=
ow quickly you want the recovery to be.=A0 I have a 800 GB database and in =
one week over 4,000 WAL logs are created.=A0 This database setup takes abou=
t 16-24 hours to recover.=A0 but at least 8 hours of that is restoring the =
"full" and the remainder is applying the WAL logs.<br>
<br>FWIW, I send all backup data (baseline & archived WALs) to a server=
which is responsible for backing it up to tape.=A0 This allows the databas=
e server to be a database server without busying itself with actual backups=
..=A0 The standby server, eventually, will be used as a warm standby in case=
the primary crashes.<br>
<br>Greg<br><br>

--0016363ba756a9a9ef046fc2e1a9--
Greg Spiegelberg [ Di, 28 Juli 2009 14:05 ] [ ID #2010055 ]

Re: WAL backup

Le 28/07/2009 =E0 06:05:07-0600, Greg Spiegelberg a =E9crit
> On Tue, Jul 28, 2009 at 1:46 AM, Albert Shih <Albert.Shih [at] obspm.fr> wro=
te:
>
> Hi all
>
> I'm not sure I understand
>
> =A0 =A0 =A0 =A0http://www.postgresql.org/docs/8.3/interactive/
> continuous-archiving.html
>
> My problem:
>
> =A0 =A0 =A0 =A0I've huge database ~ 1To (soon 2 To) and I need back=
up.
> =A0 =A0 =A0 =A0I can use pgdump because it's too long to do the bac=
kup.
> =A0 =A0 =A0 =A0So I like to use =ABcontinous-archiving=BB.
>
> *SNIP*
> =A0 =A0 =A0 =A0before this moment I use =ABcontinuous-archiving=BB =
by put in my crontab
> =A0 =A0 =A0 =A0something like (every hours)
>
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0rsync -av /databases/pgsql /database=
s/Archives
>
> is that correct ?
>
> And more important : How long I can keep this the rsync ? I mean af=
ter how
> long it's good to make a new =ABbig backup=BB ?
>
>
>
> Up to the rsync in your crontab your configuration looked good.=A0 Lose=
the
> cron'd rsync.
>
> The first rsync executed between pg_start_backup() and pg_stop_backup()=
creates
> the baseline / "full" backup of the database.=A0 Be sure you get tables=
pace
> directories outside of the default PGDATA!=A0 The archive_command adds =
the WAL
> logs as needed after and are the "incrementals".
>
> That's all you need for the backups.

OK. Thanks you.

At this moment I don't have tablespace ;-)

>
> How often do you need to create the baseline?=A0 Depends how many WAL l=
ogs are
> being created and how quickly you want the recovery to be.=A0 I have a =
800 GB
> database and in one week over 4,000 WAL logs are created.=A0 This datab=
ase setup
> takes about 16-24 hours to recover.=A0 but at least 8 hours of that is =
restoring
> the "full" and the remainder is applying the WAL logs.

OK. For me that should be OK because my database don't have lot of
modification in time.

>
> FWIW, I send all backup data (baseline & archived WALs) to a server whi=
ch is
> responsible for backing it up to tape.=A0 This allows the database serv=
er to be a
> database server without busying itself with actual backups.=A0 The stan=
dby
> server, eventually, will be used as a warm standby in case the primary =
crashes.
>

Thanks again for your help.

Regards.
--
Albert SHIH
SIO batiment 15
Observatoire de Paris Meudon
5 Place Jules Janssen
92195 Meudon Cedex
T=E9l=E9phone : 01 45 07 76 26/06 86 69 95 71
Heure local/Local time:
Mar 28 jul 2009 15:18:25 CEST

--
Sent via pgsql-admin mailing list (pgsql-admin [at] postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
Albert Shih [ Di, 28 Juli 2009 15:20 ] [ ID #2010057 ]
Datenbanken » gmane.comp.db.postgresql.admin » WAL backup

Vorheriges Thema: Postgres/C'ish question
Nächstes Thema: force varchar column to be lowercase