Strange deletion problem

This is a multi-part message in MIME format.

------=_NextPart_000_00C6_01CACFE7.14D35670
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi, I have a table which is constantly updated through out the day with =
no problems, I'm running Postgresql 8.3.8 in ubuntu karmic. However, =
within the last week for some reason overnight it is being emptied and I =
can't work out why. I've set log_min_duration_statement to 0 so that =
postgresql dumps out every query made to the db but it hasn't brought =
anything up. During the period between from when I know the table is ok =
to the next morning when it is empty I can see entries in the logs for =
the expected 200 or so deletions, but during this period approx 15k rows =
are removed. The odd thing is that there is nothing else in the logs =
that references the table (I've looked for the obvious deletion and =
trunctate statements).

The table is modified using JDBC prepared statements so I see 3 entries =
for each of the expected delete statements (parse, bind, execute) and =
the statement is as follows:

delete from product_list where retailer_id=3D? and product_id=3D?


Therefore I have a few questions:
- Is there any other statements that could be causing the rows to be =
removed that I've missed
- Is there anything that could be deleting them without generating a log =
entry for the statement?
- Is it possible that data corruption of either the index/table is =
making my delete statements removed more rows?
- Is it a possibly bug?

Thanks

John
--

Got needs? Get Goblin'! - http://www.pricegoblin.co.uk/

------=_NextPart_000_00C6_01CACFE7.14D35670
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; =
charset=3Diso-8859-1">
<META content=3D"MSHTML 6.00.6000.16981" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff>
<DIV><FONT face=3DArial size=3D2>Hi, I have a table which is constantly =
updated
through out the day with no problems, I'm running Postgresql =
8.3.8 in
ubuntu karmic. However, within the last week for some reason overnight =
it is
being emptied and I can't work out why. I've set =
log_min_duration_statement to 0
so that postgresql dumps out every query made to the db but it hasn't =
brought
anything up. During the period between from when I know the table is ok =
to the
next morning when it is empty I can see entries in the logs =
for the
expected 200 or so deletions, but during this period approx 15k rows are =

removed. The odd thing is that there is nothing else in the logs that =
references
the table (I've looked for the obvious deletion and trunctate
statements).</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>The table is modified using JDBC =
prepared
statements so I see 3 entries for each of the expected delete statements =
(parse,
bind, execute) and the statement is as follows:</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>delete from product_list where =
retailer_id=3D? and
product_id=3D?</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>Therefore I have a few =
questions:</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>- Is there any other statements that =
could be
causing the rows to be removed that I've missed</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>- Is there anything that could be =
deleting them
without generating a log entry for the statement?</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>- Is it possible that data corruption =
of either the
index/table is making my delete statements removed more =
rows?</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>- Is it a possibly bug?</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>Thanks</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>John</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>--</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>Got needs? Get Goblin'! - <A
href=3D"http://www.pricegoblin.co.uk/">http://www.pricegobli n.co.uk/</A><=
BR></FONT></DIV></BODY></HTML>

------=_NextPart_000_00C6_01CACFE7.14D35670--
John Lister [ Di, 30 März 2010 09:57 ] [ ID #2037081 ]

Re: Strange deletion problem

--0016e6d77ed36b9ca10483005a83
Content-Type: text/plain; charset=UTF-8

2010/3/30 John Lister <john.lister-ps [at] kickstone.com>

> Hi, I have a table which is constantly updated through out the day with
> no problems, I'm running Postgresql 8.3.8 in ubuntu karmic. However, within
> the last week for some reason overnight it is being emptied and I can't work
> out why. I've set log_min_duration_statement to 0 so that postgresql dumps
> out every query made to the db but it hasn't brought anything up. During the
> period between from when I know the table is ok to the next morning when
> it is empty I can see entries in the logs for the expected 200 or so
> deletions, but during this period approx 15k rows are removed. The odd thing
> is that there is nothing else in the logs that references the table (I've
> looked for the obvious deletion and trunctate statements).
>
> The table is modified using JDBC prepared statements so I see 3 entries for
> each of the expected delete statements (parse, bind, execute) and the
> statement is as follows:
>
> delete from product_list where retailer_id=? and product_id=?
>
>
> Therefore I have a few questions:
> - Is there any other statements that could be causing the rows to be
> removed that I've missed
> - Is there anything that could be deleting them without generating a log
> entry for the statement?
> - Is it possible that data corruption of either the index/table is making
> my delete statements removed more rows?
> - Is it a possibly bug?
>
>

Hi,
- maybe there are some other settings for this database (they are not stored
in the file), check the pg_settings table in the database
- maybe you're looking in a wrong log file - sometimes do (they change from
time to time - depending on the configuration)
- ensure that the logging is done to file and to THIS file, because there
are more logging settings than just log_min_duration and sometimes it can be
messed up
- try to restart the database and see if there isn't any other file created
as usually I observe that after deleting current log file, the database
doesn't recreate while logging so the logs are not stored.

regards
Szymon Guz

--0016e6d77ed36b9ca10483005a83
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<br><br><div class=3D"gmail_quote">2010/3/30 John Lister <span dir=3D"ltr">=
<<a href=3D"mailto:john.lister-ps [at] kickstone.com">john.lister-ps [at] kickston=
e.com</a>></span><br><blockquote class=3D"gmail_quote" style=3D"margin:0=
0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">






<div bgcolor=3D"#ffffff">
<div><font face=3D"Arial" size=3D"2">Hi, I have a table which is constantly=
updated
through out the day with no problems, I'm=C2=A0running=C2=A0Postgresql =
8.3.8 in
ubuntu karmic. However, within the last week for some reason overnight it i=
s
being emptied and I can't work out why. I've set log_min_duration_s=
tatement to 0
so that postgresql dumps out every query made to the db but it hasn't b=
rought
anything up. During the period between from when I know the table is ok to =
the
next morning when it=C2=A0is empty=C2=A0I can see entries in the logs for t=
he
expected 200 or so deletions, but during this period approx 15k rows are
removed. The odd thing is that there is nothing else in the logs that refer=
ences
the table (I've looked for the obvious deletion and trunctate
statements).</font></div>
<div><font face=3D"Arial" size=3D"2"></font>=C2=A0</div>
<div><font face=3D"Arial" size=3D"2">The table is modified using JDBC prepa=
red
statements so I see 3 entries for each of the expected delete statements (p=
arse,
bind, execute) and the statement=C2=A0is as follows:</font></div>
<div><font face=3D"Arial" size=3D"2"></font>=C2=A0</div>
<div><font face=3D"Arial" size=3D"2">delete from product_list where retaile=
r_id=3D? and
product_id=3D?</font></div>
<div><font face=3D"Arial" size=3D"2"></font>=C2=A0</div>
<div><font face=3D"Arial" size=3D"2"></font>=C2=A0</div>
<div><font face=3D"Arial" size=3D"2">Therefore I have a few questions:</fon=
t></div>
<div><font face=3D"Arial" size=3D"2">- Is there any other statements that c=
ould be
causing the rows to be removed that I've missed</font></div>
<div><font face=3D"Arial" size=3D"2">- Is there anything that could be dele=
ting them
without generating a log entry for the statement?</font></div>
<div><font face=3D"Arial" size=3D"2">- Is it possible that data corruption =
of either the
index/table is making my delete statements removed more rows?</font></div>
<div><font face=3D"Arial" size=3D"2">- Is it a possibly bug?</font></div>
<div><font face=3D"Arial" size=3D"2"></font>=C2=A0</div></div></blockquote>=
<div><br></div><div>Hi,</div><div>- maybe there are some other settings for=
this database (they are not stored in the file), check the pg_settings tab=
le in the database</div>
<div>- maybe you're looking in a wrong log file - sometimes do (they ch=
ange from time to time - depending on the configuration)</div><div>- ensure=
that the logging is done to file and to THIS file, because there are more =
logging settings than just log_min_duration and sometimes it can be messed =
up</div>
<div>- try to restart the database and see if there isn't any other fil=
e created as usually I observe that after deleting current log file, the da=
tabase doesn't recreate while logging so the logs are not stored.</div>
<div><br></div><div>regards<br>Szymon Guz</div></div>

--0016e6d77ed36b9ca10483005a83--
Szymon Guz [ Di, 30 März 2010 10:24 ] [ ID #2037082 ]

Re: Strange deletion problem

This is a multi-part message in MIME format.
--------------020201030100030601040108
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: quoted-printable

hello, John

2010.03.30 10:57, John Lister ra=C5=A1=C4=97:
> - Is there any other statements that could be causing the rows to be
> removed that I've missed
Please see |TRUNCATE.|
> - Is there anything that could be deleting them without generating a
> log entry for the statement?
Don't think so...
> - Is it possible that data corruption of either the index/table is
> making my delete statements removed more rows?
Maybe you have foreign key with "On delete cascade" ?
> - Is it a possibly bug?
I don't know of such bug, but I'm no bug expert as well. I guess
everything is possible.


--
Julius Tuskenis
Programavimo skyriaus vadovas
UAB nSoft
mob. +37068233050


--------------020201030100030601040108
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content=3D"text/html; charset=3DUTF-8" http-equiv=3D"Content-Type=
">
</head>
<body bgcolor=3D"#ffffff" text=3D"#000000">
hello, John<br>
<br>
2010.03.30 10:57, John Lister ra=C5=A1=C4=97:
<blockquote cite=3D"mid:D29C98141027402A85C7801C69954C11 [at] squarepi.com"
type=3D"cite">
<div><font size=3D"2" face=3D"Arial">- Is there any other statements th=
at
could be causing the rows to be removed that I've missed</font></div>
</blockquote>
Please see <code class=3D"command">TRUNCATE.</code>
<blockquote cite=3D"mid:D29C98141027402A85C7801C69954C11 [at] squarepi.com"
type=3D"cite">
<div><font size=3D"2" face=3D"Arial">- Is there anything that could be
deleting them without generating a log entry for the statement?</font></d=
iv>
</blockquote>
Don't think so...<br>
<blockquote cite=3D"mid:D29C98141027402A85C7801C69954C11 [at] squarepi.com"
type=3D"cite">
<div><font size=3D"2" face=3D"Arial">- Is it possible that data
corruption of either the index/table is making my delete statements
removed more rows?</font></div>
</blockquote>
Maybe you have foreign key with "On delete cascade" ?<br>
<blockquote cite=3D"mid:D29C98141027402A85C7801C69954C11 [at] squarepi.com"
type=3D"cite">
<div><font size=3D"2" face=3D"Arial">- Is it a possibly bug?</font></di=
v>
</blockquote>
I don't know of such bug, but I'm no bug expert as well. I guess
everything is possible.<br>
<br>
<br>
<pre class=3D"moz-signature" cols=3D"72">--
Julius Tuskenis
Programavimo skyriaus vadovas
UAB nSoft
mob. +37068233050</pre>
</body>
</html>

--------------020201030100030601040108--
Julius Tuskenis [ Di, 30 März 2010 10:28 ] [ ID #2037083 ]

Re: Strange deletion problem

This is a multi-part message in MIME format.

------=_NextPart_000_0022_01CAD05B.932BC100
Content-Type: text/plain;
charset="UTF-8"
Content-Transfer-Encoding: quoted-printable


2010/3/30 John Lister <john.lister-ps [at] kickstone.com>

Hi, I have a table which is constantly updated through out the day =
with no problems, I'm running Postgresql 8.3.8 in ubuntu karmic. =
However, within the last week for some reason overnight it is being =
emptied and I can't work out why. I've set log_min_duration_statement to =
0 so that postgresql dumps out every query made to the db but it hasn't =
brought anything up. During the period between from when I know the =
table is ok to the next morning when it is empty I can see entries in =
the logs for the expected 200 or so deletions, but during this period =
approx 15k rows are removed. The odd thing is that there is nothing else =
in the logs that references the table (I've looked for the obvious =
deletion and trunctate statements).

The table is modified using JDBC prepared statements so I see 3 =
entries for each of the expected delete statements (parse, bind, =
execute) and the statement is as follows:

delete from product_list where retailer_id=3D? and product_id=3D?


Therefore I have a few questions:
- Is there any other statements that could be causing the rows to be =
removed that I've missed
- Is there anything that could be deleting them without generating a =
log entry for the statement?
- Is it possible that data corruption of either the index/table is =
making my delete statements removed more rows?
- Is it a possibly bug?



Hi,
- maybe there are some other settings for this database (they are not =
stored in the file), check the pg_settings table in the database
- maybe you're looking in a wrong log file - sometimes do (they change =
from time to time - depending on the configuration)
- ensure that the logging is done to file and to THIS file, because =
there are more logging settings than just log_min_duration and sometimes =
it can be messed up
- try to restart the database and see if there isn't any other file =
created as usually I observe that after deleting current log file, the =
database doesn't recreate while logging so the logs are not stored.

Cheers for replying, I've checked the config and nothing seems to be =
amiss, as I'm running ubuntu the defaults seem to be to dump to stderr =
and somehow this is redirect to the log file, there doesn't seem to be =
any other log files used - although it is possible the ubuntu startup =
scripts inject the logfile on startup?
Unfortunately I can't restart the database easily, whatever changed =
seemed to have happened on friday without a restart so I'm hoping I can =
find and undo it...

John

------=_NextPart_000_0022_01CAD05B.932BC100
Content-Type: text/html;
charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

=EF=BB=BF<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8">
<META content=3D"MSHTML 6.00.6000.16981" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<BLOCKQUOTE dir=3Dltr
style=3D"PADDING-RIGHT: 0px; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; =
BORDER-LEFT: #000000 2px solid; MARGIN-RIGHT: 0px">
<DIV class=3Dgmail_quote>2010/3/30 John Lister <SPAN dir=3Dltr><<A
=
href=3D"mailto:john.lister-ps [at] kickstone.com">john.lister-ps [at] kickstone.com=
</A>></SPAN><BR>
<BLOCKQUOTE class=3Dgmail_quote
style=3D"PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: =
#ccc 1px solid">
<DIV bgcolor=3D"#ffffff">
<DIV><FONT face=3DArial size=3D2>Hi, I have a table which is =
constantly updated
through out the day with no problems, =
I'm running Postgresql 8.3.8
in ubuntu karmic. However, within the last week for some reason =
overnight it
is being emptied and I can't work out why. I've set
log_min_duration_statement to 0 so that postgresql dumps out every =
query
made to the db but it hasn't brought anything up. During the period =
between
from when I know the table is ok to the next morning when it is =

empty I can see entries in the logs for the expected 200 or so
deletions, but during this period approx 15k rows are removed. The =
odd thing
is that there is nothing else in the logs that references the table =
(I've
looked for the obvious deletion and trunctate =
statements).</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>The table is modified using JDBC =
prepared
statements so I see 3 entries for each of the expected delete =
statements
(parse, bind, execute) and the statement is as =
follows:</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>delete from product_list where =
retailer_id=3D?
and product_id=3D?</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>Therefore I have a few =
questions:</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>- Is there any other statements =
that could be
causing the rows to be removed that I've missed</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>- Is there anything that could be =
deleting them
without generating a log entry for the statement?</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>- Is it possible that data =
corruption of either
the index/table is making my delete statements removed more
rows?</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>- Is it a possibly =
bug?</FONT></DIV>
<DIV><FONT face=3DArial =
size=3D2></FONT> </DIV></DIV></BLOCKQUOTE>
<DIV><BR></DIV>
<DIV>Hi,</DIV>
<DIV>- maybe there are some other settings for this database (they are =
not
stored in the file), check the pg_settings table in the database</DIV>
<DIV>- maybe you're looking in a wrong log file - sometimes do (they =
change
from time to time - depending on the configuration)</DIV>
<DIV>- ensure that the logging is done to file and to THIS file, =
because there
are more logging settings than just log_min_duration and sometimes it =
can be
messed up</DIV>
<DIV>- try to restart the database and see if there isn't any other =
file
created as usually I observe that after deleting current log file, the =

database doesn't recreate while logging so the logs are not
stored.</DIV></DIV></BLOCKQUOTE>
<DIV dir=3Dltr><FONT face=3DArial size=3D2></FONT><FONT face=3DArial
size=3D2></FONT><BR><FONT face=3DArial size=3D2>Cheers for replying, =
I've checked the
config and nothing seems to be amiss, as I'm running ubuntu the defaults =
seem to
be to dump to stderr and somehow this is redirect to the log file, there =
doesn't
seem to be any other log files used - although it is possible the ubuntu =
startup
scripts inject the logfile on startup?</FONT></DIV>
<DIV dir=3Dltr><FONT face=3DArial size=3D2>Unfortunately I can't restart =
the database
easily, whatever changed seemed to have happened on friday without a =
restart so
I'm hoping I can find and undo it...</FONT></DIV>
<DIV dir=3Dltr><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV dir=3Dltr><FONT face=3DArial size=3D2>John</FONT></DIV>
<DIV dir=3Dltr> </DIV></BODY></HTML>

------=_NextPart_000_0022_01CAD05B.932BC100--
John Lister [ Di, 30 März 2010 23:51 ] [ ID #2037109 ]

Re: Strange deletion problem

You could create a statement level delete trigger on the relevant table,
then get it to snapshot all the contents of the pg_stat_activity table
(which will show all running queries) into some sort of log table.

If you look at the plpgsql documentation (part of the postgresql manual
for your release) there is a section on triggers and an example on
maintaining a summary table; this is sort of the pattern you want to adop=
t,
except that what you insert into the "summary" table will be something
like:

"insert into my_table_delete_log (select * from pg_stat_activity where
current_query not like '%<IDLE>%');"

If you want to run something similar at the row level instead, beware tha=
t
you may then see it trigger 15K times and thus you could end up with 15K
copies of the active queries ...

You might also want to add a sequence column to your log table so you can
be sure which order the queries were inserted, although if you're using
statement level logging, you'll should see distinct query and backend sta=
rt
timestamps anyway.

Just an idea - never tried it myself - so probably best to test it out on
a test database of some sort first!

Cheers,
Robin


On Tue, 30 Mar 2010 22:51:46 +0100, "John Lister"
<john.lister-ps [at] kickstone.com> wrote:
> 2010/3/30 John Lister <john.lister-ps [at] kickstone.com>
>
> Hi, I have a table which is constantly updated through out the day
> with no problems, I'm running Postgresql 8.3.8 in ubuntu karmic.
> However, within the last week for some reason overnight it is being
> emptied and I can't work out why. I've set
log_min_duration_statement
> to 0 so that postgresql dumps out every query made to the db but it
> hasn't brought anything up. During the period between from when I
know
> the table is ok to the next morning when it is empty I can see
entries
> in the logs for the expected 200 or so deletions, but during this
> period approx 15k rows are removed. The odd thing is that there is
> nothing else in the logs that references the table (I've looked for
the
> obvious deletion and trunctate statements).
>
> The table is modified using JDBC prepared statements so I see 3
> entries for each of the expected delete statements (parse, bind,
> execute) and the statement is as follows:
>
> delete from product_list where retailer_id=3D? and product_id=3D?
>
>
> Therefore I have a few questions:
> - Is there any other statements that could be causing the rows to b=
e
> removed that I've missed
> - Is there anything that could be deleting them without generating =
a
> log entry for the statement?
> - Is it possible that data corruption of either the index/table is
> making my delete statements removed more rows?
> - Is it a possibly bug?
>
>
>
> Hi,
> - maybe there are some other settings for this database (they are not
> stored in the file), check the pg_settings table in the database
> - maybe you're looking in a wrong log file - sometimes do (they chang=
e
> from time to time - depending on the configuration)
> - ensure that the logging is done to file and to THIS file, because
> there are more logging settings than just log_min_duration and
sometimes
> it can be messed up
> - try to restart the database and see if there isn't any other file
> created as usually I observe that after deleting current log file, th=
e
> database doesn't recreate while logging so the logs are not stored.
>
> Cheers for replying, I've checked the config and nothing seems to be
> amiss, as I'm running ubuntu the defaults seem to be to dump to stderr
and
> somehow this is redirect to the log file, there doesn't seem to be any
> other log files used - although it is possible the ubuntu startup
scripts
> inject the logfile on startup?
> Unfortunately I can't restart the database easily, whatever changed
seemed
> to have happened on friday without a restart so I'm hoping I can find
and
> undo it...
>
> John

--
Sent via pgsql-admin mailing list (pgsql-admin [at] postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
Robin Iddon [ Mi, 31 März 2010 09:00 ] [ ID #2037224 ]

Re: Strange deletion problem

Thanks to all that helped, I've eventually solved it, I set up a cron job to
monitor the table every minute to help me narrow down the time frame when
the event happened - the thought of scanning 20+Gb log files for the 6hr
period it happened again didn't fill me with joy. I also added
log_statement=all and changed log_prefix to give me a clue, additionally I
added a trigger to log all changes to an audit_table in case I still missed
it. I was using row level triggers as I didn't realise that postgresql had
statement level ones, so you learn something new every day.

It turns out one of the developers had for some insane reason put "delete
from product_list" into an overnight batch job. Bizarrely this wasn't
appearing in the logs when I used log_min_duration_statement=0, but
log_statement=all, which is something else I've learnt.

Right off to shoot the developer in the leg as we speak..

John
----- Original Message -----
From: "robin" <robin [at] edesix.com>
To: "John Lister" <john.lister-ps [at] kickstone.com>
Cc: <pgsql-admin [at] postgresql.org>
Sent: Wednesday, March 31, 2010 8:00 AM
Subject: Re: [ADMIN] Strange deletion problem


> You could create a statement level delete trigger on the relevant table,
> then get it to snapshot all the contents of the pg_stat_activity table
> (which will show all running queries) into some sort of log table.
>
> If you look at the plpgsql documentation (part of the postgresql manual
> for your release) there is a section on triggers and an example on
> maintaining a summary table; this is sort of the pattern you want to
> adopt,
> except that what you insert into the "summary" table will be something
> like:
>
> "insert into my_table_delete_log (select * from pg_stat_activity where
> current_query not like '%<IDLE>%');"
>
> If you want to run something similar at the row level instead, beware that
> you may then see it trigger 15K times and thus you could end up with 15K
> copies of the active queries ...
>
> You might also want to add a sequence column to your log table so you can
> be sure which order the queries were inserted, although if you're using
> statement level logging, you'll should see distinct query and backend
> start
> timestamps anyway.
>
> Just an idea - never tried it myself - so probably best to test it out on
> a test database of some sort first!
>
> Cheers,
> Robin
>
>
> On Tue, 30 Mar 2010 22:51:46 +0100, "John Lister"
> <john.lister-ps [at] kickstone.com> wrote:
>> 2010/3/30 John Lister <john.lister-ps [at] kickstone.com>
>>
>> Hi, I have a table which is constantly updated through out the day
>> with no problems, I'm running Postgresql 8.3.8 in ubuntu karmic.
>> However, within the last week for some reason overnight it is being
>> emptied and I can't work out why. I've set
> log_min_duration_statement
>> to 0 so that postgresql dumps out every query made to the db but it
>> hasn't brought anything up. During the period between from when I
> know
>> the table is ok to the next morning when it is empty I can see
> entries
>> in the logs for the expected 200 or so deletions, but during this
>> period approx 15k rows are removed. The odd thing is that there is
>> nothing else in the logs that references the table (I've looked for
> the
>> obvious deletion and trunctate statements).
>>
>> The table is modified using JDBC prepared statements so I see 3
>> entries for each of the expected delete statements (parse, bind,
>> execute) and the statement is as follows:
>>
>> delete from product_list where retailer_id=? and product_id=?
>>
>>
>> Therefore I have a few questions:
>> - Is there any other statements that could be causing the rows to be
>> removed that I've missed
>> - Is there anything that could be deleting them without generating a
>> log entry for the statement?
>> - Is it possible that data corruption of either the index/table is
>> making my delete statements removed more rows?
>> - Is it a possibly bug?
>>
>>
>>
>> Hi,
>> - maybe there are some other settings for this database (they are not
>> stored in the file), check the pg_settings table in the database
>> - maybe you're looking in a wrong log file - sometimes do (they change
>> from time to time - depending on the configuration)
>> - ensure that the logging is done to file and to THIS file, because
>> there are more logging settings than just log_min_duration and
> sometimes
>> it can be messed up
>> - try to restart the database and see if there isn't any other file
>> created as usually I observe that after deleting current log file, the
>> database doesn't recreate while logging so the logs are not stored.
>>
>> Cheers for replying, I've checked the config and nothing seems to be
>> amiss, as I'm running ubuntu the defaults seem to be to dump to stderr
> and
>> somehow this is redirect to the log file, there doesn't seem to be any
>> other log files used - although it is possible the ubuntu startup
> scripts
>> inject the logfile on startup?
>> Unfortunately I can't restart the database easily, whatever changed
> seemed
>> to have happened on friday without a restart so I'm hoping I can find
> and
>> undo it...
>>
>> John
>


--
Sent via pgsql-admin mailing list (pgsql-admin [at] postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
John Lister [ Mi, 31 März 2010 12:36 ] [ ID #2037237 ]
Datenbanken » gmane.comp.db.postgresql.admin » Strange deletion problem

Vorheriges Thema: postgres 8.2.9 transaction id wraparound failure
Nächstes Thema: Reclaiming space from a toast table