Apache intermittent crash

--000e0cd25ac24ff8ee047adae1a7
Content-Type: text/plain; charset=ISO-8859-1

Dear all,

1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(R) 4 CPU
3.00GHz GNU/Linux
2. The below info are my machine config and log output
3. Kindly advise how to resolve intermittent apache crash even after
performance tuning

*httpd.conf output
*# prefork MPM [DEFAULT IF USE=-threads]
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule prefork.c>
StartServers 2
MinSpareServers 4
MaxSpareServers 6
MaxClients 100
MaxRequestsPerChild 500
</IfModule>

# worker MPM [DEFAULT IF USE=threads]
# StartServers: initial number of server processes to start
# MaxClients: maximum number of simultaneous client connections
# MinSpareThreads: minimum number of worker threads which are kept spare
# MaxSpareThreads: maximum number of worker threads which are kept spare
# ThreadsPerChild: constant number of worker threads in each server process
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule worker.c>
StartServers 2
MaxClients 100
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 500
</IfModule>

*top output
*top - 00:09:00 up 157 days, 23:12, 2 users, load average: 0.67, 0.85,
0.80
Tasks: 47 total, 1 running, 46 sleeping, 0 stopped, 0 zombie
Cpu(s): 8.3% us, 1.4% sy, 0.0% ni, 88.0% id, 2.2% wa, 0.0% hi, 0.0% si
Mem: 917652k total, 858880k used, 58772k free, 142820k buffers
Swap: 1048568k total, 37844k used, 1010724k free, 434476k cached

*apache2 error log
**** glibc detected *** /usr/sbin/apache2: double free or corruption
(fasttop): 0x08b4faf0 ***
[Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting,
consider raising the MaxClients setting
*** glibc detected *** /usr/sbin/apache2: double free or corruption
(fasttop): 0x086a1170 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption
(fasttop): 0x08b48c08 ***
[Wed Dec 16 23:53:07 2009] [warn] child process 1325 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:07 2009] [warn] child process 10569 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:07 2009] [warn] child process 16964 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:09 2009] [warn] child process 1325 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:09 2009] [warn] child process 19910 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:09 2009] [warn] child process 16964 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:09 2009] [warn] child process 19718 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:11 2009] [warn] child process 19910 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:11 2009] [warn] child process 10569 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:11 2009] [warn] child process 19718 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:13 2009] [error] child process 1325 still did not exit,
sending a SIGKILL
[Wed Dec 16 23:53:13 2009] [error] child process 19910 still did not exit,
sending a SIGKILL
[Wed Dec 16 23:53:13 2009] [error] child process 10569 still did not exit,
sending a SIGKILL
[Wed Dec 16 23:53:13 2009] [error] child process 16964 still did not exit,
sending a SIGKILL

*Joomla / PHP / Apache
*System Information
*PHP built On:* Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23
19:20:15 SGT 2006 i686 *Database Version:* 5.0.24a-log *PHP
Version:* 5.1.6-pl4-gentoo
*Web Server:* Apache *WebServer to PHP interface:* apache2handler
*Joomla!
Version:* Joomla! 1.0.15 Stable [ Daytime ] 22 February 2008 23:00 UTC *User
Agent:* Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.15)
Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729)

*Mysql
*mysql Ver 14.12 Distrib 5.0.24a, for pc-linux-gnu (i686) using readline
5.1

--000e0cd25ac24ff8ee047adae1a7
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Dear all, 1. I'm using Linux 2.6.16.29 and running on i686 Intel=
(R) Pentium(R) 4 CPU 3.00GHz GNU/Linux 2. The below info are my machine =
config and log output 3. Kindly advise how to resolve intermittent apach=
e crash even after performance tuning 
 httpd.conf output # prefork MPM [DEFAULT IF USE=3D-threads]<b=
r># StartServers: number of server processes to start # MinSpareServers:=
minimum number of server processes which are kept spare # MaxSpareServe=
rs: maximum number of server processes which are kept spare 
# MaxClients: maximum number of server processes allowed to start # MaxR=
equestsPerChild: maximum number of requests a server process serves <=
IfModule prefork.c> =A0=A0=A0 StartServers=A0=A0=A0=A0=A0=A0=A0=A0 2<=
br>=A0=A0=A0 MinSpareServers=A0=A0=A0=A0=A0 4 
=A0=A0=A0 MaxSpareServers=A0=A0=A0=A0 6 =A0=A0=A0 MaxClients=A0=A0=A0=A0=
=A0=A0=A0=A0 100 =A0=A0=A0 MaxRequestsPerChild=A0 500 </IfModule&g=
t; # worker MPM [DEFAULT IF USE=3Dthreads] # StartServers: initia=
l number of server processes to start # MaxClients: maximum number of si=
multaneous client connections 
# MinSpareThreads: minimum number of worker threads which are kept spare<br=
># MaxSpareThreads: maximum number of worker threads which are kept spare<b=
r># ThreadsPerChild: constant number of worker threads in each server proce=
ss 
# MaxRequestsPerChild: maximum number of requests a server process serves<b=
r><IfModule worker.c> =A0=A0=A0 StartServers=A0=A0=A0=A0=A0=A0=A0=
=A0 2 =A0=A0=A0 MaxClients=A0=A0=A0=A0=A0=A0=A0=A0 100 =A0=A0=A0 MinS=
pareThreads=A0=A0=A0=A0 25 =A0=A0=A0 MaxSpareThreads=A0=A0=A0=A0 75 
=A0=A0=A0 ThreadsPerChild=A0=A0=A0=A0 25 =A0=A0=A0 MaxRequestsPerChild=
=A0 500 </IfModule> top output top - 00:0=
9:00 up 157 days, 23:12,=A0 2 users,=A0 load average: 0.67, 0.85, 0.80 T=
asks:=A0 47 total,=A0=A0 1 running,=A0 46 sleeping,=A0=A0 0 stopped,=A0=A0 =
0 zombie 
Cpu(s):=A0 8.3% us,=A0 1.4% sy,=A0 0.0% ni, 88.0% id,=A0 2.2% wa,=A0 0.0% h=
i,=A0 0.0% si Mem:=A0=A0=A0 917652k total,=A0=A0 858880k used,=A0=A0=A0 =
58772k free,=A0=A0 142820k buffers Swap:=A0 1048568k total,=A0=A0=A0 378=
44k used,=A0 1010724k free,=A0=A0 434476k cached 
 apache2 error log *** glibc detected *** /usr/sbin/apache2: d=
ouble free or corruption (fasttop): 0x08b4faf0 *** [Tue Dec 15 13:04:22 =
2009] [error] server reached MaxClients setting, consider raising the MaxCl=
ients setting 
*** glibc detected *** /usr/sbin/apache2: double free or corruption (fastto=
p): 0x086a1170 *** *** glibc detected *** /usr/sbin/apache2: double free=
or corruption (fasttop): 0x08b48c08 *** [Wed Dec 16 23:53:07 2009] [war=
n] child process 1325 still did not exit, sending a SIGTERM 
[Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not exit, s=
ending a SIGTERM [Wed Dec 16 23:53:07 2009] [warn] child process 10569 s=
till did not exit, sending a SIGTERM [Wed Dec 16 23:53:07 2009] [warn] c=
hild process 16964 still did not exit, sending a SIGTERM 
[Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not exit, s=
ending a SIGTERM [Wed Dec 16 23:53:09 2009] [warn] child process 1325 st=
ill did not exit, sending a SIGTERM [Wed Dec 16 23:53:09 2009] [warn] ch=
ild process 19910 still did not exit, sending a SIGTERM 
[Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not exit, s=
ending a SIGTERM [Wed Dec 16 23:53:09 2009] [warn] child process 16964 s=
till did not exit, sending a SIGTERM [Wed Dec 16 23:53:09 2009] [warn] c=
hild process 19718 still did not exit, sending a SIGTERM 
[Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not exit, se=
nding a SIGTERM [Wed Dec 16 23:53:11 2009] [warn] child process 19910 st=
ill did not exit, sending a SIGTERM [Wed Dec 16 23:53:11 2009] [warn] ch=
ild process 10569 still did not exit, sending a SIGTERM 
[Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not exit, s=
ending a SIGTERM [Wed Dec 16 23:53:11 2009] [warn] child process 19718 s=
till did not exit, sending a SIGTERM [Wed Dec 16 23:53:13 2009] [error] =
child process 1325 still did not exit, sending a SIGKILL 
[Wed Dec 16 23:53:13 2009] [error] child process 19910 still did not exit, =
sending a SIGKILL [Wed Dec 16 23:53:13 2009] [error] child process 10569=
still did not exit, sending a SIGKILL [Wed Dec 16 23:53:13 2009] [error=
] child process 16964 still did not exit, sending a SIGKILL 
 Joomla / PHP / Apache <table class=3D"adminform"><tbody><=
tr><th colspan=3D"2">System Information
</th>
</tr>
<tr>
<td colspan=3D"2">
 </td>
</tr>
<tr>
<td valign=3D"top" width=3D"250">
PHP built On:
</td>
<td>
Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23 19:20:15 SGT 2006 =
i686 </td>
</tr>
<tr>
<td>
Database Version:
</td>
<td>
5.0.24a-log </td>
</tr>
<tr>
<td>
PHP Version:
</td>
<td>
5.1.6-pl4-gentoo </td>
</tr>
<tr>
<td>
Web Server:
</td>
<td>
Apache </td>
</tr>
<tr>
<td>
WebServer to PHP interface:
</td>
<td>
apache2handler </td>
</tr>
<tr>
<td>
Joomla! Version:
</td>
<td>
Joomla! 1.0.15 Stable [ Daytime ] 22 February 2008 23:00 UTC </td>
</tr>
<tr>
<td>
User Agent:
</td>
<td>
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.15) Gecko/200=
9101601 Firefox/3.0.15 (.NET CLR 3.5.30729) </td></tr></tbody></table><b=
r> Mysql mysql=A0 Ver 14.12 Distrib 5.0.24a, for pc-linux-gnu =
(i686) using readline 5.1 
 

--000e0cd25ac24ff8ee047adae1a7--

On Wed, Dec 16, 2009 at 4:20 PM, gary lim <garylim.in [at] gmail.com> wrote:
>
> Dear all,
>
> 1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(R) 4 CP=
U 3.00GHz GNU/Linux
> 2. The below info are my machine config and log output
> 3. Kindly advise how to resolve intermittent apache crash even after perf=
ormance tuning
>
> httpd.conf output
> # prefork MPM [DEFAULT IF USE=3D-threads]
> # StartServers: number of server processes to start
> # MinSpareServers: minimum number of server processes which are kept spar=
e
> # MaxSpareServers: maximum number of server processes which are kept spar=
e
> # MaxClients: maximum number of server processes allowed to start
> # MaxRequestsPerChild: maximum number of requests a server process serves
> <IfModule prefork.c>
> =C2=A0=C2=A0=C2=A0 StartServers=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 2
> =C2=A0=C2=A0=C2=A0 MinSpareServers=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4
> =C2=A0=C2=A0=C2=A0 MaxSpareServers=C2=A0=C2=A0=C2=A0=C2=A0 6
> =C2=A0=C2=A0=C2=A0 MaxClients=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 100
> =C2=A0=C2=A0=C2=A0 MaxRequestsPerChild=C2=A0 500
> </IfModule>
>
> # worker MPM [DEFAULT IF USE=3Dthreads]
> # StartServers: initial number of server processes to start
> # MaxClients: maximum number of simultaneous client connections
> # MinSpareThreads: minimum number of worker threads which are kept spare
> # MaxSpareThreads: maximum number of worker threads which are kept spare
> # ThreadsPerChild: constant number of worker threads in each server proce=
ss
> # MaxRequestsPerChild: maximum number of requests a server process serves
> <IfModule worker.c>
> =C2=A0=C2=A0=C2=A0 StartServers=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 2
> =C2=A0=C2=A0=C2=A0 MaxClients=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 100
> =C2=A0=C2=A0=C2=A0 MinSpareThreads=C2=A0=C2=A0=C2=A0=C2=A0 25
> =C2=A0=C2=A0=C2=A0 MaxSpareThreads=C2=A0=C2=A0=C2=A0=C2=A0 75
> =C2=A0=C2=A0=C2=A0 ThreadsPerChild=C2=A0=C2=A0=C2=A0=C2=A0 25
> =C2=A0=C2=A0=C2=A0 MaxRequestsPerChild=C2=A0 500
> </IfModule>
>
>
>
> top output
> top - 00:09:00 up 157 days, 23:12,=C2=A0 2 users,=C2=A0 load average: 0.6=
7, 0.85, 0.80
> Tasks:=C2=A0 47 total,=C2=A0=C2=A0 1 running,=C2=A0 46 sleeping,=C2=A0=C2=
=A0 0 stopped,=C2=A0=C2=A0 0 zombie
> Cpu(s):=C2=A0 8.3% us,=C2=A0 1.4% sy,=C2=A0 0.0% ni, 88.0% id,=C2=A0 2.2%=
wa,=C2=A0 0.0% hi,=C2=A0 0.0% si
> Mem:=C2=A0=C2=A0=C2=A0 917652k total,=C2=A0=C2=A0 858880k used,=C2=A0=C2=
=A0=C2=A0 58772k free,=C2=A0=C2=A0 142820k buffers
> Swap:=C2=A0 1048568k total,=C2=A0=C2=A0=C2=A0 37844k used,=C2=A0 1010724k=
free,=C2=A0=C2=A0 434476k cached
>
> apache2 error log
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (fast=
top): 0x08b4faf0 ***
> [Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting, con=
sider raising the MaxClients setting
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (fast=
top): 0x086a1170 ***
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (fast=
top): 0x08b48c08 ***
> [Wed Dec 16 23:53:07 2009] [warn] child process 1325 still did not exit, =
sending a SIGTERM
> [Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not exit,=
sending a SIGTERM
> [Wed Dec 16 23:53:07 2009] [warn] child process 10569 still did not exit,=
sending a SIGTERM
> [Wed Dec 16 23:53:07 2009] [warn] child process 16964 still did not exit,=
sending a SIGTERM
> [Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not exit,=
sending a SIGTERM
> [Wed Dec 16 23:53:09 2009] [warn] child process 1325 still did not exit, =
sending a SIGTERM
> [Wed Dec 16 23:53:09 2009] [warn] child process 19910 still did not exit,=
sending a SIGTERM
> [Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not exit,=
sending a SIGTERM
> [Wed Dec 16 23:53:09 2009] [warn] child process 16964 still did not exit,=
sending a SIGTERM
> [Wed Dec 16 23:53:09 2009] [warn] child process 19718 still did not exit,=
sending a SIGTERM
> [Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not exit, =
sending a SIGTERM
> [Wed Dec 16 23:53:11 2009] [warn] child process 19910 still did not exit,=
sending a SIGTERM
> [Wed Dec 16 23:53:11 2009] [warn] child process 10569 still did not exit,=
sending a SIGTERM
> [Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not exit,=
sending a SIGTERM
> [Wed Dec 16 23:53:11 2009] [warn] child process 19718 still did not exit,=
sending a SIGTERM
> [Wed Dec 16 23:53:13 2009] [error] child process 1325 still did not exit,=
sending a SIGKILL
> [Wed Dec 16 23:53:13 2009] [error] child process 19910 still did not exit=
, sending a SIGKILL
> [Wed Dec 16 23:53:13 2009] [error] child process 10569 still did not exit=
, sending a SIGKILL
> [Wed Dec 16 23:53:13 2009] [error] child process 16964 still did not exit=
, sending a SIGKILL
>
>
> Joomla / PHP / Apache
> System Information
> PHP built On: Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23 19:20:1=
5 SGT 2006 i686
> Database Version: 5.0.24a-log
> PHP Version: 5.1.6-pl4-gentoo
> Web Server: Apache
> WebServer to PHP interface: apache2handler
> Joomla! Version: Joomla! 1.0.15 Stable [ Daytime ] 22 February 2008 23:00=
UTC
> User Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.15) =
Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729)
>
> Mysql
> mysql=C2=A0 Ver 14.12 Distrib 5.0.24a, for pc-linux-gnu (i686) using read=
line 5.1
>

Does it work if you disable PHP? Does it still exhibit problems if you
run PHP out of process, eg using fastcgi or scgi (can PHP talk SCGI?).
For my money, the double free()s are coming from PHP or a PHP module.

Cheers

Tom

------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
" from the digest: users-digest-unsubscribe [at] httpd.apache.org
For additional commands, e-mail: users-help [at] httpd.apache.org

On 16 Dec 2009, at 16:20, gary lim wrote:

> Dear all,
>
> 1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(R) 4 CPU 3.00GHz GNU/Linux

but no sign of your APR and Apache versions.

--
Nick Kew

------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
" from the digest: users-digest-unsubscribe [at] httpd.apache.org
For additional commands, e-mail: users-help [at] httpd.apache.org

--000e0cd348581ab64c047ae278f6
Content-Type: text/plain; charset=ISO-8859-1

Hi,

1. Unfortunately, our website depend heavily on PHP to operate, so it not
possible for us to disable it. We also highly suspect that the crash is
related to one of our php module because we are running a lot of Joomla
component.

2. unfortunately, we are unable to track which module is causing this issue.
Any idea how we can check which script is causing the problem because this
has been going on for weeks and we are still unable to determine the source.

3. Kindly advise how we can perform the migration of our existing codebase
to FastCGI/SCGI without breaking the current website?

rgds

On Thu, Dec 17, 2009 at 12:28 AM, Tom Evans <tevans.uk [at] googlemail.com>wrote:

> On Wed, Dec 16, 2009 at 4:20 PM, gary lim <garylim.in [at] gmail.com> wrote:
> >
> > Dear all,
> >
> > 1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(R) 4
> CPU 3.00GHz GNU/Linux
> > 2. The below info are my machine config and log output
> > 3. Kindly advise how to resolve intermittent apache crash even after
> performance tuning
> >
> > httpd.conf output
> > # prefork MPM [DEFAULT IF USE=-threads]
> > # StartServers: number of server processes to start
> > # MinSpareServers: minimum number of server processes which are kept
> spare
> > # MaxSpareServers: maximum number of server processes which are kept
> spare
> > # MaxClients: maximum number of server processes allowed to start
> > # MaxRequestsPerChild: maximum number of requests a server process serves
> > <IfModule prefork.c>
> > StartServers 2
> > MinSpareServers 4
> > MaxSpareServers 6
> > MaxClients 100
> > MaxRequestsPerChild 500
> > </IfModule>
> >
> > # worker MPM [DEFAULT IF USE=threads]
> > # StartServers: initial number of server processes to start
> > # MaxClients: maximum number of simultaneous client connections
> > # MinSpareThreads: minimum number of worker threads which are kept spare
> > # MaxSpareThreads: maximum number of worker threads which are kept spare
> > # ThreadsPerChild: constant number of worker threads in each server
> process
> > # MaxRequestsPerChild: maximum number of requests a server process serves
> > <IfModule worker.c>
> > StartServers 2
> > MaxClients 100
> > MinSpareThreads 25
> > MaxSpareThreads 75
> > ThreadsPerChild 25
> > MaxRequestsPerChild 500
> > </IfModule>
> >
> >
> >
> > top output
> > top - 00:09:00 up 157 days, 23:12, 2 users, load average: 0.67, 0.85,
> 0.80
> > Tasks: 47 total, 1 running, 46 sleeping, 0 stopped, 0 zombie
> > Cpu(s): 8.3% us, 1.4% sy, 0.0% ni, 88.0% id, 2.2% wa, 0.0% hi, 0.0%
> si
> > Mem: 917652k total, 858880k used, 58772k free, 142820k buffers
> > Swap: 1048568k total, 37844k used, 1010724k free, 434476k cached
> >
> > apache2 error log
> > *** glibc detected *** /usr/sbin/apache2: double free or corruption
> (fasttop): 0x08b4faf0 ***
> > [Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting,
> consider raising the MaxClients setting
> > *** glibc detected *** /usr/sbin/apache2: double free or corruption
> (fasttop): 0x086a1170 ***
> > *** glibc detected *** /usr/sbin/apache2: double free or corruption
> (fasttop): 0x08b48c08 ***
> > [Wed Dec 16 23:53:07 2009] [warn] child process 1325 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:07 2009] [warn] child process 10569 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:07 2009] [warn] child process 16964 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:09 2009] [warn] child process 1325 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:09 2009] [warn] child process 19910 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:09 2009] [warn] child process 16964 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:09 2009] [warn] child process 19718 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:11 2009] [warn] child process 19910 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:11 2009] [warn] child process 10569 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:11 2009] [warn] child process 19718 still did not exit,
> sending a SIGTERM
> > [Wed Dec 16 23:53:13 2009] [error] child process 1325 still did not exit,
> sending a SIGKILL
> > [Wed Dec 16 23:53:13 2009] [error] child process 19910 still did not
> exit, sending a SIGKILL
> > [Wed Dec 16 23:53:13 2009] [error] child process 10569 still did not
> exit, sending a SIGKILL
> > [Wed Dec 16 23:53:13 2009] [error] child process 16964 still did not
> exit, sending a SIGKILL
> >
> >
> > Joomla / PHP / Apache
> > System Information
> > PHP built On: Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23
> 19:20:15 SGT 2006 i686
> > Database Version: 5.0.24a-log
> > PHP Version: 5.1.6-pl4-gentoo
> > Web Server: Apache
> > WebServer to PHP interface: apache2handler
> > Joomla! Version: Joomla! 1.0.15 Stable [ Daytime ] 22 February 2008 23:00
> UTC
> > User Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.15)
> Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729)
> >
> > Mysql
> > mysql Ver 14.12 Distrib 5.0.24a, for pc-linux-gnu (i686) using readline
> 5.1
> >
>
> Does it work if you disable PHP? Does it still exhibit problems if you
> run PHP out of process, eg using fastcgi or scgi (can PHP talk SCGI?).
> For my money, the double free()s are coming from PHP or a PHP module.
>
> Cheers
>
> Tom
>
> ------------------------------------------------------------ ---------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
> " from the digest: users-digest-unsubscribe [at] httpd.apache.org
> For additional commands, e-mail: users-help [at] httpd.apache.org
>
>

--000e0cd348581ab64c047ae278f6
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi, 1. Unfortunately, our website depend heavily on PHP to operate, =
so it not possible for us to disable it. We also highly suspect that the cr=
ash is related to one of our php module because we are running a lot of Joo=
mla component. 
 2. unfortunately, we are unable to track which module is causing this i=
ssue. Any idea how we can check which script is causing the problem because=
this has been going on for weeks and we are still unable to determine the =
source. 
 3. Kindly advise how we can perform the migration of our existing codeb=
ase to FastCGI/SCGI without breaking the current website? rgds<b=
r> <div class=3D"gmail_quote">On Thu, Dec 17, 2009 at 12:28 AM, Tom =
Evans <<a href=3D"http://tevans.uk">tevans.uk</a> [at] <a h=
ref=3D"http://googlemail.com">googlemail.com</a>> wrote: 
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><=
div class=3D"h5">On Wed, Dec 16, 2009 at 4:20 PM, gary lim <<a href=3D"h=
ttp://garylim.in" target=3D"_blank">garylim.in</a> [at] <a href=3D"http://gmail.=
com" target=3D"_blank">gmail.com</a>> wrote: 

> 
> Dear all, 
> 
> 1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(=
R) 4 CPU 3.00GHz GNU/Linux 
> 2. The below info are my machine config and log output 
> 3. Kindly advise how to resolve intermittent apache crash even after p=
erformance tuning 
> 
> httpd.conf output 
> # prefork MPM [DEFAULT IF USE=3D-threads] 
> # StartServers: number of server processes to start 
> # MinSpareServers: minimum number of server processes which are kept s=
pare 
> # MaxSpareServers: maximum number of server processes which are kept s=
pare 
> # MaxClients: maximum number of server processes allowed to start 
> # MaxRequestsPerChild: maximum number of requests a server process ser=
ves 
> <IfModule prefork.c> 
> =A0=A0=A0 StartServers=A0=A0=A0=A0=A0=A0=A0=A0 2 
> =A0=A0=A0 MinSpareServers=A0=A0=A0=A0=A0 4 
> =A0=A0=A0 MaxSpareServers=A0=A0=A0=A0 6 
> =A0=A0=A0 MaxClients=A0=A0=A0=A0=A0=A0=A0=A0 100 
> =A0=A0=A0 MaxRequestsPerChild=A0 500 
> </IfModule> 
> 
> # worker MPM [DEFAULT IF USE=3Dthreads] 
> # StartServers: initial number of server processes to start 
> # MaxClients: maximum number of simultaneous client connections 
> # MinSpareThreads: minimum number of worker threads which are kept spa=
re 
> # MaxSpareThreads: maximum number of worker threads which are kept spa=
re 
> # ThreadsPerChild: constant number of worker threads in each server pr=
ocess 
> # MaxRequestsPerChild: maximum number of requests a server process ser=
ves 
> <IfModule worker.c> 
> =A0=A0=A0 StartServers=A0=A0=A0=A0=A0=A0=A0=A0 2 
> =A0=A0=A0 MaxClients=A0=A0=A0=A0=A0=A0=A0=A0 100 
> =A0=A0=A0 MinSpareThreads=A0=A0=A0=A0 25 
> =A0=A0=A0 MaxSpareThreads=A0=A0=A0=A0 75 
> =A0=A0=A0 ThreadsPerChild=A0=A0=A0=A0 25 
> =A0=A0=A0 MaxRequestsPerChild=A0 500 
> </IfModule> 
> 
> 
> 
> top output 
> top - 00:09:00 up 157 days, 23:12,=A0 2 users,=A0 load average: 0.67, =
0.85, 0.80 
> Tasks:=A0 47 total,=A0=A0 1 running,=A0 46 sleeping,=A0=A0 0 stopped,=
=A0=A0 0 zombie 
> Cpu(s):=A0 8.3% us,=A0 1.4% sy,=A0 0.0% ni, 88.0% id,=A0 2.2% wa,=A0 0=
..0% hi,=A0 0.0% si 
> Mem:=A0=A0=A0 917652k total,=A0=A0 858880k used,=A0=A0=A0 58772k free,=
=A0=A0 142820k buffers 
> Swap:=A0 1048568k total,=A0=A0=A0 37844k used,=A0 1010724k free,=A0=A0=
434476k cached 
> 
> apache2 error log 
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f=
asttop): 0x08b4faf0 *** 
> [Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting, =
consider raising the MaxClients setting 
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f=
asttop): 0x086a1170 *** 
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f=
asttop): 0x08b48c08 *** 
> [Wed Dec 16 23:53:07 2009] [warn] child process 1325 still did not exi=
t, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 10569 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 16964 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 1325 still did not exi=
t, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 19910 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 16964 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 19718 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not exi=
t, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 19910 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 10569 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 19718 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:13 2009] [error] child process 1325 still did not ex=
it, sending a SIGKILL 
> [Wed Dec 16 23:53:13 2009] [error] child process 19910 still did not e=
xit, sending a SIGKILL 
> [Wed Dec 16 23:53:13 2009] [error] child process 10569 still did not e=
xit, sending a SIGKILL 
> [Wed Dec 16 23:53:13 2009] [error] child process 16964 still did not e=
xit, sending a SIGKILL 
> 
> 
> Joomla / PHP / Apache 
> System Information 
> PHP built On: Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23 19:2=
0:15 SGT 2006 i686 
> Database Version: 5.0.24a-log 
> PHP Version: 5.1.6-pl4-gentoo 
> Web Server: Apache 
> WebServer to PHP interface: apache2handler 
> Joomla! Version: Joomla! 1.0.15 Stable [ Daytime ] 22 February 2008 23=
:00 UTC 
> User Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1=
5) Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729) 
> 
> Mysql 
> mysql=A0 Ver 14.12 Distrib 5.0.24a, for pc-linux-gnu (i686) using read=
line 5.1 
> 
 
</div></div>Does it work if you disable PHP? Does it still exhibit problems=
if you 
run PHP out of process, eg using fastcgi or scgi (can PHP talk SCGI?). 
For my money, the double free()s are coming from PHP or a PHP module. 
 
Cheers 
 
Tom 
 
------------------------------------------------------------ --------- 
The official User-To-User support forum of the Apache HTTP Server Project.<=
br>
See <URL:<a href=3D"http://httpd.apache.org/userslist.html" target=3D"_b=
lank">http://httpd.apache.org/userslist.html</a>> for more info. 
To unsubscribe, e-mail: <a href=3D"mailto:users-unsubscribe [at] httpd.apache.or=
g">users-unsubscribe [at] httpd.apache.org</a> 
=A0 " =A0 from the digest: <a href=3D"mailto:users-digest-unsubscribe=
[at] httpd.apache.org">users-digest-unsubscribe [at] httpd.apache.org</a> 
For additional commands, e-mail: <a href=3D"mailto:users-help [at] httpd.apache.=
org">users-help [at] httpd.apache.org</a> 
 
</blockquote></div> 

--000e0cd348581ab64c047ae278f6--

--000e0cd60b12fa5fe0047ae32a47
Content-Type: text/plain; charset=ISO-8859-1

Gary:

The same thing happened to one of my servers. After long research I got to
the conclusion that the problem was caused by the XCache module. If you have
it, try to disable it.

Rodrigo

2009/12/16 gary lim <garylim.in [at] gmail.com>

> Hi,
>
> 1. Unfortunately, our website depend heavily on PHP to operate, so it not
> possible for us to disable it. We also highly suspect that the crash is
> related to one of our php module because we are running a lot of Joomla
> component.
>
> 2. unfortunately, we are unable to track which module is causing this
> issue. Any idea how we can check which script is causing the problem because
> this has been going on for weeks and we are still unable to determine the
> source.
>
> 3. Kindly advise how we can perform the migration of our existing codebase
> to FastCGI/SCGI without breaking the current website?
>
>
> rgds
>
>
> On Thu, Dec 17, 2009 at 12:28 AM, Tom Evans <tevans.uk [at] googlemail.com>wrote:
>
>> On Wed, Dec 16, 2009 at 4:20 PM, gary lim <garylim.in [at] gmail.com> wrote:
>> >
>> > Dear all,
>> >
>> > 1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(R) 4
>> CPU 3.00GHz GNU/Linux
>> > 2. The below info are my machine config and log output
>> > 3. Kindly advise how to resolve intermittent apache crash even after
>> performance tuning
>> >
>> > httpd.conf output
>> > # prefork MPM [DEFAULT IF USE=-threads]
>> > # StartServers: number of server processes to start
>> > # MinSpareServers: minimum number of server processes which are kept
>> spare
>> > # MaxSpareServers: maximum number of server processes which are kept
>> spare
>> > # MaxClients: maximum number of server processes allowed to start
>> > # MaxRequestsPerChild: maximum number of requests a server process
>> serves
>> > <IfModule prefork.c>
>> > StartServers 2
>> > MinSpareServers 4
>> > MaxSpareServers 6
>> > MaxClients 100
>> > MaxRequestsPerChild 500
>> > </IfModule>
>> >
>> > # worker MPM [DEFAULT IF USE=threads]
>> > # StartServers: initial number of server processes to start
>> > # MaxClients: maximum number of simultaneous client connections
>> > # MinSpareThreads: minimum number of worker threads which are kept spare
>> > # MaxSpareThreads: maximum number of worker threads which are kept spare
>> > # ThreadsPerChild: constant number of worker threads in each server
>> process
>> > # MaxRequestsPerChild: maximum number of requests a server process
>> serves
>> > <IfModule worker.c>
>> > StartServers 2
>> > MaxClients 100
>> > MinSpareThreads 25
>> > MaxSpareThreads 75
>> > ThreadsPerChild 25
>> > MaxRequestsPerChild 500
>> > </IfModule>
>> >
>> >
>> >
>> > top output
>> > top - 00:09:00 up 157 days, 23:12, 2 users, load average: 0.67, 0.85,
>> 0.80
>> > Tasks: 47 total, 1 running, 46 sleeping, 0 stopped, 0 zombie
>> > Cpu(s): 8.3% us, 1.4% sy, 0.0% ni, 88.0% id, 2.2% wa, 0.0% hi,
>> 0.0% si
>> > Mem: 917652k total, 858880k used, 58772k free, 142820k buffers
>> > Swap: 1048568k total, 37844k used, 1010724k free, 434476k cached
>> >
>> > apache2 error log
>> > *** glibc detected *** /usr/sbin/apache2: double free or corruption
>> (fasttop): 0x08b4faf0 ***
>> > [Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting,
>> consider raising the MaxClients setting
>> > *** glibc detected *** /usr/sbin/apache2: double free or corruption
>> (fasttop): 0x086a1170 ***
>> > *** glibc detected *** /usr/sbin/apache2: double free or corruption
>> (fasttop): 0x08b48c08 ***
>> > [Wed Dec 16 23:53:07 2009] [warn] child process 1325 still did not exit,
>> sending a SIGTERM
>> > [Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not
>> exit, sending a SIGTERM
>> > [Wed Dec 16 23:53:07 2009] [warn] child process 10569 still did not
>> exit, sending a SIGTERM
>> > [Wed Dec 16 23:53:07 2009] [warn] child process 16964 still did not
>> exit, sending a SIGTERM
>> > [Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not
>> exit, sending a SIGTERM
>> > [Wed Dec 16 23:53:09 2009] [warn] child process 1325 still did not exit,
>> sending a SIGTERM
>> > [Wed Dec 16 23:53:09 2009] [warn] child process 19910 still did not
>> exit, sending a SIGTERM
>> > [Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not
>> exit, sending a SIGTERM
>> > [Wed Dec 16 23:53:09 2009] [warn] child process 16964 still did not
>> exit, sending a SIGTERM
>> > [Wed Dec 16 23:53:09 2009] [warn] child process 19718 still did not
>> exit, sending a SIGTERM
>> > [Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not exit,
>> sending a SIGTERM
>> > [Wed Dec 16 23:53:11 2009] [warn] child process 19910 still did not
>> exit, sending a SIGTERM
>> > [Wed Dec 16 23:53:11 2009] [warn] child process 10569 still did not
>> exit, sending a SIGTERM
>> > [Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not
>> exit, sending a SIGTERM
>> > [Wed Dec 16 23:53:11 2009] [warn] child process 19718 still did not
>> exit, sending a SIGTERM
>> > [Wed Dec 16 23:53:13 2009] [error] child process 1325 still did not
>> exit, sending a SIGKILL
>> > [Wed Dec 16 23:53:13 2009] [error] child process 19910 still did not
>> exit, sending a SIGKILL
>> > [Wed Dec 16 23:53:13 2009] [error] child process 10569 still did not
>> exit, sending a SIGKILL
>> > [Wed Dec 16 23:53:13 2009] [error] child process 16964 still did not
>> exit, sending a SIGKILL
>> >
>> >
>> > Joomla / PHP / Apache
>> > System Information
>> > PHP built On: Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23
>> 19:20:15 SGT 2006 i686
>> > Database Version: 5.0.24a-log
>> > PHP Version: 5.1.6-pl4-gentoo
>> > Web Server: Apache
>> > WebServer to PHP interface: apache2handler
>> > Joomla! Version: Joomla! 1.0.15 Stable [ Daytime ] 22 February 2008
>> 23:00 UTC
>> > User Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.15)
>> Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729)
>> >
>> > Mysql
>> > mysql Ver 14.12 Distrib 5.0.24a, for pc-linux-gnu (i686) using readline
>> 5.1
>> >
>>
>> Does it work if you disable PHP? Does it still exhibit problems if you
>> run PHP out of process, eg using fastcgi or scgi (can PHP talk SCGI?).
>> For my money, the double free()s are coming from PHP or a PHP module.
>>
>> Cheers
>>
>> Tom
>>
>> ------------------------------------------------------------ ---------
>> The official User-To-User support forum of the Apache HTTP Server Project.
>> See <URL:http://httpd.apache.org/userslist.html> for more info.
>> To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
>> " from the digest: users-digest-unsubscribe [at] httpd.apache.org
>> For additional commands, e-mail: users-help [at] httpd.apache.org
>>
>>
>

--
Rodrigo Aliste P.

--000e0cd60b12fa5fe0047ae32a47
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Gary: The same thing happened to one of my servers. After long resea=
rch I got to the conclusion that the problem was caused by the XCache modul=
e. If you have it, try to disable it. Rodrigo <div class=
=3D"gmail_quote">

2009/12/16 gary lim <<a href=3D"http://garylim.in">gar=
ylim.in</a> [at] <a href=3D"http://gmail.com">gmail.com</a>> <block=
quote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, 204, 2=
04); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">

Hi, 1. Unfortunately, our website depend heavily on PHP to operate, =
so it not possible for us to disable it. We also highly suspect that the cr=
ash is related to one of our php module because we are running a lot of Joo=
mla component. 

 2. unfortunately, we are unable to track which module is causing this i=
ssue. Any idea how we can check which script is causing the problem because=
this has been going on for weeks and we are still unable to determine the =
source. 

 3. Kindly advise how we can perform the migration of our existing codeb=
ase to FastCGI/SCGI without breaking the current website? rgds<b=
r> <div class=3D"gmail_quote">On Thu, Dec 17, 2009 at 12:28 AM, Tom =
Evans <<a href=3D"http://tevans.uk" target=3D"_blank">=
tevans.uk</a> [at] <a href=3D"http://googlemail.com" target=3D"_blank">googlemai=
l.com</a>> wrote: 

<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><=
div>On Wed, Dec 16, 2009 at 4:20 PM, gary lim <<a href=3D"http://garylim=
..in" target=3D"_blank">garylim.in</a> [at] <a href=3D"http://gmail.com" target=
=3D"_blank">gmail.com</a>> wrote: 

> 
> Dear all, 
> 
> 1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(=
R) 4 CPU 3.00GHz GNU/Linux 
> 2. The below info are my machine config and log output 
> 3. Kindly advise how to resolve intermittent apache crash even after p=
erformance tuning 
> 
> httpd.conf output 
> # prefork MPM [DEFAULT IF USE=3D-threads] 
> # StartServers: number of server processes to start 
> # MinSpareServers: minimum number of server processes which are kept s=
pare 
> # MaxSpareServers: maximum number of server processes which are kept s=
pare 
> # MaxClients: maximum number of server processes allowed to start 
> # MaxRequestsPerChild: maximum number of requests a server process ser=
ves 
> <IfModule prefork.c> 
> =A0=A0=A0 StartServers=A0=A0=A0=A0=A0=A0=A0=A0 2 
> =A0=A0=A0 MinSpareServers=A0=A0=A0=A0=A0 4 
> =A0=A0=A0 MaxSpareServers=A0=A0=A0=A0 6 
> =A0=A0=A0 MaxClients=A0=A0=A0=A0=A0=A0=A0=A0 100 
> =A0=A0=A0 MaxRequestsPerChild=A0 500 
> </IfModule> 
> 
> # worker MPM [DEFAULT IF USE=3Dthreads] 
> # StartServers: initial number of server processes to start 
> # MaxClients: maximum number of simultaneous client connections 
> # MinSpareThreads: minimum number of worker threads which are kept spa=
re 
> # MaxSpareThreads: maximum number of worker threads which are kept spa=
re 
> # ThreadsPerChild: constant number of worker threads in each server pr=
ocess 
> # MaxRequestsPerChild: maximum number of requests a server process ser=
ves 
> <IfModule worker.c> 
> =A0=A0=A0 StartServers=A0=A0=A0=A0=A0=A0=A0=A0 2 
> =A0=A0=A0 MaxClients=A0=A0=A0=A0=A0=A0=A0=A0 100 
> =A0=A0=A0 MinSpareThreads=A0=A0=A0=A0 25 
> =A0=A0=A0 MaxSpareThreads=A0=A0=A0=A0 75 
> =A0=A0=A0 ThreadsPerChild=A0=A0=A0=A0 25 
> =A0=A0=A0 MaxRequestsPerChild=A0 500 
> </IfModule> 
> 
> 
> 
> top output 
> top - 00:09:00 up 157 days, 23:12,=A0 2 users,=A0 load average: 0.67, =
0.85, 0.80 
> Tasks:=A0 47 total,=A0=A0 1 running,=A0 46 sleeping,=A0=A0 0 stopped,=
=A0=A0 0 zombie 
> Cpu(s):=A0 8.3% us,=A0 1.4% sy,=A0 0.0% ni, 88.0% id,=A0 2.2% wa,=A0 0=
..0% hi,=A0 0.0% si 
> Mem:=A0=A0=A0 917652k total,=A0=A0 858880k used,=A0=A0=A0 58772k free,=
=A0=A0 142820k buffers 
> Swap:=A0 1048568k total,=A0=A0=A0 37844k used,=A0 1010724k free,=A0=A0=
434476k cached 
> 
> apache2 error log 
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f=
asttop): 0x08b4faf0 *** 
> [Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting, =
consider raising the MaxClients setting 
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f=
asttop): 0x086a1170 *** 
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f=
asttop): 0x08b48c08 *** 
> [Wed Dec 16 23:53:07 2009] [warn] child process 1325 still did not exi=
t, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 10569 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 16964 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 1325 still did not exi=
t, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 19910 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 16964 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 19718 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not exi=
t, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 19910 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 10569 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 19718 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:13 2009] [error] child process 1325 still did not ex=
it, sending a SIGKILL 
> [Wed Dec 16 23:53:13 2009] [error] child process 19910 still did not e=
xit, sending a SIGKILL 
> [Wed Dec 16 23:53:13 2009] [error] child process 10569 still did not e=
xit, sending a SIGKILL 
> [Wed Dec 16 23:53:13 2009] [error] child process 16964 still did not e=
xit, sending a SIGKILL 
> 
> 
> Joomla / PHP / Apache 
> System Information 
> PHP built On: Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23 19:2=
0:15 SGT 2006 i686 
> Database Version: 5.0.24a-log 
> PHP Version: 5.1.6-pl4-gentoo 
> Web Server: Apache 
> WebServer to PHP interface: apache2handler 
> Joomla! Version: Joomla! 1.0.15 Stable [ Daytime ] 22 February 2008 23=
:00 UTC 
> User Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1=
5) Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729) 
> 
> Mysql 
> mysql=A0 Ver 14.12 Distrib 5.0.24a, for pc-linux-gnu (i686) using read=
line 5.1 
> 
 
</div></div>Does it work if you disable PHP? Does it still exhibit problems=
if you 
run PHP out of process, eg using fastcgi or scgi (can PHP talk SCGI?). 
For my money, the double free()s are coming from PHP or a PHP module. 
 
Cheers 
 
Tom 
 
------------------------------------------------------------ --------- 
The official User-To-User support forum of the Apache HTTP Server Project.<=
br>
See <URL:<a href=3D"http://httpd.apache.org/userslist.html" target=3D"_b=
lank">http://httpd.apache.org/userslist.html</a>> for more info. 
To unsubscribe, e-mail: <a href=3D"mailto:users-unsubscribe [at] httpd.apache.or=
g" target=3D"_blank">users-unsubscribe [at] httpd.apache.org</a> 
=A0 " =A0 from the digest: <a href=3D"mailto:users-digest-unsubscribe=
[at] httpd.apache.org" target=3D"_blank">users-digest-unsubscribe [at] httpd.apache.=
org</a> 
For additional commands, e-mail: <a href=3D"mailto:users-help [at] httpd.apache.=
org" target=3D"_blank">users-help [at] httpd.apache.org</a> 
 
</blockquote></div> 
</blockquote></div> -- Rodrigo Aliste P. <b=
r>

--000e0cd60b12fa5fe0047ae32a47--

--000e0cd7618088ff0a047ae33ade
Content-Type: text/plain; charset=ISO-8859-1

I just noticed that your error log shows:

[Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting,
consider raising the MaxClients setting

so try adjusting your MaxClient setting and your prefork settings. (
http://httpd.apache.org/docs/2.0/mod/prefork.html)

I'm not an expert on tuning those settings, but I'm sure someone else here
could help you.

2009/12/16 Rodrigo Aliste P. <raliste [at] gmail.com>

> Gary:
>
> The same thing happened to one of my servers. After long research I got to
> the conclusion that the problem was caused by the XCache module. If you have
> it, try to disable it.
>
>
> Rodrigo
>
> 2009/12/16 gary lim <garylim.in [at] gmail.com>
>
> Hi,
>>
>> 1. Unfortunately, our website depend heavily on PHP to operate, so it not
>> possible for us to disable it. We also highly suspect that the crash is
>> related to one of our php module because we are running a lot of Joomla
>> component.
>>
>> 2. unfortunately, we are unable to track which module is causing this
>> issue. Any idea how we can check which script is causing the problem because
>> this has been going on for weeks and we are still unable to determine the
>> source.
>>
>> 3. Kindly advise how we can perform the migration of our existing codebase
>> to FastCGI/SCGI without breaking the current website?
>>
>>
>> rgds
>>
>>
>> On Thu, Dec 17, 2009 at 12:28 AM, Tom Evans <tevans.uk [at] googlemail.com>wrote:
>>
>>> On Wed, Dec 16, 2009 at 4:20 PM, gary lim <garylim.in [at] gmail.com> wrote:
>>> >
>>> > Dear all,
>>> >
>>> > 1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(R) 4
>>> CPU 3.00GHz GNU/Linux
>>> > 2. The below info are my machine config and log output
>>> > 3. Kindly advise how to resolve intermittent apache crash even after
>>> performance tuning
>>> >
>>> > httpd.conf output
>>> > # prefork MPM [DEFAULT IF USE=-threads]
>>> > # StartServers: number of server processes to start
>>> > # MinSpareServers: minimum number of server processes which are kept
>>> spare
>>> > # MaxSpareServers: maximum number of server processes which are kept
>>> spare
>>> > # MaxClients: maximum number of server processes allowed to start
>>> > # MaxRequestsPerChild: maximum number of requests a server process
>>> serves
>>> > <IfModule prefork.c>
>>> > StartServers 2
>>> > MinSpareServers 4
>>> > MaxSpareServers 6
>>> > MaxClients 100
>>> > MaxRequestsPerChild 500
>>> > </IfModule>
>>> >
>>> > # worker MPM [DEFAULT IF USE=threads]
>>> > # StartServers: initial number of server processes to start
>>> > # MaxClients: maximum number of simultaneous client connections
>>> > # MinSpareThreads: minimum number of worker threads which are kept
>>> spare
>>> > # MaxSpareThreads: maximum number of worker threads which are kept
>>> spare
>>> > # ThreadsPerChild: constant number of worker threads in each server
>>> process
>>> > # MaxRequestsPerChild: maximum number of requests a server process
>>> serves
>>> > <IfModule worker.c>
>>> > StartServers 2
>>> > MaxClients 100
>>> > MinSpareThreads 25
>>> > MaxSpareThreads 75
>>> > ThreadsPerChild 25
>>> > MaxRequestsPerChild 500
>>> > </IfModule>
>>> >
>>> >
>>> >
>>> > top output
>>> > top - 00:09:00 up 157 days, 23:12, 2 users, load average: 0.67, 0.85,
>>> 0.80
>>> > Tasks: 47 total, 1 running, 46 sleeping, 0 stopped, 0 zombie
>>> > Cpu(s): 8.3% us, 1.4% sy, 0.0% ni, 88.0% id, 2.2% wa, 0.0% hi,
>>> 0.0% si
>>> > Mem: 917652k total, 858880k used, 58772k free, 142820k
>>> buffers
>>> > Swap: 1048568k total, 37844k used, 1010724k free, 434476k cached
>>> >
>>> > apache2 error log
>>> > *** glibc detected *** /usr/sbin/apache2: double free or corruption
>>> (fasttop): 0x08b4faf0 ***
>>> > [Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting,
>>> consider raising the MaxClients setting
>>> > *** glibc detected *** /usr/sbin/apache2: double free or corruption
>>> (fasttop): 0x086a1170 ***
>>> > *** glibc detected *** /usr/sbin/apache2: double free or corruption
>>> (fasttop): 0x08b48c08 ***
>>> > [Wed Dec 16 23:53:07 2009] [warn] child process 1325 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:07 2009] [warn] child process 10569 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:07 2009] [warn] child process 16964 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:09 2009] [warn] child process 1325 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:09 2009] [warn] child process 19910 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:09 2009] [warn] child process 16964 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:09 2009] [warn] child process 19718 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:11 2009] [warn] child process 19910 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:11 2009] [warn] child process 10569 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:11 2009] [warn] child process 19718 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:13 2009] [error] child process 1325 still did not
>>> exit, sending a SIGKILL
>>> > [Wed Dec 16 23:53:13 2009] [error] child process 19910 still did not
>>> exit, sending a SIGKILL
>>> > [Wed Dec 16 23:53:13 2009] [error] child process 10569 still did not
>>> exit, sending a SIGKILL
>>> > [Wed Dec 16 23:53:13 2009] [error] child process 16964 still did not
>>> exit, sending a SIGKILL
>>> >
>>> >
>>> > Joomla / PHP / Apache
>>> > System Information
>>> > PHP built On: Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23
>>> 19:20:15 SGT 2006 i686
>>> > Database Version: 5.0.24a-log
>>> > PHP Version: 5.1.6-pl4-gentoo
>>> > Web Server: Apache
>>> > WebServer to PHP interface: apache2handler
>>> > Joomla! Version: Joomla! 1.0.15 Stable [ Daytime ] 22 February 2008
>>> 23:00 UTC
>>> > User Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
>>> rv:1.9.0.15) Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729)
>>> >
>>> > Mysql
>>> > mysql Ver 14.12 Distrib 5.0.24a, for pc-linux-gnu (i686) using
>>> readline 5.1
>>> >
>>>
>>> Does it work if you disable PHP? Does it still exhibit problems if you
>>> run PHP out of process, eg using fastcgi or scgi (can PHP talk SCGI?).
>>> For my money, the double free()s are coming from PHP or a PHP module.
>>>
>>> Cheers
>>>
>>> Tom
>>>
>>> ------------------------------------------------------------ ---------
>>> The official User-To-User support forum of the Apache HTTP Server
>>> Project.
>>> See <URL:http://httpd.apache.org/userslist.html> for more info.
>>> To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
>>> " from the digest: users-digest-unsubscribe [at] httpd.apache.org
>>> For additional commands, e-mail: users-help [at] httpd.apache.org
>>>
>>>
>>
>
>
> --
> Rodrigo Aliste P.
>
>

--
Rodrigo Aliste P.

--000e0cd7618088ff0a047ae33ade
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I just noticed that your error log shows: [Tue Dec 15 13:04:22 2009=
] [error] server reached MaxClients setting, consider raising the MaxClient=
s setting so try adjusting your MaxClient setting and your prefork s=
ettings. (<a href=3D"http://httpd.apache.org/docs/2.0/mod/prefork.html">htt=
p://httpd.apache.org/docs/2.0/mod/prefork.html</a>) 

 I'm not an expert on tuning those settings, but I'm sure someon=
e else here could help you. <div class=3D"gmail_quote">2009/12/1=
6 Rodrigo Aliste P. <<a href=3D"mailto:raliste [at] gmail.c=
om">raliste [at] gmail.com</a>> 

<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Gary: The =
same thing happened to one of my servers. After long research I got to the =
conclusion that the problem was caused by the XCache module. If you have it=
, try to disable it. 

 Rodrigo <div class=3D"gmail_quote">
2009/12/16 gary lim <<a href=3D"http://garylim.in" tar=
get=3D"_blank">garylim.in</a> [at] <a href=3D"http://gmail.com" target=3D"_blank=
">gmail.com</a>><div><div></div><div class=3D"h5"> <blockquote=
class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, 204, 204); =
margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">

Hi, 1. Unfortunately, our website depend heavily on PHP to operate, =
so it not possible for us to disable it. We also highly suspect that the cr=
ash is related to one of our php module because we are running a lot of Joo=
mla component. 

 2. unfortunately, we are unable to track which module is causing this i=
ssue. Any idea how we can check which script is causing the problem because=
this has been going on for weeks and we are still unable to determine the =
source. 

 3. Kindly advise how we can perform the migration of our existing codeb=
ase to FastCGI/SCGI without breaking the current website? rgds<b=
r> <div class=3D"gmail_quote">On Thu, Dec 17, 2009 at 12:28 AM, Tom =
Evans <<a href=3D"http://tevans.uk" target=3D"_blank">=
tevans.uk</a> [at] <a href=3D"http://googlemail.com" target=3D"_blank">googlemai=
l.com</a>> wrote: 

<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><=
div>On Wed, Dec 16, 2009 at 4:20 PM, gary lim <<a href=3D"http://garylim=
..in" target=3D"_blank">garylim.in</a> [at] <a href=3D"http://gmail.com" target=
=3D"_blank">gmail.com</a>> wrote: 

> 
> Dear all, 
> 
> 1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(=
R) 4 CPU 3.00GHz GNU/Linux 
> 2. The below info are my machine config and log output 
> 3. Kindly advise how to resolve intermittent apache crash even after p=
erformance tuning 
> 
> httpd.conf output 
> # prefork MPM [DEFAULT IF USE=3D-threads] 
> # StartServers: number of server processes to start 
> # MinSpareServers: minimum number of server processes which are kept s=
pare 
> # MaxSpareServers: maximum number of server processes which are kept s=
pare 
> # MaxClients: maximum number of server processes allowed to start 
> # MaxRequestsPerChild: maximum number of requests a server process ser=
ves 
> <IfModule prefork.c> 
> =A0=A0=A0 StartServers=A0=A0=A0=A0=A0=A0=A0=A0 2 
> =A0=A0=A0 MinSpareServers=A0=A0=A0=A0=A0 4 
> =A0=A0=A0 MaxSpareServers=A0=A0=A0=A0 6 
> =A0=A0=A0 MaxClients=A0=A0=A0=A0=A0=A0=A0=A0 100 
> =A0=A0=A0 MaxRequestsPerChild=A0 500 
> </IfModule> 
> 
> # worker MPM [DEFAULT IF USE=3Dthreads] 
> # StartServers: initial number of server processes to start 
> # MaxClients: maximum number of simultaneous client connections 
> # MinSpareThreads: minimum number of worker threads which are kept spa=
re 
> # MaxSpareThreads: maximum number of worker threads which are kept spa=
re 
> # ThreadsPerChild: constant number of worker threads in each server pr=
ocess 
> # MaxRequestsPerChild: maximum number of requests a server process ser=
ves 
> <IfModule worker.c> 
> =A0=A0=A0 StartServers=A0=A0=A0=A0=A0=A0=A0=A0 2 
> =A0=A0=A0 MaxClients=A0=A0=A0=A0=A0=A0=A0=A0 100 
> =A0=A0=A0 MinSpareThreads=A0=A0=A0=A0 25 
> =A0=A0=A0 MaxSpareThreads=A0=A0=A0=A0 75 
> =A0=A0=A0 ThreadsPerChild=A0=A0=A0=A0 25 
> =A0=A0=A0 MaxRequestsPerChild=A0 500 
> </IfModule> 
> 
> 
> 
> top output 
> top - 00:09:00 up 157 days, 23:12,=A0 2 users,=A0 load average: 0.67, =
0.85, 0.80 
> Tasks:=A0 47 total,=A0=A0 1 running,=A0 46 sleeping,=A0=A0 0 stopped,=
=A0=A0 0 zombie 
> Cpu(s):=A0 8.3% us,=A0 1.4% sy,=A0 0.0% ni, 88.0% id,=A0 2.2% wa,=A0 0=
..0% hi,=A0 0.0% si 
> Mem:=A0=A0=A0 917652k total,=A0=A0 858880k used,=A0=A0=A0 58772k free,=
=A0=A0 142820k buffers 
> Swap:=A0 1048568k total,=A0=A0=A0 37844k used,=A0 1010724k free,=A0=A0=
434476k cached 
> 
> apache2 error log 
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f=
asttop): 0x08b4faf0 *** 
> [Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting, =
consider raising the MaxClients setting 
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f=
asttop): 0x086a1170 *** 
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f=
asttop): 0x08b48c08 *** 
> [Wed Dec 16 23:53:07 2009] [warn] child process 1325 still did not exi=
t, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 10569 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 16964 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 1325 still did not exi=
t, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 19910 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 16964 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 19718 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not exi=
t, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 19910 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 10569 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 19718 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:13 2009] [error] child process 1325 still did not ex=
it, sending a SIGKILL 
> [Wed Dec 16 23:53:13 2009] [error] child process 19910 still did not e=
xit, sending a SIGKILL 
> [Wed Dec 16 23:53:13 2009] [error] child process 10569 still did not e=
xit, sending a SIGKILL 
> [Wed Dec 16 23:53:13 2009] [error] child process 16964 still did not e=
xit, sending a SIGKILL 
> 
> 
> Joomla / PHP / Apache 
> System Information 
> PHP built On: Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23 19:2=
0:15 SGT 2006 i686 
> Database Version: 5.0.24a-log 
> PHP Version: 5.1.6-pl4-gentoo 
> Web Server: Apache 
> WebServer to PHP interface: apache2handler 
> Joomla! Version: Joomla! 1.0.15 Stable [ Daytime ] 22 February 2008 23=
:00 UTC 
> User Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1=
5) Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729) 
> 
> Mysql 
> mysql=A0 Ver 14.12 Distrib 5.0.24a, for pc-linux-gnu (i686) using read=
line 5.1 
> 
 
</div></div>Does it work if you disable PHP? Does it still exhibit problems=
if you 
run PHP out of process, eg using fastcgi or scgi (can PHP talk SCGI?). 
For my money, the double free()s are coming from PHP or a PHP module. 
 
Cheers 
 
Tom 
 
------------------------------------------------------------ --------- 
The official User-To-User support forum of the Apache HTTP Server Project.<=
br>
See <URL:<a href=3D"http://httpd.apache.org/userslist.html" target=3D"_b=
lank">http://httpd.apache.org/userslist.html</a>> for more info. 
To unsubscribe, e-mail: <a href=3D"mailto:users-unsubscribe [at] httpd.apache.or=
g" target=3D"_blank">users-unsubscribe [at] httpd.apache.org</a> 
=A0 " =A0 from the digest: <a href=3D"mailto:users-digest-unsubscribe=
[at] httpd.apache.org" target=3D"_blank">users-digest-unsubscribe [at] httpd.apache.=
org</a> 
For additional commands, e-mail: <a href=3D"mailto:users-help [at] httpd.apache.=
org" target=3D"_blank">users-help [at] httpd.apache.org</a> 
 
</blockquote></div> 
</blockquote></div></div></div> -- Rodrigo Aliste P. 
</blockquote></div> -- Rodrigo Aliste P=
.. 

--000e0cd7618088ff0a047ae33ade--

-->From: gary lim
-->Sent: 16 December, 2009 16:20
-->To: users [at] httpd.apache.org
-->Subject: [users [at] httpd] Apache intermittent crash - HELP!!!
-->
-->
-->Dear all,
-->
-->1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(R) 4
CPU 3.00GHz GNU/Linux
-->2. The below info are my machine config and log output
-->3. Kindly advise how to resolve intermittent apache crash even after
performance tuning
-->
-->apache2 error log
-->*** glibc detected *** /usr/sbin/apache2: double free or corruption
(fasttop): 0x08b4faf0 ***
-->[Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting,
consider raising the MaxClients setting
-->*** glibc detected *** /usr/sbin/apache2: double free or corruption
(fasttop): 0x086a1170 ***
-->*** glibc detected *** /usr/sbin/apache2: double free or corruption
(fasttop): 0x08b48c08 ***

Gary,

What version of glibc do you have? The glibc detected could be causing this
since its producing a dump code. I would revert back to a default setup
with your httpd.conf file, and just use the basics you need for your site to
run. If you get no crashes then its an issue with your performance tuning.

Thanks,
Daniel

------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
" from the digest: users-digest-unsubscribe [at] httpd.apache.org
For additional commands, e-mail: users-help [at] httpd.apache.org

--000e0cd32fb626f051047ae5939d
Content-Type: text/plain; charset=ISO-8859-1

Hi daniel,

1. the intermittent crash happens even before we did any performance tuning
of the apache.

2. the reason why we did the tuning is because we want to limit the apache
memory usage which previously was due to apache using up all the swap mem
causing the crash.

rgds

On Thu, Dec 17, 2009 at 12:42 PM, Daniel Reinhardt
<cryptodan [at] cryptodan.net>wrote:

> -->From: gary lim
> -->Sent: 16 December, 2009 16:20
> -->To: users [at] httpd.apache.org
> -->Subject: [users [at] httpd] Apache intermittent crash - HELP!!!
> -->
> -->
> -->Dear all,
>
> -->
> -->1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(R) 4
> CPU 3.00GHz GNU/Linux
> -->2. The below info are my machine config and log output
> -->3. Kindly advise how to resolve intermittent apache crash even after
> performance tuning
> -->
> -->apache2 error log
>
> -->*** glibc detected *** /usr/sbin/apache2: double free or corruption
> (fasttop): 0x08b4faf0 ***
> -->[Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting,
> consider raising the MaxClients setting
> -->*** glibc detected *** /usr/sbin/apache2: double free or corruption
> (fasttop): 0x086a1170 ***
> -->*** glibc detected *** /usr/sbin/apache2: double free or corruption
> (fasttop): 0x08b48c08 ***
>
>
>
> Gary,
>
> What version of glibc do you have? The glibc detected could be causing
> this since its producing a dump code. I would revert back to a default
> setup with your httpd.conf file, and just use the basics you need for your
> site to run. If you get no crashes then its an issue with your performance
> tuning.
>
> Thanks,
> Daniel
>
> ------------------------------------------------------------ ---------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
> " from the digest: users-digest-unsubscribe [at] httpd.apache.org
> For additional commands, e-mail: users-help [at] httpd.apache.org
>
>

--000e0cd32fb626f051047ae5939d
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi daniel, 1. the intermittent crash happens even before we did any =
performance tuning of the apache. 2. the reason why we did the tunin=
g is because we want to limit the apache memory usage which previously was =
due to apache using up all the swap mem causing the crash. 
 rgds <div class=3D"gmail_quote">On Thu, Dec 17, 2009 at 12:4=
2 PM, Daniel Reinhardt <<a href=3D"mailto:cryptodan [at] cr=
yptodan.net">cryptodan [at] cryptodan.net</a>> wrote: <blockquote c=
lass=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, 204, 204); ma=
rgin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
-->From: gary lim 
-->Sent: 16 December, 2009 16:20 
-->To: <a href=3D"mailto:users [at] httpd.apache.org" target=3D"_blank">users=
[at] httpd.apache.org</a> 
-->Subject: [users [at] httpd] Apache intermittent crash - HELP!!! 
--> 
--> 
-->Dear all,<div class=3D"im"> 
--> 
-->1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium=
(R) 4 CPU 3.00GHz GNU/Linux 
-->2. The below info are my machine config and log output 
-->3. Kindly advise how to resolve intermittent apache crash even after =
performance tuning </div>
--> 
-->apache2 error log<div class=3D"im"> 
-->*** glibc detected *** /usr/sbin/apache2: double free or corruption (=
fasttop): 0x08b4faf0 *** 
-->[Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting,=
consider raising the MaxClients setting 
-->*** glibc detected *** /usr/sbin/apache2: double free or corruption (=
fasttop): 0x086a1170 *** 
-->*** glibc detected *** /usr/sbin/apache2: double free or corruption (=
fasttop): 0x08b48c08 *** 
 
 
 </div>
Gary, 
 
What version of glibc do you have? =A0The glibc detected could be causing t=
his since its producing a dump code. =A0I would revert back to a default se=
tup with your httpd.conf file, and just use the basics you need for your si=
te to run. =A0If you get no crashes then its an issue with your performance=
tuning. 

 
Thanks, 
Daniel <div><div></div><div class=3D"h5">
 
------------------------------------------------------------ --------- 
The official User-To-User support forum of the Apache HTTP Server Project.<=
br>
See <URL:<a href=3D"http://httpd.apache.org/userslist.html" target=3D"_b=
lank">http://httpd.apache.org/userslist.html</a>> for more info. 
To unsubscribe, e-mail: <a href=3D"mailto:users-unsubscribe [at] httpd.apache.or=
g" target=3D"_blank">users-unsubscribe [at] httpd.apache.org</a> 
=A0" =A0 from the digest: <a href=3D"mailto:users-digest-unsubscribe [at] =
httpd.apache.org" target=3D"_blank">users-digest-unsubscribe [at] httpd.apache.o=
rg</a> 
For additional commands, e-mail: <a href=3D"mailto:users-help [at] httpd.apache.=
org" target=3D"_blank">users-help [at] httpd.apache.org</a> 
 
</div></div></blockquote></div> 

--000e0cd32fb626f051047ae5939d--

------=_NextPart_000_0015_01CA7F24.6BFE2B50
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

From: gary lim
Sent: 17 December, 2009 5:05
To: users [at] httpd.apache.org
Subject: Re: [users [at] httpd] Apache intermittent crash - HELP!!!

Hi daniel,

1. the intermittent crash happens even before we did any performance =
tuning of the apache.

2. the reason why we did the tuning is because we want to limit the =
apache memory usage which previously was due to apache using up all the =
swap mem causing the crash.

rgds

On Thu, Dec 17, 2009 at 12:42 PM, Daniel Reinhardt =
<cryptodan [at] cryptodan.net> wrote:

-->From: gary lim
-->Sent: 16 December, 2009 16:20
-->To: users [at] httpd.apache.org
-->Subject: [users [at] httpd] Apache intermittent crash - HELP!!!
-->
-->
-->Dear all,

-->
-->1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) =
Pentium(R) 4 CPU 3.00GHz GNU/Linux
-->2. The below info are my machine config and log output
-->3. Kindly advise how to resolve intermittent apache crash even =
after performance tuning

-->
-->apache2 error log

-->*** glibc detected *** /usr/sbin/apache2: double free or corruption =
(fasttop): 0x08b4faf0 ***
-->[Tue Dec 15 13:04:22 2009] [error] server reached MaxClients =
setting, consider raising the MaxClients setting
-->*** glibc detected *** /usr/sbin/apache2: double free or corruption =
(fasttop): 0x086a1170 ***
-->*** glibc detected *** /usr/sbin/apache2: double free or corruption =
(fasttop): 0x08b48c08 ***

Gary,

What version of glibc do you have? The glibc detected could be =
causing this since its producing a dump code. I would revert back to a =
default setup with your httpd.conf file, and just use the basics you =
need for your site to run. If you get no crashes then its an issue with =
your performance tuning.

Thanks,
Daniel

------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server =
Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
" from the digest: users-digest-unsubscribe [at] httpd.apache.org
For additional commands, e-mail: users-help [at] httpd.apache.org

Gary,

If your application is using all of the SWAP memory maybe its time to =
rebuild the system with more swap space or upgrade the RAM. Can you =
post the server specs for us? I have never seen an application take =
full use of all the memory and all the allocated swap space. You have =
hardware issues that need to be dealt with before attempting to solve =
the software ones.

When you list your system specs, can you also tell us if your Kernel has =
support for higher memory system configs?

Thanks,
Daniel
------=_NextPart_000_0015_01CA7F24.6BFE2B50
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content=3Dtext/html;charset=3Diso-8859-1 =
http-equiv=3DContent-Type>
<META name=3DGENERATOR content=3D"MSHTML 8.00.6001.18865"></HEAD>
<BODY style=3D"PADDING-LEFT: 10px; PADDING-RIGHT: 10px; PADDING-TOP: =
15px"
id=3DMailContainerBody leftMargin=3D0 topMargin=3D0 =
CanvasTabStop=3D"true"
name=3D"Compose message area">
<DIV style=3D"FONT: 10pt Tahoma">
<DIV> </DIV>
<DIV style=3D"BACKGROUND: #f5f5f5">
<DIV style=3D"font-color: black">From: <A =
title=3Dgarylkc [at] gmail.com
href=3D"mailto:garylkc [at] gmail.com">gary lim</A> </DIV>
<DIV>Sent: 17 December, 2009 5:05</DIV>
<DIV>To: <A title=3Dusers [at] httpd.apache.org
href=3D"mailto:users [at] httpd.apache.org">users [at] httpd.apache.or g</A> </DIV>
<DIV>Subject: Re: [users [at] httpd] Apache intermittent crash -
HELP!!!</DIV></DIV></DIV>
<DIV> </DIV>Hi daniel, 1. the intermittent crash happens even =
before
we did any performance tuning of the apache. 2. the reason why we =
did the
tuning is because we want to limit the apache memory usage which =
previously was
due to apache using up all the swap mem causing the
crash. rgds 
<DIV class=3Dgmail_quote>On Thu, Dec 17, 2009 at 12:42 PM, Daniel =
Reinhardt <<A
href=3D"mailto:cryptodan [at] cryptodan.net">cryptodan [at] cryptodan. net</A>></=
SPAN>
wrote: 
<BLOCKQUOTE
style=3D"BORDER-LEFT: rgb(204,204,204) 1px solid; MARGIN: 0pt 0pt 0pt =
0.8ex; PADDING-LEFT: 1ex"
class=3Dgmail_quote>-->From: gary lim -->Sent: 16 December, =
2009
16:20 -->To: <A href=3D"mailto:users [at] httpd.apache.org"
target=3D_blank>users [at] httpd.apache.org</A> -->Subject: =
[users [at] httpd]
Apache intermittent crash - HELP!!! --> --> -->Dear =
all,
<DIV class=3Dim> --> -->1. I'm using Linux 2.6.16.29 and =
running on
i686 Intel(R) Pentium(R) 4 CPU 3.00GHz GNU/Linux -->2. The below =
info
are my machine config and log output -->3. Kindly advise how to =
resolve
intermittent apache crash even after performance
tuning </DIV>--> -->apache2 error log
<DIV class=3Dim> -->*** glibc detected *** /usr/sbin/apache2: =
double free
or corruption (fasttop): 0x08b4faf0 *** -->[Tue Dec 15 13:04:22 =
2009]
[error] server reached MaxClients setting, consider raising the =
MaxClients
setting -->*** glibc detected *** /usr/sbin/apache2: double free =
or
corruption (fasttop): 0x086a1170 *** -->*** glibc detected ***
/usr/sbin/apache2: double free or corruption (fasttop): 0x08b48c08
*** </DIV>Gary, What version of glibc do you =
have?
The glibc detected could be causing this since its producing a =
dump
code. I would revert back to a default setup with your =
httpd.conf file,
and just use the basics you need for your site to run. If you =
get no
crashes then its an issue with your performance
tuning. Thanks, Daniel 
<DIV>
<DIV></DIV>
<DIV
=
class=3Dh5> ----------------------------------------------------------=
----------- The
official User-To-User support forum of the Apache HTTP Server =
Project. See
<URL:<A
title=3D"http://httpd.apache.org/userslist.html CTRL + Click to =
follow link"
href=3D"http://httpd.apache.org/userslist.html"
target=3D_blank>http://httpd.apache.org/userslist.html</A>> for =
more
info. To unsubscribe, e-mail: <A
href=3D"mailto:users-unsubscribe [at] httpd.apache.org"
target=3D_blank>users-unsubscribe [at] httpd.apache.org</A> " =
from
the digest: <A =
href=3D"mailto:users-digest-unsubscribe [at] httpd.apache.org"
target=3D_blank>users-digest-unsubscribe [at] httpd.apache.org</A> For =
additional
commands, e-mail: <A href=3D"mailto:users-help [at] httpd.apache.org"
=
target=3D_blank>users-help [at] httpd.apache.org</A> </DIV></DIV></BLOC=
KQUOTE></DIV> 
<DIV>Gary,</DIV>
<DIV> </DIV>
<DIV>If your application is using all of the SWAP =
memory
maybe its time to rebuild the system with more swap space or upgrade the =

RAM. Can you post the server specs for us? I have never seen =
an
application take full use of all the memory and all the allocated swap
space. You have hardware issues that need to be dealt with before
attempting to solve the software ones.</DIV>
<DIV> </DIV>
<DIV>When you list your system specs, can you also =
tell us if
your Kernel has support for higher memory system configs?</DIV>
<DIV> </DIV>
<DIV>Thanks,</DIV>
<DIV>Daniel</DIV></BODY></HTML>

------=_NextPart_000_0015_01CA7F24.6BFE2B50--

--000e0cd3512a52d531047b070dc7
Content-Type: text/plain; charset=ISO-8859-1

Hi Daniel,

After performance tuning of apache, the mem usage is now more managable.
However, the apache still experience intermittent crash which we suspect is
due to certain PHP module running in our joomla website.

*top output
*top - 04:58:52 up 160 days, 4:02, 1 user, load average: 0.08, 0.04, 0.05
Tasks: 69 total, 2 running, 67 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.9% us, 5.0% sy, 0.0% ni, 93.2% id, 0.0% wa, 0.0% hi, 0.0% si
Mem: 917652k total, 908144k used, 9508k free, 32660k buffers
Swap: 1048568k total, 37856k used, 1010712k free, 179480k cached

Model: *Dell PowerEdge 850*
CPU:Intel *Pentium IV 3.0GHz* / 2MB, 800MHz FSB
RAM: *2GB **DDR-2 SDRAM* 533MHz w/ ECC*
*Harddisk: *02 x SATA* (7200RPM) HDD (non-RAID)

*Apache Version
*Server version: Apache/2.0.58
Server built: Dec 12 2006 21:41:17

Server version: Apache/2.0.58
Server built: Dec 12 2006 21:41:17
Server's Module Magic Number: 20020903:12
Server loaded: APR 0.9.12, APR-UTIL 0.9.12
Compiled using: APR 0.9.12, APR-UTIL 0.9.12
Architecture: 32-bit
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/prefork"
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D HTTPD_ROOT="/usr"
-D SUEXEC_BIN="/usr/sbin/suexec2"
-D DEFAULT_PIDLOG="/var/run/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="/var/run/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="/etc/apache2/mime.types"
-D SERVER_CONFIG_FILE="/etc/apache2/httpd.conf"

Compiled in modules:
core.c
prefork.c
http_core.c
mod_so.c

On Thu, Dec 17, 2009 at 10:22 PM, Daniel Reinhardt
<cryptodan [at] cryptodan.net>wrote:

>
> *From:* gary lim <garylkc [at] gmail.com>
> *Sent:* 17 December, 2009 5:05
> *To:* users [at] httpd.apache.org
> *Subject:* Re: [users [at] httpd] Apache intermittent crash - HELP!!!
>
> Hi daniel,
>
> 1. the intermittent crash happens even before we did any performance tuning
> of the apache.
>
> 2. the reason why we did the tuning is because we want to limit the apache
> memory usage which previously was due to apache using up all the swap mem
> causing the crash.
>
> rgds
>
>
> On Thu, Dec 17, 2009 at 12:42 PM, Daniel Reinhardt <
> cryptodan [at] cryptodan.net> wrote:
>
>> -->From: gary lim
>> -->Sent: 16 December, 2009 16:20
>> -->To: users [at] httpd.apache.org
>> -->Subject: [users [at] httpd] Apache intermittent crash - HELP!!!
>> -->
>> -->
>> -->Dear all,
>>
>> -->
>> -->1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(R) 4
>> CPU 3.00GHz GNU/Linux
>> -->2. The below info are my machine config and log output
>> -->3. Kindly advise how to resolve intermittent apache crash even after
>> performance tuning
>> -->
>> -->apache2 error log
>>
>> -->*** glibc detected *** /usr/sbin/apache2: double free or corruption
>> (fasttop): 0x08b4faf0 ***
>> -->[Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting,
>> consider raising the MaxClients setting
>> -->*** glibc detected *** /usr/sbin/apache2: double free or corruption
>> (fasttop): 0x086a1170 ***
>> -->*** glibc detected *** /usr/sbin/apache2: double free or corruption
>> (fasttop): 0x08b48c08 ***
>>
>>
>>
>> Gary,
>>
>> What version of glibc do you have? The glibc detected could be causing
>> this since its producing a dump code. I would revert back to a default
>> setup with your httpd.conf file, and just use the basics you need for your
>> site to run. If you get no crashes then its an issue with your performance
>> tuning.
>>
>> Thanks,
>> Daniel
>>
>> ------------------------------------------------------------ ---------
>> The official User-To-User support forum of the Apache HTTP Server Project.
>> See <URL:http://httpd.apache.org/userslist.html> for more info.
>> To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
>> " from the digest: users-digest-unsubscribe [at] httpd.apache.org
>> For additional commands, e-mail: users-help [at] httpd.apache.org
>>
>>
>
> Gary,
>
> If your application is using all of the SWAP memory maybe its time to
> rebuild the system with more swap space or upgrade the RAM. Can you post
> the server specs for us? I have never seen an application take full use of
> all the memory and all the allocated swap space. You have hardware issues
> that need to be dealt with before attempting to solve the software ones.
>
> When you list your system specs, can you also tell us if your Kernel has
> support for higher memory system configs?
>
> Thanks,
> Daniel
>

--000e0cd3512a52d531047b070dc7
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Daniel, After performance tuning of apache, the mem usage is now =
more managable. However, the apache still experience intermittent crash whi=
ch we suspect is due to certain PHP module running in our joomla website.<b=
r>
 top output top - 04:58:52 up 160 days,=A0 4:02,=A0 1 user,=A0=
load average: 0.08, 0.04, 0.05 Tasks:=A0 69 total,=A0=A0 2 running,=A0 =
67 sleeping,=A0=A0 0 stopped,=A0=A0 0 zombie Cpu(s):=A0 1.9% us,=A0 5.0%=
sy,=A0 0.0% ni, 93.2% id,=A0 0.0% wa,=A0 0.0% hi,=A0 0.0% si 
Mem:=A0=A0=A0 917652k total,=A0=A0 908144k used,=A0=A0=A0=A0 9508k free,=A0=
=A0=A0 32660k buffers Swap:=A0 1048568k total,=A0=A0=A0 37856k used,=A0 =
1010712k free,=A0=A0 179480k cached Model: Dell PowerEdge 850 

CPU:Intel Pentium IV 3.0GHz / 2MB, 800MHz FSB 
RAM: 2GB DDR-2 SDRAM </fo=
nt>533MHz w/ ECC 

Harddisk: 02 x SATA (7200RPM) HDD (non-RAID) =
 Apache Version Server version: Apache/2.0.58 
Server built:=A0=A0 Dec 12 2006 21:41:=
17 Server version: Apache/2.0.58</spa=
n> 
Server built:=A0=A0 Dec 12 2006 21:41:=
17 Server's Module Magic Number: 20020903:12 
Server loaded:=A0 APR 0.9.12, APR-UTIL=
0.9.12 Compiled using: APR 0.9.12, APR-UTIL 0.9.12 
Architecture:=A0=A0 32-bit Server c=
ompiled with.... =A0-D APACHE_MPM_DIR=3D"server/mpm/prefork"</=
span> 
=A0-D APR_HAS_SENDFILE =A0-D APR_HA=
S_MMAP =A0-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) 
=A0-D APR_USE_SYSVSEM_SERIALIZE=
 =A0=
-D APR_USE_PTHREAD_SERIALIZE <span=
style=3D"color: rgb(0, 0, 0);">=A0-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT</sp=
an> 
=A0-D APR_HAS_OTHER_CHILD =A0-D AP_=
HAVE_RELIABLE_PIPED_LOGS =A0-D HTTPD_ROOT=3D"/usr" 
=A0-D SUEXEC_BIN=3D"/usr/sbin/sue=
xec2" =A0-D DEFAULT_PIDLOG=3D"/var/run/httpd.pid"<=
br style=3D"color: rgb(0, 0, 0);">
=A0-D DEFAULT_SCOREBOARD=3D"logs/=
apache_runtime_status" =A0-D DEFAULT_LOCKFILE=3D"/var/run/acce=
pt.lock" 
=A0-D DEFAULT_ERRORLOG=3D"logs/er=
ror_log" =A0-D AP_TYPES_CONFIG_FILE=3D"/etc/apache2/mime.types=
" 
=A0-D SERVER_CONFIG_FILE=3D"/etc/=
apache2/httpd.conf" Compiled in =
modules: 
=A0 core.c =A0 prefork.c =A0 htt=
p_core.c 
=A0 mod_so.c <d=
iv class=3D"gmail_quote">On Thu, Dec 17, 2009 at 10:22 PM, Daniel Reinhardt=
<<a href=3D"mailto:cryptodan [at] cryptodan.net">cryptodan=
[at] cryptodan.net</a>> wrote: 
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">

<div style=3D"padding-left: 10px; padding-right: 10px; padding-top: 15px;" =
name=3D"Compose message area">
<div style=3D"font-family: Tahoma; font-style: normal; font-variant: normal=
; font-weight: normal; font-size: 10pt; line-height: normal; font-size-adju=
st: none; font-stretch: normal;">
<div> </div>
<div style=3D"background: rgb(245, 245, 245) none repeat scroll 0% 0%; -moz=
-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-=
background-inline-policy: -moz-initial;">
<div>From: <a title=3D"garylkc [at] gmail.com" href=3D"mailto:garylkc [at] gma=
il.com" target=3D"_blank">gary lim</a> </div>
<div>Sent: 17 December, 2009 5:05</div><div class=3D"im">
<div>To: <a title=3D"users [at] httpd.apache.org" href=3D"mailto:users [at] ht=
tpd.apache.org" target=3D"_blank">users [at] httpd.apache.org</a> </div>
</div><div>Subject: Re: [users [at] httpd] Apache intermittent crash -
HELP!!!</div></div></div><div><div></div><div class=3D"h5">
<div> </div>Hi daniel, 1. the intermittent crash happens even bef=
ore
we did any performance tuning of the apache. 2. the reason why we di=
d the
tuning is because we want to limit the apache memory usage which previously=
was
due to apache using up all the swap mem causing the
crash. rgds 
<div class=3D"gmail_quote">On Thu, Dec 17, 2009 at 12:42 PM, Daniel Reinhar=
dt <<a href=3D"mailto:cryptodan [at] cryptodan.net" target=
=3D"_blank">cryptodan [at] cryptodan.net</a>>
wrote: 
<blockquote style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt=
0pt 0pt 0.8ex; padding-left: 1ex;" class=3D"gmail_quote">-->From: gary =
lim -->Sent: 16 December, 2009
16:20 -->To: <a href=3D"mailto:users [at] httpd.apache.org" target=3D"_b=
lank">users [at] httpd.apache.org</a> -->Subject: [users [at] httpd]
Apache intermittent crash - HELP!!! --> --> -->Dear all=
,
<div> --> -->1. I'm using Linux 2.6.16.29 and running on=

i686 Intel(R) Pentium(R) 4 CPU 3.00GHz GNU/Linux -->2. The below in=
fo
are my machine config and log output -->3. Kindly advise how to res=
olve
intermittent apache crash even after performance
tuning </div>--> -->apache2 error log
<div> -->*** glibc detected *** /usr/sbin/apache2: double free
or corruption (fasttop): 0x08b4faf0 *** -->[Tue Dec 15 13:04:22 200=
9]
[error] server reached MaxClients setting, consider raising the MaxClient=
s
setting -->*** glibc detected *** /usr/sbin/apache2: double free or=

corruption (fasttop): 0x086a1170 *** -->*** glibc detected ***
/usr/sbin/apache2: double free or corruption (fasttop): 0x08b48c08
*** </div>Gary, What version of glibc do you have?=

=A0The glibc detected could be causing this since its producing a dump
code. =A0I would revert back to a default setup with your httpd.conf file=
,
and just use the basics you need for your site to run. =A0If you get no=

crashes then its an issue with your performance
tuning. Thanks, Daniel 
<div>
<div></div>
<div> ------------------------------------------------------------ ----=
----- The
official User-To-User support forum of the Apache HTTP Server Project.<br=
>See
<URL:<a title=3D"http://httpd.apache.org/userslist.html
CTRL + Click to follow link" href=3D"http://httpd.apache.org/userslist.html=
" target=3D"_blank">http://httpd.apache.org/userslist.html</a>> for more=

info. To unsubscribe, e-mail: <a href=3D"mailto:users-unsubscribe [at] http=
d.apache.org" target=3D"_blank">users-unsubscribe [at] httpd.apache.org</a> =
=A0" =A0 from
the digest: <a href=3D"mailto:users-digest-unsubscribe [at] httpd.apache.org" =
target=3D"_blank">users-digest-unsubscribe [at] httpd.apache.org</a> For addi=
tional
commands, e-mail: <a href=3D"mailto:users-help [at] httpd.apache.org" target=
=3D"_blank">users-help [at] httpd.apache.org</a> </div></div></blockquote=
></div> 
</div></div><div>Gary,</div>
<div>=A0</div>
<div>If your application is using all of the SWAP me=
mory
maybe its time to rebuild the system with more swap space or upgrade the
RAM.=A0 Can you post the server specs for us?=A0 I have never seen an
application take full use of all the memory and all the allocated swap
space.=A0 You have hardware issues that need to be dealt with before
attempting to solve the software ones.</div>
<div>=A0</div>
<div>When you list your system specs, can you also t=
ell us if
your Kernel has support for higher memory system configs?</div>
<div>=A0</div>
<div>Thanks,</div>
<div>Daniel</div></div>
</blockquote></div> 

--000e0cd3512a52d531047b070dc7--

Just a couple things to point out (but nothing specific to fixing your
probem) ...

gary lim wrote:
> Hi Daniel,
>
> After performance tuning of apache, the mem usage is now more
> managable. However, the apache still experience intermittent crash
> which we suspect is due to certain PHP module running in our joomla
> website.

If it really is the PHP module that is crashing it, there's really no
other way to test that theory unless you disable it and see if the
crashes persist.

> *top output
> *top - 04:58:52 up 160 days, 4:02, 1 user, load average: 0.08,
> 0.04, 0.05
> Tasks: 69 total, 2 running, 67 sleeping, 0 stopped, 0 zombie
> Cpu(s): 1.9% us, 5.0% sy, 0.0% ni, 93.2% id, 0.0% wa, 0.0% hi,
> 0.0% si
> Mem: 917652k total, 908144k used, 9508k free, 32660k buffers
> Swap: 1048568k total, 37856k used, 1010712k free, 179480k cached
>
>
> Model: *Dell PowerEdge 850*
> CPU:Intel *Pentium IV 3.0GHz* / 2MB, 800MHz FSB
> RAM: *2GB **DDR-2 SDRAM* 533MHz w/ ECC*
> *Harddisk: *02 x SATA* (7200RPM) HDD (non-RAID)

Not sure if you noticed or not, but all of your memory is not actually
showing up as available to the kernel. Are you running a kernel with
"bigmem"-style support? If not, you're missing out on half of your RAM.

> *Apache Version
> *Server version: Apache/2.0.58
> Server built: Dec 12 2006 21:41:17

There is always the 2.2 branch, which I'd recommend trying out if
possible. Perhaps this is a locked down distro release with only the
older version available (RHEL/CentOS?).

--
Justin Pasher

------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
" from the digest: users-digest-unsubscribe [at] httpd.apache.org
For additional commands, e-mail: users-help [at] httpd.apache.org

--000e0cd5c9a6ea5192047b084ded
Content-Type: text/plain; charset=ISO-8859-1

Hu justin,

1. Because its a production site, we are unable to disable the PHP
modules/script......From the log output below, we are having difficulty to
pinpoint the source of error. I just spent the last couple of hours to run
through the pages and see if the log shows up any systematic
error...unfortunately, wasn't able to replicate the source of problem
causing the glibc issue. Not sure if this only happen during high
loading....

*** glibc detected *** /usr/sbin/apache2: double free or corruption
(fasttop): 0x086a1170 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption
(fasttop): 0x08b48c08 ***
[Wed Dec 16 23:53:07 2009] [warn] child process 1325 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:07 2009] [warn] child process 10569 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:07 2009] [warn] child process 16964 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:09 2009] [warn] child process 1325 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:09 2009] [warn] child process 19910 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:09 2009] [warn] child process 16964 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:09 2009] [warn] child process 19718 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:11 2009] [warn] child process 19910 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:11 2009] [warn] child process 10569 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not exit,
sending a SIGTERM
[Wed Dec 16 23:53:11 2009] [warn] child process 19718 still did not exit,
sending a SIGTERM

2. The mem usage is correct because we are running the web server as a Xen
Virtual Machine.

On Sat, Dec 19, 2009 at 5:20 AM, Justin Pasher
<justinp [at] newmediagateway.com>wrote:

> Just a couple things to point out (but nothing specific to fixing your
> probem) ...
>
>
> gary lim wrote:
>
>> Hi Daniel,
>>
>> After performance tuning of apache, the mem usage is now more managable.
>> However, the apache still experience intermittent crash which we suspect is
>> due to certain PHP module running in our joomla website.
>>
>
> If it really is the PHP module that is crashing it, there's really no other
> way to test that theory unless you disable it and see if the crashes
> persist.
>
> *top output
>>
>> *top - 04:58:52 up 160 days, 4:02, 1 user, load average: 0.08, 0.04,
>> 0.05
>> Tasks: 69 total, 2 running, 67 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 1.9% us, 5.0% sy, 0.0% ni, 93.2% id, 0.0% wa, 0.0% hi, 0.0%
>> si
>> Mem: 917652k total, 908144k used, 9508k free, 32660k buffers
>> Swap: 1048568k total, 37856k used, 1010712k free, 179480k cached
>>
>>
>> Model: *Dell PowerEdge 850*
>> CPU:Intel *Pentium IV 3.0GHz* / 2MB, 800MHz FSB
>> RAM: *2GB **DDR-2 SDRAM* 533MHz w/ ECC*
>> *Harddisk: *02 x SATA* (7200RPM) HDD (non-RAID)
>>
>
> Not sure if you noticed or not, but all of your memory is not actually
> showing up as available to the kernel. Are you running a kernel with
> "bigmem"-style support? If not, you're missing out on half of your RAM.
>
> *Apache Version
>>
>> *Server version: Apache/2.0.58
>> Server built: Dec 12 2006 21:41:17
>>
>
> There is always the 2.2 branch, which I'd recommend trying out if possible.
> Perhaps this is a locked down distro release with only the older version
> available (RHEL/CentOS?).
>
>
> --
> Justin Pasher
>
>
> ------------------------------------------------------------ ---------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
> " from the digest: users-digest-unsubscribe [at] httpd.apache.org
> For additional commands, e-mail: users-help [at] httpd.apache.org
>
>

--000e0cd5c9a6ea5192047b084ded
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hu justin, 1. Because its a production site, we are unable to disabl=
e the PHP modules/script......From the log output below, we are having diff=
iculty to pinpoint the source of error. I just spent the last couple of hou=
rs to run through the pages and see if the log shows up any systematic erro=
r...unfortunately, wasn't able to replicate the source of problem causi=
ng the glibc issue. Not sure if this only happen during high loading....<br=
>
 *** glibc detected *** /usr/sbin/apache2: double free or corruption=
(fasttop): 0x086a1170 *** *** glibc detected *** /usr/sbin/apache2: dou=
ble free or corruption (fasttop): 0x08b48c08 *** [Wed Dec 16 23:53:07 20=
09] [warn] child process 1325 still did not exit, sending a SIGTERM 
[Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not exit, s=
ending a SIGTERM [Wed Dec 16 23:53:07 2009] [warn] child process 10569 s=
till did not exit, sending a SIGTERM [Wed Dec 16 23:53:07 2009] [warn] c=
hild process 16964 still did not exit, sending a SIGTERM 
[Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not exit, s=
ending a SIGTERM [Wed Dec 16 23:53:09 2009] [warn] child process 1325 st=
ill did not exit, sending a SIGTERM [Wed Dec 16 23:53:09 2009] [warn] ch=
ild process 19910 still did not exit, sending a SIGTERM 
[Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not exit, s=
ending a SIGTERM [Wed Dec 16 23:53:09 2009] [warn] child process 16964 s=
till did not exit, sending a SIGTERM [Wed Dec 16 23:53:09 2009] [warn] c=
hild process 19718 still did not exit, sending a SIGTERM 
[Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not exit, se=
nding a SIGTERM [Wed Dec 16 23:53:11 2009] [warn] child process 19910 st=
ill did not exit, sending a SIGTERM [Wed Dec 16 23:53:11 2009] [warn] ch=
ild process 10569 still did not exit, sending a SIGTERM 
[Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not exit, s=
ending a SIGTERM [Wed Dec 16 23:53:11 2009] [warn] child process 19718 s=
till did not exit, sending a SIGTERM 2. The mem usage is correct bec=
ause we are running the web server as a Xen Virtual Machine. 
 <div class=3D"gmail_quote">On Sat, Dec 19, 2009 at 5:20 AM, Justin =
Pasher <<a href=3D"mailto:justinp [at] newmediagateway.com"=
>justinp [at] newmediagateway.com</a>> wrote: <blockquote class=3D"=
gmail_quote" style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0p=
t 0pt 0pt 0.8ex; padding-left: 1ex;">
Just a couple things to point out (but nothing specific to fixing your prob=
em) ...<div class=3D"im"> 
 
gary lim wrote: 
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi Daniel, 
 
After performance tuning of apache, the mem usage is now more managable. Ho=
wever, the apache still experience intermittent crash which we suspect is d=
ue to certain PHP module running in our joomla website. 
</blockquote>
 </div>
If it really is the PHP module that is crashing it, there's really no o=
ther way to test that theory unless you disable it and see if the crashes p=
ersist. 
 
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
*top output<div class=3D"im"> 
*top - 04:58:52 up 160 days, =A04:02, =A01 user, =A0load average: 0.08, 0.0=
4, 0.05 
Tasks: =A069 total, =A0 2 running, =A067 sleeping, =A0 0 stopped, =A0 0 zom=
bie 
Cpu(s): =A01.9% us, =A05.0% sy, =A00.0% ni, 93.2% id, =A00.0% wa, =A00.0% h=
i, =A00.0% si 
Mem: =A0 =A0917652k total, =A0 908144k used, =A0 =A0 9508k free, =A0 =A0326=
60k buffers 
Swap: =A01048568k total, =A0 =A037856k used, =A01010712k free, =A0 179480k =
cached 
 
 
Model: *Dell PowerEdge 850* 
CPU:Intel *Pentium IV 3.0GHz* / 2MB, 800MHz FSB 
RAM: *2GB **DDR-2 SDRAM* 533MHz w/ ECC* 
*Harddisk: *02 x SATA* (7200RPM) HDD (non-RAID) 
</div></blockquote>
 
Not sure if you noticed or not, but all of your memory is not actually show=
ing up as available to the kernel. Are you running a kernel with "bigm=
em"-style support? If not, you're missing out on half of your RAM.=
 

 
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
*Apache Version<div class=3D"im"> 
*Server version: Apache/2.0.58 
Server built: =A0 Dec 12 2006 21:41:17 
</div></blockquote>
 
There is always the 2.2 branch, which I'd recommend trying out if possi=
ble. Perhaps this is a locked down distro release with only the older versi=
on available (RHEL/CentOS?). 
 
 
-- 
Justin Pasher<div><div></div><div class=3D"h5"> 
 
------------------------------------------------------------ --------- 
The official User-To-User support forum of the Apache HTTP Server Project.<=
br>
See <URL:<a href=3D"http://httpd.apache.org/userslist.html" target=3D"_b=
lank">http://httpd.apache.org/userslist.html</a>> for more info. 
To unsubscribe, e-mail: <a href=3D"mailto:users-unsubscribe [at] httpd.apache.or=
g" target=3D"_blank">users-unsubscribe [at] httpd.apache.org</a> 
=A0" =A0 from the digest: <a href=3D"mailto:users-digest-unsubscribe [at] =
httpd.apache.org" target=3D"_blank">users-digest-unsubscribe [at] httpd.apache.o=
rg</a> 
For additional commands, e-mail: <a href=3D"mailto:users-help [at] httpd.apache.=
org" target=3D"_blank">users-help [at] httpd.apache.org</a> 
 
</div></div></blockquote></div> 

--000e0cd5c9a6ea5192047b084ded--

gary lim wrote:
> Hu justin,
>
> 1. Because its a production site, we are unable to disable the PHP
> modules/script......From the log output below, we are having
> difficulty to pinpoint the source of error. I just spent the last
> couple of hours to run through the pages and see if the log shows up
> any systematic error...unfortunately, wasn't able to replicate the
> source of problem causing the glibc issue. Not sure if this only
> happen during high loading....

I knew you would say that, but that doesn't change what I suggested.
It's like the situation where someone has suspected flaky hardware (e.g.
bad RAM), but they can't reboot the server to run memtest86 because it's
production. It won't get you any closer to a solution when you can't
test all possible avenues, especially when it's very difficult to even
replicate the issue consistently.

--
Justin Pasher

------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
" from the digest: users-digest-unsubscribe [at] httpd.apache.org
For additional commands, e-mail: users-help [at] httpd.apache.org

Justin Pasher wrote:
> gary lim wrote:
>> Hu justin,
>>
>> 1. Because its a production site, we are unable to disable the PHP
>> modules/script......From the log output below, we are having
>> difficulty to pinpoint the source of error. I just spent the last
>> couple of hours to run through the pages and see if the log shows up
>> any systematic error...unfortunately, wasn't able to replicate the
>> source of problem causing the glibc issue. Not sure if this only
>> happen during high loading....

One other thought... You said the server is running under Xen. Can you
not just clone the virtual machine instance and play around with the
test one? I'm not sure what sort of online cloning capabilities Xen
supports, so I don't know if it would require a shutdown or not.

--
Justin Pasher

------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
" from the digest: users-digest-unsubscribe [at] httpd.apache.org
For additional commands, e-mail: users-help [at] httpd.apache.org

--000e0cd34858c1f17b047b08cd94
Content-Type: text/plain; charset=ISO-8859-1

Hi justin,

yes, we are working on this avenue which will take some time....in the
meantime, we need to try to resolve the crash issue in our production
system.

On Sat, Dec 19, 2009 at 7:00 AM, Justin Pasher
<justinp [at] newmediagateway.com>wrote:

> Justin Pasher wrote:
>
>> gary lim wrote:
>>
>>> Hu justin,
>>>
>>> 1. Because its a production site, we are unable to disable the PHP
>>> modules/script......From the log output below, we are having difficulty to
>>> pinpoint the source of error. I just spent the last couple of hours to run
>>> through the pages and see if the log shows up any systematic
>>> error...unfortunately, wasn't able to replicate the source of problem
>>> causing the glibc issue. Not sure if this only happen during high
>>> loading....
>>>
>>
> One other thought... You said the server is running under Xen. Can you not
> just clone the virtual machine instance and play around with the test one?
> I'm not sure what sort of online cloning capabilities Xen supports, so I
> don't know if it would require a shutdown or not.
>
> --
> Justin Pasher
>
> ------------------------------------------------------------ ---------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
> " from the digest: users-digest-unsubscribe [at] httpd.apache.org
> For additional commands, e-mail: users-help [at] httpd.apache.org
>
>

--000e0cd34858c1f17b047b08cd94
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi justin, yes, we are working on this avenue which will take some t=
ime....in the meantime, we need to try to resolve the crash issue in our pr=
oduction system. <div class=3D"gmail_quote">On Sat, Dec 19, 2009=
at 7:00 AM, Justin Pasher <<a href=3D"mailto:justinp [at] =
newmediagateway.com">justinp [at] newmediagateway.com</a>> wrote: 
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class=3D"im"=
>Justin Pasher wrote: 
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
gary lim wrote: 
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hu justin, 
 
1. Because its a production site, we are unable to disable the PHP modules/=
script......From the log output below, we are having difficulty to pinpoint=
the source of error. I just spent the last couple of hours to run through =
the pages and see if the log shows up any systematic error...unfortunately,=
wasn't able to replicate the source of problem causing the glibc issue=
.. Not sure if this only happen during high loading.... 

</blockquote></blockquote>
 </div>
One other thought... You said the server is running under Xen. Can you not =
just clone the virtual machine instance and play around with the test one? =
I'm not sure what sort of online cloning capabilities Xen supports, so =
I don't know if it would require a shutdown or not. 

 
-- <div><div></div><div class=3D"h5">
Justin Pasher 
 
------------------------------------------------------------ --------- 
The official User-To-User support forum of the Apache HTTP Server Project.<=
br>
See <URL:<a href=3D"http://httpd.apache.org/userslist.html" target=3D"_b=
lank">http://httpd.apache.org/userslist.html</a>> for more info. 
To unsubscribe, e-mail: <a href=3D"mailto:users-unsubscribe [at] httpd.apache.or=
g" target=3D"_blank">users-unsubscribe [at] httpd.apache.org</a> 
=A0" =A0 from the digest: <a href=3D"mailto:users-digest-unsubscribe [at] =
httpd.apache.org" target=3D"_blank">users-digest-unsubscribe [at] httpd.apache.o=
rg</a> 
For additional commands, e-mail: <a href=3D"mailto:users-help [at] httpd.apache.=
org" target=3D"_blank">users-help [at] httpd.apache.org</a> 
 
</div></div></blockquote></div> 

--000e0cd34858c1f17b047b08cd94--

--000e0cd6742c02d77c047b08e359
Content-Type: text/plain; charset=ISO-8859-1

yes, before we disable any PHP module, we want to be careful to only do it
only if necessary since it under production

On Sat, Dec 19, 2009 at 6:55 AM, Justin Pasher
<justinp [at] newmediagateway.com>wrote:

> gary lim wrote:
>
>> Hu justin,
>>
>> 1. Because its a production site, we are unable to disable the PHP
>> modules/script......From the log output below, we are having difficulty to
>> pinpoint the source of error. I just spent the last couple of hours to run
>> through the pages and see if the log shows up any systematic
>> error...unfortunately, wasn't able to replicate the source of problem
>> causing the glibc issue. Not sure if this only happen during high
>> loading....
>>
>
> I knew you would say that, but that doesn't change what I suggested. It's
> like the situation where someone has suspected flaky hardware (e.g. bad
> RAM), but they can't reboot the server to run memtest86 because it's
> production. It won't get you any closer to a solution when you can't test
> all possible avenues, especially when it's very difficult to even replicate
> the issue consistently.
>
> --
>
> Justin Pasher
>
> ------------------------------------------------------------ ---------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
> " from the digest: users-digest-unsubscribe [at] httpd.apache.org
> For additional commands, e-mail: users-help [at] httpd.apache.org
>
>

--000e0cd6742c02d77c047b08e359
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

yes, before we disable any PHP module, we want to be careful to only do it =
only if necessary since it under production <div class=3D"gmail_quot=
e">On Sat, Dec 19, 2009 at 6:55 AM, Justin Pasher <<a =
href=3D"mailto:justinp [at] newmediagateway.com">justinp [at] newmedia gateway.com</a>=
> wrote: 
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class=3D"im"=
>gary lim wrote: 
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hu justin, 
 
1. Because its a production site, we are unable to disable the PHP modules/=
script......From the log output below, we are having difficulty to pinpoint=
the source of error. I just spent the last couple of hours to run through =
the pages and see if the log shows up any systematic error...unfortunately,=
wasn't able to replicate the source of problem causing the glibc issue=
.. Not sure if this only happen during high loading.... 

</blockquote>
 </div>
I knew you would say that, but that doesn't change what I suggested. It=
's like the situation where someone has suspected flaky hardware (e.g. =
bad RAM), but they can't reboot the server to run memtest86 because it&=
#39;s production. It won't get you any closer to a solution when you ca=
n't test all possible avenues, especially when it's very difficult =
to even replicate the issue consistently. 

 
--<div><div></div><div class=3D"h5"> 
Justin Pasher 
 
------------------------------------------------------------ --------- 
The official User-To-User support forum of the Apache HTTP Server Project.<=
br>
See <URL:<a href=3D"http://httpd.apache.org/userslist.html" target=3D"_b=
lank">http://httpd.apache.org/userslist.html</a>> for more info. 
To unsubscribe, e-mail: <a href=3D"mailto:users-unsubscribe [at] httpd.apache.or=
g" target=3D"_blank">users-unsubscribe [at] httpd.apache.org</a> 
=A0" =A0 from the digest: <a href=3D"mailto:users-digest-unsubscribe [at] =
httpd.apache.org" target=3D"_blank">users-digest-unsubscribe [at] httpd.apache.o=
rg</a> 
For additional commands, e-mail: <a href=3D"mailto:users-help [at] httpd.apache.=
org" target=3D"_blank">users-help [at] httpd.apache.org</a> 
 
</div></div></blockquote></div> 

--000e0cd6742c02d77c047b08e359--

--000e0cd3512af22670047b092b3f
Content-Type: text/plain; charset=ISO-8859-1

hi Rodrigo,

XCache doesn't seem to be installed in our setup....

*phpinfo()
*PHP Version => 5.1.6-pl4-gentoo

System => Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23 19:20:15 SGT
2006 i686
Build Date => Dec 11 2006 15:41:50
Configure Command => './configure' '--prefix=/usr/lib/php5'
'--host=i686-pc-linux-gnu' '--mandir=/usr/lib/php5/man' '
--infodir=/usr/lib/php5/info' '--sysconfdir=/etc'
'--cache-file=./config.cache' '--enable-cli' '--disable-cgi' '--with
-config-file-path=/etc/php/cli-php5'
'--with-config-file-scan-dir=/etc/php/cli-php5/ext-active' '--without-pear'
'--di
sable-bcmath' '--with-bz2' '--disable-calendar' '--disable-ctype'
'--without-curl' '--without-curlwrappers' '--disable
-dbase' '--disable-exif' '--without-fbsql' '--without-fdftk'
'--disable-filepro' '--disable-ftp' '--with-gettext' '--w
ithout-gmp' '--disable-hash' '--without-hwapi' '--without-iconv'
'--without-informix' '--without-kerberos' '--disable-
mbstring' '--with-mcrypt' '--disable-memory-limit' '--without-mhash'
'--without-ming' '--without-msql' '--without-mssq
l' '--with-ncurses' '--with-openssl' '--with-openssl-dir=/usr'
'--disable-pcntl' '--disable-pdo' '--without-pgsql' '--
disable-posix' '--with-pspell' '--without-recode' '--disable-simplexml'
'--disable-shmop' '--without-snmp' '--disable-
soap' '--disable-sockets' '--without-sybase' '--without-sybase-ct'
'--disable-sysvmsg' '--disable-sysvsem' '--disable-
sysvshm' '--without-tidy' '--disable-tokenizer' '--disable-wddx'
'--disable-xmlreader' '--disable-xmlwriter' '--withou
t-xmlrpc' '--without-xsl' '--with-zlib' '--disable-debug' '--enable-dba'
'--without-cdb' '--with-db4' '--without-flatf
ile' '--with-gdbm' '--without-inifile' '--without-qdbm'
'--with-freetype-dir=/usr' '--with-t1lib=/usr' '--disable-gd-j
is-conv' '--enable-gd-native-ttf' '--with-jpeg-dir=/usr'
'--with-png-dir=/usr' '--without-xpm-dir' '--with-gd' '--with
-mysql=/usr/lib/mysql' '--with-mysql-sock=/var/run/mysqld/mysqld.sock'
'--without-mysqli' '--with-readline' '--without
-libedit' '--without-mm' '--without-sqlite'
Server API => Command Line Interface
Virtual Directory Support => disabled
Configuration File (php.ini) Path => /etc/php/cli-php5/php.ini
Scan this dir for additional .ini files => /etc/php/cli-php5/ext-active
PHP API => 20041225
PHP Extension => 20050922
Zend Extension => 220051025
Debug Build => no
Thread Safety => disabled
Zend Memory Manager => enabled
IPv6 Support => enabled
Registered PHP Streams => php, file, http, ftp, compress.bzip2,
compress.zlib, https, ftps
Registered Stream Socket Transports => tcp, udp, unix, udg, ssl, sslv3,
sslv2, tls
Registered Stream Filters => string.rot13, string.toupper, string.tolower,
string.strip_tags, convert.*, consumed, bzi
p2.*, zlib.*

This program makes use of the Zend Scripting Language Engine:
Zend Engine v2.1.0, Copyright (c) 1998-2006 Zend Technologies

Configuration

PHP Core

Directive => Local Value => Master Value
allow_call_time_pass_reference => On => On
allow_url_fopen => Off => Off
always_populate_raw_post_data => Off => Off
arg_separator.input => & => &
arg_separator.output => & => &
asp_tags => Off => Off
auto_append_file => no value => no value
auto_globals_jit => On => On
auto_prepend_file => no value => no value
browscap => no value => no value
default_charset => no value => no value
default_mimetype => text/html => text/html
define_syslog_variables => Off => Off
disable_classes => no value => no value
disable_functions => no value => no value
display_errors => On => On
display_startup_errors => Off => Off
doc_root => no value => no value
docref_ext => no value => no value
docref_root => no value => no value
enable_dl => On => On
error_append_string => no value => no value
error_log => no value => no value
error_prepend_string => no value => no value
error_reporting => 2039 => 2039
expose_php => On => On
extension_dir => /usr/lib/php5/lib/php/extensions/no-debug-non-zts-20050922
=> /usr/lib/php5/lib/php/extensions/no-deb
ug-non-zts-20050922
file_uploads => On => On
highlight.bg => #FFFFFF => #FFFFFF
highlight.comment => #FF8000 => #FF8000
highlight.default => #0000BB => #0000BB
highlight.html => #000000 => #000000
highlight.keyword => #007700 => #007700
highlight.string => #DD0000 => #DD0000
html_errors => Off => On
ignore_repeated_errors => Off => Off
ignore_repeated_source => Off => Off
ignore_user_abort => Off => Off
implicit_flush => On => Off
include_path => .:/usr/share/php5:/usr/share/php =>
..:/usr/share/php5:/usr/share/php
log_errors => Off => Off
log_errors_max_len => 1024 => 1024
magic_quotes_gpc => On => On
magic_quotes_runtime => Off => Off
magic_quotes_sybase => Off => Off
mail.force_extra_parameters => no value => no value
max_execution_time => 0 => 30
max_input_time => -1 => 60
open_basedir => no value => no value
output_buffering => 0 => no value
output_handler => no value => no value
post_max_size => 8M => 8M
precision => 12 => 12
realpath_cache_size => 16K => 16K
realpath_cache_ttl => 120 => 120
register_argc_argv => On => On
register_globals => Off => Off
register_long_arrays => On => On
report_memleaks => On => On
report_zend_debug => Off => Off
safe_mode => Off => Off
safe_mode_exec_dir => no value => no value
safe_mode_gid => Off => Off
safe_mode_include_dir => no value => no value
sendmail_from => no value => no value
sendmail_path => /usr/sbin/sendmail -t -i => /usr/sbin/sendmail -t -i
serialize_precision => 100 => 100
short_open_tag => On => On
SMTP => localhost => localhost
smtp_port => 25 => 25
sql.safe_mode => Off => Off
track_errors => Off => Off
unserialize_callback_func => no value => no value
upload_max_filesize => 2M => 2M
upload_tmp_dir => no value => no value
user_dir => no value => no value
variables_order => EGPCS => EGPCS
xmlrpc_error_number => 0 => 0
xmlrpc_errors => Off => Off
y2k_compliance => On => On
zend.ze1_compatibility_mode => Off => Off

bz2

BZip2 Support => Enabled
Stream Wrapper support => compress.bz2://
Stream Filter support => bzip2.decompress, bzip2.compress
BZip2 Version => 1.0.3, 15-Feb-2005

date

date/time support => enabled
Timezone Database Version => 2006.1
Timezone Database => internal
Default timezone => Asia/Singapore

Directive => Local Value => Master Value
date.default_latitude => 31.7667 => 31.7667
date.default_longitude => 35.2333 => 35.2333
date.sunrise_zenith => 90.583333 => 90.583333
date.sunset_zenith => 90.583333 => 90.583333
date.timezone => no value => no value

dba

DBA support => enabled
Supported handlers => gdbm db4

dom

DOM/XML => enabled
DOM/XML API Version => 20031129
libxml Version => 2.6.26
HTML Support => enabled
XPath Support => enabled
XPointer Support => enabled
Schema Support => enabled
RelaxNG Support => enabled

gd

GD Support => enabled
GD Version => bundled (2.0.28 compatible)
FreeType Support => enabled
FreeType Linkage => with freetype
FreeType Version => 2.1.10
T1Lib Support => enabled
GIF Read Support => enabled
GIF Create Support => enabled
JPG Support => enabled
PNG Support => enabled
WBMP Support => enabled
XBM Support => enabled

gettext

GetText Support => enabled

libxml

libXML support => active
libXML Version => 2.6.26
libXML streams => enabled

mcrypt

mcrypt support => enabled
Version => 2.5.7
Api No => 20021217
Supported ciphers => cast-128 gost rijndael-128 twofish arcfour cast-256
loki97 rijndael-192 saferplus wake blowfish-c
ompat des rijndael-256 serpent xtea blowfish enigma rc2 tripledes
Supported modes => cbc cfb ctr ecb ncfb nofb ofb stream

Directive => Local Value => Master Value
mcrypt.algorithms_dir => no value => no value
mcrypt.modes_dir => no value => no value

mysql
MySQL Support => enabled
Active Persistent Links => 0
Active Links => 0
Client API version => 5.0.24a
MYSQL_MODULE_TYPE => external
MYSQL_SOCKET => /var/run/mysqld/mysqld.sock
MYSQL_INCLUDE => -I/usr/include/mysql
MYSQL_LIBS => -L/usr/lib -lmysqlclient

Directive => Local Value => Master Value
mysql.allow_persistent => On => On
mysql.connect_timeout => 60 => 60
mysql.default_host => no value => no value
mysql.default_password => no value => no value
mysql.default_port => no value => no value
mysql.default_socket => no value => no value
mysql.default_user => no value => no value
mysql.max_links => Unlimited => Unlimited
mysql.max_persistent => Unlimited => Unlimited
mysql.trace_mode => Off => Off

ncurses

ncurses support => enabled
ncurses library version => 5.5
color support => yes

openssl

OpenSSL support => enabled
OpenSSL Version => OpenSSL 0.9.8c 05 Sep 2006

pcre

PCRE (Perl Compatible Regular Expressions) Support => enabled
PCRE Library Version => 6.6 06-Feb-2006

pspell

PSpell Support => enabled

Reflection

Reflection => enabled
Version => $Id: php_reflection.c,v 1.164.2.33 2006/03/29 14:28:42 tony2001
Exp $

session

Session Support => enabled
Registered save handlers => files user
Registered serializer handlers => php php_binary

Directive => Local Value => Master Value
session.auto_start => Off => Off
session.bug_compat_42 => On => On
session.bug_compat_warn => On => On
session.cache_expire => 180 => 180
session.cache_limiter => nocache => nocache
session.cookie_domain => no value => no value
session.cookie_lifetime => 0 => 0
session.cookie_path => / => /
session.cookie_secure => Off => Off
session.entropy_file => no value => no value
session.entropy_length => 0 => 0
session.gc_divisor => 100 => 100
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.hash_bits_per_character => 4 => 4
session.hash_function => 0 => 0
session.name => PHPSESSID => PHPSESSID
session.referer_check => no value => no value
session.save_handler => files => files
session.save_path => no value => no value
session.serialize_handler => php => php
session.use_cookies => On => On
session.use_only_cookies => Off => Off
session.use_trans_sid => 0 => 0

SPL

SPL support => enabled
Interfaces => Countable, OuterIterator, RecursiveIterator, SeekableIterator,
SplObserver, SplSubject
Classes => AppendIterator, ArrayIterator, ArrayObject,
BadFunctionCallException, BadMethodCallException, CachingIterat
or, DirectoryIterator, DomainException, EmptyIterator, FilterIterator,
InfiniteIterator, InvalidArgumentException, Ite
ratorIterator, LengthException, LimitIterator, LogicException,
NoRewindIterator, OutOfBoundsException, OutOfRangeExcep
tion, OverflowException, ParentIterator, RangeException,
RecursiveArrayIterator, RecursiveCachingIterator, RecursiveDi
rectoryIterator, RecursiveFilterIterator, RecursiveIteratorIterator,
RuntimeException, SplFileInfo, SplFileObject, Spl
ObjectStorage, SplTempFileObject, UnderflowException,
UnexpectedValueException

standard

Regex Library => Bundled library enabled
Dynamic Library Support => enabled
Path to sendmail => /usr/sbin/sendmail -t -i

Directive => Local Value => Master Value
assert.active => 1 => 1
assert.bail => 0 => 0
assert.callback => no value => no value
assert.quiet_eval => 0 => 0
assert.warning => 1 => 1
auto_detect_line_endings => 0 => 0
default_socket_timeout => 60 => 60
safe_mode_allowed_env_vars => PHP_ => PHP_
safe_mode_protected_env_vars => LD_LIBRARY_PATH => LD_LIBRARY_PATH
url_rewriter.tags => a=href,area=href,frame=src,input=src,form=,fieldset= =>
a=href,area=href,frame=src,input=src,form
=,fieldset=
user_agent => no value => no value

xml

XML Support => active
XML Namespace Support => active
libxml2 Version => 2.6.26

zlib

ZLib Support => enabled
Stream Wrapper support => compress.zlib://
Stream Filter support => zlib.inflate, zlib.deflate
Compiled Version => 1.2.3
Linked Version => 1.2.3

Directive => Local Value => Master Value
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value

Additional Modules

Module Name
readline

On Thu, Dec 17, 2009 at 10:13 AM, Rodrigo Aliste P. <raliste [at] gmail.com>wrote:

> Gary:
>
> The same thing happened to one of my servers. After long research I got to
> the conclusion that the problem was caused by the XCache module. If you have
> it, try to disable it.
>
>
> Rodrigo
>
> 2009/12/16 gary lim <garylim.in [at] gmail.com>
>
> Hi,
>>
>> 1. Unfortunately, our website depend heavily on PHP to operate, so it not
>> possible for us to disable it. We also highly suspect that the crash is
>> related to one of our php module because we are running a lot of Joomla
>> component.
>>
>> 2. unfortunately, we are unable to track which module is causing this
>> issue. Any idea how we can check which script is causing the problem because
>> this has been going on for weeks and we are still unable to determine the
>> source.
>>
>> 3. Kindly advise how we can perform the migration of our existing codebase
>> to FastCGI/SCGI without breaking the current website?
>>
>>
>> rgds
>>
>>
>> On Thu, Dec 17, 2009 at 12:28 AM, Tom Evans <tevans.uk [at] googlemail.com>wrote:
>>
>>> On Wed, Dec 16, 2009 at 4:20 PM, gary lim <garylim.in [at] gmail.com> wrote:
>>> >
>>> > Dear all,
>>> >
>>> > 1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(R) 4
>>> CPU 3.00GHz GNU/Linux
>>> > 2. The below info are my machine config and log output
>>> > 3. Kindly advise how to resolve intermittent apache crash even after
>>> performance tuning
>>> >
>>> > httpd.conf output
>>> > # prefork MPM [DEFAULT IF USE=-threads]
>>> > # StartServers: number of server processes to start
>>> > # MinSpareServers: minimum number of server processes which are kept
>>> spare
>>> > # MaxSpareServers: maximum number of server processes which are kept
>>> spare
>>> > # MaxClients: maximum number of server processes allowed to start
>>> > # MaxRequestsPerChild: maximum number of requests a server process
>>> serves
>>> > <IfModule prefork.c>
>>> > StartServers 2
>>> > MinSpareServers 4
>>> > MaxSpareServers 6
>>> > MaxClients 100
>>> > MaxRequestsPerChild 500
>>> > </IfModule>
>>> >
>>> > # worker MPM [DEFAULT IF USE=threads]
>>> > # StartServers: initial number of server processes to start
>>> > # MaxClients: maximum number of simultaneous client connections
>>> > # MinSpareThreads: minimum number of worker threads which are kept
>>> spare
>>> > # MaxSpareThreads: maximum number of worker threads which are kept
>>> spare
>>> > # ThreadsPerChild: constant number of worker threads in each server
>>> process
>>> > # MaxRequestsPerChild: maximum number of requests a server process
>>> serves
>>> > <IfModule worker.c>
>>> > StartServers 2
>>> > MaxClients 100
>>> > MinSpareThreads 25
>>> > MaxSpareThreads 75
>>> > ThreadsPerChild 25
>>> > MaxRequestsPerChild 500
>>> > </IfModule>
>>> >
>>> >
>>> >
>>> > top output
>>> > top - 00:09:00 up 157 days, 23:12, 2 users, load average: 0.67, 0.85,
>>> 0.80
>>> > Tasks: 47 total, 1 running, 46 sleeping, 0 stopped, 0 zombie
>>> > Cpu(s): 8.3% us, 1.4% sy, 0.0% ni, 88.0% id, 2.2% wa, 0.0% hi,
>>> 0.0% si
>>> > Mem: 917652k total, 858880k used, 58772k free, 142820k
>>> buffers
>>> > Swap: 1048568k total, 37844k used, 1010724k free, 434476k cached
>>> >
>>> > apache2 error log
>>> > *** glibc detected *** /usr/sbin/apache2: double free or corruption
>>> (fasttop): 0x08b4faf0 ***
>>> > [Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting,
>>> consider raising the MaxClients setting
>>> > *** glibc detected *** /usr/sbin/apache2: double free or corruption
>>> (fasttop): 0x086a1170 ***
>>> > *** glibc detected *** /usr/sbin/apache2: double free or corruption
>>> (fasttop): 0x08b48c08 ***
>>> > [Wed Dec 16 23:53:07 2009] [warn] child process 1325 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:07 2009] [warn] child process 10569 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:07 2009] [warn] child process 16964 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:09 2009] [warn] child process 1325 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:09 2009] [warn] child process 19910 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:09 2009] [warn] child process 16964 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:09 2009] [warn] child process 19718 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:11 2009] [warn] child process 19910 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:11 2009] [warn] child process 10569 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:11 2009] [warn] child process 19718 still did not
>>> exit, sending a SIGTERM
>>> > [Wed Dec 16 23:53:13 2009] [error] child process 1325 still did not
>>> exit, sending a SIGKILL
>>> > [Wed Dec 16 23:53:13 2009] [error] child process 19910 still did not
>>> exit, sending a SIGKILL
>>> > [Wed Dec 16 23:53:13 2009] [error] child process 10569 still did not
>>> exit, sending a SIGKILL
>>> > [Wed Dec 16 23:53:13 2009] [error] child process 16964 still did not
>>> exit, sending a SIGKILL
>>> >
>>> >
>>> > Joomla / PHP / Apache
>>> > System Information
>>> > PHP built On: Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23
>>> 19:20:15 SGT 2006 i686
>>> > Database Version: 5.0.24a-log
>>> > PHP Version: 5.1.6-pl4-gentoo
>>> > Web Server: Apache
>>> > WebServer to PHP interface: apache2handler
>>> > Joomla! Version: Joomla! 1.0.15 Stable [ Daytime ] 22 February 2008
>>> 23:00 UTC
>>> > User Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
>>> rv:1.9.0.15) Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729)
>>> >
>>> > Mysql
>>> > mysql Ver 14.12 Distrib 5.0.24a, for pc-linux-gnu (i686) using
>>> readline 5.1
>>> >
>>>
>>> Does it work if you disable PHP? Does it still exhibit problems if you
>>> run PHP out of process, eg using fastcgi or scgi (can PHP talk SCGI?).
>>> For my money, the double free()s are coming from PHP or a PHP module.
>>>
>>> Cheers
>>>
>>> Tom
>>>
>>> ------------------------------------------------------------ ---------
>>> The official User-To-User support forum of the Apache HTTP Server
>>> Project.
>>> See <URL:http://httpd.apache.org/userslist.html> for more info.
>>> To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
>>> " from the digest: users-digest-unsubscribe [at] httpd.apache.org
>>> For additional commands, e-mail: users-help [at] httpd.apache.org
>>>
>>>
>>
>
>
> --
> Rodrigo Aliste P.
>
>

--000e0cd3512af22670047b092b3f
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

hi Rodrigo, XCache doesn't seem to be installed in our setup....=
 phpinfo() PHP Version =3D> 5.1.6-pl4-gentoo Sys=
tem =3D> Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23 19:20:15 SG=
T 2006 i686 
Build Date =3D> Dec 11 2006 15:41:50 Configure Command =3D>=A0 =
9;./configure' '--prefix=3D/usr/lib/php5' '--host=3Di686-pc=
-linux-gnu' '--mandir=3D/usr/lib/php5/man' ' --infodir=
=3D/usr/lib/php5/info' '--sysconfdir=3D/etc' '--cache-file=
=3D./config.cache' '--enable-cli' '--disable-cgi' '=
--with 
-config-file-path=3D/etc/php/cli-php5' '--with-config-file-scan-dir=
=3D/etc/php/cli-php5/ext-active' '--without-pear' '--di =
sable-bcmath' '--with-bz2' '--disable-calendar' '--=
disable-ctype' '--without-curl' '--without-curlwrappers'=
; '--disable 
-dbase' '--disable-exif' '--without-fbsql' '--witho=
ut-fdftk' '--disable-filepro' '--disable-ftp' '--wi=
th-gettext' '--w ithout-gmp' '--disable-hash' '-=
-without-hwapi' '--without-iconv' '--without-informix' =
'--without-kerberos' '--disable- 
mbstring' '--with-mcrypt' '--disable-memory-limit' '=
;--without-mhash' '--without-ming' '--without-msql' =
9;--without-mssq l' '--with-ncurses' '--with-openssl'=
; '--with-openssl-dir=3D/usr' '--disable-pcntl' '--disa=
ble-pdo' '--without-pgsql' '-- 
disable-posix' '--with-pspell' '--without-recode' '=
--disable-simplexml' '--disable-shmop' '--without-snmp'=
'--disable- soap' '--disable-sockets' '--without-sy=
base' '--without-sybase-ct' '--disable-sysvmsg' '--=
disable-sysvsem' '--disable- 
sysvshm' '--without-tidy' '--disable-tokenizer' '--=
disable-wddx' '--disable-xmlreader' '--disable-xmlwriter=
9; '--withou t-xmlrpc' '--without-xsl' '--with-zlib&=
#39; '--disable-debug' '--enable-dba' '--without-cdb=
9; '--with-db4' '--without-flatf 
ile' '--with-gdbm' '--without-inifile' '--without-q=
dbm' '--with-freetype-dir=3D/usr' '--with-t1lib=3D/usr'=
'--disable-gd-j is-conv' '--enable-gd-native-ttf' '=
--with-jpeg-dir=3D/usr' '--with-png-dir=3D/usr' '--without-=
xpm-dir' '--with-gd' '--with 
-mysql=3D/usr/lib/mysql' '--with-mysql-sock=3D/var/run/mysqld/mysql=
d.sock' '--without-mysqli' '--with-readline' '--wit=
hout -libedit' '--without-mm' '--without-sqlite' 
Server API =3D> Command Line Interface Virtual Directory Support =3D&=
gt; disabled Configuration File (php.ini) Path =3D> /etc/php/cli-php5=
/php.ini Scan this dir for additional .ini files =3D> /etc/php/cli-ph=
p5/ext-active 
PHP API =3D> 20041225 PHP Extension =3D> 20050922 Zend Extensio=
n =3D> 220051025 Debug Build =3D> no Thread Safety =3D> disa=
bled Zend Memory Manager =3D> enabled IPv6 Support =3D> enabled=
 Registered PHP Streams =3D> php, file, http, ftp, compress.bzip2, co=
mpress.zlib, https, ftps 
Registered Stream Socket Transports =3D> tcp, udp, unix, udg, ssl, sslv3=
, sslv2, tls Registered Stream Filters =3D> string.rot13, string.toup=
per, string.tolower, string.strip_tags, convert.*, consumed, bzi p2.*, z=
lib.* 
 This program makes use of the Zend Scripting Language Engine: Ze=
nd Engine v2.1.0, Copyright (c) 1998-2006 Zend Technologies Conf=
iguration PHP Core Directive =3D> Local Value =3D> Mast=
er Value 
allow_call_time_pass_reference =3D> On =3D> On allow_url_fopen =3D=
> Off =3D> Off always_populate_raw_post_data =3D> Off =3D> O=
ff arg_separator.input =3D> & =3D> & arg_separator.outp=
ut =3D> & =3D> & 
asp_tags =3D> Off =3D> Off auto_append_file =3D> no value =3D&g=
t; no value auto_globals_jit =3D> On =3D> On auto_prepend_file =
=3D> no value =3D> no value browscap =3D> no value =3D> no v=
alue default_charset =3D> no value =3D> no value 
default_mimetype =3D> text/html =3D> text/html define_syslog_varia=
bles =3D> Off =3D> Off disable_classes =3D> no value =3D> no=
value disable_functions =3D> no value =3D> no value display_er=
rors =3D> On =3D> On 
display_startup_errors =3D> Off =3D> Off doc_root =3D> no value=
=3D> no value docref_ext =3D> no value =3D> no value docref=
_root =3D> no value =3D> no value enable_dl =3D> On =3D> On<=
br>error_append_string =3D> no value =3D> no value 
error_log =3D> no value =3D> no value error_prepend_string =3D>=
no value =3D> no value error_reporting =3D> 2039 =3D> 2039 =
expose_php =3D> On =3D> On extension_dir =3D> /usr/lib/php5/lib=
/php/extensions/no-debug-non-zts-20050922 =3D> /usr/lib/php5/lib/php/ext=
ensions/no-deb 
ug-non-zts-20050922 file_uploads =3D> On =3D> On <a href=3D"htt=
p://highlight.bg">highlight.bg</a> =3D> #FFFFFF =3D> #FFFFFF highl=
ight.comment =3D> #FF8000 =3D> #FF8000 highlight.default =3D> #=
0000BB =3D> #0000BB 
highlight.html =3D> #000000 =3D> #000000 highlight.keyword =3D>=
#007700 =3D> #007700 highlight.string =3D> #DD0000 =3D> #DD000=
0 html_errors =3D> Off =3D> On ignore_repeated_errors =3D> O=
ff =3D> Off 
ignore_repeated_source =3D> Off =3D> Off ignore_user_abort =3D>=
Off =3D> Off implicit_flush =3D> On =3D> Off include_path =
=3D> .:/usr/share/php5:/usr/share/php =3D> .:/usr/share/php5:/usr/sha=
re/php log_errors =3D> Off =3D> Off 
log_errors_max_len =3D> 1024 =3D> 1024 magic_quotes_gpc =3D> On=
=3D> On magic_quotes_runtime =3D> Off =3D> Off magic_quotes=
_sybase =3D> Off =3D> Off mail.force_extra_parameters =3D> no v=
alue =3D> no value 
max_execution_time =3D> 0 =3D> 30 max_input_time =3D> -1 =3D>=
; 60 open_basedir =3D> no value =3D> no value output_buffering =
=3D> 0 =3D> no value output_handler =3D> no value =3D> no va=
lue post_max_size =3D> 8M =3D> 8M 
precision =3D> 12 =3D> 12 realpath_cache_size =3D> 16K =3D> =
16K realpath_cache_ttl =3D> 120 =3D> 120 register_argc_argv =3D=
> On =3D> On register_globals =3D> Off =3D> Off register_=
long_arrays =3D> On =3D> On 
report_memleaks =3D> On =3D> On report_zend_debug =3D> Off =3D&=
gt; Off safe_mode =3D> Off =3D> Off safe_mode_exec_dir =3D> =
no value =3D> no value safe_mode_gid =3D> Off =3D> Off safe_=
mode_include_dir =3D> no value =3D> no value 
sendmail_from =3D> no value =3D> no value sendmail_path =3D> /u=
sr/sbin/sendmail -t -i=A0 =3D> /usr/sbin/sendmail -t -i serialize_pre=
cision =3D> 100 =3D> 100 short_open_tag =3D> On =3D> On S=
MTP =3D> localhost =3D> localhost 
smtp_port =3D> 25 =3D> 25 sql.safe_mode =3D> Off =3D> Off<br=
>track_errors =3D> Off =3D> Off unserialize_callback_func =3D> =
no value =3D> no value upload_max_filesize =3D> 2M =3D> 2M u=
pload_tmp_dir =3D> no value =3D> no value 
user_dir =3D> no value =3D> no value variables_order =3D> EGPCS=
=3D> EGPCS xmlrpc_error_number =3D> 0 =3D> 0 xmlrpc_errors =
=3D> Off =3D> Off y2k_compliance =3D> On =3D> On zend.ze1=
_compatibility_mode =3D> Off =3D> Off 
 bz2 BZip2 Support =3D> Enabled Stream Wrapper support =3D&=
gt; compress.bz2:// Stream Filter support =3D> bzip2.decompress, bzip=
2.compress BZip2 Version =3D> 1.0.3, 15-Feb-2005 date d=
ate/time support =3D> enabled 
Timezone Database Version =3D> 2006.1 Timezone Database =3D> inter=
nal Default timezone =3D> Asia/Singapore Directive =3D> Loc=
al Value =3D> Master Value date.default_latitude =3D> 31.7667 =3D&=
gt; 31.7667 
date.default_longitude =3D> 35.2333 =3D> 35.2333 date.sunrise_zeni=
th =3D> 90.583333 =3D> 90.583333 date.sunset_zenith =3D> 90.583=
333 =3D> 90.583333 date.timezone =3D> no value =3D> no value<br=
> dba 
 DBA support =3D> enabled Supported handlers =3D> gdbm db4 <=
br>dom DOM/XML =3D> enabled DOM/XML API Version =3D> 200311=
29 libxml Version =3D> 2.6.26 HTML Support =3D> enabled XPat=
h Support =3D> enabled 
XPointer Support =3D> enabled Schema Support =3D> enabled Relax=
NG Support =3D> enabled gd GD Support =3D> enabled G=
D Version =3D> bundled (2.0.28 compatible) FreeType Support =3D> e=
nabled 
FreeType Linkage =3D> with freetype FreeType Version =3D> 2.1.10<b=
r>T1Lib Support =3D> enabled GIF Read Support =3D> enabled GIF =
Create Support =3D> enabled JPG Support =3D> enabled PNG Suppor=
t =3D> enabled 
WBMP Support =3D> enabled XBM Support =3D> enabled gettext<=
br> GetText Support =3D> enabled libxml libXML support =
=3D> active libXML Version =3D> 2.6.26 libXML streams =3D> e=
nabled 
 mcrypt mcrypt support =3D> enabled Version =3D> 2.5.7<b=
r>Api No =3D> 20021217 Supported ciphers =3D> cast-128 gost rijnda=
el-128 twofish arcfour cast-256 loki97 rijndael-192 saferplus wake blowfish=
-c 
ompat des rijndael-256 serpent xtea blowfish enigma rc2 tripledes Suppor=
ted modes =3D> cbc cfb ctr ecb ncfb nofb ofb stream Directive =3D=
> Local Value =3D> Master Value mcrypt.algorithms_dir =3D> no v=
alue =3D> no value 
mcrypt.modes_dir =3D> no value =3D> no value mysql MySQL Su=
pport =3D> enabled Active Persistent Links =3D> 0 Active Links =
=3D> 0 Client API version =3D> 5.0.24a MYSQL_MODULE_TYPE =3D>=
; external 
MYSQL_SOCKET =3D> /var/run/mysqld/mysqld.sock MYSQL_INCLUDE =3D> -=
I/usr/include/mysql MYSQL_LIBS =3D> -L/usr/lib -lmysqlclient D=
irective =3D> Local Value =3D> Master Value mysql.allow_persistent=
=3D> On =3D> On 
mysql.connect_timeout =3D> 60 =3D> 60 mysql.default_host =3D> n=
o value =3D> no value mysql.default_password =3D> no value =3D>=
no value mysql.default_port =3D> no value =3D> no value mysql.=
default_socket =3D> no value =3D> no value 
mysql.default_user =3D> no value =3D> no value mysql.max_links =3D=
> Unlimited =3D> Unlimited mysql.max_persistent =3D> Unlimited =
=3D> Unlimited mysql.trace_mode =3D> Off =3D> Off ncurse=
s ncurses support =3D> enabled 
ncurses library version =3D> 5.5 color support =3D> yes ope=
nssl OpenSSL support =3D> enabled OpenSSL Version =3D> Open=
SSL 0.9.8c 05 Sep 2006 pcre PCRE (Perl Compatible Regular Exp=
ressions) Support =3D> enabled 
PCRE Library Version =3D> 6.6 06-Feb-2006 pspell PSpell Su=
pport =3D> enabled Reflection Reflection =3D> enabled<b=
r>Version =3D> $Id: php_reflection.c,v 1.164.2.33 2006/03/29 14:28:42 to=
ny2001 Exp $ 
 session Session Support =3D> enabled Registered save handl=
ers =3D> files user Registered serializer handlers =3D> php php_bi=
nary Directive =3D> Local Value =3D> Master Value session.a=
uto_start =3D> Off =3D> Off 
session.bug_compat_42 =3D> On =3D> On session.bug_compat_warn =3D&=
gt; On =3D> On session.cache_expire =3D> 180 =3D> 180 sessio=
n.cache_limiter =3D> nocache =3D> nocache session.cookie_domain =
=3D> no value =3D> no value 
session.cookie_lifetime =3D> 0 =3D> 0 session.cookie_path =3D> =
/ =3D> / session.cookie_secure =3D> Off =3D> Off session.ent=
ropy_file =3D> no value =3D> no value session.entropy_length =3D&g=
t; 0 =3D> 0 
session.gc_divisor =3D> 100 =3D> 100 session.gc_maxlifetime =3D>=
; 1440 =3D> 1440 session.gc_probability =3D> 1 =3D> 1 sessio=
n.hash_bits_per_character =3D> 4 =3D> 4 session.hash_function =3D&=
gt; 0 =3D> 0 
<a href=3D"http://session.name">session.name</a> =3D> PHPSESSID =3D> =
PHPSESSID session.referer_check =3D> no value =3D> no value ses=
sion.save_handler =3D> files =3D> files session.save_path =3D> =
no value =3D> no value 
session.serialize_handler =3D> php =3D> php session.use_cookies =
=3D> On =3D> On session.use_only_cookies =3D> Off =3D> Off<b=
r>session.use_trans_sid =3D> 0 =3D> 0 SPL SPL support =
=3D> enabled 
Interfaces =3D> Countable, OuterIterator, RecursiveIterator, SeekableIte=
rator, SplObserver, SplSubject Classes =3D> AppendIterator, ArrayIter=
ator, ArrayObject, BadFunctionCallException, BadMethodCallException, Cachin=
gIterat 
or, DirectoryIterator, DomainException, EmptyIterator, FilterIterator, Infi=
niteIterator, InvalidArgumentException, Ite ratorIterator, LengthExcepti=
on, LimitIterator, LogicException, NoRewindIterator, OutOfBoundsException, =
OutOfRangeExcep 
tion, OverflowException, ParentIterator, RangeException, RecursiveArrayIter=
ator, RecursiveCachingIterator, RecursiveDi rectoryIterator, RecursiveFi=
lterIterator, RecursiveIteratorIterator, RuntimeException, SplFileInfo, Spl=
FileObject, Spl 
ObjectStorage, SplTempFileObject, UnderflowException, UnexpectedValueExcept=
ion standard Regex Library =3D> Bundled library enabled<br=
>Dynamic Library Support =3D> enabled Path to sendmail =3D> /usr/s=
bin/sendmail -t -i 
 Directive =3D> Local Value =3D> Master Value assert.active =3D=
> 1 =3D> 1 assert.bail =3D> 0 =3D> 0 assert.callback =3D&=
gt; no value =3D> no value assert.quiet_eval =3D> 0 =3D> 0 a=
ssert.warning =3D> 1 =3D> 1 
auto_detect_line_endings =3D> 0 =3D> 0 default_socket_timeout =3D&=
gt; 60 =3D> 60 safe_mode_allowed_env_vars =3D> PHP_ =3D> PHP_<b=
r>safe_mode_protected_env_vars =3D> LD_LIBRARY_PATH =3D> LD_LIBRARY_P=
ATH url_rewriter.tags =3D> a=3Dhref,area=3Dhref,frame=3Dsrc,input=3Ds=
rc,form=3D,fieldset=3D =3D> a=3Dhref,area=3Dhref,frame=3Dsrc,input=3Dsrc=
,form 
=3D,fieldset=3D user_agent =3D> no value =3D> no value xml<=
br> XML Support =3D> active XML Namespace Support =3D> active<b=
r>libxml2 Version =3D> 2.6.26 zlib ZLib Support =3D> en=
abled Stream Wrapper support =3D> compress.zlib:// 
Stream Filter support =3D> zlib.inflate, zlib.deflate Compiled Versio=
n =3D> 1.2.3 Linked Version =3D> 1.2.3 Directive =3D> Lo=
cal Value =3D> Master Value zlib.output_compression =3D> Off =3D&g=
t; Off 
zlib.output_compression_level =3D> -1 =3D> -1 zlib.output_handler =
=3D> no value =3D> no value Additional Modules Module N=
ame readline <div class=3D"gmail_quote">On Thu, D=
ec 17, 2009 at 10:13 AM, Rodrigo Aliste P. <<a href=3D=
"mailto:raliste [at] gmail.com">raliste [at] gmail.com</a>> wrote: 
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Gary: The =
same thing happened to one of my servers. After long research I got to the =
conclusion that the problem was caused by the XCache module. If you have it=
, try to disable it. 
 Rodrigo <div class=3D"gmail_quote">

2009/12/16 gary lim <<a href=3D"http://garylim.in" tar=
get=3D"_blank">garylim.in</a> [at] <a href=3D"http://gmail.com" target=3D"_blank=
">gmail.com</a>><div><div></div><div class=3D"h5"> <blockquote=
class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, 204, 204); =
margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">

Hi, 1. Unfortunately, our website depend heavily on PHP to operate, =
so it not possible for us to disable it. We also highly suspect that the cr=
ash is related to one of our php module because we are running a lot of Joo=
mla component. 

 2. unfortunately, we are unable to track which module is causing this i=
ssue. Any idea how we can check which script is causing the problem because=
this has been going on for weeks and we are still unable to determine the =
source. 

 3. Kindly advise how we can perform the migration of our existing codeb=
ase to FastCGI/SCGI without breaking the current website? rgds<b=
r> <div class=3D"gmail_quote">On Thu, Dec 17, 2009 at 12:28 AM, Tom =
Evans <<a href=3D"http://tevans.uk" target=3D"_blank">=
tevans.uk</a> [at] <a href=3D"http://googlemail.com" target=3D"_blank">googlemai=
l.com</a>> wrote: 

<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><=
div>On Wed, Dec 16, 2009 at 4:20 PM, gary lim <<a href=3D"http://garylim=
..in" target=3D"_blank">garylim.in</a> [at] <a href=3D"http://gmail.com" target=
=3D"_blank">gmail.com</a>> wrote: 

> 
> Dear all, 
> 
> 1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(=
R) 4 CPU 3.00GHz GNU/Linux 
> 2. The below info are my machine config and log output 
> 3. Kindly advise how to resolve intermittent apache crash even after p=
erformance tuning 
> 
> httpd.conf output 
> # prefork MPM [DEFAULT IF USE=3D-threads] 
> # StartServers: number of server processes to start 
> # MinSpareServers: minimum number of server processes which are kept s=
pare 
> # MaxSpareServers: maximum number of server processes which are kept s=
pare 
> # MaxClients: maximum number of server processes allowed to start 
> # MaxRequestsPerChild: maximum number of requests a server process ser=
ves 
> <IfModule prefork.c> 
> =A0=A0=A0 StartServers=A0=A0=A0=A0=A0=A0=A0=A0 2 
> =A0=A0=A0 MinSpareServers=A0=A0=A0=A0=A0 4 
> =A0=A0=A0 MaxSpareServers=A0=A0=A0=A0 6 
> =A0=A0=A0 MaxClients=A0=A0=A0=A0=A0=A0=A0=A0 100 
> =A0=A0=A0 MaxRequestsPerChild=A0 500 
> </IfModule> 
> 
> # worker MPM [DEFAULT IF USE=3Dthreads] 
> # StartServers: initial number of server processes to start 
> # MaxClients: maximum number of simultaneous client connections 
> # MinSpareThreads: minimum number of worker threads which are kept spa=
re 
> # MaxSpareThreads: maximum number of worker threads which are kept spa=
re 
> # ThreadsPerChild: constant number of worker threads in each server pr=
ocess 
> # MaxRequestsPerChild: maximum number of requests a server process ser=
ves 
> <IfModule worker.c> 
> =A0=A0=A0 StartServers=A0=A0=A0=A0=A0=A0=A0=A0 2 
> =A0=A0=A0 MaxClients=A0=A0=A0=A0=A0=A0=A0=A0 100 
> =A0=A0=A0 MinSpareThreads=A0=A0=A0=A0 25 
> =A0=A0=A0 MaxSpareThreads=A0=A0=A0=A0 75 
> =A0=A0=A0 ThreadsPerChild=A0=A0=A0=A0 25 
> =A0=A0=A0 MaxRequestsPerChild=A0 500 
> </IfModule> 
> 
> 
> 
> top output 
> top - 00:09:00 up 157 days, 23:12,=A0 2 users,=A0 load average: 0.67, =
0.85, 0.80 
> Tasks:=A0 47 total,=A0=A0 1 running,=A0 46 sleeping,=A0=A0 0 stopped,=
=A0=A0 0 zombie 
> Cpu(s):=A0 8.3% us,=A0 1.4% sy,=A0 0.0% ni, 88.0% id,=A0 2.2% wa,=A0 0=
..0% hi,=A0 0.0% si 
> Mem:=A0=A0=A0 917652k total,=A0=A0 858880k used,=A0=A0=A0 58772k free,=
=A0=A0 142820k buffers 
> Swap:=A0 1048568k total,=A0=A0=A0 37844k used,=A0 1010724k free,=A0=A0=
434476k cached 
> 
> apache2 error log 
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f=
asttop): 0x08b4faf0 *** 
> [Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting, =
consider raising the MaxClients setting 
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f=
asttop): 0x086a1170 *** 
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f=
asttop): 0x08b48c08 *** 
> [Wed Dec 16 23:53:07 2009] [warn] child process 1325 still did not exi=
t, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 10569 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 16964 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 1325 still did not exi=
t, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 19910 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 16964 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:09 2009] [warn] child process 19718 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not exi=
t, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 19910 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 10569 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:11 2009] [warn] child process 19718 still did not ex=
it, sending a SIGTERM 
> [Wed Dec 16 23:53:13 2009] [error] child process 1325 still did not ex=
it, sending a SIGKILL 
> [Wed Dec 16 23:53:13 2009] [error] child process 19910 still did not e=
xit, sending a SIGKILL 
> [Wed Dec 16 23:53:13 2009] [error] child process 10569 still did not e=
xit, sending a SIGKILL 
> [Wed Dec 16 23:53:13 2009] [error] child process 16964 still did not e=
xit, sending a SIGKILL 
> 
> 
> Joomla / PHP / Apache 
> System Information 
> PHP built On: Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23 19:2=
0:15 SGT 2006 i686 
> Database Version: 5.0.24a-log 
> PHP Version: 5.1.6-pl4-gentoo 
> Web Server: Apache 
> WebServer to PHP interface: apache2handler 
> Joomla! Version: Joomla! 1.0.15 Stable [ Daytime ] 22 February 2008 23=
:00 UTC 
> User Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1=
5) Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729) 
> 
> Mysql 
> mysql=A0 Ver 14.12 Distrib 5.0.24a, for pc-linux-gnu (i686) using read=
line 5.1 
> 
 
</div></div>Does it work if you disable PHP? Does it still exhibit problems=
if you 
run PHP out of process, eg using fastcgi or scgi (can PHP talk SCGI?). 
For my money, the double free()s are coming from PHP or a PHP module. 
 
Cheers 
 
Tom 
 
------------------------------------------------------------ --------- 
The official User-To-User support forum of the Apache HTTP Server Project.<=
br>
See <URL:<a href=3D"http://httpd.apache.org/userslist.html" target=3D"_b=
lank">http://httpd.apache.org/userslist.html</a>> for more info. 
To unsubscribe, e-mail: <a href=3D"mailto:users-unsubscribe [at] httpd.apache.or=
g" target=3D"_blank">users-unsubscribe [at] httpd.apache.org</a> 
=A0 " =A0 from the digest: <a href=3D"mailto:users-digest-unsubscribe=
[at] httpd.apache.org" target=3D"_blank">users-digest-unsubscribe [at] httpd.apache.=
org</a> 
For additional commands, e-mail: <a href=3D"mailto:users-help [at] httpd.apache.=
org" target=3D"_blank">users-help [at] httpd.apache.org</a> 
 
</blockquote></div> 
</blockquote></div></div></div> -- Rodrigo Aliste P. 
</blockquote></div> 

--000e0cd3512af22670047b092b3f--

--000e0cd6742cb6bfa7047b096c90
Content-Type: text/plain; charset=ISO-8859-1

Hi daniel,

1. Glibc version - (GNU libc) 2.4

2. Our default http.conf is experiencing the intermittent apache crash,
hence the performance tuning when we detected that apache has use up all the
swap probably due to memory leak

gary

On Thu, Dec 17, 2009 at 12:42 PM, Daniel Reinhardt
<cryptodan [at] cryptodan.net>wrote:

> -->From: gary lim
> -->Sent: 16 December, 2009 16:20
> -->To: users [at] httpd.apache.org
> -->Subject: [users [at] httpd] Apache intermittent crash - HELP!!!
> -->
> -->
> -->Dear all,
>
> -->
> -->1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(R) 4
> CPU 3.00GHz GNU/Linux
> -->2. The below info are my machine config and log output
> -->3. Kindly advise how to resolve intermittent apache crash even after
> performance tuning
> -->
> -->apache2 error log
>
> -->*** glibc detected *** /usr/sbin/apache2: double free or corruption
> (fasttop): 0x08b4faf0 ***
> -->[Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting,
> consider raising the MaxClients setting
> -->*** glibc detected *** /usr/sbin/apache2: double free or corruption
> (fasttop): 0x086a1170 ***
> -->*** glibc detected *** /usr/sbin/apache2: double free or corruption
> (fasttop): 0x08b48c08 ***
>
>
>
> Gary,
>
> What version of glibc do you have? The glibc detected could be causing
> this since its producing a dump code. I would revert back to a default
> setup with your httpd.conf file, and just use the basics you need for your
> site to run. If you get no crashes then its an issue with your performance
> tuning.
>
> Thanks,
> Daniel
>
> ------------------------------------------------------------ ---------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
> " from the digest: users-digest-unsubscribe [at] httpd.apache.org
> For additional commands, e-mail: users-help [at] httpd.apache.org
>
>

--000e0cd6742cb6bfa7047b096c90
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi daniel, 1. Glibc version - (GNU libc) 2.4 2. Our default h=
ttp.conf is experiencing the intermittent apache crash, hence the performan=
ce tuning when we detected that apache has use up all the swap probably due=
to memory leak 
 gary <div class=3D"gmail_quote">On Thu, Dec 17, 200=
9 at 12:42 PM, Daniel Reinhardt <<a href=3D"mailto:cry=
ptodan [at] cryptodan.net">cryptodan [at] cryptodan.net</a>> wrote: <blo=
ckquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, 204,=
204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
-->From: gary lim 
-->Sent: 16 December, 2009 16:20 
-->To: <a href=3D"mailto:users [at] httpd.apache.org" target=3D"_blank">users=
[at] httpd.apache.org</a> 
-->Subject: [users [at] httpd] Apache intermittent crash - HELP!!! 
--> 
--> 
-->Dear all,<div class=3D"im"> 
--> 
-->1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium=
(R) 4 CPU 3.00GHz GNU/Linux 
-->2. The below info are my machine config and log output 
-->3. Kindly advise how to resolve intermittent apache crash even after =
performance tuning </div>
--> 
-->apache2 error log<div class=3D"im"> 
-->*** glibc detected *** /usr/sbin/apache2: double free or corruption (=
fasttop): 0x08b4faf0 *** 
-->[Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting,=
consider raising the MaxClients setting 
-->*** glibc detected *** /usr/sbin/apache2: double free or corruption (=
fasttop): 0x086a1170 *** 
-->*** glibc detected *** /usr/sbin/apache2: double free or corruption (=
fasttop): 0x08b48c08 *** 
 
 
 </div>
Gary, 
 
What version of glibc do you have? =A0The glibc detected could be causing t=
his since its producing a dump code. =A0I would revert back to a default se=
tup with your httpd.conf file, and just use the basics you need for your si=
te to run. =A0If you get no crashes then its an issue with your performance=
tuning. 

 
Thanks, 
Daniel <div><div></div><div class=3D"h5">
 
------------------------------------------------------------ --------- 
The official User-To-User support forum of the Apache HTTP Server Project.<=
br>
See <URL:<a href=3D"http://httpd.apache.org/userslist.html" target=3D"_b=
lank">http://httpd.apache.org/userslist.html</a>> for more info. 
To unsubscribe, e-mail: <a href=3D"mailto:users-unsubscribe [at] httpd.apache.or=
g" target=3D"_blank">users-unsubscribe [at] httpd.apache.org</a> 
=A0" =A0 from the digest: <a href=3D"mailto:users-digest-unsubscribe [at] =
httpd.apache.org" target=3D"_blank">users-digest-unsubscribe [at] httpd.apache.o=
rg</a> 
For additional commands, e-mail: <a href=3D"mailto:users-help [at] httpd.apache.=
org" target=3D"_blank">users-help [at] httpd.apache.org</a> 
 
</div></div></blockquote></div> 

--000e0cd6742cb6bfa7047b096c90--

--------------------------------------------------
From: "Justin Pasher" <justinp [at] newmediagateway.com>
Sent: 18 December, 2009 21:20
To: <users [at] httpd.apache.org>
Subject: Re: [users [at] httpd] Apache intermittent crash - HELP!!!

> Just a couple things to point out (but nothing specific to fixing your probem)
> ...
>
> gary lim wrote:
>> Hi Daniel,

>> *top output
>> *top - 04:58:52 up 160 days, 4:02, 1 user, load average: 0.08, 0.04, 0.05
>> Tasks: 69 total, 2 running, 67 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 1.9% us, 5.0% sy, 0.0% ni, 93.2% id, 0.0% wa, 0.0% hi, 0.0% si
>> Mem: 917652k total, 908144k used, 9508k free, 32660k buffers
>> Swap: 1048568k total, 37856k used, 1010712k free, 179480k cached
>>
>>
>> Model: *Dell PowerEdge 850*
>> CPU:Intel *Pentium IV 3.0GHz* / 2MB, 800MHz FSB
>> RAM: *2GB **DDR-2 SDRAM* 533MHz w/ ECC*
>> *Harddisk: *02 x SATA* (7200RPM) HDD (non-RAID)
>
> Not sure if you noticed or not, but all of your memory is not actually <
> showing up as available to the kernel. Are you running a kernel with <
> "bigmem"-style support? If not, you're missing out on half of your RAM. <

I would be willing to guess, that his kernel is not using his full ram, and if
your system is running out of 10Gigs of Swap something else needs to be done
with the computer. I have a similar setup, and my swap hardly gets used.
However, I am sure my site/server doesn't get anywhere near the hits yours does.
I am, however, running Ubuntu 9.10, a few game servers, webmin, mysql, php,
apache 2.2.12, ssh, postfix, and a few other process. I am also running Gnome
Desktop Environment, because I like to use it when I am cleaning my main
gaming/workstation PC.

I would look into recompiling your Kernel with RAM support upto 4gig or greater,
seeing your machine is only 32bit, I would recommend using upto 4Gig support in
the memory section. After that see if it makes a difference, if you do not see
a difference, then post what you are running on the machine. The more
information you provide the better we can diagnose your issue with your server,
and get down to the root cause of your crashes.

Here are my system specs: http://www.cryptodan.net/sys/index.php?disp=dynamic

Thanks,
Daniel

------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe [at] httpd.apache.org
" from the digest: users-digest-unsubscribe [at] httpd.apache.org
For additional commands, e-mail: users-help [at] httpd.apache.org

Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!

Re: Apache intermittent crash - HELP!!!