--001485f5ce4a5ef9f70486f86da1
Content-Type: text/plain; charset=ISO-8859-1

Hey all,

I am trying to port a module I wrote in C to one that uses mod_perl. The C
module would spawn a thread in the child_init process and poll a file. The
file polled contained data that the header parser phase would use. I
protected the data via a RW Lock.

Now when doing this in mod_perl I see the following things:

if I have a ChildInitHandler, any changes I make to globals are not
reflected in a HeaderParserHandler. My globals do not have the :shared
attribute - must they?

I've had better luck getting rid of the ChildInit directive and initializing
globals like a normal script. Then my handler routine can see what I've
initialized. I can even spawn a thread. Unfortunately, changes to a shared
array that my thread makes are not picked up by the handler.

I'm definitely missing something here as I'm approaching this from a C
module perspective where I had access to the right hooks and knew what
process I'd be running in. Is there a way to do what I want in mod_perl -
namely have a HeaderParserHandler that uses data for some logic where that
data is updated from some other thread that I spawn?

Thanks,

SB

--001485f5ce4a5ef9f70486f86da1
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>Hey all,</div>
<div>=A0</div>
<div>I am trying to port a module I wrote in C to one that uses mod_perl.=
=A0 The C module would spawn a thread in the child_init process and poll a =
file.=A0 The file polled contained data that the header parser phase=A0woul=
d use.=A0 I protected the data via a RW Lock.</div>

<div>=A0</div>
<div>Now when doing this in mod_perl I see the following things:</div>
<div>=A0</div>
<div>if I have a ChildInitHandler, any changes I make to globals are not re=
flected in a HeaderParserHandler.=A0 My globals do not have the :shared att=
ribute - must they?</div>
<div>=A0</div>
<div>I've had better luck getting rid of the ChildInit directive and in=
itializing globals like a normal script.=A0 Then my handler routine can see=
what I've initialized.=A0 I can even spawn a thread.=A0 Unfortunately,=
changes to a shared array that my thread makes are not picked up by the ha=
ndler.</div>

<div>=A0</div>
<div>I'm definitely missing something here as I'm approaching this =
from a C module perspective where I had access to the right hooks and knew =
what process I'd be running in.=A0 Is there a way to do what I want in =
mod_perl - namely have a HeaderParserHandler that uses data for some logic =
where that data is updated from some other thread that I spawn?</div>

<div>=A0</div>
<div>Thanks,</div>
<div>=A0</div>
<div>SB</div>

--001485f5ce4a5ef9f70486f86da1--

On Wed, May 19, 2010 at 4:37 PM, Some Guy <teknosoul [at] gmail.com> wrote:
> if I have a ChildInitHandler, any changes I make to globals are not
> reflected in a HeaderParserHandler.=A0 My globals do not have the :shared
> attribute - must they?

Only if you're trying to update them from a different thread. If
there aren't multiple threads involved, you can update globals from
anywhere. If that's not working for you there's probably a mistake in
the way you're referring to the variables. If you post some code we
may be able to help.

> Is there a way to do what I want in mod_perl -
> namely have a HeaderParserHandler that uses data for some logic where tha=
t
> data is updated from some other thread that I spawn?

A more common approach with perl is to make a handler that runs in an
early phase which reloads the data as needed, and not use a separate
thread. If you're concerned about reloading the data adding too much
delay to a web request, you could do this in a cleanup handler.

- Perrin

--001485f72658f91d800487061233
Content-Type: text/plain; charset=ISO-8859-1

The cleanup handler is a brilliant idea. That removes the need for a
polling thread. If I attach the cleanup to the request's pool and run in a
threaded MPM, this led to contention for the globals that I'm updating in
C. Is this the case with PerlModules, or can I get away without locking
them?

Thanks,

SB

On Wed, May 19, 2010 at 6:13 PM, Perrin Harkins <perrin [at] elem.com> wrote:

> On Wed, May 19, 2010 at 4:37 PM, Some Guy <teknosoul [at] gmail.com> wrote:
> > if I have a ChildInitHandler, any changes I make to globals are not
> > reflected in a HeaderParserHandler. My globals do not have the :shared
> > attribute - must they?
>
> Only if you're trying to update them from a different thread. If
> there aren't multiple threads involved, you can update globals from
> anywhere. If that's not working for you there's probably a mistake in
> the way you're referring to the variables. If you post some code we
> may be able to help.
>
> > Is there a way to do what I want in mod_perl -
> > namely have a HeaderParserHandler that uses data for some logic where
> that
> > data is updated from some other thread that I spawn?
>
> A more common approach with perl is to make a handler that runs in an
> early phase which reloads the data as needed, and not use a separate
> thread. If you're concerned about reloading the data adding too much
> delay to a web request, you could do this in a cleanup handler.
>
> - Perrin
>

--001485f72658f91d800487061233
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

The cleanup handler is a brilliant idea.=A0 That removes the need for a pol=
ling thread.=A0 If I attach the cleanup to the request's pool and run i=
n a threaded MPM, this led to contention for the globals that I'm updat=
ing in C.=A0 Is this the case with PerlModules, or can I get away without l=
ocking them?<br>
=A0<br>Thanks,<br>=A0<br>SB<br><br>
<div class=3D"gmail_quote">On Wed, May 19, 2010 at 6:13 PM, Perrin Harkins =
<span dir=3D"ltr"><<a href=3D"mailto:perrin [at] elem.com">perrin [at] elem.com</a=
>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"PADDING-LEFT: 1ex; MARGIN: 0px 0=
px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<div class=3D"im">On Wed, May 19, 2010 at 4:37 PM, Some Guy <<a href=3D"=
mailto:teknosoul [at] gmail.com">teknosoul [at] gmail.com</a>> wrote:<br>> if I=
have a ChildInitHandler, any changes I make to globals are not<br>> ref=
lected in a HeaderParserHandler.=A0 My globals do not have the :shared<br>
> attribute - must they?<br><br></div>Only if you're trying to updat=
e them from a different thread. =A0If<br>there aren't multiple threads =
involved, you can update globals from<br>anywhere. =A0If that's not wor=
king for you there's probably a mistake in<br>
the way you're referring to the variables. =A0If you post some code we<=
br>may be able to help.<br>
<div class=3D"im"><br>> Is there a way to do what I want in mod_perl -<b=
r>> namely have a HeaderParserHandler that uses data for some logic wher=
e that<br>> data is updated from some other thread that I spawn?<br><br>
</div>A more common approach with perl is to make a handler that runs in an=
<br>early phase which reloads the data as needed, and not use a separate<br=
>thread. =A0If you're concerned about reloading the data adding too muc=
h<br>
delay to a web request, you could do this in a cleanup handler.<br><font co=
lor=3D"#888888"><br>- Perrin<br></font></blockquote></div><br>

--001485f72658f91d800487061233--

On 05/20/2010 08:54 AM, Some Guy wrote:
> The cleanup handler is a brilliant idea. That removes the need for a
> polling thread. If I attach the cleanup to the request's pool and run
> in a threaded MPM, this led to contention for the globals that I'm
> updating in C. Is this the case with PerlModules, or can I get away
> without locking them?

Why use a threaded MPM? Is there something else you're running that
relies on it? We generally recommend doing with the prefork mpm because
you're running in share-nothing mode which performs better (and uses
less memory on decent operating systems).

--
Michael Peters
Plus Three, LP

--001485f5cef4dcc003048706f9c1
Content-Type: text/plain; charset=ISO-8859-1

This module is meant for other parties to use, so I can't define what MPM
they use. I would like it to work in either case. I don't mind throwing in
locking logic based on the results of querying whether the MPM is threaded
or not - I just want to better understand that scenario and how it applies
to modules written for mod_perl.

Thanks,

SB
On Thu, May 20, 2010 at 9:20 AM, Michael Peters <mpeters [at] plusthree.com>wrote:

> On 05/20/2010 08:54 AM, Some Guy wrote:
>
>> The cleanup handler is a brilliant idea. That removes the need for a
>> polling thread. If I attach the cleanup to the request's pool and run
>> in a threaded MPM, this led to contention for the globals that I'm
>> updating in C. Is this the case with PerlModules, or can I get away
>> without locking them?
>>
>
> Why use a threaded MPM? Is there something else you're running that relies
> on it? We generally recommend doing with the prefork mpm because you're
> running in share-nothing mode which performs better (and uses less memory on
> decent operating systems).
>
> --
> Michael Peters
> Plus Three, LP
>

--001485f5cef4dcc003048706f9c1
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>This module is meant for other parties to use, so I can't define w=
hat MPM they use.=A0 I would like it to work in either case.=A0 I don't=
mind throwing in locking logic based on the results of querying whether th=
e MPM is threaded or not - I just want to better understand that scenario a=
nd how it applies to modules written for mod_perl.</div>

<div>=A0</div>
<div>Thanks,</div>
<div>=A0</div>
<div>SB<br></div>
<div class=3D"gmail_quote">On Thu, May 20, 2010 at 9:20 AM, Michael Peters =
<span dir=3D"ltr"><<a href=3D"mailto:mpeters [at] plusthree.com">mpeters [at] plus=
three.com</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"PADDING-LEFT: 1ex; MARGIN: 0px 0=
px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<div class=3D"im">On 05/20/2010 08:54 AM, Some Guy wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"PADDING-LEFT: 1ex; MARGIN: 0px 0=
px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">The cleanup handler is a brillia=
nt idea. =A0That removes the need for a<br>polling thread. =A0If I attach t=
he cleanup to the request's pool and run<br>
in a threaded MPM, this led to contention for the globals that I'm<br>u=
pdating in C. =A0Is this the case with PerlModules, or can I get away<br>wi=
thout locking them?<br></blockquote><br></div>Why use a threaded MPM? Is th=
ere something else you're running that relies on it? We generally recom=
mend doing with the prefork mpm because you're running in share-nothing=
mode which performs better (and uses less memory on decent operating syste=
ms).<br>
<font color=3D"#888888"><br>-- <br>Michael Peters<br>Plus Three, LP<br></fo=
nt></blockquote></div><br>

--001485f5cef4dcc003048706f9c1--

--0016363b9f2ab7a5b8048708121e
Content-Type: text/plain; charset=ISO-8859-1

I tried the cleanup handler solution, and threw a sleep in there for ha
has. Basically I do the following in my handler:

Register a cleanup on the request's pool
Check if I need to do something to the user agent if it is found in an array
that is backed by a data file
If it's not in the array, decline the request, else, do my thing.
In the cleanup:
Check if the file has changed since i last checked
Reload the file into the array if necessary

If the MPM is threaded, a rwlock is used where the cleanup obtains the
writelock and request handlers are obtaining a read lock.

I threw a sleep in the cleanup to see what happens, and it appears to delay
sending the results of the next request. I issued a request, then issued
another. I didn't get the results of the second request until the sleep
from the first returned. No locks used by the request handler are held
during sleep time, and the MPM is threaded. I had expected the request to
be flushed to the client before the cleanup code is run. Is this wrong?

Thanks,

SB

On Thu, May 20, 2010 at 9:58 AM, Some Guy <teknosoul [at] gmail.com> wrote:

> This module is meant for other parties to use, so I can't define what MPM
> they use. I would like it to work in either case. I don't mind throwing in
> locking logic based on the results of querying whether the MPM is threaded
> or not - I just want to better understand that scenario and how it applies
> to modules written for mod_perl.
>
> Thanks,
>
> SB
> On Thu, May 20, 2010 at 9:20 AM, Michael Peters <mpeters [at] plusthree.com>wrote:
>
>> On 05/20/2010 08:54 AM, Some Guy wrote:
>>
>>> The cleanup handler is a brilliant idea. That removes the need for a
>>> polling thread. If I attach the cleanup to the request's pool and run
>>> in a threaded MPM, this led to contention for the globals that I'm
>>> updating in C. Is this the case with PerlModules, or can I get away
>>> without locking them?
>>>
>>
>> Why use a threaded MPM? Is there something else you're running that relies
>> on it? We generally recommend doing with the prefork mpm because you're
>> running in share-nothing mode which performs better (and uses less memory on
>> decent operating systems).
>>
>> --
>> Michael Peters
>> Plus Three, LP
>>
>
>

--0016363b9f2ab7a5b8048708121e
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>I tried the cleanup handler solution, and threw a sleep in there for h=
a has.=A0 Basically I do the following in my handler:</div>
<div>=A0</div>
<div>Register a cleanup on the request's pool</div>
<div>Check if I need to do something to the user agent if it is found in an=
array that is backed by a data file</div>
<div>If it's not in the array, decline the request, else, do my thing.<=
br></div>
<div>In the cleanup:</div>
<div>Check if the file has changed since i last checked</div>
<div>Reload the file into the array if necessary</div>
<div>=A0</div>
<div>If the MPM is threaded, a rwlock is used where the cleanup obtains the=
writelock and request handlers are obtaining a read lock.</div>
<div>=A0</div>
<div>I threw a sleep in the cleanup to see what happens, and it appears to =
delay sending the results of the next request.=A0 I issued a request, then =
issued another.=A0 I didn't get the results of the second request until=
the sleep from the first returned.=A0 No locks used by the request handler=
are held during sleep time, and the MPM is threaded.=A0 I had expected the=
request to be flushed to the client before the cleanup code is run.=A0 Is =
this wrong?</div>

<div>=A0</div>
<div>Thanks,</div>
<div>=A0</div>
<div>SB</div>
<div>=A0</div>
<div class=3D"gmail_quote">On Thu, May 20, 2010 at 9:58 AM, Some Guy <span =
dir=3D"ltr"><<a href=3D"mailto:teknosoul [at] gmail.com">teknosoul [at] gmail.com<=
/a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"PADDING-LEFT: 1ex; MARGIN: 0px 0=
px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<div>This module is meant for other parties to use, so I can't define w=
hat MPM they use.=A0 I would like it to work in either case.=A0 I don't=
mind throwing in locking logic based on the results of querying whether th=
e MPM is threaded or not - I just want to better understand that scenario a=
nd how it applies to modules written for mod_perl.</div>

<div>=A0</div>
<div>Thanks,</div>
<div>=A0</div>
<div>SB<br></div>
<div>
<div></div>
<div class=3D"h5">
<div class=3D"gmail_quote">On Thu, May 20, 2010 at 9:20 AM, Michael Peters =
<span dir=3D"ltr"><<a href=3D"mailto:mpeters [at] plusthree.com" target=3D"_b=
lank">mpeters [at] plusthree.com</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"PADDING-LEFT: 1ex; MARGIN: 0px 0=
px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<div>On 05/20/2010 08:54 AM, Some Guy wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"PADDING-LEFT: 1ex; MARGIN: 0px 0=
px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">The cleanup handler is a brillia=
nt idea. =A0That removes the need for a<br>polling thread. =A0If I attach t=
he cleanup to the request's pool and run<br>
in a threaded MPM, this led to contention for the globals that I'm<br>u=
pdating in C. =A0Is this the case with PerlModules, or can I get away<br>wi=
thout locking them?<br></blockquote><br></div>Why use a threaded MPM? Is th=
ere something else you're running that relies on it? We generally recom=
mend doing with the prefork mpm because you're running in share-nothing=
mode which performs better (and uses less memory on decent operating syste=
ms).<br>
<font color=3D"#888888"><br>-- <br>Michael Peters<br>Plus Three, LP<br></fo=
nt></blockquote></div><br></div></div></blockquote></div><br>

--0016363b9f2ab7a5b8048708121e--

On Thu, May 20, 2010 at 11:17 AM, Some Guy <teknosoul [at] gmail.com> wrote:
> If the MPM is threaded, a rwlock is used where the cleanup obtains the
> writelock and request handlers are obtaining a read lock.

You don't need to do this. Variables are not shared between perl
threads unless you make them shared. If you avoid this, your code
will work with the more common prefork MPM as well.

> I threw a sleep in the cleanup to see what happens, and it appears to delay
> sending the results of the next request.

It doesn't do that for me. Maybe you're running with only one
process/thread/interpreter?

- Perrin