Storable Freeze/thaw - Size Limits - 65535 (?)

When using Storable freeze/thaw to work with basic object (hash) storage
within a mysql db (blob column), I can't seem to get any more than 65535
bytes stored.

Since 65535 appears to be a "magic" number, I've been digging for where my
problem might reside (Storable, Perl, etc). but knowing many on this list
use Storable to do lots of cool things, can someone provide a clue?

Any other comments are much appreciated.

Thanks in advance,
Bill

Re: Storable Freeze/thaw - Size Limits - 65535 (?)

Bill,

It might be a MySQL limitation:

http://dev.mysql.com/doc/mysql/en/storage-requirements.html

If you're using a BLOB column type, you're limited to 2^16 bytes, which is
the barrier you're hitting. Try a MEDIUMBLOB column and see if that
helps.

--pete


On Sun, 18 Sep 2005, Bill Whillers wrote:

> When using Storable freeze/thaw to work with basic object (hash) storage
> within a mysql db (blob column), I can't seem to get any more than 65535
> bytes stored.
>
> Since 65535 appears to be a "magic" number, I've been digging for where my
> problem might reside (Storable, Perl, etc). but knowing many on this list
> use Storable to do lots of cool things, can someone provide a clue?
>
> Any other comments are much appreciated.
>
> Thanks in advance,
> Bill
>
>
>

----
Peter Leonard
pete@peteleonard.com

Re: Storable Freeze/thaw - Size Limits - 65535 (?)

On 9/18/05, Bill Whillers wrote:
> When using Storable freeze/thaw to work with basic object (hash) storage
> within a mysql db (blob column), I can't seem to get any more than 65535
> bytes stored.
>=20
> Since 65535 appears to be a "magic" number, I've been digging for where m=
y
> problem might reside (Storable, Perl, etc). but knowing many on this lis=
t
> use Storable to do lots of cool things, can someone provide a clue?

I'm going to guess that max_allowed_packet is set to 64K. Thereby
limiting the size of blob that you can read or set. See
http://dev.mysql.com/doc/mysql/en/blob.html for details.

If you need only a small amount more, you could compress the string in
memory before storing it and reverse that the other way. If you need
a lot more, then you're going to have to do some real work, sorry.

Cheers,
Ben

Re: Storable Freeze/thaw - Size Limits - 65535 (?)

On Sun, Sep 18, 2005 at 10:43:09PM -0700, Bill Whillers wrote:
> When using Storable freeze/thaw to work with basic object (hash) storage
> within a mysql db (blob column), I can't seem to get any more than 65535
> bytes stored.

You touched the limit of a BLOB, which is limited to 2^16 bytes. Change
your column type to MEDIUMBLOB (2^24) or LONGBLOB (2^32). Got the same
problem a couple of weeks ago and searched myself crazy. A simple
solution though ;-)

HTH/Frank

Re: Storable Freeze/thaw - Size Limits - 65535 (?)

I should have had "sanity check" in my subject -- This was exactly the
problem.. my first asumption a couple hours ago was 2^24 ... not 16 -- (wrong
table) -- Grief!!

Thanks _alot_ guys.





On Sunday 18 September 2005 22:45, you wrote:
> Bill,
>
> It might be a MySQL limitation:
>
> http://dev.mysql.com/doc/mysql/en/storage-requirements.html
>
> If you're using a BLOB column type, you're limited to 2^16 bytes, which is
> the barrier you're hitting. Try a MEDIUMBLOB column and see if that
> helps.
>
> --pete
>
> On Sun, 18 Sep 2005, Bill Whillers wrote:
> > When using Storable freeze/thaw to work with basic object (hash) storage
> > within a mysql db (blob column), I can't seem to get any more than 65535
> > bytes stored.
> >
> > Since 65535 appears to be a "magic" number, I've been digging for where
> > my problem might reside (Storable, Perl, etc). but knowing many on this
> > list use Storable to do lots of cool things, can someone provide a clue?
> >
> > Any other comments are much appreciated.
> >
> > Thanks in advance,
> > Bill
>
> ----
> Peter Leonard
> pete@peteleonard.com