Hi,

Does anyone happen to know if there's a convenient module which will analyse
at least two XML files and list the most frequently-used words?

(It would have to be able to reject tags and certain words such as "the" and
"is")

"DVH" <dvh [at] dvhdvhdvhdvdh.dvh> wrote:

> Hi,
>
> Does anyone happen to know if there's a convenient module which will
> analyse at least two XML files and list the most frequently-used
> words?
>
> (It would have to be able to reject tags

XML::Parser, which a Char handler.

> and certain words such as
> "the" and "is")

split in the handler on non-words, use a hash for counting. Delete
afterwards all occurences of the, is, etc.

Note that this is a very simplistic approach, since it words are hypenated,
it counts them as two different ones.

--
John Small Perl scripts: http://johnbokma.com/perl/
Perl programmer available: http://castleamber.com/
I ploink googlegroups.com :-)

John Bokma <john [at] castleamber.com> writes:

> "DVH" <dvh [at] dvhdvhdvhdvdh.dvh> wrote:
>
>> Hi,
>>
>> Does anyone happen to know if there's a convenient module which will
>> analyse at least two XML files and list the most frequently-used
>> words?
>>
>> (It would have to be able to reject tags
>
> XML::Parser, which a Char handler.
>
>> and certain words such as
>> "the" and "is")
>
> split in the handler on non-words, use a hash for counting. Delete
> afterwards all occurences of the, is, etc.
>
> Note that this is a very simplistic approach, since it words are hypenated,
> it counts them as two different ones.

Seaching for "word frequency" on search.cpan.org turns up some modules
that are designed for this sort of thing, and may take some of the
trickier issues into account.

----Scott.

"DVH" <dvh [at] dvhdvhdvhdvdh.dvh> wrote:

> Hi,
>
> Does anyone happen to know if there's a convenient module which will
> analyse at least two XML files and list the most frequently-used
> words?
>
> (It would have to be able to reject tags

XML::Parser, which a Char handler.

> and certain words such as
> "the" and "is")

split in the handler on non-words, use a hash for counting. Delete
afterwards all occurences of the, is, etc.

Note that this is a very simplistic approach, since it words are hypenated,
it counts them as two different ones.

--
John Small Perl scripts: http://johnbokma.com/perl/
Perl programmer available: http://castleamber.com/
I ploink googlegroups.com :-)

John Bokma <john [at] castleamber.com> writes:

> "DVH" <dvh [at] dvhdvhdvhdvdh.dvh> wrote:
>
>> Hi,
>>
>> Does anyone happen to know if there's a convenient module which will
>> analyse at least two XML files and list the most frequently-used
>> words?
>>
>> (It would have to be able to reject tags
>
> XML::Parser, which a Char handler.
>
>> and certain words such as
>> "the" and "is")
>
> split in the handler on non-words, use a hash for counting. Delete
> afterwards all occurences of the, is, etc.
>
> Note that this is a very simplistic approach, since it words are hypenated,
> it counts them as two different ones.

Seaching for "word frequency" on search.cpan.org turns up some modules
that are designed for this sort of thing, and may take some of the
trickier issues into account.

----Scott.

On Tue, 25 Oct 2005 01:30:58 +1000, Scott W Gifford wrote:

Hi Folks

A list of stop word, courtesy of MySQL, can be downloaded from:

http://savage.net.au/Ron/mysql-stop-words.txt

On Tue, 25 Oct 2005 01:30:58 +1000, Scott W Gifford wrote:

Hi Folks

A list of stop word, courtesy of MySQL, can be downloaded from:

http://savage.net.au/Ron/mysql-stop-words.txt