[Israel.pm] how much memory each Perl variable uses

Amir E. Aharoni amir.aharoni at mail.huji.ac.il
Wed Sep 9 03:46:02 PDT 2009


Hello,

I am using Parse::MediaWikiDump to analyze a static XML dump of
Wikipedia. I am reading every page and saving some interesting info
about it to a text file. (If it's relevant, i'm running Cygwin Perl
5.10 on Windows XP.)

The memory usage of this program keeps growing quite quickly as it
progresses through the dump, even though i believe that i am not
aggregating any info in variables - only in files. Of course, i might
be wrong - it is possible that i am aggregating something without
noticing (There are 1,600 lines of code, and maybe i am missing some
rogue variable...). And maybe Perl's garbage collector isn't doing its
job. And maybe some internal variable in Parse::MediaWikiDump is
aggregating data.

I can start sprinkling Devel::Size::size() calls around the code, but
that would be rather annoying, because if i understand correctly it
only works per variable, which means that i'll have to write such a
line for every variable, and i have a lot of them, not to mention the
variables in the external module.

Is there any convenient tool which can produce a detailed list of all
the memory usage of a Perl program during runtime?

I also tried periodically checking the value of Devel::Leak::NoteSV(),
and it indeed keeps growing, but i don't really what can i do with it.

Thanks in advance for any help.

If anyone is curious, the code is available at
http://perlwikibot.svn.sourceforge.net/viewvc/perlwikibot/trunk/no-interwiki/

(The relevant file name is prepare_noiw_list.pl.)

(N.B.: I posted the same question at
http://www.perlmonks.org/?node_id=794209 but the answers there have
been slow, so i am trying my luck here. You are certainly welcome to
reply at PerlMonks.)

--
אמיר אלישע אהרוני
Amir Elisha Aharoni

http://aharoni.wordpress.com

"We're living in pieces,
I want to live in peace." - T. Moore


More information about the Perl mailing list