[Israel.pm] Scraping data via Perl from ASP websites

Yuval Kogman nothingmuch at woobling.org
Thu Aug 28 12:42:40 PDT 2008

On Thu, Aug 28, 2008 at 04:08:17 -0700, Yossi Klein wrote:
> I have had much success scraping data from Perl websites using the
> HTTP modules (HTTP:Request, HTTP::Response, etc.). However, I now
> have a need to use Perl to scrape data off of ASP sites and am not
> having much success.

Did you try WWW::Mechanize? Web::Scraper? They are higher level
wrappers over LWP, maybe they can take care of whatever weird
redirection is happenning.

Also, I can't believe i'm saying this, but Evan Carroll's module
sounds related:


He's sometimes difficult to deal with so take this with a grain of

Apparently some ASP.NET pages rely on javascript to function

  Yuval Kogman <nothingmuch at woobling.org>
http://nothingmuch.woobling.org  0xEBD27418

More information about the Perl mailing list