[Israel.pm] Site Crawler

Yuval Yaari yuval at windax.com
Sun Nov 14 09:17:14 PST 2004


Guy Malachi wrote:

>Hey,
>Anybody have any tips on how I can create a site crawler that will
>extract all the links on a remote site and see if the site links to my
>site?
>Basically I have a list of urls that I want to check for each url if
>somewhere on the site (extracting all links and following onsite links
>recursively) there is a link to my site.
>
>Oh yea, it must run on Windows.
>  
>
Look at WWW::Mechanize (I managed to run it on Windows).
Shouldn't be too difficult. Good luck.

  --Yuval



More information about the Perl mailing list