[Israel.pm] LWP and the Jewish/Hebrew question

Mikhael Goikhman migo at homemail.com
Mon Jun 28 03:10:16 PDT 2010

On 28 Jun 2010 12:21:37 +0300, Meir Guttman wrote:
> The LWP (Lib WWW for Perl) package seems to be very unfriendly or
> even right out hostile to Unicode, RTL text or just Hebrew.

I can't say that I can agree with this. :)

> There are many examples. For one: the HTML::TreeBuilder package
> ->Dump and ->as_HTML methods totally garble Hebrew UTF-8 text.

Maybe it is garbled by your text viewer or web browser when you
view unmarked text with mixed direction parts?

Without an example it is difficult to believe that HTML::TreeBuilder
does something special with Hebrew.

> But let's start first with a very simple Cookie_jar case which I
> first posted
> <http://www.mail-archive.com/libwww@perl.org/msg06768.html> on the
> libwww at perl.org mailing list (with no results!) The following code
> to the Google site suggested by the LWP originator and maintainer
> Gisle Aas worked fine:
> use LWP::UserAgent;
> use HTTP::Cookies;
> my $jar = HTTP::Cookies->new(file => "lwp.jar", autosave => 1);
> my $ua = LWP::UserAgent->new(cookie_jar => $jar);
> $ua->get("http://www.google.com")->dump;
> And indeed, it fills the 'lwp.jar' file with a few entries.
> But just changing the URL to http://www.magna.isa.gov.il
> <http://www.magna.isa.gov.il/>  produced a cookie jar file with just a
> comment headline and no entries at all.
> As you might see for yourself, the MAGNA site does send a Set-Cookie:
> ASP.NET_SessionId=ddkkv245c14tol45bgk35m45; path=/; HttpOnly line.
> So, in the spirit of the last week's events, why is the Israeli site
> discriminated against???

Maybe because this site defines such behaviour? :)

The cookie set by this page is transient (i.e. valid for one session
only), because it does not contain the "expires" field.

So HTTP::Cookies correctly does not store it in the persistent cookie
file "lwp.jar" specified by you.

But although temporary cookies are not stored in the persistent
cookie files, they still take effect. You can verify this by:

  print $jar->as_string;


perl -e 'print+chr(64+hex)for+split//,d9b815c07f9b8d1e'

More information about the Perl mailing list