[Israel.pm] Parallel jobs

Avishalom Shalit avishalom at gmail.com
Mon Jun 8 07:15:27 PDT 2009


ok, how's this then.

spawn the threads right at the beginning ,
each thread runs an individual process, and dies when finished.
(each process can  call the same external perl script that does not
need to be thread safe.)
so  the experts will correct me if i am wrong, but i don't  think that
anything encapsulated in that thread needs to be compatible with
threads.  so no problem there. no safety problems either.

so, you get all your answers in the original main thread, ta-da

still , i may have done it with running different processes and using
the file system as a flagging mechanism for telling the main program
that wakes up once in a while when things are done and a report can be
generated.

works, but ugly as a cow.
2009/6/8 Yossi Itzkovich <Yossi.Itzkovich at ecitele.com>:
> Because I want to collect the results , send a nice report etc.
>
>
> -----Original Message-----
> From: perl-bounces at perl.org.il [mailto:perl-bounces at perl.org.il] On Behalf Of Avishalom Shalit
> Sent: Monday, June 08, 2009 3:55 PM
> To: Perl in Israel
> Subject: Re: [Israel.pm] Parallel jobs
>
> why does it have to be in the same program , if they don't communicate much?
> i mean, why not run a script to invoke many instances of the program
> to the stations ?
>
>
> 2009/6/8 Yossi Itzkovich <Yossi.Itzkovich at ecitele.com>:
>> Thanks,
>>
>> I need them basically to run (more or less unrelated ) tasks in parallel.  For example: Connect to dozens of stations and run there a long operation.  I don't want to wait long time by doing them sequentially.
>>
>> I prefer threads, but I wrote in the original mail why I can't.
>>
>>
>> Yossi
>>
>> -----Original Message-----
>> From: perl-bounces at perl.org.il [mailto:perl-bounces at perl.org.il] On Behalf Of Avishalom Shalit
>> Sent: Monday, June 08, 2009 3:01 PM
>> To: Perl in Israel
>> Subject: Re: [Israel.pm] Parallel jobs
>>
>> resending
>>
>> 2009/6/8 Avishalom Shalit <avishalom at gmail.com>
>>>
>>> i had used pearl threads,
>>> it isn't too complicated and most things i needed were threadsafe.
>>> however , they are not lightweight at all,
>>> they are masked forks, and like forks, you really want to spawn them as early as possible .
>>>
>>> what do you need them for ?
>>>
>>> 2009/6/8 Yossi Itzkovich <Yossi.Itzkovich at ecitele.com>
>>>>
>>>> Hi,
>>>>
>>>> I would like to ask how you people work with parallel jobs (processes/threads) in Perl (>=5.8.4).
>>>> I know there are simple built in commands (open, fork etc.) but I am asking about higher order managers.
>>>> Threads might be good - but not enabled by default in Perl build -so I am limited (also not all modules are thread safe).
>>>> I have been using Parallel::Jobs for a while, and I read a little about Parallel::Forker, but I wanted to know if there are better alternatives out there.
>>>>
>>>>
>>>> Thanks for your help.
>>>>
>>>> Yossi
>>>> _______________________________________________
>>>> Perl mailing list
>>>> Perl at perl.org.il
>>>> http://mail.perl.org.il/mailman/listinfo/perl
>>>
>>>
>>>
>>> --
>>> -- vish
>>
>>
>>
>> --
>> -- vish
>> _______________________________________________
>> Perl mailing list
>> Perl at perl.org.il
>> http://mail.perl.org.il/mailman/listinfo/perl
>> _______________________________________________
>> Perl mailing list
>> Perl at perl.org.il
>> http://mail.perl.org.il/mailman/listinfo/perl
>>
>
>
>
> --
> -- vish
> _______________________________________________
> Perl mailing list
> Perl at perl.org.il
> http://mail.perl.org.il/mailman/listinfo/perl
> _______________________________________________
> Perl mailing list
> Perl at perl.org.il
> http://mail.perl.org.il/mailman/listinfo/perl



-- 
-- vish


More information about the Perl mailing list