[Israel.pm] Parallel jobs

Berler Chanan chananb at centerity.com
Tue Jun 9 02:05:15 PDT 2009


I return with my idea, of creating an array as queue for running process.
You can use this array to control the command: program & program &.... using
memory size / (memory needed - 1memory needed) = no. of threads can be ran
(ps I would have used: (memory needed - 1memory needed unit) so I won't
overload the memory).

PS: I never assume threads are an issue on perl. Can you use C/C++ to spin
the perl modules running ? 
    I know for sure that C/C++ supports threads quite easily

Chanan

-----Original Message-----
From: perl-bounces at perl.org.il [mailto:perl-bounces at perl.org.il] On Behalf
Of Issac Goldstand
Sent: Tuesday, June 09, 2009 9:16 AM
To: Perl in Israel
Subject: Re: [Israel.pm] Parallel jobs

Yeah, but if each system call is to the same thing, and that "thing"
happens to weigh, say, 25MB of RAM, and you want 50 of them, that's
gonna make problems.  The same overhead in a single multi-threaded
process might be 30MB total process size, vs >1GB with forked processes

Avishalom Shalit wrote:
> well,
> system calls to
> program & program & program &
> and waking occasionally to check the file system,
> can run it in a single thread.
>
>
> 2009/6/9 Issac Goldstand <margol at beamartyr.net>:
>   
>> Yossi's right.  Solaris's perl (and the one in Coolstack) come without
>> threads (Sun says they're still too experimental), unlike almost every
>> other vendor distribution which does have thread support by default.  We
>> ended up compiling our own Perl to get around this.
>>
>> Note that ActivePerl (IIRC) has thread support in Solaris
>> out-of-the-box.  (The reason we didn't use this is because we would have
>> had to compile Net::SSLeay ourselves, and we decided we'd rather compile
>> everything ourselves if we had to touch compilers already).  We ran
>> benchmarks, and IIRC our Perl actually came out ahead, though not by
much.
>>
>>  Yitzchak
>>
>> Yossi Itzkovich wrote:
>>     
>>>  The first one (Perl 5.8.4) is the one which comes with Solaris 10:
>>>     usethreads=undef use5005threads=undef useithreads=undef
usemultiplicity=undef
>>>
>>> The second one is one 5.8.8 compiled by our system group:
>>>     usethreads=undef use5005threads=undef useithreads=undef
usemultiplicity=undef
>>>
>>> Yossi
>>>
>>> -----Original Message-----
>>> From: perl-bounces at perl.org.il [mailto:perl-bounces at perl.org.il] On
Behalf Of Gabor Szabo
>>> Sent: Monday, June 08, 2009 9:24 PM
>>> To: Perl in Israel
>>> Subject: Re: [Israel.pm] Parallel jobs
>>>
>>> On Mon, Jun 8, 2009 at 9:22 PM, Yossi
>>> Itzkovich<Yossi.Itzkovich at ecitele.com> wrote:
>>>
>>>       
>>>> Well, you missed one very important point: Perl is compiled by default
**without** threads support. I don't/can't run a personal Perl build- I must
use the system's one.
>>>>
>>>>         
>>> which OS are you talking about and which version of perl?
>>>
>>> Gabor
>>> _______________________________________________
>>> Perl mailing list
>>> Perl at perl.org.il
>>> http://mail.perl.org.il/mailman/listinfo/perl
>>> _______________________________________________
>>> Perl mailing list
>>> Perl at perl.org.il
>>> http://mail.perl.org.il/mailman/listinfo/perl
>>>
>>>       
>> _______________________________________________
>> Perl mailing list
>> Perl at perl.org.il
>> http://mail.perl.org.il/mailman/listinfo/perl
>>
>>     
>
>
>
>   

_______________________________________________
Perl mailing list
Perl at perl.org.il
http://mail.perl.org.il/mailman/listinfo/perl
No virus found in this incoming message.
Checked by AVG - www.avg.com 
Version: 8.5.339 / Virus Database: 270.12.56/2161 - Release Date: 06/08/09
17:59:00



More information about the Perl mailing list