[Israel.pm] binary vectors representation

Shlomo Yona shlomo at cs.haifa.ac.il
Sun Jun 13 00:21:54 PDT 2004


On Sun, 13 Jun 2004, Offer Kaye wrote:

> Some possible approaches to compact and handle your bit vectors:
> 1. Use bzip2/gzip on your data file. The simplest and perhaps easiest
> approach.

That's probably very space efficient, but I'm not sure how
simple it is to implement.

> 2. Use http://search.cpan.org/~amruta/Sparse-0.02/

This is very nice. I'll look more deeply into it.

> 3. Turn data into a piddle (PDL data structure) and use
> http://search.cpan.org/~kwilliams/PDL-Sparse-0.01/
> (this might be your best bet- should be simple yet very quick and powerful).

This sounds like an overkill just to get a sparse vector...
Also, I don't know PDL... 

> 4. If @strings is very large, you might consider using a C library to read
> and handle the data, wrapped in Perl using SWIG for example. There are
> efficient libraries for specific architectures such as Intel's MKL, or you
> can try a non-specific library. If you decide to try this route let me know
> and I'll help if I can...

@strings is very large but no one forces me to read it all
at once into memory.

-- 
Shlomo Yona
shlomo at cs.haifa.ac.il
http://cs.haifa.ac.il/~shlomo/



More information about the Perl mailing list