[Israel.pm] content of big file on windows

Offer Kaye oferk at oren.co.il
Tue Mar 16 09:40:04 PST 2004


>
> Tie::File.
>
> --
> Gaal Yahas <gaal at forum2.org>
> http://gaal.livejournal.com/

While using Tie::File is a very elegant solution, when it comes to large
files it isn't very efficient.
Here are two solutions, one based on Tie::File and another based on an
advice I found on perlfaq5:

*************************************** <tie_solution>
***************************************
use strict;
use warnings;
use Tie::File;
use Fcntl 'O_RDONLY';

my @array;
my $bigfile = "/path/to/big/file";
my $count = 4 - 1; # 4 is "N" arg to tail, e.g. "tail -4 file"

tie @array, 'Tie::File', $bigfile, mode => O_RDONLY or die "Couldn't tie
$bigfile for reading: $!\n";
print $array[$#array - $_],"\n" for reverse 0 .. $count
*************************************** </tie_solution>
***************************************

*************************************** <other_solution>
***************************************
use strict;
use warnings;

my $count;
my $bigfile = "/path/to/big/file";
my $lines_from_end = 4;

open(IN,"$bigfile") or die "Couldn't open $bigfile for reading: $!\n";
$count += tr/\n/\n/ while sysread(IN, $_, 1048576);
close(IN) or die "Couldn't close big file after reading: $!\n";

open(IN,"$bigfile") or die "Couldn't open $bigfile for reading: $!\n";
while(<IN>){print if ($. > ($count - $lines_from_end))}
close(IN) or die "Couldn't close big file after reading: $!\n";
*************************************** </other_solution>
***************************************

62MB file  - other_solution finished in 1.3 secs, tie_solution finished in
32 secs.
213MB file - other_solution finished in 5 secs,   tie_solution finished in 2
minutes.

So I would advise against using Tie::File for this case (large input file).

Regards,
Offer Kaye




More information about the Perl mailing list