Perl: Read the last 10 lines of a file

[meikel]

Hi,

I just need a fast routine to read the last few lines from a large textfile.

A good example would be to read the last 10 lines from the current logfile which can be many MB.

I just don't want to scan the whole file to just get the last few lines. Is there an easy way to do this?

Greetings from Bonn, Germany
[nbsp][nbsp]Meikel Weber
www.meikel.com (http://www.meikel.com)

[Del]

Meikel,

Use the tail program. Give me a couple minutes and I'll find my notes on how to do it :)

<edit>
This should do it

open (LOG, &quot;tail -10[nbsp][nbsp]/path/to/log.txt | tail -10 |&quot;) || die $!;
print &quot;<PRE><FONT SIZE=-1>
while (<LOG>) {
[nbsp][nbsp]print &quot;$_\n&quot;;
}
close (LOG);
print &quot;</FONT></PRE>\n&quot;;

</edit>

<edit2>
Whoops, guess I'll use the 'disable smileys' box
</edit2>

Del

(try man tail to see what it does)<!-- NO_AUTO_LINK -->
[This message has been edited by Del (edited 05-05-99)]

[meikel]

Hi Del,

thanks for this solution. I'm not sure if this is exactly what I need. I have to get some idea on the tail program and how it works.

I need this routine for some speed and memory things. I don't know if opening a new process that runs tail twice helps a lot. I have to test this a bit more. Maybe I'll play with binary file access from perl a bit :)

Greetings from Bonn, Germany
[nbsp][nbsp]Meikel Weber
www.meikel.com (http://www.meikel.com)

[Del]

Meikel,

I've never used it for anything, just had that script kicking around in my zip disk (for some reason or another, I'm sure I had a reason when I kept it)


Maybe I'll play with binary file access from perl a bit


If you need to tail a binary file, the Perl Cookbook has a program to pull the tail off a binary log file (since tail won't work on binary stuff). If that's what you're going for, I imagine the prog is available at perl.com or oreilly.com (maybe). If not, I can post it here for you if you want it. Or, if I'm misreading what you're saying, then slap me and I'll shut up.

Del

[meikel]

Hi Del,

The problem is defined in the first post. It's just ascii files (not binary). But as a programmer, I have the feeling that there must be an easy solution. And starting a new child process usually is not an &quot;easy&quot; solution.

Binary access means:
Reading the last 2KB and doing a split at Character #13 to find the last few lines. So it's much easier to find the last 10 lines in 2KB than in 2MB.

But usually that involves knowing the original data structure (to tell that 2KB is at least covering the last 10 lines).

I'll post here if I find an easy way. Or again, if someone did that already: You could save me some programming work :)

Greetings from a party guy
[nbsp][nbsp]Meikel Weber
www.tickerland.com (http://www.tickerland.com)

[Justin]

Hey party guy,

Since we're on Linux, you might want to look for chr #10 (line feed) as opposed to 13 (carriage return). Won't find any of those unless you uploaded a text file in binary - and of course the log files are created by Linux so chr #13 won't be there :)

I recently wrote a text editor (MDI, supports large files, etc) that can convert between Unix and Windows/DOS formats and save as either one too (so really when my site says created in Notepad, it's not really true - should be created in ScratchPad :))

Ok, way past my bed time (huh?) so me go now :)

------------------
Justin Nelson
FutureQuest Support

[Terra]

Simple problem, difficult to implement in pure perl code...

My honest advice, stick with a 'tail -10 log' and read the resultant output...

I once tried to duplicate the Linux tail function in perl code and tossed it out the window when the complexity far outweighed the convenience of a self-piped child read of the forked tail...

It's a deceptively difficult little utility to emulate... ;)

--
Terra
--Tail looks simple eh?[nbsp][nbsp]Muhahahahaha-
FutureQuest

PS: One more kewl tool - 'tac'

[Terra]

Del,

Why are you using 2 tails (piping 1 to another)???

The first one 'tail -10 /blah/log' will do the trick...

--
Terra

[Del]

Actually, I'm not :) That's just a file I had kicking around on my zip disk, don't even remember why I saved it. It just happened to be the only reference I could find that mentioned the tail program...

Nine flippin Perl books sitting at arm's lenght, not one of em had something like that in it. Closest I got was in the Cookbook, 'how to trail a growing logfile' and one to emulate the tail program but on a binary file. The 'trail growing logfile' wasn't it either, it would hit EOF, sleep, clear EOF and read some more.

For the life of me I can't remember where I got that program now though. It's been sitting on one drive or another since my days of free hosting at VR9 *shudder*

Del

[meikel]

Hi again,

thanks for all that insight. Of course, the problem is not that easy. At the moment it's just &quot;get the last 10 or 50 lines&quot;. But I have the feeling that a nice and fast routine would enable me to implement much better programs.

If I'm capable of nearly targetting &quot;any&quot; line in a large ascii file, I could implement a binary search for some specific data-lines (if the file is sorted) or things like this.

Unfortunately I don't have much time in the very near future to play around with some ideas on this.

Greetings from Bonn, Germany
[nbsp][nbsp]Meikel Weber
www.meikel.com (http://www.meikel.com)