Searching the logs?

[Kara]

I would like to get some info from today's log. Seeing as I connect at basically a snail's pace, is there anyway that I can search it through telnet or something? I know I could wait till tomorrow when it is zipped up, but I'm impatient and want the info now. If I downloaded the today file, right now it would be 10-15 MB and that might take a few hours and with my luck my ISP would kick me off when it's almost done.

I'm not a telnet pro, but I can follow directions well.

Kara

[Arthur]

Kara, it depends on what you want to look for, but you can use "grep" and "more" to search through the log.
To look for a referrer from let's say Korea ;) you could do something like;
grep "somedomain.kr" access.today | more

The "more" is so you can look at it page by page (press space to advance, q to quit, b to go back).
There's so many possibilities, without some more information about what you're searching for, it's too much to list. You can of course copy the log from today to another directory, do "gzip access.today" in that other directory and then download, if you don't want to wait for the stats-run.

Arthur

[Justin]

Sometimes less is more...

The 'less' command is similar to the 'more' command, but allows you to scroll up, and much more... it's my preferred paginator anyway.

'grep' is your friend; type 'man grep' for a TON of options. Though it's typically used for regex matching, it's also fine for searching plain text strings as well -- provided no special characters are used...

Less is most useful when you have a good telnet program (eg, where PGUP/PGDN etc work properly)... but there are keyboard shortcuts as well (I think 'h' is home, etc... 'man less' for more... er... you know what I mean)

[Kara]

I want to check the activity of a certain IP number. So I could type
grep 216.12.123.123 access.today | less (or more) and it would go to the first occurance of that IP number?

Is a good telnet client the one that comes with win98 or is the one in CNC better?
Justin when you say special characters you mean things like &^% right? You should teach a class in this. You could prolly make a few extra bucks :QTmoney:

Kara

[manish]

Here are some things you can play around with:
cat, zcat, grep, cut, sort, uniq, less, more, wc

I use zcat on old gzipped files. grep is used to search for some content in the log. I use cut to extract the first field which is the IP address. sort is used for sorting. uniq will get rid of duplicate entries after sorting. Less acts like your paging tool. wc can count the number of lines. You can also redirect your output to a file using ">" (overwrite) or ">>" (append)

cat access.today | grep <filename> | cut -f-1 -d " " | sort | uniq | less
will give you all the ip addresses that accessed <filename>

cat access.today | grep <filename> | cut -f-1 -d " " | sort | uniq | wc -l
will count the number of people who accessed filename.

Here's a small script

date > ~/report.txt
echo "Number of entries:" >> ~/report.txt
cat access.today | grep <filename> | cut -f-1 -d " " | sort | uniq | wc -l >> ~/report.txt
cat access.today | grep <filename> | cut -f-1 -d " " | sort | uniq >> ~/report.txt


You can play around with pipes and other commands. Don't you love the flexibility you get with Unix :)

Manish

[Arthur]

Originally posted by Kara:
I want to check the activity of a certain IP number.
Try this;
grep 123.123.123.123 access.today | awk '{print $7}' | sort | uniq -i -c | sort -r | less
That gets you all the files accessed by that IP sorted and ranked by number of accesses. By changing "$7" you can get different fields in the log, e.g. "$11", gets the referrer field.
If you use "zgrep" instead of "grep" you can grep from gzipped files (e.g. access.20011029.gz).

Is a good telnet client the one that comes with win98 or is the one in CNC better? The one that comes with Win98 isn't too bad.
It's not much different from the one in the CNC I think.