grep, the search wonder

Lets start with a basic explanaition of a command we have not used in the previous tutorial, but is a very important command in this one: grep. The command is used in the next way: grep 'search pattern' filename

To take a closer look at what grep can do we are going to make a list with nicks, and flags on a channel that they have. On the shell type pico nicklist and enter the next text in that file:

Asmo flags=vpof
Revenger flags=vpof
tojoe flags=vpof
tween flags=vpof
nickmenza flags=vf
fubik flags=vf
Azmo flags=vf
TheScorp flags=vf

Exit pico and click yes to accept it saving the changes.

Now lets use the grep command on this list. We have some problems in the channel, becuase there seems to be two Asmo's, one is called Asmo, the other is Azmo, and they dont have the same flags. Now to use grep to quickly get both nick we have to look for a pattern that is unique to these two lines. In both cases the nick is almost the same, except for the second character. In grep expressions we can use a . meaning any letter or number. Type the next line on the shell and hit enter:
grep 'A.mo' nicklist

It will give the next list back:
Asmo flags=vpof
Azmo flags=vf

Now, if we would want to see all ops in the list, we can use grep with the next search pattern:

grep 'flags=.*o' nicklist

Which will give us the next list back:
Asmo flags=vpof
Revenger flags=vpof
tojoe flags=vpof
tween flags=vpof

To account for any characters between the flags= and the o (voice and/or partyline flags here) we use a . which means any character, and a * which means, any number of the previous character (the .).

Now if we would want to know how many voices or ops there are, we can use the -c option to count the found results, instead of displaying the actual lines.
grep -c 'flags=.*v' nicklist

Will give us back:
8

Ofcourse we could have used 'flags=v' as well as search pattern, since the v character would always directly follow the =.

Sofar for a quick introduction on this command. In the script we will be using the command more extensively, so if you want to experiment a bit more with grep, then go ahead. Type man grep on the shell to see more options and possibilities.

the eggdrop logfile

It's hopefully obvious to you already we would need a logfile to work on to begin with. The default location for the eggdrop bot to write the logfiles to is the logs directory in the eggdropbot's directory. The bot writes for a day to a file called eggdrop.log, after which this file will be renamed to eggdrop.log.yesterday and a new eggdrop.log will be made. The old eggdrop.log.yesterday will be removed then. Make sure you know the location of your logfile, and the name of it in case they are different from the ones I used in this tutorial. And be sure your logfile includes commands performed in the partyline, per MSG, private messages and CTCP's. If you decide to add more in this script, then obviously add those as well. Check the Eggdrop Config file tutorial if you need more help with this.
This whole script needs to be placed in the same directory as the logfile. If not, you also need to supply the path to the eggdrop.log where ever that file is mentioned in the script.

the Eggdrop Logfile Analyzer

Time to start with the actual scripting! I am going to assume you did the first bash scripting tutorial, so commands we explained there, I wont spent any more time in this tutorial. The green lines are the actual lines from the script. type pico eggdropAnalyzer in the directory where your bot writes his eggdrop.log file to startup the pico text editor and make the file eggdropAnalyzer.

#!/bin/bash
Location of bash, modify path if needed.
touch eggdropLogTemp
Here we are making the temporary file which will hold all the text we will mail out in the end.

echo "Subject: eggdropLog `date +%d-%B-%Y`" >> eggdropLogTemp
Here we echo the text Subject: eggdropLog `date +%d-%B-%Y`, and send it (with >>) to the temporary file. You might have recognised the date command captioned in `` there, as we did in the previous tutorial.
We use Subject: here, becuase using this, this will be the actual subject in the final email.

echo "eggdropLog `date +%d-%B-%Y`" >> eggdropLogTemp
This will be our first actual line of text in the email body. Again, send to append in the eggdropLogTemp file with the >>.

echo "" >> eggdropLogTemp
Add a empty line for clarity and readability :)

echo "Commands performed on the partyline:" >> eggdropLogTemp
First group of lines from the eggdrop log file we want to collect are the commands performed on the partyline. Ofcourse we add a line clearly pointing out in the email what they are ;)

grep '^\[.*\] #.*#' eggdrop.log >> eggdropLogTemp
Lets take a look at how those lines look like:
[09:52] #Asmo# +host asmo *!*asmo@.is.a.irc-junkie.org
Here the user with handle Asmo used the +host command to add a new hostmask to his record. We see a pattern here we can filter out with grep: the line always starts with [ ] holding the time, followed with a space, followed by a double # holding the nick.
So, translated into a grep expression we use:

^\[
^ means the line must start with the next character, in this case [. As [ is a special character for grep (it is used in ranges as well, use man grep on the shell for more information), we need to preceed it with a \, meaning this character needs to be taken litteraly.

.*\]
As explained before, .* means any number of any characters, which is sufficient to represent the displayed time between the [ ]'s. And we need to close again with \].

" "
Simply a space, just what it is.

#.*#
Two #'s which in between the .* again to represent the nick being between the two #'s.
And ofcourse the whole expression is encaptioned in ' '.

echo "" >> eggdropLogTemp
echo "Commands send in MSG:" >> eggdropLogTemp
grep '^\[.*\] (.*) !' eggdrop.log >> eggdropLogTemp
First two lines do what we saw before, first a empty line for clarity, next a line telling us what section is ahead. Now we are going to use grep to search up all commands send to the bot in /MSG. Lets see how such a line looks like in the logfile:
[09:51] (Asmo!asmo@www.irc-junkie.org) !Asmo! NOTES read ...
User Asmo used the NOTES READ ALL command here to pickup any messages left to him by others users on the bot. Other commands you might see often here are OP, IDENT etc.
As you see we have the brackets holding the time again, followed by the hostmask again, but this time enclosed by ( )'s, followed by the nick that is enclodes by !'s. Enough material to filter out all lines that have the commands in msg, lets take a look how:

^\[.*\]
Same as with the commands on the partyline. The ^ means the next character has to be the very first one on the line, which is [. The [ character is represented as \[, because the [ ] characters have a special meaning in grep.
Next we use .* to represent the time, followed by \] again for the ].

" "
A space.

(.*)
We use .* again for any string of characters that are inside ( )'s

" !"
A space again, followed by a !.

echo "" >> eggdropLogTemp
echo "Private messages to the bot:" >> eggdropLogTemp
Should be self explanatory by now! Private messages to the bot are next!

Now lets take a look at the next part:
if [ `grep -c '^\[.*\] \[.*\]' eggdrop.log` -gt 100 ]
then
echo "More then 100 messages, probably flooded" >> eggdropLogTemp
else
grep '^\[.*\] \[.*\]' eggdrop.log >> eggdropLogTemp
fi
Here we are going to do things with a extra step. In the case of heavy message floods we dont want this script to send us a email that is very large, but instead, send a warning that the bot is probally flooded with messages. Let me explain each step:

if [ `grep -c '^\[.*\] \[.*\]' eggdrop.log` -gt 100 ]
If [Number of private messages is greater then 100]. Note how grep is enclosed again in ` `, which does the command first. We use the -c option to give the number of hits found back in this case, instead of giving us the actual lines. We then compare it to 100 with the -gt (greater then)
then
Then do:
echo "More then 100 messages, probably flooded" >> eggdropLogTemp
only put in the email: "The amount of private messages in the logfile exceeds 100, probably flooded"
else
If the number of private messages is NOT over 100 , then do:
grep '^\[.*\] \[.*\]' eggdrop.log >> eggdropLogTemp
Do the grep command like explained above for the other sections.
fi
End of the if section.

echo "" >> eggdropLogTemp
echo "CTCP's to the bot:" >> eggdropLogTemp
This should be no problem nomore :)

if [ `grep -c '^\[.*\] CTCP' eggdrop.log` -gt 100 ]
then
echo "More then 100 CTCP's, probably flooded" >> eggdropLogTemp
else
grep '^\[.*\] CTCP' eggdrop.log >> eggdropLogTemp
fi
And the same story as with the private messages again. Here we are going to collect the CTCP lines. These can hold CTCP Ping, CTCP DCC Chat, CTCP Version, etc etc. And, as you probally know, abused a lot in floods. So we defenitally want to use a IF to see if we exceed 100 CTCP's.
We do it the same way as with private messages. We use the -c option in grep to see if the total lines that match our expression exceed 100. If it does, we just put the line "The amount of CTCP's in the logfile exceeds 100, probably flooded" in the email, if its below 100, then we perform the command grep like the other sections, and put the output in the temporary file.

less eggdropLogTemp | sendmail your@email.adress
We use the program less, a text file reader to read all the contents of our temporary file, and "pipe" them using the character | ( SHIFT \ ) to sendmail. With sendmail we can send the contents of our file to the email adress which we put behind the command.
In case sendmail does not work (bash: sendmail: command not found), try whereis sendmail to see where the program sendmail is installed. Usually this will be in /usr/sbin/sendmail Then simply supply this whole path instead of just "sendmail".

rm eggdropLogTemp
And finally, we remove the temporary file, since its not necessary nomore.

Thats it! Exit Pico and chmod +x eggdropAnalyzer to make the file executable. Type ./eggdropAnalyzer and pick up your mail to see if the script is working correctly.

Completing the setup

As you see, once you understand how the grep command works its fairly easy to add more, or setup analyzers for other programs. If you want to go fancy, you can even add the number of kicks, or for example joins a day now with the use of the -wc in grep.
Only one thing is left to do now, setting up crontab to run this script daily.
We want to run this script just before a new logfile is started. So you have to make sure you know what time the eggdrop switches logfiles. This is easy enough by reading a logfile, and looking at the end:
[03:00] Switching logfiles...
In this case I want crontab to run my eggdropAnalyzer at 2:59 each day. For more information on how to setup crontab read the previous bash scripting tutorial.

? Discuss This Article here.