Dr. StrangeLog or: How I Learned to Stop Worrying and Love the SmartLog

To make amends for not posting all that time I’m writing the next post right away:

I love SmartLog. Its quick, its (really!) dirty, and it optimized our daily work by at least 100%!

Smartview Tracker is probably still one of the best Firewall Built-In Log-Viewer out there but it always left me aching for more functionality. Also everyone out there who knows that Checkpoint switches logs at 2GB by default will know how slow searches become the farther the day progresses.

So Checkpoint introduced SmartLog with R75.40 and I LOVE it! I guess everyone knows how the GUI looks like and how great the performance is, however Checkpoints documentation on it is really horrible.

Sadly I had to learn this quite soon after Upgrading to R75.40. SmartLog seems to have quite a lot of issues. For example it will stop to show new logs every now and then without any apparent reason nor a helpfull error message in the log you could base troubleshooting on.

So I want to use this post to summarize everything I (had to) learned about SmartLog so far:

Lets begin with some Basic:

• SmartLog resides under /opt/CPSmartLog-R75.40VS (path varies with further versions of course)
• As usual the path has an environment variable: $SMARTLOGDIR
• In $SMARTLOGDIR/log you will find a logfile called “smartlog_server.elg” which focusses mostly indexing operation but also logs the search queries users type in
• $SMARTLOGDIR/conf contains the main configuration file for SmartLog: smart log_settings.txt
• SmartLog will index your regular FW Logs from $FWDIR/log but will create its own Index files under $SMARTLOGDIR/data
• SmartLog IS NOT a forensical Log-Sink that will represent 100% of all Logfiles your firewalls produce! It even purges logfiles after a given time or HDD space limit!
• SmartLog seems to have glitches sometimes that prevent it to index a freshly switched fw.log. A manual logswitch triggered in Smartview Tracker fixed this for me  a couple of times now (remember the dirty part?).

Now lets focus on the Config file ($SMARTLOGDIR/conf/smartlog_settings.txt):

The config file comes in a familiar checkpoint config file structure (e.g. comparable to objects_5_0.c file). Some usefull and interesting values are:

• :min_disk_space (100240) – represents the minimal  disk space SmartLog should keep free thus shrinking the Index when this limit is reached which results in logs being dropped from the SmartLog index and you will not be able to search for them anymore. However it will leave your original Logfiles untouched!
  
• :max_index_size (250000) – is the limit the SmartLog index will grow to. And if you have a couple of months worth of Logfiles it will grab all Performance the Management Station can give it and will do exactly do that build and Index up to the size specified here! Now slow growing but reindexing all you old logfiles instead, which is great as this gives you a lot of backlog you can search through in an instant.
• :tops_num_limit (1000) – specifies how many rows of log a search query will return initially. More results are being pulled up when you scroll down. Mind that if you have a big screen and and the initial query returns less rows than your SmartLog GUI windows can display you wont get a scrollbar. Resize the SmartLog GUI window to get the scrollbar to get to older logs by mouse.
• :tops_time_limit (20) – will limit the time the SmartLog GUI will spend on fetching the initial search query results. Should only play a role if you have a weak management station or search for really old log entries in a big Index
• :field_synonyms – If for some reason you are annoyed by the search operators you can add your own synonyms here (e.g. “:from (source)” creates the operator “from” which can be used instead of “src” or “source”)
• :num_days_restriction_for_fetch_all_integrated (3) – speciefies how far back the indexer goes to index logfiles. In this example the indexer will only index logfiels that are not older than 3 days. I guess this setting is only interesting afert and upgrade when you want to index old logfiles.
• There are a couple of more settings in there which I had no need to tweak until now, but just take a look and get familiar with the file…
• If you change settings in the conf file use “smartlogstop;smartlogstart” to restart SmartLog. Keep in mind that the GUI will close for all Administrators!

By now there are also a couple of SK articles about SmartLog so just search the Checkpoint KB for “smartlog” and read through them!

Also check out this blog: http://dreezman.wordpress.com. There are some interesting SmartLog Posts over there.

I hope this can help someone out there who is new to SmartLog. Personally I think this and way more should be in Checkpoints SmartLog documentation!

Regards
Sebastian