LOGMUNCHER(5) File Formats Manual LOGMUNCHER(5) NAME logmuncher - configuration files for logmuncher DESCRIPTION Logmuncher(1) works by examining one or more configuration files that tell it how to detect system problems. This document describes the format of a configuration file. General Format Each configuration file consists of one or more lines, which are pro- cessed in order. Blank lines and lines beginning with "#" are ignored. Other lines contain a keyword and a single argument (usually a file- name), as specified below. Arguments may contain blanks. Relative pathnames are interpreted with respect to the configuration directory, either /usr/local/etc/logmuncher/conf or the directory specified with the -d switch. Many of the commands in the configuration file refer to "pattern" files. These files contain patterns to be fed to fgrep(1) or egrep(1). In all pattern files, all-blank lines and lines beginning with "#" are ignored. In a regular-expression pattern file (see below), a pattern beginning with "#" can be given by writing "[#]". Patterns beginning with "#" cannot be written in a fixed-pattern file; if a constant pat- tern beginning with "#" is needed, it must be placed in a regular- expression file. Percent Expansion Certain keywords in the configuration file support "percent expansion" in the manner of printf(3). This allows variable information to be dynamically inserted in their arguments. The following forms are rec- ognized: %C The name of the configuration file itself, with all directory information and extensions stripped. This allows identical con- figuration files (e.g., for multiple hosts logged onto one machine) to be maintained via symlinks. %c The name of the configuration file with directory information removed. %d The date logmuncher was invoked. %h The hostname of the machine that logmuncher is running on. %t The time logmuncher was invoked. %% A single percent sign. Keywords The logmuncher configuration file supports three general types of key- words. The first type (include) exercise high-level control over the processing of the configuration file. The second type (logfile, tail- file, mlogfile, logdir, and taildir) specify log files to be processed. If multiple files are selected (e.g., by specifying a keyword more than once), they are concatenated for further processing. The third type (all other keywords) tells what is to be done with the log information. Quite reasonably, no keywords of the second type can appear until at least one log file has been selected using a keyword of the first type. The following keywords are recognized by logmuncher in configuration files: include The argument gives the name of a configuration file to be inserted at this point. File inclusion may be nested arbitrar- ily (warning: self-recursive inclusion will fill your disk!). logfile The argument gives the name of a log file to be checked. The file will be scanned in its entirety for interesting informa- tion. Percent expansion is performed on the filename. mlogfile The arguments give the names of log files to be checked. Unlike logfile, shell wildcard notation is allowed. Percent expansion is performed before wildcard expansion. For convenience, this keyword will ignore any file whose name ends in ".offset" or ".gz". tailfile The argument gives the name of a log file to be checked. Only the part of each file that has changed since the last logmuncher run will be scanned for interesting information (using log- tail(8). Percent expansion is performed on the filename. mtailfile The arguments give the names of log files to be checked. Unlike tailfile, shell wildcard notation is allowed. Percent expansion is performed before wildcard expansion. For convenience, this keyword will ignore any file whose name ends in ".offset" or ".gz". logdir The argument gives the name of a directory tree to be searched for log files. Any file modified in the last 24 hours, and whose name does not end in ".offset" or ".gz", will be treated as if it had been supplied to a logfile directive. Percent expansion is performed on the directory name before the search is performed. WARNING: because of how the arguments to logdir are processed, it is possible for an attacker to execute arbitrary shell com- mands with the privileges of logmuncher by placing carefully crafted filenames into the directory tree. Do not use logdir or taildir on trees in which non-privileged users can cause files to be created. taildir The argument gives the name of a directory tree to be searched for log files. Any file modified in the last 24 hours, and whose name does not end in ".offset" or ".gz", will be treated as if it had been supplied to a tailfile directive. Percent expansion is performed on the directory name before the search is performed. WARNING: See the warning under logdir. fixed-ignore The name of a file containing fgrep(1) patterns to be removed from the log file. These patterns are matched in case-sensitive fashion. fixed-ignore-i As fixed-ignore, but case is ignored. fixed-report The name of a file containing fgrep(1) lines that should be kept in the log file. All other lines are removed. Matching is case-sensitive. fixed-report-i As fixed-report, but case is ignored. re-ignore The name of a file containing egrep(1) patterns to be removed from the log file. These patterns are matched in case-sensitive fashion. See below for egrep extensions. re-ignore-i As re-ignore, but case is ignored. See below for egrep exten- sions. re-report The name of a file containing egrep(1) lines that should be kept in the log file. All other lines are removed. Matching is case-sensitive. See below for egrep extensions. re-report-i As re-report, but case is ignored. See below for egrep exten- sions. filter The argument is evaluated as a shell command that is then used to filter the current log file. For example, filter egrep -v -i -f foo is approximately equivalent to re-ignore-i foo (except that the latter allows comments in the pattern file). WARNING: since the arguments to this command are passed to the shell eval built-in, be careful of shell metacharacters. subject Specifies a subject line to be put on any e-mailed reports. Percent expansion is performed on the subject line. If no sub- ject directive appears in the configuration file, a default sub- ject is constructed from the host name, date, time, and configu- ration file name. header Specifies a header line that should precede any reports sent via e-mail or to stdout. Percent expansion is performed on the header line. If no header directive is given, no header will precede the reports. If there are multiple header keywords, they will be combined to produce a multi-line header. After a report is sent with one of send-report, cat-report, or filter- report, a subsequent header command will reinitialize the header. save-partial-report Causes the current report to be saved for later sending. This is useful when only one e-mail message is desired for several reports. The results of all save-partial-report commands are concatenated to produce a final report when send-report is exe- cuted. Note that save-partial-report does NOT modify the cur- rent report in any way; usually it will be followed by a restore command and further processing. This command takes no argument. clear-partial-report Clears the current partial report, if any. The partial report is cleared only when this command is issued or a new configura- tion file is processed. This command takes no argument. send-report Causes the current report (if non-empty) to be e-mailed to the address named in the argument. If any non-empty partial reports have been saved with save-partial-report, they are prepended to the current report before sending. Multiple reports can be sent by repeating this line. If no argument is given, the report is sent to root or to the address given in the -t switch to log- muncher(1). If one of the pattern-matching commands above gen- erated output, and none of the *-report commands has been exe- cuted since that output was generated, a report will automati- cally be e-mailed to root (or the address named in the -t switch). cat-report Causes the current report, prepended with any partial reports, to be sent to logmuncher's standard output (if the concatenation is non-empty). If one of the pattern-matching commands above generated output, and none of the *-report commands has been executed since that output was generated, a report will automat- ically be e-mailed to root (or the address named in the -t switch). filter-report Causes the current report, prepended with any partial reports to be sent to the standard input of the shell command given by the arguments (if the concatenation is non-empty). If one of the pattern-matching commands above generated output, and none of the *-report commands has been executed since that output was generated, a report will automatically be e-mailed to root (or the address named in the -t switch). For example, filter-report mail -s "Security violations for $HOSTNAME $DATE $TIME" $SYSAD- MIN is roughly equivalent to writing send-report with a properly selected subject line. WARNING: since the arguments to this command are passed to the shell eval built-in, be careful of shell metacharacters. restore Causes the current set of messages to be reset to the original messages collected by all preceding logfile or tailfile com- mands, as modified by any subsequent replace-original commands. This is useful for generating reports at different levels from the same log file. This command takes no argument. replace-original Causes the current (presumably modified) set of messages to replace the original messages collected by the logfile or tail- file commands. This is useful for generating reports at differ- ent levels from the same log file. reset A deprecated synonym for replace-original. EGREP EXTENSIONS (This idea was suggested by Nicole King.) Logmuncher extends the normal egrep(1) syntax to simplify matching of patterns commonly found in log files. In addition to all the normal egrep(1) patterns, the following extensions are supported: [:digits:] Matches any string of digits; same as ([[:digit:]]+). [:xdigits:] Matches any string of hex digits; same as ([[:xdigit:]]+). [:ipaddr:] Matches any IPv4 or IPv6 address, as well as some other strings; same as ([:.[:xdigit:]]+). [:ipv4addr:] Matches any IPv4 address; same as (([[:digit]]{1,3}\.){3}[[digit]]{1,3}). [:hostname:] Matches common hostname patterns, as well as some other strings; same as ([-_.[:alnum:]]+). [:email:] Matches common e-mail addresses, as well as some other strings; same as ([-_.=+!%[:alnum:]]+(@[[:hostname:]]+)?). WARNING: This pattern does not match all legal RFC 2821 addresses, since RFC2821 specifies that the local-part of an address is com- pletely uninterpreted. However, it matches most addresses in common use. [:decimal:] Matches floating-point numbers with or without a fractional part; same as ([[:digit:]]+(\.[[:digit:]]+)?|\.[[:digit:]]+). [:pid:] Matches the square-bracketed PID commonly found in log messages; same as (\[[[:digit:]]+\]). [:month:] Matches English-language month names; same as (Jan(uary)?|Feb(ruary)?|Mar(ch)?|Apr(il)?|May|June?|July?|Aug(ust)?|Sep(tem- ber)?|Oct(ober)?|Nov(ember)?|Dec(ember)?). [:day:] Matches English-language day names; same as (Mon(day)?|Tue(sday?)|Wed(nes- day)?|Thu(rsday)?|Fri(day)?|Sat(urday)?|Sun(day)?) EXAMPLE The following example configuration file illustrates most of the fea- tures of a logmuncher configuration file. Two files are collected, and several reports are sent at varying security levels. # # Control file for finding trouble in /var/log/messages # and /var/log/mail # mtailfile /var/log/messages /var/log/mail # # Look for major hack attempts # re-report-i ../patterns/messages.hacking.re subject %h %d %t Active system attack! header ----- Attacks in /var/log/{messages,mail} ----- send-report # # Don't repeat the attack information in later reports. # We restore the original file and then remove the # messages reported above. # restore re-ignore-i ../patterns/messages.hacking.re reset # # Look for security violations. We first extract # suspicious things, and then suppress stuff we know is # OK. # re-report-i ../patterns/messages.violations.re re-ignore-i ../patterns/messages.violations.ignore.re subject %h %d %t Security violations send-report # # Again, get rid of stuff we've already reported # restore re-ignore-i ../patterns/messages.violations.re reset # # Report remaining "unusual" events. We get rid of # any stuff we know is OK, and report all the rest. # fixed-ignore-i ../patterns/messages.ignore.fixed re-ignore-i ../patterns/messages.ignore.re subject %h %d %t Unusual system events send-report FILES /usr/local/etc/logmuncher/conf Directory containing configuration files. /usr/local/etc/logmuncher/patterns Normal location of pattern files. SEE ALSO logmuncher(8), egrep(1), logtail(8) AUTHOR Geoff Kuenning (geoff@cs.hmc.edu) LOGMUNCHER(5)