logmon configuration file schema.
---
do:
# Default action configuration inherited by all logfiles.
action: smtp:alice:password123@example.com
sender: alice@example.com
receivers:
- bob@example.com
logfiles:
# a simple list if no other configuration is needed
- "/var/log/service1.log"
- "/var/log/service2.log"
default:
# Default logfile configuration inherited by all logfiles.
default_error_pattern: '(?i)ERR(OR)?|CIRT(ICAL)?|EXCEPT(ION)?'
logfiles:
"/var/log/service3.log":
# It's one JSON document per line, not a plain text log.
json: true
# Handle JSON documents where the property `level` has
# the value `'ERROR'` or `'CRITICAL'`.
json_match:
level: [in, [ERROR, CRITICAL]]
# Path for the value of the {brief} template variable:
json_brief: [message]
do:
# Run multiple action, HTTP request and write matched entries to a file.
- action: https://api.example.com/v1/logs
http_method: POST
http_params:
subject: "{brief}"
entries: "{entries}"
http_content_type: JSON
oauth2_token_url: https://api.example.com/v1/oauth/token
oauth2_client_id: "23ca1cd3-a234-4719-883f-a6e509fc57f4"
oauth2_client_secret: "uBti6UENQnU0M1ZxM2IF0meGfovarZ5RRdzfdQe9pga/Vu5KK2vRFtlfcxP0ooMQftfUJeMOkl4Juoo+dXnwiA=="
oauth2_scope: [write_log]
- action: file:/var/logs/service3_errors.log
"systemd:SYSTEM:UNIT:cron.service":
do: "file:/var/log/cron_errors.log"
output_indent: null
output_format: JSON
"/var/log/service4.log":
do:
# The command line string is parsed into a `list[str]` before the template
# parameters are interpolated and run via `Popen(args=arg_list)`, it is not
# a shell string.
action: "command:my_command --brief={breif} --entry={...entries}"
command_env:
PATH: null # inherit $PATH
HOME: "/"
LOGMON_LOGFILE: "{logfile}" # same template variables
Default action configuration.
All actions inherit these settings if they don't overwrite them.
Email subject template.
Default: '{brief}'
Body template for the emails.
Template variables:
{entries} - All entries formatted with the output_format and output_indent options.{entries_str} - All entries for the message concatenated into a string with entries_delemeter between each (default is two newlines).{entries_raw} - Raw entries (list[str] for normal log files or list[dict] for SystemD or JSON log files).{logfile} - The path of the logfile.{entry1} - The first log entry of the message.{line1} - The first line of the first log entry.{brief} - Like {line1}, but with the entry start pattern removed.{entrynum} - The number of entries in this message.{sender} - The sender email address.{receivers} - Comma separated list of receiver email addresses.{receiver_list} - List of receiver email addresses (list[str]).{nl} - A newline character ('\n'){{ - A literal {}} - A literal }Default: '{logfile}\n\n{entries_str}'
Host to connect to for SMTP/IMAP/HTTP(S).
Default: 'localhost'
Port to connect to for SMTP/IMAP/HTTP(S) if not the standard port.
Value must be greater or equal to 0
Credentials for SMTP/IMAP, HTTP basic auth, or OAuth 2.0 password grant type.
Credentials for SMTP/IMAP, HTTP basic auth, or OAuth 2.0 password grant type.
secure option for SMTP/IMAP.
Default: null
Write messages to logmon's log instead of/in addition to performing the action.
Default: 'onerror'
Keep connection to server alive (SMTP, IMAP, HTTP(S)).
Default: false
Default: 'POST'
Default: '/'
Default: [('subject', '{subject}'), ('receivers', '{receivers}'), ('entries', '{entries_raw}')]
Each additional property must conform to the following schema
Type: stringMust contain a minimum of 2 items
Must contain a maximum of 2 items
Default: 'URL'
Additional HTTP headers. The Authorization header will be overwritten if OAuth 2.0 is used or if username and password are set.
Each additional property must conform to the following schema
Type: stringDefault: 0
Value must be greater or equal to 0
null means no timeout.
Default: null
Value must be greater or equal to 0.0
Default: 'client_credentials'
null means don't use OAuth 2.0.
Default: null
Seconds to substract from the expiration date-time when checking for access token expiration.
Default: 0.0
Command to run if action is 'COMMAND'.
The template parameters are the same as with body plus the special syntax {...entries}, which makes the argument repeat as a separate argument for each entry. E.g. if there are the entries 'foo' and 'bar' the argument list ['command', '--entry={...entries}'] will expand to ['command', '--entry=foo', '--entry=bar'].
Additional parameters:
{python} - Path of the Python binary used to execute logmon itself. (sys.executable){python_version} - Full vesrsion string of the Python binary. (sys.version){python_version_major} - sys.version_info.major.{python_version_minor} - sys.version_info.minor.{python_version_micro} - sys.version_info.micro.{env.NAME} - Environment variables of the logmon process. (os.getenv('NAME'))This string is split into the argument array using shlex.split().
[
"/path/to/command",
"--sender",
"{sender}",
"--receivers",
"{receivers}",
"--",
"{...entries}"
]
Working directory of spawned process.
Run the process as user/UID.
Run the process as group/GID.
setpgid() to apply for the sub-process.
If True use setsid() in the sub-process.
Default: False
setgroups() to apply for the sub-process.
Set the environment of the spawned process to this. Passing null as the value means to inherit that environment variable from the current environment. If this is unset the environment of the logmon process is inherited.
Each additional property must conform to the following schema
'file:/path/to/file', 'append:/path/to/file', 'inherit:', 'null:'
Default: 'null:'
'file:/path/to/file', 'append:/path/to/file', 'inherit:', 'null:', 'stdout:'
Default: 'null:'
If true the process is long-running and log entries are passed by writing them to the stdin of the process instead of command line arguments.
Default: false
Timeout in seconds. If the timeout expires the process is killed.
Default: null
Value must be greater or equal to 0.0
chroot() into the given path before the sub-process is executed.
Default: null
umask() to apply for the sub-process.
Value must be greater or equal to 0
nice() to apply for the sub-process.
Value must be greater or equal to 0
Encoding used to communicate with sub-process.
See: Python's encoding error handling
Default: 'replace'
Path of output file.
Default: 'UTF-8'
See: Python's encoding error handling
Default: 'replace'
Open file in append mode.
Default: true
Set owner of the file as user/UID.
Set owner of the file as group/GID.
Default: 'regular'
Create the file with these permissions. E.g.: rwxr-x---, u=rwx,g=rx,o=, or 0750.
Compress output file.
Default: null
Default: null
Indent JSON/YAML log entries in output. If null the JSON documents will be in a single line.
Default: 4
Value must be greater or equal to 0
Whole JSON document on a single line.
Use this format when writing JSON log entries to the output.
Default: 'YAML'
String used to delimite entries in {entries_str}.
Default: '\n\n'
Email sender address.
Default: logmon@<host>
List of email receiver addresses.
Default: <sender>
Action to perform.
This can also be a string in the form of one of these:
{smtp,imap,http,https}[:[//][<user>[:<password>]@]<host>[:<port>][/<path>[?<query>]]]
command[:<command> [<option>...]]
{file,fifo}[:<path>]
Parameters defined here overwrite values passed via other options.
For SMTP and IMAP these query parameters are supported:
senderreceiverssecureDefault: 'SMTP'
See root → do → anyOf → LogActionConfig → action for more details.
See root → do → anyOf → LogActionConfig → action for more details.
Default logfile configuration.
All logfiles inherit these settings if they don't overwrite them.
Default: '^\\[\\d\\d\\d\\d-\\d\\d-\\d\\d[T ]\\d\\d:\\d\\d:\\d\\d(?:\\.\\d+)?(?: ?(?:[-+]\\d\\d:?\\d\\d|Z))?\\]'
Python regular expression.
List of Python regular expressions that will be joined with | into a single expression.
Even if the error_pattern matches, if this pattern also matches the log entry is ignored.
Seconds to wait for more data if the line wasn't ended with a newline character.
Default: 0.04
Value must be greater or equal to 0
Seconds to wait before trying to re-open the file if it was not found and if inotify isn't used.
Default: 30
Value must be greater or equal to 0
Seconds to wait when there are no entries if inotify is not used.
Default: 5
Value must be greater or equal to 0
Seconds to wait for more messages before the action is performed.
Default: 0.08
Value must be greater or equal to 0
Seconds to wait after a logfile handler has crashed before it is restarted.
Default: 10
Value must be greater or equal to 0
Default: 20
Value must be greater or equal to 0
Default: 2048
Value must be greater or equal to 0
If the inotify package is available this defaults to true.
Seek to end of log file on open.
Default: true
If true parses each line of the log file as a JSON document. Empty lines and lines starting with // are skipped.
Default: false
JSON property paths and values to compare them to. A log entry will only be processed if all properties match. Per default all log entries are processed.
Operators:
= - equals!= - not equals< - less than> - greater than<= - less than or equal>= - greater than or equal~ - match regular expressionin - value in a list or range of given valuesnot in - value not in a list or range of given valuesThe argument to in and not in can be a list like ["foo", "bar"] or a range definition like {"start": 0, "stop": 10}. Start is inclusive, stop is exclusive.
Per default no filter is defined.
Each additional property must conform to the following schema
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Must contain a minimum of 2 items
Must contain a maximum of 2 items
"~"
Even if json_match matches, if this matches then the log entry is ignored.
Each additional property must conform to the following schema
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Must contain a minimum of 2 items
Must contain a maximum of 2 items
"~"
Use property at this path as the {brief} template variable.
Default: ['message']
Default: 'UTF-8'
See: Python's encoding error handling
Default: 'replace'
If true the last segment of a logfile path is a glob pattern. The rest of the path is just a normal path still. This way multiple logfiles can be processed at once and the directory is monitored for changes for when other matching files appear.
Default: false
Read compressed logfiles.
Default: null
Match log entries of this or higher priority.
Each additional property must conform to the following schema
Even if a log entry is matched via systemd_match, if it also matches via systemd_ignore it is ignored.
Default: null
The mapping keys or entries in the array of strings is the path of the log file.
You can read from a SystemD journal instead of a file by specifying a path in the form of:
systemd:[<open_flag>(+<open_flag>)*][:{UNIT,SYSLOG}:<identifier>]
Where open_flag can be one of:
LOCAL_ONLYRUNTIME_ONLYSYSTEMCURRENT_USERExamples:
systemd:
systemd:SYSTEM+LOCAL_ONLY:SYSLOG:sshd
systemd::UNIT:sshd.service
Mapping from logfiles to their configurations.
Each additional property must conform to the following schema
Default: '^\\[\\d\\d\\d\\d-\\d\\d-\\d\\d[T ]\\d\\d:\\d\\d:\\d\\d(?:\\.\\d+)?(?: ?(?:[-+]\\d\\d:?\\d\\d|Z))?\\]'
Even if the error_pattern matches, if this pattern also matches the log entry is ignored.
Seconds to wait for more data if the line wasn't ended with a newline character.
Default: 0.04
Value must be greater or equal to 0
Seconds to wait before trying to re-open the file if it was not found and if inotify isn't used.
Default: 30
Value must be greater or equal to 0
Seconds to wait when there are no entries if inotify is not used.
Default: 5
Value must be greater or equal to 0
Seconds to wait for more messages before the action is performed.
Default: 0.08
Value must be greater or equal to 0
Seconds to wait after a logfile handler has crashed before it is restarted.
Default: 10
Value must be greater or equal to 0
Default: 20
Value must be greater or equal to 0
Default: 2048
Value must be greater or equal to 0
If the inotify package is available this defaults to true.
Seek to end of log file on open.
Default: true
If true parses each line of the log file as a JSON document. Empty lines and lines starting with // are skipped.
Default: false
JSON property paths and values to compare them to. A log entry will only be processed if all properties match. Per default all log entries are processed.
Operators:
= - equals!= - not equals< - less than> - greater than<= - less than or equal>= - greater than or equal~ - match regular expressionin - value in a list or range of given valuesnot in - value not in a list or range of given valuesThe argument to in and not in can be a list like ["foo", "bar"] or a range definition like {"start": 0, "stop": 10}. Start is inclusive, stop is exclusive.
Per default no filter is defined.
Each additional property must conform to the following schema
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Must contain a minimum of 2 items
Must contain a maximum of 2 items
"~"
Even if json_match matches, if this matches then the log entry is ignored.
Each additional property must conform to the following schema
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Must contain a minimum of 2 items
Must contain a maximum of 2 items
"~"
Use property at this path as the {brief} template variable.
Default: ['message']
Default: 'UTF-8'
See: Python's encoding error handling
Default: 'replace'
If true the last segment of a logfile path is a glob pattern. The rest of the path is just a normal path still. This way multiple logfiles can be processed at once and the directory is monitored for changes for when other matching files appear.
Default: false
Read compressed logfiles.
Default: null
Match log entries of this or higher priority.
Even if a log entry is matched via systemd_match, if it also matches via systemd_ignore it is ignored.
Default: null
See root → do → anyOf → LogActionConfig → action for more details.
See root → do → anyOf → LogActionConfig → action for more details.
See root → do → anyOf → LogActionConfig → action for more details.
See root → do → anyOf → LogActionConfig → action for more details.
All the configuration is taken from the global settings.
No Additional ItemsMap of action limiters that can be assigned to actions. You can set a limiter to null to make it unlimited.
Default: {"default": {"max_actions_per_minute": 6, "max_actions_per_hour": 60}}
Each additional property must conform to the following schema
Default: 6
Value must be strictly greater than 0
Default: 60
Value must be strictly greater than 0
Log configuration of logmon itself.
Default: '[%(asctime)s] [%(process)d] %(levelname)s: %(message)s'
Default: '%Y-%m-%dT%H:%M:%S%z'
Write the process Id of the logmon process to this file.