Rsyslog¶

The rsyslog Collector binary is an executable that generates Tornado Events from rsyslog inputs.

This Collector is meant to be integrated with rsyslog’s own logging through the omprog module. Consequently, it is never started manually, but instead will be started, and managed, directly by rsyslog itself.

Here is an example rsyslog configuration template that pipes logs to the rsyslog-collector (the parameters are explained in the configuration file tornado_rsyslog_collector/conf/rsyslog_collector.toml) only logs with severity higher than warning:

module(load="omprog")

action(type="omprog"
       binary="/usr/lib64/tornado/bin/tornado_rsyslog_collector --logger=warn")

An example of a fully instantiated startup setup is:

module(load="omprog")

action(type="omprog"
       binary="/usr/lib64/tornado/bin/tornado_rsyslog_collector --config-dir=/tornado-rsyslog-collector/config --tornado-event-socket-ip=192.168.123.12 --tornado-event-socket-port=4747")

Note that all parameters for the binary option must be on the same line. You will need to place this configuration in a file in your rsyslog directory, for instance:

/etc/rsyslog.d/tornado.conf

In this example the Collector will:

Reads the configuration from the /tornado-rsyslog-collector/config directory
Write outgoing Events to the TCP socket at tornado_server_ip:4747

The Collector will need to be run in parallel with the Tornado Engine before any events will be processed, for example:

/opt/tornado/bin/tornado --tornado-event-socket-ip=tornado_server_ip

Under this configuration, rsyslog is in charge of starting the Collector when needed and piping the incoming logs to it. As the last stage, the Tornado Events generated by the Collector are forwarded to the Tornado Engine’s TCP socket.

This integration strategy is the best option for supporting high performance given massive amounts of log data.

Because the Collector expects the input to be in JSON format, rsyslog should be pre-configured to properly pipe its inputs in this form.