Raspberry Pi Centralized Log Server

Setting up a Pi to be a centralized log store is amazingly simple. If you are using Raspbian it comes with rsyslog installed by default, so all that’s left is to setup the config and tailor log rotation.

First, you should create a directory under /var/log for all the remote logs.

sudo mkdir /var/log/central

Then edit the configuration for rsyslog, which can be found in /etc/rsyslog.conf
Find and uncomment the following line(s):

# provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514

# provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514

You may see a different set of configuration options on newer versions of rsyslog

# provides UDP syslog reception
module(load="imudp")
input(type="imudp" port="514")

#provides TCP syslog reception
#module(load="imtcp")
#input(type="imtcp" port="514")

Then create a template file under /etc/rsyslog.d/central.conf with the following contents:

$template RemoteLogs,"/var/log/central/%HOSTNAME%.log" 
*.*  ?RemoteLogs
& ~

The first line tells rsyslog to place incoming logs into their own file based on hostname. The RemoteLogs line is just a name for the template, it can be any name, the official documentation uses RemoteHost in the examples. You can use whatever makes sense. However, it should match the preceding line. The third line tells rsyslog to stop processing messages and not write anything to the local files.

For any configuration change to become active you will need to restart the daemon

service rsyslog restart

This will open port 514 on your machine. If you have a firewall configured, then adjust it accordingly.

If you do not set up log rotation, you will eventually have very large files and a full disk. This is easily remedied by configuring logrotate.

Create a configuration file so logrotate knows about your logs:

sudo vim /etc/logrotate.d/central

Add the configuration details for your environment. Here is an excerpt from my config file.

/var/log/central/pfsense.fakelabs.org.log
{
rotate 52
weekly
size 25M
missingok
notifempty
compress
delaycompress
sharedscripts
postrotate
  invoke-rc.d rsyslog rotate > /dev/null
endscript
}

/var/log/central/switch0.log
/var/log/central/fileserver.log
/var/log/central/logpi.log
{
rotate 12
monthly
size 10M
missingok
notifempty
compress
delaycompress
sharedscripts
postrotate
  invoke-rc.d rsyslog rotate > /dev/null
endscript
}

The script /etc/cron.daily/logrotate is run daily by cron, so no need to restart anything for changes to take effect. If you want to validate the changes, you can run the following command:

sudo logrotate --debug

This will not rotate your logs, but it will give you a sanity check of your configuration.

Once you are satisfied with the above, you are all done. Now you need to configure all your devices to send their logs to the Raspberry Pi.

If you are using syslog, edit /etc/syslog.conf and add the following at the end of the file:

*.* @log.fakelabs.local

if you have rsyslog, edit /etc/rsyslog.conf and append the following :

*.* @log.fakelabs.local
*.* @@log.fakelabs.local #TCP

if you have syslog-ng, edit /etc/syslog-ng.conf and insert the following:

# Collect system and internal logs
source src {
  system();
  internal();
};

# Where to send the logs and the transport
destination logbox {
  udp("1og.fakelabs.local" port(514));
};

# What to send and where
log {
  source(src);
  destination(logbox);
};

Change domain to your log server IP or domain name, then restart your log daemon.

After each host you configure, you should see the logs under /var/log/central on the Raspberry Pi.

NOTE:

I’m not encrypting the logs sent to the pi on my LAN. However, the logs are being encrypted from my network out to my remote Splunk server in the cloud. Maybe in another blog post, I’ll cover encrypted remote logging.

That’s all there is to it! Thanks for reading.