Custom-made, luxury planet building.
Install Splunk Lite on CentOS 7

I decided to play with Splunk on the VPS.  Splunk is a huge and powerful tool, but my needs are simple.  I want to dump all my log files for all my servers somewhere.  I want the logs easily browsed and searched.  Kudos for automated alerts to key events.  Splunk Light may fit the bill. Time to play with it.

My install was onto an inexpensive linode virtual server, running CentOS 7.

Download and install Splunk Lite

Create an account with splunk.com

Splunk requires a registered account to download their software. Visit splunk.com and create an account.

Download Splunk Lite for Linux

Visit the Splunk Lite download page.  Select the linux download, and the 64-bit installer with the .RPM filename extension.  On the right side, you will see a link that says Got wget?  Get this url.  Click this, and copy the long url it provdes you, and paste it into your server terminal window.

``` {.EnlighterJSRAW data-enlighter-language="shell"} wget ...super log url from above... sudo rpm -i ./splunklight-[version]-linux-2.6-x86_64.rpm

This will install splunk into /opt/splunk. A user/group is created for
splunk to run under as an unprivileged user.

### Agree to the Splunk terms of service

Start the splunk service. Page through the terms and conditions, and
agree to the terms of service. Then shut it down for now.

``` sh
sudo /opt/splunk/bin/splunk start
sudo /opt/splunk/bin/splunk stop

Configure splunk as a system service

Create a systemd service control file as /etc/systemd/system/splunk.service

[Unit]
Description=Splunk
After=network.target

[Service]
RemainAfterExit=yes
ExecStart=/opt/splunk/bin/splunk start
ExecStop=/opt/splunk/bin/splunk/stop
ExecReload=/opt/splunk/bin/splunk restart

[Install]
WantedBy=multi-user.target

Start the splunk server, and enable loading on system reboot

sudo systemctl enable splunk
sudo systemctl start splunk

Make splunk accessible through your firewall

Allow splunk through your firewall

Splunk, by default, operates a web interface on the server at port 8000. You could open up this port in the fireall, and visit it in your web browser. CentOS 7 runs the new firewalld.

Create a firewall service definition file as /etc/firewalld/services/splunk.xml

<?xml version="1.0" encoding="utf-8"?>
<service>
  <short>splunk</short>
  <description>Splunk web console is hosted as a web service on port 8000</description>
  <port protocol="tcp" port="8000"/>
</service>

Enable the splunk firewall service and reload the firewall rules

sudo firewall-cmd --permanent --zone=public --add-service=splunk
sudo systemctl reload firewalld

(Alternative) Redirect splunk web server to a named virtualhost

I operate several websites from a single VPS. I don't want port 8000 on every domain or subdomain on this server to be open, serving up splunk for all to find. The apache webserver has a simple solution to this. The Reverse Proxy.

Ignore the previous subsection. We're going to leave port 8000 closed in our firewall, making it inaccessible to the outside world.

Update your dns to create a subdomain for splunk.

Use the appropriate control panel from your domain registrar, or your hosting company. Create a new DNS record for your server giving it a new subdomain. splunk.yourdomainname.com could be an easy choice.

Create an apache VirtualHost configuration

Ask apache to pass requests to your new subdomain onto the splunk server behind the firewall. This could happen in /etc/httpd/conf/httpd.conf like this:

<VirtualHost *:80>
  ServerName splunk.omgfurry.com
  CustomLog /var/log/httpd/splunk.omgfurry.com.log combined
  ErrorLog  /var/log/httpd/splunk.omgfurry.com.error.log

  ProxyPass "/" "http://localhost:8000/"
  ProxyPassReverse "/" "http://localhost:8000/"
</VirtualHost>

You may wish to only serve this site via SSL, but I'll not go into that here.

Always ask apache if you broke the config file:

[mcp@li6553-511 httpd]# sudo httpd -t
Syntax OK

Start the Splunk server and restart the web server.

sudo systemctl start splunk
sudo systemctl restart httpd

Log into and configure your splunk installation

If all the peices fit, point your browser at your splunk url. This could be http://yourhostname.com:8000/ or http://splunk.yourhostname.com/ depending on which route you took through your firewall.

What to do next:

  • Enable the *nix plugin.  This will configure many default options for monitoring server log files and health metrics.  more info
  • Add web server log files for indexing
    • Click the Add Data button.
    • Ask splunk to index /var/log/httpd/*log
  • Add server security log
    • Click the Add Data button
    • Ask splunk to index /var/log/secure
  • To monitor log files, the splunk user must have write access to those files.