Security Information and Event Management (SIEM) with Splunk

Security and data privacy have been always important, but given the significant breaches that have taken place over the last two years, they’re growing increasingly more important with each passing day. If you’ve decided to up your commitment to security, you can start by taking better advantage of the data your system is already logging. You can also capture even more data so that you have a better understanding of your system and will be more easily able to spot anything suspicious going on.

In either case, it’s necessary to use a tool that specializes in working with log data so that you don’t have to read unstructured data from many different sources. That’s where Splunk comes in.

Splunk is best known as a centralized logging management tool, but it can be very helpful when you need to work with security in a proactive (and even reactive) manner. But how you use Splunk for security analysis? Stay tuned; we’ll get into it. Let’s start by establishing some of the basics first.

Security Information and Event Management

Security Information and Event Management (SIEM) is a software product focused on the security of systems. It’s a combination of security information management (SIM) and security event management (SEM) tools. This combination allows you to do real-time analysis and offline analysis with persisted data that you can retain for a long time.

Everything starts with data collection, and that’s where SIM comes into play. Depending on the specific tool you use, you can actively move data or upload data on demand to a centralized place. Choosing one tool over another will determine whether you’re doing real-time analysis or forensic analysis using data from the past. Once you load data, you can perform searches for troubleshooting and create reports and visualizations to make sense of the collected data.

Things then get interesting when SEM comes to the table. After you’ve identified patterns in the data, you can correlate the data to automate notifications and actions based on the rules you define. For example, you can set it to trigger an alert because there are too many 404 errors. By looking at the logs, you can then correlate the requests to see that someone is trying to find a hole in your system.

You can implement everything I’ve mentioned here yourself. Still, there are tools that will help you get started quickly so you can focus your attention on using the data to improve the security of your system. Splunk is the SIEM tool I’m recommending—but why should you choose it?

Why Splunk for SIEM?

As this Gartner report states, there are lots of tools for SIEM, but Splunk stands out above the rest.

With Splunk, you can collect, parse, and store data in a standard format so that it’s easy to analyze. You can also configure automatic notifications with alerts and reports, correlate data with searches, and create visualizations with dashboards. It doesn’t matter really what the sources of the data are; you can collect data in real time or on demand. You can install and configure Splunk on any cloud provider, on-premises, or using a hybrid of those two. If you don’t want to administrate Splunk yourself, they also have an as-a-service option.

But what makes Splunk really stand out, according to the Gartner report, is what Splunk calls “apps” and other specialized services for security. These apps provide a set of searches, preset alerts, reports, and dashboards so that you can start analyzing data quickly. The apps include PCI Compliance, Stream, Security Essentials, Analytics for Hadoop, and Machine Learning Toolkit. You can also use services like Enterprise Security or User Behavior Analytics.

We don’t have time in this single post to get into all the details of all these apps. But we do have time to go through some of Splunk’s key features and how to use them.

Centralized Data Repository

If you want to be proactive about security, you need to store all your data in a centralized location. Reading and trying to understand different formats from different sources is demanding work. Instead, use Splunk to store your data so you can analyze it all in one place. Having a centralized repository becomes even more important if you need to support long-term storage for compliance reasons.

Splunk makes it easier to analyze that centrally stored data by converting data into events with timestamps. It starts by parsing the data to identify break lines and default fields, encoding characters, setting a timestamp if there’s no date field, and even masking certain data. Then, all this data is effectively distributed into the cluster so that search and indexing speeds remain fast. This process is called indexing; Splunk charges you by index volume.

You can move your data to Splunk using forwarders or, as I mentioned before, you can manually upload data on demand. I recommend that you start uploading data manually and familiarize yourself with how Splunk interprets your data.

Once you load your data into Splunk, you can perform searches, reports, and visualizations.

Real-Time Security Analysis

If security issues arise, you need to handle them as quickly as possible. What if you could take advantage of real-time monitoring to act on security issues as soon as they happen? Splunk lets you do that.

Splunk can warn you of brute-force attacks and other intrusions with its alerting feature. You can configure alerts to notify you via email when a saved search returns results. For example, that saved search could be a regular expression that looks for sensitive data like social security numbers or passwords; if you log sensitive data that might violate compliance, you’ll be notified.

As you continue learning how your app is used, you can create more rules that will let you respond quickly to vulnerabilities or attacks.

Security Improved by Using Log Data

Finally, remember that SIEM is neither a tool nor a software, but having a good tool (like Splunk) will make SIEM easier to implement.

As Grady Booch says, “A tool with a fool is still a fool.” Just because someone knows the inner workings of Splunk doesn’t mean they know how to apply SIEM to the collected data. You need to understand both the how and why of SIEM, and to know what sort of data matters for your business. It’s your responsibility to create the correlation rules from all this data. You’re the one that can best interpret your own data.

Keep watching the data you’re collecting. Some data might stop being relevant, in which case you should get rid of that data to save some costs and reduce the noise. But don’t let a fear of “too much data” stop you from logging. If you care about security, I encourage you to continue logging the data you think will be useful.