• Skip to primary navigation
  • Skip to main content
  • Skip to footer

Cisco Umbrella

Enterprise network security

  • Free Trial
  • Contact us
  • Blog
  • Login
    • Umbrella Login
    • Cloudlock Login
  • Products
    • Product
      • Cisco Umbrella Cloud Security Service
      • Cisco Umbrella Investigate
      • Product Packages
      • Support Packages
    • Functionality
      • DNS-Layer Security
      • Secure Web Gateway
      • Cloud Access Security Broker (CASB)
      • Interactive Intelligence
      • Cloud-Delivered Firewall
    •  
    • Webinar signup
  • Solutions
    • By Need
      • Protect Mobile Users
      • Fast Incident Response
      • Web Content Filtering
      • Shadow IT Discovery & App Blocking
      • Unified Threat Enforcement
      • Reduce Security Infections
      • Secure Direct Internet Access
      • Securing Remote and Roaming Users
    • By Network
      • Protect Guest Wi-Fi
      • SD-WAN Security
      • Off-Network Endpoint Security
    • By Industry
      • Higher Education Security
      • K-12 Schools Security
      • Healthcare, Retail and Hospitality Security
      • Enterprise Cloud Security
      • Small Business Cybersecurity
      • Our Customers
      • Customer Stories
    • Ransomware Defense for Dummies book
  • Why Us
    • Fast Reliable Cloud
      • Cloud Security Infrastructure
      • Cloud Network Status
      • Cloud Network Activity
      • Recursive DNS Services
      • Top Reasons to Trial
      • Getting Started
    • Unmatched Intelligence
      • Cyber Attack Prevention
      • Interactive Intelligence
    • Extensive Integrations
      • IT Security Integrations
      • Hardware Integrations
      • Meraki Integration
      • Cisco SD-WAN
    • Navigation-dropdown-promo-free-trial_102820
  • Resources
    • Content Library
      • Top Resources
      • Analyst Reports
      • Case Studies
      • Customer Videos
      • Datasheets
      • eBooks
      • Infographics
      • Solution Briefs
    • International Documents
      • Deutsch/German
      • Español/Spanish
      • Français/French
      • Italiano/Italian
      • 日本語/Japanese
    • Cisco Umbrella Blog
      • Latest Posts
      • Security Posts
      • Research Posts
      • Threats Posts
      • Product Posts
      • Spotlight
    • For Customers
      • Support
      • Customer Success Hub
      • Umbrella Deployment Hub
      • Customer Success Webinars
      • What’s New
      • Cisco Umbrella Studio
  • Trends & Threats
    • Market Trends
      • Rise of Remote Workers
      • Secure Internet Gateway (SIG)
      • Secure Access Service Edge (SASE)
    • Security Threats
      • Ransomware
      • Cryptomining Malware Protection
      • Cybersecurity Threat Landscape
    •  
    • 2020 Cybersecurity trends
  • Partners
    • Channel Partners
      • Partner Program
      • Become a Partner
    • Service Providers
      • Secure Connectivity
      • Managed Security for MSSPs
      • Managed IT for MSPs
    •  
    • Become a partner
  • Free Trial Signup
  • Umbrella Login
  • Cloudlock Login
  • Contact Us
Research

Big Data Driven Security with Splunk

By OpenDNS Security Research
Posted on May 24, 2013
Updated on October 15, 2020

Share

Facebook0Tweet0LinkedIn0

In order to deliver predictive threat protection to our customers, the Umbrella Security Labs research team has to collect and correlate data from various sources in innovative ways. We’ve shared in previous posts how our team applies proprietary algorithms to data from the OpenDNS Global Network, but we’re constantly on the hunt for  easy-to-use data platforms that allow for real-time and interactive data visibility.

That’s why we wanted to share a bit about our experience with Splunk, a big data management system that provides fast machine data parsing, indexing, searching and data analyses. The GUI interface, dashboard and availability of security-related add-ons make for a neat out-of-the-box solution for enhanced data visibility.

Splunk Basic Usage

Installation of Splunk base is rather straightforward. Check out their official docs for installation instructions. When you’re getting started, these are some of the basic ways to use Splunk: add data to splunk (data input), search, delete, data aggregation, data transformation, and charting.

If you’re using customized data, you’ll likely find input to be the trickiest part. That’s where Splunk will have to figure out the correct data format, and properly parse it to extract fields. Splunk tries to automatically break the raw blob of textual input into EVENTS based on default or customized event breaking settings, and recognize the timestamp for each event. These settings can be customized both via Splunk GUI or command line interface (CLI). Make changes props.conf file to tell Splunk how to treat your data with correct configurations.

An example of extracting tab delimited fields from my input data:

For data queries and other operations (aggregating, data transforming etc.), Splunk’s pipe syntax seems pretty straightforward.

The following query that maps out a number of IP addresses that fits certain criteria serve as a good example of basic query syntaxes. The example requires the geoIP mapping app provided by Maxmind, and amMap, a mapping app.

sourcetype=mute* | rex "(?d+.d+.d+.d+)"| search ip!=192.168* ip!=0.0.* ip!=10.*|  stats count by ip | eval count_label="Event" | eval iterator="ip" | eval iterator_label="IP" |  eval zoom = "zoom="334%" zoom_x="-128.58%" zoom_y="-113.11%""| eval movie_color="#FF0000" | eval output_file="home_threat_data.xml" | eval app="amMap" | lookup geoip clientip as ip | search client_country!=^$ | mapit

Splunk data forwarding and receiving

Install the universal forwarder if your have remote data. The universal forwarder gathers data from servers where your input data reside and forwards them to your main Splunk server for indexing and searching.

./splunk add forward-server [splunk server:port]
/opt/splunkforwarder/bin/splunk add monitor /path/to/app/logs/ -index main -sourcetype %app%

At the same time, enable receiver – the main Splunk server and indexer by going to Splunk GUI, in forwarding and receiving->add new -> TCP port [port]

To troubleshoot the deployment, check these internal logs at the receiving indexer:

$SPLUNK_HOME/var/log/splunk/splunkd.log
$SPLUNK_HOME/var/log/splunk/license_audit.log

Use cases for Splunk security apps

Splunk base has a set of charting choices. In the following example, we made a pie chart of user agent distribution of our mobile clients data.

Snort app has been a great tool for quick network threat monitoring and alerting. We can easily retrieve all the entries that triggered snort, and perform in-depth investigations given the source IP addresses and contextual network data.

Snort and amMap makes use of Maxmind’s geo-ip mapping to give us an instant global look at the threat’s scale and spreading patterns.

Conclusion

We have yet to explore Splunk’s other interesting capabilities, such as real-time correlation making and alerting, or its distributed system deployment scheme (with Hadoop integration). We’ve spent lots of time with Hadoop and Hbase, which are largely back-end systems. As far as our primitive use of Splunk goes, it seems to serve quite well as a front-end portal for internal search, query and reporting. Data parsing for customized data is not as intuitive. It would be great if it provided pipe-like syntax for data input, as well.

Previous Post:

Previous Article

Next Post:

Next Article

Follow Us

  • Twitter
  • Facebook
  • LinkedIn
  • YouTube

Footer Sections

What we make

  • Cloud Security Service
  • DNS-Layer Network Security
  • Secure Web Gateway
  • Security Packages

Who we are

  • Cisco Umbrella Blog
  • Cloud Network Status
  • Cloud Network Activity
  • OpenDNS is now Cisco Umbrella

Learn more

  • Events
  • Careers
  • Support
  • Cisco Umbrella Live Demo
  • Contact Sales
Umbrella by Cisco
208.67.222.222+208.67.220.220
2620:119:35::35+2620:119:53::53
Sign up for a Free Trial
  • Cisco Online Privacy Statement
  • Terms of Service
  • Sitemap

© 2021 Cisco Umbrella