• Skip to primary navigation
  • Skip to main content
  • Skip to footer

Cisco Umbrella

Enterprise network security

  • Free Trial
  • Contact us
  • Blog
  • Login
    • Umbrella Login
    • Cloudlock Login
  • Products
    • Product
      • Cisco Umbrella Cloud Security Service
      • Cisco Umbrella Investigate
      • Product Packages
      • Support Packages
    • Functionality
      • DNS-Layer Security
      • Secure Web Gateway
      • Cloud Access Security Broker (CASB)
      • Interactive Intelligence
      • Cloud-Delivered Firewall
    •  
    • Webinar signup
  • Solutions
    • By Need
      • Protect Mobile Users
      • Fast Incident Response
      • Web Content Filtering
      • Shadow IT Discovery & App Blocking
      • Unified Threat Enforcement
      • Reduce Security Infections
      • Secure Direct Internet Access
      • Securing Remote and Roaming Users
    • By Network
      • Protect Guest Wi-Fi
      • SD-WAN Security
      • Off-Network Endpoint Security
    • By Industry
      • Higher Education Security
      • K-12 Schools Security
      • Healthcare, Retail and Hospitality Security
      • Enterprise Cloud Security
      • Small Business Cybersecurity
      • Our Customers
      • Customer Stories
    • Ransomware Defense for Dummies book
  • Why Us
    • Fast Reliable Cloud
      • Global Cloud Architecture
      • Cloud Network Status
      • Cloud Network Activity
      • Recursive DNS Services
      • Top Reasons to Trial
      • Getting Started
    • Unmatched Intelligence
      • Cyber Attack Prevention
      • Interactive Intelligence
    • Extensive Integrations
      • IT Security Integrations
      • Hardware Integrations
      • Meraki Integration
      • Cisco SD-WAN
    • Navigation-dropdown-promo-free-trial_102820
  • Resources
    • Content Library
      • Top Resources
      • Cybersecurity Webinars
      • Events
      • Analyst Reports
      • Case Studies
      • Customer Videos
      • Datasheets
      • eBooks
      • Infographics
      • Solution Briefs
    • International Documents
      • Deutsch/German
      • Español/Spanish
      • Français/French
      • Italiano/Italian
      • 日本語/Japanese
    • Cisco Umbrella Blog
      • Latest Posts
      • Security Posts
      • Research Posts
      • Threats Posts
      • Product Posts
      • Spotlight
    • For Customers
      • Support
      • Customer Success Hub
      • Umbrella Deployment Hub
      • Customer Success Webinars
      • What’s New
      • Cisco Umbrella Studio
  • Trends & Threats
    • Market Trends
      • Rise of Remote Workers
      • Secure Internet Gateway (SIG)
      • Secure Access Service Edge (SASE)
    • Security Threats
      • Ransomware
      • Cryptomining Malware Protection
      • Cybersecurity Threat Landscape
    •  
    • Navigation-dropdown-promo-threat-report_020521
  • Partners
    • Channel Partners
      • Partner Program
      • Become a Partner
    • Service Providers
      • Secure Connectivity
      • Managed Security for MSSPs
      • Managed IT for MSPs
    •  
    • Become a partner
  • Free Trial Signup
  • Umbrella Login
  • Cloudlock Login
  • Contact Us
Product

Poseidon: Real-Time HTTP Log Analyzer

By Jeremiah O'Connor
Posted on August 19, 2015
Updated on March 5, 2020

Share

Facebook0Tweet0LinkedIn0

OpenDNS is known for being a cloud-delivered DNS security company, analyzing around 70 billion DNS requests per day. We also monitor about 10.1 million daily HTTP traffic requests on our proxy, which is maintained by our awesome Cloud Enforcement Team. Since this is a relatively new data set for us, building new data pipelines and processing proper metrics is an important step to gaining visibility and building intuitions about the data. With this in mind, we have created Poseidon, a statistics tool to monitor the network behavior on our proxy.
The primary purpose of Poseidon is to process the HTTP logs ingested from our Kafka stream and provide meaningful metrics in order to gain actionable insights over a sliding one-hour window. Some of the statistics it displays are top non-blocked domains, blocked URLs counts, and a trending system over the past hour (updating every five minutes).
Poseidon’s trending system is based on time series analysis algorithms. We take the relative average standard deviation per domain over the last hour, then sort through these domains by trending score to obtain the top trending domains and URLs in the observed window (60 minutes), and identify items with the sharpest trends for deeper inspection.
Figure 1 is an overview of how Poseidon’s windowing system works:
(Figure 1)
Screen Shot 2015-08-14 at 1.32.01 PM
Poseidon provides supplemental data for our researchers by retrieving classification data from Investigate, for example specific threat attribution scores (e.g., ASN scores, IP scores, and DGA scores). We have also created a display for time series data in a histogram to see the traffic behavior within the time window. Poseidon also monitors and generates statistics for OpenDNS’s new IP blocking feature.
Figure 2 is an example of time series histogram, where we can observe the overall counts over the last 1 hr. along with each 5 min. intervals, this helps with identifying sharp changes in traffic and network behavior:
(Figure 2)
Screen Shot 2015-08-14 at 2.31.02 PM
Some of the other fields in the proxy traffic that we are conducting analysis on are the HTTP Referer and User-Agent fields. For example, if there was a specific referer sending users to multiple malicious pages, this may be suspicious. We have deployed a set of rules to match malicious user-agent strings, and have also tied in useragentstring.com’s API to help identify and analyze unknown user-agents.
Figures 3 and 4 are screenshots of Poseidon displaying top-counts and displaying overall trending. From this data we can examine user behavior, trends in traffic, and identify outliers in the data:
(Figure 3)
Screen Shot 2015-08-14 at 11.34.32 AM
(Figure 4)
Screen Shot 2015-08-14 at 11.53.26 AM
The next step for the development of Poseidon is to rebuild it with an analytics platform like Apache Spark to distribute the computations as we deploy more rules, build HTTP detection models, and harvest more network statistics. In addition, we can proxy traffic for domains found from our DNS models and gain more information about them at the HTTP level. This will also be attached to our email alert system in order to send out daily digests of proxy statistics to the team for more in-depth analysis.
Using Poseidon, OpenDNS Security Labs increases its ability to detect new and emerging threats. As we discover suspicious domains from our proprietary DNS models, we can selectively proxy the traffic to gain greater insight – including subdomains, paths, filenames, and file extensions. Any identified indicators can then be fed back into our existing threat models, training sets, and alerting systems. The discovered indicators may also serve as the basis for the creation of new threat models or an entirely new vein of research.
These are exciting times in the OpenDNS Security Labs. Keep checking back with us as we continue to discover new and interesting threats using our innovative systems.

Previous Post:

Previous Article

Next Post:

Next Article

Follow Us

  • Twitter
  • Facebook
  • LinkedIn
  • YouTube

Footer Sections

What we make

  • Cloud Security Service
  • DNS-Layer Network Security
  • Secure Web Gateway
  • Security Packages

Who we are

  • Global Cloud Architecture
  • Cloud Network Status
  • Cloud Network Activity
  • OpenDNS is now Umbrella
  • Cisco Umbrella Blog

Learn more

  • Webinars
  • Careers
  • Support
  • Cisco Umbrella Live Demo
  • Contact Sales
Umbrella by Cisco
208.67.222.222+208.67.220.220
2620:119:35::35+2620:119:53::53
Sign up for a Free Trial
  • Cisco Online Privacy Statement
  • Terms of Service
  • Sitemap

© 2021 Cisco Umbrella