Here at Cisco Umbrella, we have a culture and passion for giving back to the technical community. This has included projects such as: our free consumer OpenDNS service, Phishtank, DNSStream, BGPStream, DNSCrypt, and several other data sources.
With that, we are very excited to announce the Cisco Umbrella 1 Million — a free list of the top 1 million most popular domains. This project came from our most recent hack-a-thon where we had more than 300 participants across 3 different countries hack for 24 hours. Hack-a-thons are an important piece of our engineering team’s culture and showcases their passion to build. On the heels of the announcement that the Alexa 1 Million list was not going to be available for free anymore, the idea was that we have the data and the means to provide an alternative.
The data itself is based on the Cisco Umbrella global cloud architecture that processes approximately 620 Billion requests per day, across more than 24K customers, in more than 165 countries. Although the data source is quite different from Alexa’s, we believe it’s arguably more accurate as it’s not based on only HTTP requests from users with browser additions. The way the ranking is computed is not as simple as the net sum of all DNS queries.
We have built our own popularity algorithm which uses the number of unique client IPs visiting this domain, relative to the sum of all requests to all domains. We take a score of how many different client/unique IPs go to this domain compared to others, and then rank the domains based on that. Our domain (vs site) ranking reflects the popularity of internet activity over any port or protocol from any application and not just web activity over port 80 from browsers.
Calculated and published daily
So, moving forward we will be calculating and publishing the list of the top 1M domains daily! This will be in order of popularity. This also included the TLDs (top level domains). For example did you know that google.com is more popular than aggregating all queries for the entire .org TLD?
List Format
- One comma delimited tuple per line
- Rank first, followed by domain
Example
1,com
2,net
3,google.com
4,org
5,googleapis.com
6,facebook.com
7,microsoft.com
8,www.google.com
9,doubleclick.net
10,g.doubleclick.net
We hope you enjoy our new data set and please follow us on Twitter for updated news on the latest Cisco Umbrella 1M lists and features.
The Cisco Umbrella 1M data is available free of charge here.
Share