Here at OpenDNS we have been working on a next-generation data visualization tool. We have been using this tool extensively to provide new insights into, and unique perspectives of, our intelligence database. The project has now reached a certain level of maturity and we are extremely proud to announce OpenGraphiti!
So, what is it ?
OpenGraphiti is a new data visualization engine focused on 3D rendering. It can visualize any relational data by connecting various entities with ‘like’ relationships. In addition to the visualization tool, we also provide all of the required tools and libraries to create your own custom datasets. The engine exposes an API, written in C, with wrappers for Python and Javascript. Any data scientist with basic skills in Python can add visualizations to their framework and connect it with their favorite data analysis tools. The API offers the possibility to directly play with the visual representation of the data – making it easier to explore, analyze, and present complex data to a visually inclined audience.
How does it work ?
When it comes to data visualization, there are multiple approaches to the problem. As the main purpose of the engine is to analyze the topology of any relational dataset, we need to gravitate towards visualization techniques that will let the data drive the layout – and not the other way around. For that kind of visualization, the de facto standard tends to be the use of force-directed layouts.
The general concept is relatively simple in that every node is treated as a particle and every edge as a force on the particles. By implementing an engine capable of running a particle physics model we can transform relational data, however loosely related, into a 2D or 3D structure – completely defined by the shape of the relational structure. This relational structure serves to highlight hidden clusters or topological patterns that may have previously gone unnoticed
OpenGraphiti is written in C/C++ and takes full advantage of the GPU power using technologies such as OpenGL, GLSL shaders and OpenCL parallel programming library. Most of the geometrical math is calculated with the GLM math library and the python scripts leverage the amazing NetworkX graph library. The project can be compiled with CLang++ for a native binary (Best performances) but also with LLVM compilers like Emscripten to create a Javascript bytecode that can be easily integrated in a web page.
Most data visualization tools suffer from latency or performance issues when dealing with large datasets. Our goal is to significantly push the limits by using modern acceleration and parallelization techniques. Everybody can appreciate the evolution of 3D graphic cards just by looking at the last video games : Indeed several hardware companies have invested a lot of efforts in improving the quality and performance of their GPUs and they have become insanely good at 3D rendering. Then why not apply it to data visualization?
Where can I get it ?
Don’t miss our official release at BlackHat! This will be the chance to discover OpenGraphiti and ask any questions you may have. Andrew Hay and I (Thibault Reuille) will be presenting on August 6th 2014 in the Jasmine Ballroom from 14:15-15:15 (2:15pm to 3:15pm). The git repository containing all of the OpenGraphiti code and associated tools will be made available before the talk.
BlackHat Talk : Unveiling the open source visualization engine for busy hackers
