Full immersive experience in a virtual world has always been a source of interest in litterature, movies, and technology. Famous cyberpunk writers such as William Gibson in Neuromancer or Neal Stephenson in Snowcrash have explored the concept artisticly. Many movies have attempted to represent the infinite possibilities and dangers of virtual reality : The Matrix, Tron, Hackers, and so many others. A lot of us still remember the days of the mighty Power Glove and the first adaptations to video games. All of these references, which are now classics, had one thing in common: A true fascination for an alternate digital reality.
Fast forward to our modern technology era, the 3D reality landscape has significantly changed. Data centers can now store a massive amount of information and data visualization has become the new playground for building accurate visual analytics ready for interpretation. One not-so-used approach to handle large quantities of entry points is to accelerate the computation and rendering on GPUs. These methods have been extensively used in physics simulation or medical imaging and it’s no suprise that we’re progressively seeing them becoming mainstream for more abstract-oriented data visualization. Recent graphic cards can render several millions of triangles per second in realtime, they can also run physics and audio effects in parallel. Minus the game design work, data visualization requirements are definitely not so far from gaming ones but yet the technology remains under-used. So what are we waiting for? Clearly, there is still a long way to go.
A Common Misconception
When you hear the words “3D Graphics”, chances are you are thinking of “projected 3D”. The first things that come to mind are 3D movies, 3D TV, video games or maybe augmented reality. All of those examples actually referring to animations projected onto a flat surface. Indeed, there is a difference: Real 3D refers to a true three-dimensionnal space similar to the one you experience everyday, on the other hand projected 3D defines a set of methods to create the illusion of a real 3D space. Subtle difference but nonetheless important : In projected 3D, you are always limited by the rendering device, the size of your TV, the definition of your monitor.
Now what does that tell us? As we integrate state-of-the-art 3D rendering techniques into our modern data visualization engines, the exploration of this data will always suffer from the device limitations, therefore our interpretation biased. The human brain is arguably the most powerful visual processor but its performance is certainly not infinite. There is a limit of what we can perceive and understand: Most of the time, data scientists have to deal with multi-dimensional sets of sophisticated features. We can only efficiently use 3 spatial + 1 temporal dimension, some spectral ones (Color, Temperature…) and it is unfortunately not enough.
If the first barrier is visual, the second one is more conceptual. The true potential of 3D visualization will only be unlocked with a smart combination of realistic 3D graphics, flawless virtual reality devices, powerful data analytics skills, and intuitive UI/UX design.
Here are 2 famous futuristic predictions made by Raymond Kurzweil, Director of Engineering at Google:
“In 2019, the computational capacity of a $4,000 computing device (in 1999 dollars) is approximately equal to the computational capability of the human brain (20 quadrillion calculations per second).”
“In 2019, people experience 3-D virtual reality through glasses and contact lenses that beam images directly to their retinas (retinal display). Coupled with an auditory source (headphones), users can remotely communicate with other people and access the Internet.”
The Beginning of a New Era
This evolution will redefine the limits of what we currently know. The majority of the interfaces we use today are designed to work in two dimensions and we have come a long way. With the advances of virtual & augmented reality, it feels pretty obvious that our interfaces will follow the movement. As the devices and technology evolves, the feeling of immersion will improve and the user will constantly navigate between 2 worlds. Now this leads to interesting questions, especially for data visualization:
- How would you take advantage of the 3 dimensions to efficiently structure & represent data?
- What physics laws would you keep, remove, modify or enhance to make navigation intuitive?
- How does the user interact with his world?
About 80% of the population is composed of visual learners, it is pretty safe to say that the visual aspect of an interface is the most important one. However it is not enough for a true illusion of alternate reality. Integrating other senses are mandatory for a realistic immersion feeling. Sound is a big one as it defines our perception of space, we all can pretty accurately locate a source of sound even if our eyes are closed.
Also, that last question leads to interesting thoughts: User interaction is mainly about mapping human movements or states to virtual actions but how does the user perceive his own self in a virtual world? For instance, if I am wearing a head mounted display (Such as the Oculus Rift), chances are that I can’t see my real hands or the mouse/keyboard. It makes the immersion illusion even more challenging. Similarly, speech is our easiest way to communicate elaborated information between humans, should we develop the same ability with our human-machine interfaces?
Clearly, the possibilities are endless and the more we learn about the human body, the more we will be capable of designing ergonomic interfaces. A lot of existing and emerging new technologies are on the verge of enabling such capabilities and there is no doubt that they will be part of the workstation of the next generation.
For the occasion of the RSA conference in San Francisco, we had to bring our A game and demonstrate our improvements of our data visualization engine. The novelty this year is the support of the Oculus Rift inside OpenGraphiti. If you are curious and interested in giving it a try, get in touch with us at www.opengraphiti.com