mixed reality

When Chemistry Meets Mixed Reality

By Rodney Guzman, CEO/Co-Founder/Owner at InterKnowlogy

holocule mixed reality


There is a lot of chatter about the future and promise of mixed reality. Devices like Hololens are in their infancy but still demonstrate the art of the possible. Are devices like these ready for primetime? When someone puts on a Hololens for the first time, after the initial awe and wonder passes, they usually start paying attention to only how heavy and uncomfortable the device is, how limited the field of view is, and how restrictive the interactions to drive experiences are. These devices do have to evolve before they can attempt to be as universal as smart phones, but the potential is intoxicating.

These current limitations in mixed reality devices might lead you to believe that the only experiences that can be delivered are those that inspire the potential for the future and not experiences that can actually be functional today. I would be the first one to raise my hand to ask why bother doing something in mixed reality when it would be better served on a tablet or even with a mouse and keyboard. If we are talking productivity apps and not games, then ease of interaction will trump the glitz and sexiness of the experience. It has to make sense in 3D, and it has to be so much better in 3D that you would want to experience it that way versus on a flat screen. We believe we found such a scenario in the field of chemistry.

Like many people reading this, I have never had to take an organic chemistry class. I recall my pre-med roommates suffering through this class while carrying around a plastic tin full of bonds and atoms. We are spatial animals and 3D is very natural to us. Understanding the nature and composition of molecules is better when it occurs in 3D and is why students physically build molecules with their plastic pieces. Something is lost in translation when visualizing a molecule on a 2D screen. The 3D structures that you can freely rotate in your hands and see the depth of connections and structures just do not permeate the level of understanding through a 2D screen. We have spoken to a number of chemists, and there is a belief that mixed reality has a role to play here.

We decided to undergo an internal R&D effort to visualize molecules in a 3D space with Hololens. Unity is the go to technology for Hololens, but it has some technical weaknesses that proved to be too challenging to overcome. You can put a molecular structure into Unity and render it just fine, but that has to be done before the app is compiled. If you attempt to load a molecular structure from a protein database file (PDB) and dynamically render it, Unity’s optimizations are much more difficult to manifest and the performance of the app (on Hololens) suffers to the point of making it unusable. We decided that we needed more performance and took the DirectX route. After a few internal versions, we finally found an optimized sweet spot for Hololens.

We recently updated Holocule in the Windows Store and it is free to download. It is in beta form and we are seeking feedback. Holocule renders PDB files and it is integrated with third party services to load molecules from the Internet. Molecules can be fully manipulated including zoom, rotate and voice commands. This app is a work in progress and was released to inspire a new generation of productivity apps only made possible with mixed reality. If you have any questions please contact us at curious@interknowlogy.com

Hololens Showroom

Tagged on: