Hands up who’s seen the provocative Stephen Spielberg sci-fi thriller Minority Report? In the movie, the main protagonist, chief of ‘pre-crime’ John Anderton played by Tom Cruise, investigates a future crime via a cool gesture-based holographic virtual reality (VR) interface. Whilst current VR technology isn’t quite that far into the future, it’s certainly not far off. Indeed, virtual reality is now becoming a reality in microscopy as researchers strive to improve their 3D understanding of complex biological samples. As creator of both the confocal microscope and the head-mounted display, a forerunner of the VR headset, Marvin Minsky would certainly approve of the convergence of these two technologies. The potential is enormous: imagine, for example, being able to take a virtual tour inside a tumour, to climb into an intestinal crypt or to peel apart the posterior parietal cortex – and all without getting your hands dirty!
‘Immersive microscopy’, as it is now known, is an area of imaging in which Zeiss in partnership with software developers arivis are currently leading the field (you can learn more here). To get in on the act, the Bioimaging Research Hub at Cardiff School of Biosciences has been developing a VR application of our own for visualisation and manipulation of volume datasets generated by the Hub’s various 3D imaging modalities. We anticipate that this technology will have significant relevance not only to imaging research within the school, but also to teaching and science outreach and engagement.
We’ve been using the affordable Oculus Go VR standalone headset and controller in association with the Unreal 4 games engine to create VR environments allowing interaction with our whole range of surface rendered 3D models. These range from microscopic biological samples imaged by confocal or lightsheet microscopy, such as cells or pollen grains, to large, photo-realistic anatomical models generated via photogrammetry.
As proof of principle we’ve developed a working prototype that allows users to manipulate 3D models of pollen grains in virtual space. You can see this in action in the movie above. We’re planning further developments of the system including new virtual 3D environments, different 3D models and object physics, and features such as interactive sample annotation via pop up GUIs. The great thing about VR of course is that we’re limited only by our imagination. To borrow a quote from John Lennnon, if ‘reality leaves a lot to the imagination’ then VR leaves a lot more!
AJH & MDI
Further reading
- Theart et al. (2017) Virtual reality assisted microscopy data visualization and colocalization analysis. BMC Bioinformatics 18(Suppl 2): 64
- Zeiss Blog post: Virtual Reality – the New Normal in Microscopy?
N