Unmet Need
Augmented reality (AR) refers to the overlay of computer-generated graphics over a user’s view of the real world. Typically, this is achieved through a device such as a head-mounted display (HMD). HMD-based AR has been used in the medical domain for treatment, education, and surgery. Useful information, measurements and assistive overlays can be provided to the user on a see-through display. For precise tasks on small objects, such as in surgery or dentistry, loupes are commonly used for optical magnification. In previous work, a digital magnification system, in other words, a virtual loupe, was implemented on a video see-through (VST) head-mounted display for surgical applications. The system was evaluated by measuring the completion time of a suturing task performed by surgeons. Although it was accepted by surgeons as a useful functionality, it was limited by the lower quality of digital magnification and by many disadvantages of the VST technology. For example, with VST technology, latency between the real world and the processed images can cause motion sickness, making it inappropriate for critical tasks such as surgery. Improvements are sought to incorporate high quality optical magnification, via loupes, with AR overlays displayed on optical see-through (OST) HMDs, which avoid the above disadvantages of VST technology.
Technology Overview
Hopkins inventors propose a novel system that combines a head-mounted display and magnifying loupe, with calibration methods to properly provide augmented reality on such devices to allow users to see computer graphics overlaid on the magnified field-of-vision. The major components of the system hardware involve an optical see-through head-mounted display and a head-mounted magnifying. Methods were developed to accurately align the computer graphics with the objects seen by the user, both for the magnified field of vision and non-magnified field of vision. This novel technology also considers the distortion property of the magnifying loupe and accordingly adjusts the visual overlay. The system can provide an accurate overlay over a magnified field of vision for users, e.g. dentists and surgeons. This novel OST-HMD system and its software can potentially achieve better accuracy and efficiency required in clinical or other detailed AR procedures.
Stage of Development
Prototype developed
Publications
L. Qian, T. Song, M. Unberath and P. Kazanzides, "AR-Loupe: Magnified Augmented Reality by Combining an Optical See-Through Head-Mounted Display and a Loupe" in IEEE Transactions on Visualization & Computer Graphics.
doi: 10.1109/TVCG.2020.3037284
https://doi.ieeecomputersociety.org/10.1109/TVCG.2020.3037284
