A private client was looking to develop a facial tracking application with uses in medical and healthcare settings. The goal was to increase the accuracy of consultations and procedures, and provide a new standard of training for students and assistants. They tapped our team at CXR to explore the possibility of a mixed reality smart glasses application – something that had not been done before.
Engineered with Unity and C++, this revolutionary application can track up to 17 facial feature points at a time, passing camera frames through algorithms and machine learning to overlay a face with an AR user interface in real time.
The application includes a pre-loaded practice face as well, as a cadaver, for training purposes before attempting on an actual patient. This allows the user to become familiar not just with the treatment but the application itself.
The application overlays the data in real time, pinpointing the exact location for procedures. Panels of information are also presented to assist the professional in providing all relevant details to the patient.
While created for injection-based procedures, the tracking and application in general can be reconfigured for operations as important as reconstruction and as common as plastic surgery.
CXR has demonstrated that powerful mixed reality smart glasses applications are now within the realm of possibility for processors such as that of the HoloLens 2, meaning the potential use cases for MR glasses and technology are limitless.