Dentify - VR Simulator

VR Haptic Simulator for training Dentistry Students

What is Dentify?

Dentify is a VR Dentistry Simulator that I was hired to build in collaboration with the Egas Moniz School of Health and Science in Lisbon, as part of an ongoing PhD thesis by professor Pedro Rodrigues. It supports the use of a haptic pen to provide a realistic training environment for Dentistry university students first learning how to treat cavities.

It is meant as a way to complement their learning experience, before contact with real patients, and as a first step before the use of expensive and single-use training equipment is employed.

A study was conducted with university students, the results were later published in the Nature magazine: https://www.nature.com/articles/s41598-023-29537-5

Project Description

This simulator was built using Unity3D and the OpenXR Toolkit as well as Touchy, an API for the Haptic Pen hardware we had available, the 3DSystems Phantom Omni.

A custom handle was 3D-printed for this pen to simulate the shape and weight of a dental drill and also to force students to use proper posture and technique.

The pen allows its users to feel virtual surfaces, with different degrees of compliance and different textures, all with a pretty high degree of precision and resolution.

Features

Dentify is at its core a dental drilling simulator, we strived to achieve a realistic feel to the experience so we used 3D scanned models for our clinical cases, four separate meshes to represent the different tissue found inside a tooth, and tuned the resistance and effectiveness of the drill for each part separately:

Enamel, the shiny white layer at the surface, the hardest part of the tooth
Dentine, the yellowish inner layer that is slightly softer
Pulp, the connective tissue and nerves that exist inside teeth, offers almost no resistance
Decay, diseased tissue that results from enamel and dentine being broken up

Drilling powered by realtime mesh deformation

I tried several approaches for the drilling itself, (like voxelization, as seen in some video-game deformable terrain) but ended up landing on procedurally pushing the vertices of each mesh in the direction of the drilling until the lower layer was exposed.

The main reason for this technique was that the haptics API required a static mesh collision as input, and it was too costly to generate a static mesh from a voxel grid every few frames.

The final implementation was very performant so long as we were careful to make the modifications asynchronously and update the collision as sparingly as possible.

To simulate the thermo-dynamics at play in the real drilling process, the drill will overheat if it contacts tissue for too long.

This must be avoided to not cause the patient discomfort and to prevent damage to the tool itself.

To simulate this each layer has a different rate at witch it causes over-heating, and the drills temperature is represented by a red glow that will start showing if the user makes a prolonged contact.

These contacts are tracked in the user metrics.

Drill overheating after prolonged contacts

Selecting tools and working mirror

The interaction model used had to be pretty simple and straight forward as most users of the simulator would not have previous VR experience.

The users can select a tool for their left and right hand independently, click buttons via a lazer pointer selector, and use the thumb sticks in the VR controller to adjust the positioning and scale of the teeth.

The mirror tool actually reflects its surroundings (implemented with an extra camera and render texture).

The simulator is meant as a training tool and as such it comes with a lot of features meant to track the performance of its users, as well as tweak every aspect of the experience.

A metrics and analytics system was implemented to handle this need and it was eventually used during user testing to save and export user data for analysis.

While the students use the VR headset and haptic pen the computer monitor displays a dashboard that a assistant can use to tweak the settings or run commands in real time.

Editing settings on the dashboard

Treatment of Class 1 and Class 2 Cavities

The simulator implements two common types of cavities, that have different treatment requirements. Bellow is a video showcasing the typical experience a student would go through when using the simulator.

Thanks for reading!

This project was lots of fun to work on despite being slightly outside of my wheel house.
Games don't normally command the rigorous amount of precision that a simulator, let alone a simulator for the medical field, requires just to approximate the real thing.

I had to work closely with some very patient and helpful folks that were willing to help me bridge that gap in knowledge and experience. The challenges I had to overcome were very unique to this application but I think I was still able to make use of my core skills to deliver a pretty polished experience that I am very proud of.

Next project: C++ 3D Engine

Study results published in Nature Magazine

Page updated

Google Sites

Report abuse