Industrial - Bachelors

Haptikos Reality: Surgical Trainer

Haptikos is an interface that connects several technologies to help surgeons and trainees develop skills in otherwise rare or difficult surgeries, with surgeons developing models using digital twin data that trainees can practice on providing a range of data and information to help develop skills and assess trainees.

the opportunity

This project was initiated by the Herston Biofabrication Institute (HBI) to support their digital twin surgical training (DTST) initiative to be developed with medical health professionals including doctors, academics, specialists and surgeons. The goal of this project was to help educate and train surgical trainees on surgical procedures of 2 or more hours using digital twin and XR technologies.

the Research

Secondary research was done initially to inform later primary research focussing on the impacts of XR technologies on surgery and medical education. An expert interview was conducted with a surgeon with decades of experience over a recorded Zoom call for half an hour, a survey was also deployed online and distributed to clinics through email, personal medical contacts to pass on to their colleagues, and various moderated surgical groups online, however, there was only 1 survey respondent over 3 weeks so it was instead analysed as a separate interview with a medical student.

Thematic Analysis

The research gave use several design intervention areas to be investigated:

• Communication training potentially with other surgical team members such as the nursing or anaesthetic team.
• Tool practice for dexterity, physical fatigue, mental endurance, and operative steps for complex equipment.
• XR simulations that create immersion through physical feedback systems.
• XR-assisted procedural recall for use in real and simulated surgical scenarios.
• Preoperative modelling and virtual surgical planning used with XR technology.

These design interventions were used to inform the design development however some of the design interventions such as preoperative modelling and virtual surgical planning already have existing design solutions such as the StealthStation™ S8 surgical navigation system developed by Medtronic and covered in the report.
Research Report
File Type
File Size
1 MB
Download File

the Apparatus

Surgeons at Metro North Hospital who want to prepare a trainee for an operation or recall a previous operational experience that would benefit others would use digital twin data previously collected to develop anatomical models for trainees to practice. This data would be moved through Digital Anatomy™ Creator a 3D slicing software designed for the Stratasys Digital Anatomy™ J750 printer which uses composite printed materials to mimic the mechanical properties of tissues. Fiber optic shape sensors are imbedded between layers that would receive deformation or strain below the area of surgical interest and using the FBG-Scan 915-EP™ measuring device and the ILLumiSense™ visualisation software forces applied to the model through surgical instruments can be mapped. The surgeon would first work with HBI to calibrate the model for future use by operating on the model this would set a benchmark score/base procedure that would need to be imitated by trainees.

The training apparatus integrates, the Intel® RealSense™ D435 depth camera and projector positioned above the surgical table. Below, the FBG-Scan 915-EP™, coupled with a computer, collects data. The Haptikos serves as the central hub, ensuring a deep learning experience for trainees and a refined practice environment for surgeons.

The training mat is divided into four distinct zones using fiducial markers. The depth camera cross-verifies the user’s tool list, identifying any missing items as ‘active tools.’ The ‘surgical table’ hosts the anatomical model, securely mounted through the board’s tapped base, with cables connected to the computer and the FGB measuring device. The projection screen displays performance metrics, procedural guidance, and useful tips based on the anatomical model and user input. In the ‘input area,’ trainees can select models and navigate the computer peripherals.
Notably, the projector can also project directly onto the anatomical model, offering real-time guidance for procedures, including depth in bone drilling, tool speed, torque, and precise cutting guidelines. This comprehensive approach to surgical training exemplifies our commitment to innovation and excellence in medical education.

Design development and ideation

Design Development Record
File Type
File Size
252 KB
Download File

Prototype images

James Holyhead

James is an Industrial Designer, studying Design and Engineering, and enjoys model making, rapid prototyping, CAD, and mechanical design.