双目表面重建Stereo reconstruction
Magic Leap 2 双目相机结合学习式立体匹配，获取具有真实尺度的暴露肝表面。Magic Leap 2 stereo cameras and learned stereo matching recover a metric exposed surface.
← 技术平台← Platform
LIVER · 肝脏导航Liver Navigation
让术中肝脏与术前影像重新对齐Align preoperative anatomy with the liver in the operative field
面向开腹肝脏手术，利用头显自身的双目相机重建暴露表面，自动识别解剖特征，通过生物力学形变矫正将肿瘤与血管模型叠加到术野。For open hepatic surgery, headset stereo cameras reconstruct the exposed surface. Learned anatomical features guide biomechanical correction to overlay tumor and vessel models in the operative field.
研发中 · 仿体验证与术中图像评估In development · phantom and surgical-image evaluation
临床场景Clinical Context
肝脏变了形，深处的肿瘤在哪里？The liver changes shape. Where is the tumor now?
术前，医生可以在 CT 中逐层查看肿瘤、门静脉与肝静脉的关系。但打开腹腔、游离肝脏，并用牵开器或填塞物暴露术野后，眼前的肝脏已不再保持扫描时的形状。器官表面看得见，埋在内部的肿瘤和血管分支却仍藏在组织下方。Before surgery, CT lets the surgeon inspect the tumor in relation to portal and hepatic vessels. After opening the abdomen, mobilizing the liver, and exposing it with retractors or packing, the organ no longer has the shape seen in the scan. Its surface is visible, but the tumor and branching vessels remain beneath it.
此时，医生需要把屏幕上的三维解剖不断转换到实际术野中：计划的切除方向是否仍能到达靶点？预想的血管位置是否随牵拉发生了偏移？传统触笔需要接触肝表面、逐点采集特征，既占用操作时间，也可能扰动正在测量的软组织。The surgeon must continually translate screen-based anatomy into the operative field: does the planned approach still lead to the target, and has a vessel shifted with retraction? Conventional stylus acquisition samples surface landmarks point by point, adding time and potentially disturbing the soft tissue being measured.
系统利用头显采集当前可见的肝表面，将这些测量用于更新术前模型。导航希望帮助医生在同一视野里理解肿瘤、血管与当下肝脏形态之间的关系，为切除或穿刺路径判断提供空间参考。The system captures the currently visible liver surface through the headset and uses those measurements to update the preoperative model. The aim is to help the surgeon relate tumors and vessels to the present organ shape in one view, providing spatial context for resection or needle-path decisions.
- 01
术前影像Preoperative scan
肿瘤与血管位置清楚Tumor and vessels mapped
- 02
术中形变Intraoperative deformation
牵拉与填塞改变空间关系Retraction and packing alter geometry
- 03
形变后导航Deformation-aware navigation
用当前表面更新模型位置Update the model from the current surface
工作流程Workflow
从采集到术野显示From acquisition to in-field display
自动解剖特征识别Anatomical feature detection
SAM 2 分割肝表面，外观与几何双分支网络识别镰状韧带及左右下缘。SAM 2 segments the liver; an appearance-and-geometry network identifies the falciform ligament and inferior ridges.
配准与形变补偿Registration and deformation correction
Weighted-patch ICP 完成刚性初始化，LIBR 根据表面与特征约束矫正形变。Weighted-patch ICP initializes alignment; LIBR corrects deformation using surface and feature constraints.
语音驱动原位叠加Voice-driven in situ overlay
通过语音推进采集、配准与显示，查看肝脏、血管和肿瘤的空间关系。Voice commands advance acquisition, registration, and display of the liver, vessels, and tumor.
头显负责感知与显示，计算由工作站承担；流程无需外部追踪器或触笔接触器官。The headset handles sensing and display, with computation on a workstation. Acquisition requires no external tracker or stylus contact.
研究进展Research Progress
系统特点与验证边界System characteristics and validation scope
5.8 ± 3.2 mm
完整可见时非刚性靶点误差Nonrigid target error at full visibility
11.6 ± 3.8 mm
相同条件下刚性配准误差Rigid target error under the same condition
21.6 s
稿件报告的端到端处理时间Reported end-to-end processing time
研究稿件报告了 1,647 次配准、21,411 次亚表面靶点测量。端到端定位数据来自仿体；113 帧真实开腹手术图像用于独立的特征检测评估。60% 遮挡下，非刚性靶点误差为 10.0 ± 5.1 mm。上述研究结果不等同于患者术中导航精度。The manuscript reports 1,647 registrations and 21,411 subsurface target measurements. End-to-end targeting was evaluated on phantoms; 113 real surgical frames were used separately to evaluate feature detection. At 60% occlusion, nonrigid target error was 10.0 ± 5.1 mm. These findings do not establish patient-level intraoperative navigation accuracy.
研究依据 · 稿件Research basis · manuscript
Deformation-Aware Mixed Reality Surgical Navigation for Liver Intervention: A Markerless, Hands-Free, Fully Automatic, Learning-Based System
Bowen Xiang, Jon S. Heiselman, Michael I. Miga
探索其他导航方向Explore the navigation systems
LIVER
肝脏导航Liver Navigation
面向开腹肝脏手术，利用头显自身的双目相机重建暴露表面，自动识别解剖特征，通过生物力学形变矫正将肿瘤与血管模型叠加到术野。For open hepatic surgery, headset stereo cameras reconstruct the exposed surface. Learned anatomical features guide biomechanical correction to overlay tumor and vessel models in the operative field.
了解导航系统 →Explore the system →
BREAST
乳房导航Breast Navigation
面向保乳手术，将皮肤标志点识别、乳房表面采集与 MR 显示集中在同一头显，利用非刚性配准建立术前模型与术中乳房之间的空间对应。For breast-conserving surgery, a single headset combines skin-fiducial sensing, surface acquisition, and MR display. Nonrigid registration relates the preoperative model to the intraoperative breast.
了解导航系统 →Explore the system →
HEAD & NECK
头颈导航Head & Neck Navigation
针对头颈肿瘤再切除中的定位难题，将三维标本上的切缘标注经形变配准映射回创面，再通过增强现实显示目标位置。For head and neck tumor re-resection, deformable registration maps margin annotations from a scanned 3D specimen back to the wound bed, where AR displays the target location.
了解导航系统 →Explore the system →