共享 RGB-D 采集Shared RGB-D sensing
同一 Magic Leap 2 数据流同时采集带标签的皮肤标志点与部分乳房表面。One Magic Leap 2 stream captures labeled skin fiducials and a partial breast surface.
← 技术平台← Platform
BREAST · 乳房导航Breast Navigation
把形变后的肿瘤位置带回医生视野Bring the deformed tumor location into the surgeon’s view
面向保乳手术，将皮肤标志点识别、乳房表面采集与 MR 显示集中在同一头显，利用非刚性配准建立术前模型与术中乳房之间的空间对应。For breast-conserving surgery, a single headset combines skin-fiducial sensing, surface acquisition, and MR display. Nonrigid registration relates the preoperative model to the intraoperative breast.
研发中 · 系统方法与验证流程In development · system methods and validation workflow
临床场景Clinical Context
体表看不见的病灶，如何在保乳手术中定位？How do you locate a hidden lesion during breast-conserving surgery?
保乳手术中，医生面对的病灶可能既无法从体表看见，也不容易通过触诊准确判断边界。术前 MR 或 CT 可以提供病灶的位置与范围，但从影像检查到手术，体位、手臂姿势和支撑条件的变化，都会让乳房软组织重新分布。During breast-conserving surgery, a lesion may be invisible at the skin and its boundaries difficult to determine by palpation. Preoperative MR or CT describes its location and extent, but changes in body position, arm posture, and support between imaging and surgery redistribute the soft tissue.
医生既要找到病灶，又希望保留尽可能多的正常组织。一个表面位置看似正确的刚性模型，内部肿瘤却可能没有随乳房一起变形；反复在影像、体表标记和手术区域之间对照，需要医生在脑中完成这一步空间转换。The surgeon needs to locate the lesion while preserving as much normal tissue as possible. A rigid model can appear aligned at the surface while its internal tumor fails to follow the breast’s deformation. Comparing images, skin markings, and the surgical area leaves that spatial translation to the surgeon.
系统通过同一头显采集皮肤标志点和多视角乳房表面，让术前个体化模型适应当前形态。医生可在视野中查看估计的皮下肿瘤位置，并对比刚性与非刚性模型，理解形变对定位的影响。A single headset captures skin fiducials and the breast surface from multiple views so the patient-specific model can adapt to the current shape. In-view comparison of rigid and nonrigid models is designed to show the estimated subsurface tumor position and the effect of deformation on localization.
- 01
影像中的病灶Lesion in the scan
术前模型记录内部位置Model captures the internal location
- 02
体位发生变化Position changes
体表与皮下组织共同变形Surface and subsurface tissue deform
- 03
重新建立对应Restore spatial correspondence
以标志点和表面约束配准Register using fiducials and surface
工作流程Workflow
从采集到术野显示From acquisition to in-field display
多视角稳定融合Stable multi-view fusion
结合字母检测、布局匹配与跨帧投票，随头部移动累积标志点与表面点云。Letter detection, layout matching, and temporal voting consolidate fiducials and surface points across head poses.
非刚性模型对齐Nonrigid model alignment
以标志点和表面共同约束正则化 Kelvinlet 形变模型，通过 LIBR 框架优化。Fiducials and surface measurements jointly constrain regularized Kelvinlet deformation within a LIBR optimization framework.
原位模型查看In situ model viewing
通过语音推进流程，设计支持刚性与非刚性模型切换，查看皮下肿瘤位置。Voice advances the workflow, with rigid and nonrigid model views designed to show the subsurface tumor.
“单头显”指感知和显示共用设备。神经网络推理由连接的 GPU 工作站完成；系统仍使用皮肤标志点。A single headset combines sensing and display. Neural inference runs on a connected GPU workstation, and the workflow still uses skin fiducials.
研究进展Research Progress
系统特点与验证边界System characteristics and validation scope
RGB-D
一条数据流完成两类采集One stream for two sensing tasks
0
外部光学追踪器External optical trackers
MR / CT
患者个体化术前模型Patient-specific preoperative model
当前研究稿件描述了系统架构、标志点识别、表面重建及非刚性配准方法。亚表面靶点精度、表面误差和端到端时延的定量结果尚待补充。The current manuscript describes the architecture, fiducial sensing, surface reconstruction, and nonrigid registration. Quantitative subsurface targeting, surface-error, and end-to-end latency results are still pending.
研究依据 · 稿件Research basis · manuscript
A Self-Contained Mixed-Reality Surgical Navigation System for Breast Interventions with Comprehensive Validation Using Subsurface Target Localization
Bowen Xiang, Zixuan Liu, Jon S. Heiselman, Michael I. Miga
探索其他导航方向Explore the navigation systems
LIVER
肝脏导航Liver Navigation
面向开腹肝脏手术，利用头显自身的双目相机重建暴露表面，自动识别解剖特征，通过生物力学形变矫正将肿瘤与血管模型叠加到术野。For open hepatic surgery, headset stereo cameras reconstruct the exposed surface. Learned anatomical features guide biomechanical correction to overlay tumor and vessel models in the operative field.
了解导航系统 →Explore the system →BREAST
乳房导航Breast Navigation
面向保乳手术，将皮肤标志点识别、乳房表面采集与 MR 显示集中在同一头显，利用非刚性配准建立术前模型与术中乳房之间的空间对应。For breast-conserving surgery, a single headset combines skin-fiducial sensing, surface acquisition, and MR display. Nonrigid registration relates the preoperative model to the intraoperative breast.
了解导航系统 →Explore the system →
HEAD & NECK
头颈导航Head & Neck Navigation
针对头颈肿瘤再切除中的定位难题，将三维标本上的切缘标注经形变配准映射回创面，再通过增强现实显示目标位置。For head and neck tumor re-resection, deformable registration maps margin annotations from a scanned 3D specimen back to the wound bed, where AR displays the target location.
了解导航系统 →Explore the system →