EndoControlMag: Robust Endoscopic Vascular Motion Magnification with Periodic Reference Resetting and Hierarchical Tissue-aware Dual-Mask Control

An Wang*, Rulin Zhou*, Mengya Xu*, Yiru Ye, Longfei Gou, Yiting Chang, Hao Chen, Chwee Ming Lim, Jiankun Wang, Hongliang Ren
The Chinese University of Hong Kong; CUHK Shenzhen Research Institute
The First Affiliated Hospital of Wenzhou Medical University
Nanfang Hospital, Southern Medical University
Singapore General Hospital, Duke-NUS Medical School
Southern University of Science and Technology

*Equal Contribution
Corresponding Author

Abstract

Visualizing subtle vascular motions in endoscopic robotic surgery is crucial for surgical precision and decision-making, yet remains challenging due to the complex and dynamic nature of surgical scenes. To address this, we introduce EndoControlMag, a training-free, Lagrangian-based framework with mask-conditioned vascular motion magnification tailored to endoscopic environments. Our approach features two key modules: a Periodic Reference Resetting (PRR) scheme that divides videos into short overlapping clips with dynamically updated reference frames to prevent error accumulation while maintaining temporal coherence, and a Hierarchical Tissue-aware Magnification (HTM) framework with dual-mode mask dilation. HTM first tracks vessel cores using a pretrained visual tracking model to maintain accurate localization despite occlusions and view changes. It then applies one of two adaptive softening strategies to surrounding tissues: motion-based softening that modulates magnification strength proportional to observed tissue displacement, or distance-based exponential decay that simulates biomechanical force attenuation. This dual-mode approach accommodates diverse surgical scenarios—motion-based softening excels with complex tissue deformations while distance-based softening provides stability during unreliable optical flow conditions. We evaluate EndoControlMag on our EndoVMM24 dataset spanning four different surgery types and various challenging scenarios, including instrument occlusions, view changes, and vessel deformations. Quantitative metrics, visual assessments, and expert surgeon evaluations demonstrate that EndoControlMag significantly outperforms existing methods in both magnification accuracy and visual quality while maintaining robustness across challenging surgical conditions.

Graphical Abstract

Graphical abstract of our EndoControlMag.

Compare in Easy Cases

Comparison of the amplification effects of different models on the Easy Case dataset

Laparoscopic Cholecystectomy (LC)

Original

EndoControlMag

FlowMag

Axial-VMM

MDL-VMM

DMM

STB-VMM

EVM

Laparoscopic Distal Gastrectomy (LDG)

Original

EndoControlMag

FlowMag

Axial-VMM

MDL-VMM

DMM

STB-VMM

EVM

Laparoscopic Roux-en-Y Gastric Bypass (LRYGB)

Original

EndoControlMag

FlowMag

Axial-VMM

MDL-VMM

DMM

STB-VMM

EVM

Robot-assisted Radical Prostatectomy (RARP)

Original

EndoControlMag

FlowMag

Axial-VMM

MDL-VMM

DMM

STB-VMM

EVM

Compare in Hard Cases

Comparison of the amplification effects of different models on the Hard Case dataset

Occlusion

Tool Disturbance

Original

EndoControlMag(Distance)

EndoControlMag(Motion)

FlowMag

Original

EndoControlMag(Distance)

EndoControlMag(Motion)

FlowMag

Vessel Deformation

View Change

Original

EndoControlMag(Distance)

EndoControlMag(Motion)

FlowMag

Original

EndoControlMag(Distance)

EndoControlMag(Motion)

FlowMag

Compare in Different Magnification Factors

Original

x2

x4

x8

x16

x32

Poster

BibTeX

BibTex Code Here