Segment Anything in 3D Medical Images and Videos

Welcome to join our mailing list to get updates. We’re also actively looking to collaborate on annotating new large-scale 3D datasets. If you have unlabeled medical images or videos and want to share them with the community, let’s connect!

• 20250705: Release Efficient MedSAM2 baseline for FLARE 2025 Pan-cancer segmentation challenge RECIST-to-3D
• 20250423: Release lung lesion segmentation dataset LUNA25-MedSAM2 for LUNA25

• Create a virtual environment: conda create -n medsam2 python=3.12 -y and conda activate medsam2
• Install PyTorch: pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu124 (Linux CUDA 12.4)
• Download code git clone https://github.com/bowang-lab/MedSAM2.git && cd MedSAM2 and run pip install -e ".[dev]"
• Download checkpoints: bash download.sh
• Optional: Please install the following dependencies for gradio

sudo apt-get update
sudo apt-get install ffmpeg
pip install gradio==3.38.0
pip install numpy==1.26.3 
pip install ffmpeg-python 
pip install moviepy

• CT_DeepLesion-MedSAM2

• LLD-MMRI-MedSAM2

• RVENet-MedSAM2

Note: Please also cite the raw DeepLesion, LLD-MMRI and RVENET papers when using these datasets.

• Colab: MedSAM2_inference_CT_Lesion_Demo.ipynb

• CMD

python medsam2_infer_3D_CT.py -i CT_DeepLesion/images -o CT_DeepLesion/segmentation

• Colab: MedSAM2_Inference_Video_Demo.ipynb

• CMD

python medsam2_infer_video.py -i input_video_path -m input_mask_path -o output_video_path 

python app.py

Use FLARE25 pan-cancer CT dataset as an example.

• Download sam2.1_hiera_tiny.pt to checkpoints
• Add dataset information in sam2/configs/sam2.1_hiera_tiny512_FLARE_RECIST.yaml: data -> train -> datasets
• Set train_video_batch_size based on the GPU memory

sh single_node_train_medsam2.sh

• multi-node training

sbatch multi_node_train.sh

• inference with RECIST marker (simulate a box prompt on middle slice)

python medsam2_infer_CT_lesion_npz_recist.py

• Train Efficient MedSAM2 on FLARE25 pan-cancer CT dataset for CPU-based inference

sh single_node_train_eff_medsam2_FLARE25.sh

• Inference with RECIST marker on the FLARE25 pan-cancer validation dataset.

npz = np.load('path to/CT_Lesion_FLARE23Ts_0057.npz', allow_pickle=True)
print(npz.keys())
imgs = npz['imgs'] # (D, W, H), [0, 255]
recist = npz['recist'] # (D, W, H), binary RECIST marker on tumor middle slice {0, 1}
gts = npz['gts'] # (D, W, H), 3D tumor ground truth mask. It will be not available in the testing set

simulate a box prompt on middle slice

python eff_medsam2_infer_CT_lesion_npz_recist.py

• We highly appreciate all the challenge organizers and dataset owners for providing the public datasets to the community.
• We thank Meta AI for making the source code of SAM2 and EfficientTAM publicly available. Please also cite these papers when using MedSAM2.

@article{MedSAM2,
    title={MedSAM2: Segment Anything in 3D Medical Images and Videos},
    author={Ma, Jun and Yang, Zongxin and Kim, Sumin and Chen, Bihui and Baharoon, Mohammed and Fallahpour, Adibvafa and Asakereh, Reza and Lyu, Hongwei and Wang, Bo},
    journal={arXiv preprint arXiv:2504.03600},
    year={2025}
}

Please also cite SAM2

@inproceedings{SAM2,
title={{SAM} 2: Segment Anything in Images and Videos},
    author={Nikhila Ravi and Valentin Gabeur and Yuan-Ting Hu and Ronghang Hu and Chaitanya Ryali and Tengyu Ma and Haitham Khedr and Roman R{\"a}dle and Chloe Rolland and Laura Gustafson and Eric Mintun and Junting Pan and Kalyan Vasudev Alwala and Nicolas Carion and Chao-Yuan Wu and Ross Girshick and Piotr Dollar and Christoph Feichtenhofer},
    booktitle={International Conference on Learning Representations},
    year={2025}
}

and EfficientTAM

@article{xiong2024efficienttam,
    title={Efficient Track Anything},
    author={Yunyang Xiong, Chong Zhou, Xiaoyu Xiang, Lemeng Wu, Chenchen Zhu, Zechun Liu, Saksham Suri, Balakrishnan Varadarajan, Ramya Akula, Forrest Iandola, Raghuraman Krishnamoorthi, Bilge Soran, Vikas Chandra},
    journal={preprint arXiv:2411.18933},
    year={2024}
}