[Paper] [BibTeX]

MedSAM2: Segment Anything in 3D Medical Images and Videos is a foundation model towards solving promptable visual segmentation in medical images and videos. We have adopted this valuable model developed by Meta AI to detect lesions and various components in 3D medical images and videos.

Installation Guide Video

This code base relies on SAM2.1 base configuration. As 3D Slicer python version at the time of development was 3.9 and SAM2.1 requires it to be at least 3.10, the segmentation core and plugin interface should be set up separately.

Please install MedSAM 2 on a GPU machine with CUDA>=12.4 using:

• Download repository:

git clone https://github.com/bowang-lab/MedSAMSlicer.git
cd MedSAMSlicer
git checkout MedSAM2

• Create virtual environment: conda create -n medsam2 python=3.12 -y
• conda activate medsam2
• Install PyTorch 2.4: conda install pytorch torchvision pytorch-cuda=12.4 -c pytorch -c nvidia
• pip install -e .
• pip install -r requirements.txt
• Download checkpoints (various trained MedSAM2 models): python download_models.py

To ensure the successful installation of the backend, you can run the following script:

python infer_MedSAM2_slicer.py

1. Install 3D Slicer from its official website. The compatibility of our plugin has been tested with 3D Slicer >= 5.4.0
2. Select the Welcome to Slicer drop-down menu in the toolbar at the top and navigate to Developer Tools > Extension Wizard.
3. Click on select Extension and locate the MedSAM2 folder under MedSAM2/slicer. Confirm if asked to import new module.
4. Now, from the Welcome to Slicer drop-down menu, under the Segmentation sub-menu, MedSAM2 option is added. By choosing it, you can start using the plugin.

Usage Guide Video

You have to run the segmentation core to accept the incoming segmentation requests. You can do it both locally or on a remote computer:

python server.py

This runs the server on the public interface of your device on port 8080.

1. In 3D Slicer, from the Welcome to Slicer drop-down menu, under the Segmentation sub-menu, select MedSAM2.
2. Set IP to the machine running the server. For local machines use 127.0.0.1. Do the same for port.
3. After loading your image, choose the proper preprocessing method from the Preprocessing section.
4. In the Red pannel, slide to the beginning of the component you want to segment. From ROI section select Set As Start Slice.
5. Do similarly to set the end slice as well. The order of these two steps is not important and you can change your selections by selecting a new start/end as many times as you wish. The plugin automatically reposition the Red pannel to the middle of start and end slices.
6. Use Add Bounding Box to select as many component you target in the middle slice.
7. Now from the Segmentation section, Segment Middle Slice. After the loading is done, if no segment is visible, you might need to slide the Red pannel view slightly to the left.
8. You can now refine the inferred segmentation mask using Refine Middle Slice and built-in Slicer modules.
9. When you are satisfied with the middle slice, you can propagate the middle mask to other slices by clicking on Full Segmentation.

The models are licensed under the Apache 2.0 license. Please refer to our research paper for more details on the models.