ā
ā
ā
ā
- Mar 17, 2025: š We release the inference code and model weights of Step-Video-TI2V. Download
- Mar 17, 2025: š We release a new TI2V benchmark Step-Video-TI2V-Eval
- Mar 17, 2025: š Step-Video-TI2V has been integrated into ComfyUI-Stepvideo-ti2v. Enjoy!
- Mar 17, 2025: š We have made our technical report available as open source. Read
| ęé©¬č·³č· | ęé©¬č¹²äø | ę马ååå„č·ļ¼ē¶åč½¬čŗ« |
|---|---|---|
18.1.mp4 |
1.mp4 |
2.mp4 |
| äø¤åē·ååØäŗēøę³å»ļ¼é夓ēÆē»äø¤äŗŗęęć(motion_score: 2) | äø¤åē·ååØäŗēøę³å»ļ¼é夓ēÆē»äø¤äŗŗęęć(motion_score: 5) | äø¤åē·ååØäŗēøę³å»ļ¼é夓ēÆē»äø¤äŗŗęęć(motion_score: 20) |
|---|---|---|
00000000.5.1.mp4 |
00000009.1.mp4 |
00000000.6.mp4 |
šÆ Tips: The default motion_score = 5 is suitable for general use. If you need more stability, set motion_score = 2, though it may lack dynamism in certain movements. For greater movement flexibility, you can use motion_score = 10 or motion_score = 20 to enable more intense actions. Feel free to customize the motion_score based on your creative needs to fit different use cases.
| é夓ēÆē»å„³å©ļ¼å„³å©åØč·³č | é夓ē¼ę ¢ęØčæļ¼å„³å©åØč·³č | é夓ęčæļ¼å„³å©åØč·³č |
|---|---|---|
00000003.2.mp4 |
00000016.mp4 |
00000001.1.1.mp4 |
| Camera Movement | čæéę¹å¼ |
|---|---|
| Fixed Camera | åŗå®é夓 |
| Pan Up/Down/Left/Right | é夓äø/äø/å·¦/å³ē§» |
| Tilt Up/Down/Left/Right | é夓äø/äø/å·¦/å³ę |
| Zoom In/Out | é夓ę¾å¤§/ē¼©å° |
| Dolly In/Out | é夓ęØčæ/ęčæ |
| Camera Rotation | é夓ęč½¬ |
| Tracking Shot | é夓č·é |
| Orbit Shot | é夓ēÆē» |
| Rack Focus | ē¦ē¹č½¬ē§» |
š§ Motion Score Considerations: motion_score = 5 or 10 offers smoother and more accurate motion than motion_score = 2, with motion_score = 10 providing the best responsiveness and camera tracking. Choosing the suitable setting enhances motion precision and fluidity.
| 儳ēååč”čµ°ļ¼čęÆęÆčåęØ”ē³ēęę | 儳äŗŗēØē¼ļ¼ē¶ååƹēé夓åé£å»ēåØä½ć | ēøē«ę士åęē¼ē¼äøę¬ļ¼é·ēµä»ęäøåååØę©ę£ļ¼ čŗ«åēµå ½å½±åēåē¼éŖēå¼ŗå ļ¼ å¼ å¼å·Øå£ååŗä½å¼ |
|---|---|---|
11.mp4 |
00000001.mp4 |
25.mp4 |
We present Step-Video-TI2V, a state-of-the-art text-driven image-to-video generation model with 30B parameters, capable of generating videos up to 102 frames based on both text and image inputs. We build Step-Video-TI2V-Eval as a new benchmark for the text-driven image-to-video task and compare Step-Video-TI2V with open-source and commercial TI2V engines using this dataset. Experimental results demonstrate the state-of-the-art performance of Step-Video-TI2V in the image-to-video generation task.
Step-Video-TI2V is trained based on Step-Video-T2V. To incorporate the image condition as the first frame of the generated video, we encode it into latent representations using Step-Video-T2Vās Video-VAE and concatenate them along the channel dimension of the video latent. Additionally, we introduce a motion score condition, enabling users to control the dynamic level of the video generated from the image condition.
| Models | š¤ Huggingface | š¤ Modelscope | šļø ComfyUI |
|---|---|---|---|
| Step-Video-TI2V | Download | Download | Link |
git clone https://github.com/stepfun-ai/Step-Video-TI2V.git
conda create -n stepvideo python=3.10
conda activate stepvideo
cd Step-Video-TI2V
pip install -e .python api/call_remote_server.py --model_dir where_you_download_dir & ## We assume you have more than 4 GPUs available. This command will return the URL for both the caption API and the VAE API. Please use the returned URL in the following command.
parallel=1 or 4 # or parallel=8 Single GPU can also predict the results, although it will take longer
url='127.0.0.1'
model_dir=where_you_download_dir
torchrun --nproc_per_node $parallel run_parallel.py --model_dir $model_dir --vae_url $url --caption_url $url --ulysses_degree $parallel --prompt "ē¬čµ·ę„" --first_image_path ./assets/demo.png --infer_steps 50 --cfg_scale 9.0 --time_shift 13.0 --motion_score 5.0We list some more useful configurations for easy usage:
| Argument | Default | Description |
|---|---|---|
--model_dir |
None | The model checkpoint for video generation |
--prompt |
āē¬čµ·ę„ā | The text prompt for I2V generation |
first_image_path |
./assets/demo.png | The reference image path for I2V task. |
--infer_steps |
50 | The number of steps for sampling |
--cfg_scale |
9.0 | Embedded Classifier free guidance scale |
--time_shift |
7.0 | Shift factor for flow matching schedulers. |
--motion_score |
5.0 | Score to control the motion level of the video. |
--seed |
None | The random seed for generating video, if None, we init a random seed |
--use-cpu-offload |
False | Use CPU offload for the model load to save more memory, necessary for high-res video generation |
--save-path |
./results | Path to save the generated video |
To evaluate the performance of Step-Video-TI2V, We leverage VBench-I2V to systematically compare Step-Video-TI2V with recently released leading open-source models. The detailed results presented in the table below, highlight our modelās superior performance over these models. We presented two results of Step-Video-TI2V, with the motion set to 5 and 10, respectively. As expected, this mechanism effectively balances the motion dynamics and stability (or consistency) of the generated videos. Additionally, we submitted our results to the VBench-I2V leaderboard, where Step-Video-TI2V achieved the top-ranking position. We also introduce a new benchmark dataset, Step-Video-TI2V-Eval, specifically designed for the TI2V task to support future research and evaluation. The dataset includes 178 real-world and 120 anime-style prompt-image pairs, ensuring broad coverage of diverse user scenarios.
| Scores | Step-Video-TI2V (motion=10) | Step-Video-TI2V (motion=5) | OSTopA | OSTopB |
|---|---|---|---|---|
| Total Score | 87.98 | 87.80 | 87.49 | 86.77 |
| I2V Score | 95.11 | 95.50 | 94.63 | 93.25 |
| Video-Text Camera Motion | 48.15 | 49.22 | 29.58 | 46.45 |
| Video-Image Subject Consistency | 97.44 | 97.85 | 97.73 | 95.88 |
| Video-Image Background Consistency | 98.45 | 98.63 | 98.83 | 96.47 |
| Quality Score | 80.86 | 80.11 | 80.36 | 80.28 |
| Subject Consistency | 95.62 | 96.02 | 94.52 | 96.28 |
| Background Consistency | 96.92 | 97.06 | 96.47 | 97.38 |
| Motion Smoothness | 99.08 | 99.24 | 98.09 | 99.10 |
| Dynamic Degree | 48.78 | 36.58 | 53.41 | 38.13 |
| Aesthetic Quality | 61.74 | 62.29 | 61.04 | 61.82 |
| Imaging Quality | 70.17 | 70.43 | 71.12 | 70.82 |
The online version of Step-Video-TI2V is available on č·é®č§é¢, where you can also explore some impressive examples.
@misc{huang2025step,
title={Step-Video-TI2V Technical Report: A State-of-the-Art Text-Driven Image-to-Video Generation Model},
author={Haoyang Huang, Guoqing Ma, Nan Duan, Xing Chen, Changyi Wan, Ranchen
Ming, Tianyu Wang, Bo Wang, Zhiying Lu, Aojie Li, Xianfang Zeng, Xinhao
Zhang, Gang Yu, Yuhe Yin, Qiling Wu, Wen Sun, Kang An, Xin Han, Deshan Sun,
Wei Ji, Bizhu Huang, Brian Li, Chenfei Wu, Guanzhe Huang, Huixin Xiong,
Jiaxin He, Jianchang Wu, Jianlong Yuan, Jie Wu, Jiashuai Liu, Junjing Guo,
Kaijun Tan, Liangyu Chen, Qiaohui Chen, Ran Sun, Shanshan Yuan, Shengming
Yin, Sitong Liu, Wei Chen, Yaqi Dai, Yuchu Luo, Zheng Ge, Zhisheng Guan,
Xiaoniu Song, Yu Zhou, Binxing Jiao, Jiansheng Chen, Jing Li, Shuchang Zhou,
Xiangyu Zhang, Yi Xiu, Yibo Zhu, Heung-Yeung Shum, Daxin Jiang},
year={2025},
eprint={2503.11251},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2503.11251},
}
@misc{ma2025stepvideot2vtechnicalreportpractice,
title={Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model},
author={Guoqing Ma and Haoyang Huang and Kun Yan and Liangyu Chen and Nan Duan and Shengming Yin and Changyi Wan and Ranchen Ming and Xiaoniu Song and Xing Chen and Yu Zhou and Deshan Sun and Deyu Zhou and Jian Zhou and Kaijun Tan and Kang An and Mei Chen and Wei Ji and Qiling Wu and Wen Sun and Xin Han and Yanan Wei and Zheng Ge and Aojie Li and Bin Wang and Bizhu Huang and Bo Wang and Brian Li and Changxing Miao and Chen Xu and Chenfei Wu and Chenguang Yu and Dapeng Shi and Dingyuan Hu and Enle Liu and Gang Yu and Ge Yang and Guanzhe Huang and Gulin Yan and Haiyang Feng and Hao Nie and Haonan Jia and Hanpeng Hu and Hanqi Chen and Haolong Yan and Heng Wang and Hongcheng Guo and Huilin Xiong and Huixin Xiong and Jiahao Gong and Jianchang Wu and Jiaoren Wu and Jie Wu and Jie Yang and Jiashuai Liu and Jiashuo Li and Jingyang Zhang and Junjing Guo and Junzhe Lin and Kaixiang Li and Lei Liu and Lei Xia and Liang Zhao and Liguo Tan and Liwen Huang and Liying Shi and Ming Li and Mingliang Li and Muhua Cheng and Na Wang and Qiaohui Chen and Qinglin He and Qiuyan Liang and Quan Sun and Ran Sun and Rui Wang and Shaoliang Pang and Shiliang Yang and Sitong Liu and Siqi Liu and Shuli Gao and Tiancheng Cao and Tianyu Wang and Weipeng Ming and Wenqing He and Xu Zhao and Xuelin Zhang and Xianfang Zeng and Xiaojia Liu and Xuan Yang and Yaqi Dai and Yanbo Yu and Yang Li and Yineng Deng and Yingming Wang and Yilei Wang and Yuanwei Lu and Yu Chen and Yu Luo and Yuchu Luo and Yuhe Yin and Yuheng Feng and Yuxiang Yang and Zecheng Tang and Zekai Zhang and Zidong Yang and Binxing Jiao and Jiansheng Chen and Jing Li and Shuchang Zhou and Xiangyu Zhang and Xinhao Zhang and Yibo Zhu and Heung-Yeung Shum and Daxin Jiang},
year={2025},
eprint={2502.10248},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2502.10248},
}