[model] Support MiniCPM-V 4.5 by tc-mb · Pull Request #23586 · vllm-project/vllm (original) (raw)

Conversation

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})

@tc-mb tc-mb commented

Aug 25, 2025

•

edited by github-actionsbot

I’m bringing MiniCPM-V 4.5 in this PR.

Compared to the recently released MiniCPM-V 4.0, this new version puts more emphasis on performance metrics.

[ gemini-code-assist[bot] ](/apps/gemini-code-assist)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for the MiniCPM-V 4.5 model. The changes include updating documentation and test registries, adding a new chat template, and implementing the model-specific logic, particularly a new Resampler4_5 class to handle temporal data. While the implementation is comprehensive, I've identified two critical bugs in the new minicpmv.py file that could lead to runtime errors. One is related to incorrect data handling for temporal IDs, and the other is an unhandled edge case when processing these IDs, which could cause a crash. Addressing these issues is crucial for the stability of the new model support.

tc-mb mentioned this pull request

Aug 26, 2025

tc-mb requested review fromLucasWilkinson, ProExpertProg, WoosukKwon, alexm-redhat, comaniac, jikunshang, njhill, robertgshaw2-redhat, sighingnow, simon-mo, tlrmchlsmth, yewentao256, youkaichao and zou3519 as code owners

August 27, 2025 04:20

after run

pip install -U vllm
--pre
--extra-index-url https://wheels.vllm.ai/nightly

It shows that version 4.5 is not supported.

Can you add --force-reinstall to ensure the newest nightly is downloaded? It looks like the old version is cached on your system

Can you add --force-reinstall to ensure the newest nightly is downloaded? It looks like the old version is cached on your system

Thank u for your kind reply, it work！

xiao-llm pushed a commit to xiao-llm/vllm that referenced this pull request

Aug 28, 2025

Signed-off-by: tc-mb caitianchi@modelbest.cn Signed-off-by: Xin Yang xyangx@amazon.com Signed-off-by: Abatom abzhonghua@gmail.com Signed-off-by: chzhang chaojun.zhang@intel.com Signed-off-by: Pate Motter patemotter@google.com Signed-off-by: Terrencezzj terrence@cohere.ai Signed-off-by: Woosuk Kwon woosuk@thinkingmachines.ai Signed-off-by: simon-mo simon.mo@hey.com Signed-off-by: mgoin mgoin64@gmail.com Signed-off-by: Siyuan Fu siyuanf@nvidia.com Signed-off-by: siyuanf siyuanf@nvidia.com Signed-off-by: Weiliang Liu weiliangl@nvidia.com Signed-off-by: Michael Goin mgoin64@gmail.com Signed-off-by: yewentao256 zhyanwentao@126.com Signed-off-by: DarkLight1337 tlleungac@connect.ust.hk Signed-off-by: Luka Govedič ProExpertProg@users.noreply.github.com Signed-off-by: Zijing Liu liuzijing2014@gmail.com Signed-off-by: Zijing Liu liuzijing2014@users.noreply.github.com Signed-off-by: jiabin.00 jiabin.00@bytedance.com Signed-off-by: zjy0516 riverclouds.zhu@qq.com Signed-off-by: Harry Mellor 19981378+hmellor@users.noreply.github.com Signed-off-by: Jee Jee Li pandaleefree@gmail.com Signed-off-by: tc-mb 157115220+tc-mb@users.noreply.github.com Signed-off-by: Roger Wang hey@rogerw.me Signed-off-by: Roger Wang hey@rogerw.io Signed-off-by: Huy Do huydhn@gmail.com Signed-off-by: Matúš Námešný matus.namesny@ameria.com Signed-off-by: Guillaume Calmettes gcalmettes@scaleway.com Signed-off-by: Chen Zhang zhangch99@outlook.com Signed-off-by: oye93 en.ouyang93@outlook.com Signed-off-by: Julien Lin jullin@nvidia.com Signed-off-by: Didier Durand durand.didier@gmail.com Signed-off-by: Tianyu Li tianyu.li@arm.com Signed-off-by: Hongxia Yang hongxia.yang@amd.com Signed-off-by: Yuekai Zhang zhangyuekai@foxmail.com Signed-off-by: vllmellm vllm.ellm@embeddedllm.com Signed-off-by: jiang1.li jiang1.li@intel.com Signed-off-by: Zerohertz ohg3417@gmail.com Signed-off-by: Hyogeun Oh (오효근) ohg3417@gmail.com Signed-off-by: Thomas Parnell tpa@zurich.ibm.com Signed-off-by: Russell Bryant rbryant@redhat.com Signed-off-by: Isotr0py mozf@mail2.sysu.edu.cn Signed-off-by: Huzaifa Sidhpurwala huzaifas@redhat.com Signed-off-by: Federico 65908512+coval3nte@users.noreply.github.com Signed-off-by: Zixuan Zhang zixuanzhang@bytedance.com Signed-off-by: wuhang wuhang6@huawei.com Signed-off-by: czhu-cohere conway.zhu@cohere.com Signed-off-by: Wei Wei wwei6@meta.com Signed-off-by: Yiheng Xu charlesyihengxu@gmail.com Signed-off-by: Chenheli Hua huachenheli@outlook.com Signed-off-by: wangyafeng wangyafeng@baidu.com Co-authored-by: Xin Yang 105740670+xyang16@users.noreply.github.com Co-authored-by: Wentao Ye 44945378+yewentao256@users.noreply.github.com Co-authored-by: Zhonghua Deng abzhonghua@gmail.com Co-authored-by: Chaojun Zhang chaojun.zhang@intel.com Co-authored-by: Pate Motter p@temotter.com Co-authored-by: Terrence Zhao 32208165+Terrencezzj@users.noreply.github.com Co-authored-by: Woosuk Kwon woosuk.kwon@berkeley.edu Co-authored-by: Simon Mo simon.mo@hey.com Co-authored-by: Michael Goin mgoin64@gmail.com Co-authored-by: weiliang weiliangl@nvidia.com Co-authored-by: Siyuan Fu siyuanf@nvidia.com Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Cyrus Leung tlleungac@connect.ust.hk Co-authored-by: Copilot 198982749+Copilot@users.noreply.github.com Co-authored-by: ProExpertProg 11367180+ProExpertProg@users.noreply.github.com Co-authored-by: Luka Govedič ProExpertProg@users.noreply.github.com Co-authored-by: Zijing Liu liuzijing2014@users.noreply.github.com Co-authored-by: Bin Jia 45593998+FoolPlayer@users.noreply.github.com Co-authored-by: Jiangyun Zhu riverclouds.zhu@qq.com Co-authored-by: Harry Mellor 19981378+hmellor@users.noreply.github.com Co-authored-by: Raghavan oneraghavan@gmail.com Co-authored-by: Jee Jee Li pandaleefree@gmail.com Co-authored-by: Cyrus Leung cyrus.tl.leung@gmail.com Co-authored-by: Roger Wang hey@rogerw.io Co-authored-by: Roger Wang hey@rogerw.me Co-authored-by: knlnguyen1802 knlnguyen1802@gmail.com Co-authored-by: Huy Do huydhn@gmail.com Co-authored-by: Matúš Námešný matus@namesny.com Co-authored-by: Guillaume Calmettes gcalmettes@scaleway.com Co-authored-by: Chen Zhang zhangch99@outlook.com Co-authored-by: En Ouyang en.ouyang93@outlook.com Co-authored-by: Li, Jiang jiang1.li@intel.com Co-authored-by: nvjullin jullin@nvidia.com Co-authored-by: Didier Durand 2927957+didier-durand@users.noreply.github.com Co-authored-by: TianyuLi0 116711075+TianyuLi0@users.noreply.github.com Co-authored-by: Hongxia Yang 62075498+hongxiayang@users.noreply.github.com Co-authored-by: Yuekai Zhang zhangyuekai@foxmail.com Co-authored-by: vllmellm vllm.ellm@embeddedllm.com Co-authored-by: Hyogeun Oh (오효근) ohg3417@gmail.com Co-authored-by: Thomas Parnell tpa@zurich.ibm.com Co-authored-by: Russell Bryant rbryant@redhat.com Co-authored-by: Lukas Geiger lukas.geiger94@gmail.com Co-authored-by: Isotr0py mozf@mail2.sysu.edu.cn Co-authored-by: Huzaifa Sidhpurwala huzaifas@redhat.com Co-authored-by: Federico 65908512+coval3nte@users.noreply.github.com Co-authored-by: zixuanzhang226 zixuanzhang@bytedance.com Co-authored-by: wuhang wuhang6@huawei.com Co-authored-by: yzds 41983536+youzhedian@users.noreply.github.com Co-authored-by: hongchao hongchao@msh.team Co-authored-by: czhu-cohere conway.zhu@cohere.com Co-authored-by: Wei weiweinpu@gmail.com Co-authored-by: Yiheng Xu charlesyihengxu@gmail.com Co-authored-by: Aaron Pham contact@aarnphm.xyz Co-authored-by: Chenheli Hua huachenheli@outlook.com Co-authored-by: CSWYF3634076 58356743+CSWYF3634076@users.noreply.github.com Signed-off-by: Xiao Yu xiao.yu@amd.com

zhewenl pushed a commit to zhewenl/vllm that referenced this pull request

Aug 28, 2025

I still get the error:

pip install -U vllm --pre --extra-index-url https://wheels.vllm.ai/nightly --force-reinstall vllm serve "openbmb/MiniCPM-V-4_5" --tensor-parallel-size=2 --trust-remote-code ... ERROR 08-29 13:40:25 [multiproc_executor.py:559] ValueError: Currently, MiniCPMV only supports versions 2.0, 2.5, 2.6, 4.0. Got version: (4, 5)

Running on Debian 12 with Python 3.11.

rebuild the vllm ,and sometime I can got the right result, somtime I got this error but no error information in vllm serve : {"error":{"message":"EngineCore encountered an issue. See stack trace (above) for the root cause.","type":"Internal Server Error","param":null,"code":500}} How to solve this issue?

Can you show the stack trace?

tc-mb mentioned this pull request

Sep 1, 2025

tc-mb deleted the Support-MiniCPM-V-4.5 branch

September 2, 2025 05:38

zhewenl pushed a commit to zhewenl/vllm that referenced this pull request

Sep 3, 2025

tc-mb mentioned this pull request

Sep 8, 2025

Which version of Python should be used?

Any version supported by vLLM (Python 3.9-3.12) should work

Environment: Python 3.10, CUDA 12.9. The image functionality works properly, while the video functionality is not operational. However, the logs return a 200 status code with no exceptions.

vLLM was installed using the following method:
pip install -U vllm
--pre
--extra-index-url https://wheels.vllm.ai/nightly
--force-reinstall

vllm serve /data02/model/OpenBMB/MiniCPM-V-4_5 --served-model-name OpenBMB/MiniCPM-V-4_5