Online video support for VLMs by litianjian · Pull Request #10020 · vllm-project/vllm (original) (raw)
Signed-off-by: DarkLight1337 tlleungac@connect.ust.hk
Signed-off-by: DarkLight1337 tlleungac@connect.ust.hk
Isotr0py pushed a commit to Isotr0py/vllm that referenced this pull request
Signed-off-by: DarkLight1337 tlleungac@connect.ust.hk Co-authored-by: litianjian litianjian@bytedance.com Co-authored-by: DarkLight1337 tlleungac@connect.ust.hk Signed-off-by: Isotr0py 2037008807@qq.com
JC1DA pushed a commit to JC1DA/vllm that referenced this pull request
Signed-off-by: DarkLight1337 tlleungac@connect.ust.hk Co-authored-by: litianjian litianjian@bytedance.com Co-authored-by: DarkLight1337 tlleungac@connect.ust.hk Signed-off-by: Loc Huynh jc1da.3011@gmail.com
sumitd2 pushed a commit to sumitd2/vllm that referenced this pull request
Signed-off-by: DarkLight1337 tlleungac@connect.ust.hk Co-authored-by: litianjian litianjian@bytedance.com Co-authored-by: DarkLight1337 tlleungac@connect.ust.hk Signed-off-by: Sumit Dubey sumit.dubey2@ibm.com
sleepwalker2017 pushed a commit to sleepwalker2017/vllm that referenced this pull request
Signed-off-by: DarkLight1337 tlleungac@connect.ust.hk Co-authored-by: litianjian litianjian@bytedance.com Co-authored-by: DarkLight1337 tlleungac@connect.ust.hk
pang-wu added a commit to pang-wu/litellm that referenced this pull request
ghost pushed a commit to BerriAI/litellm that referenced this pull request
- Support video_url
Support VLMs that works with video. Example implemenation in vllm: vllm-project/vllm#10020
- llms openai.py: Add ChatCompletionVideoObject
Add data structures to support video_url in chat completion
- test test_completion.py: add test for video_url
ghost pushed a commit to BerriAI/litellm that referenced this pull request
- Adding VertexAI Claude 3.7 Sonnet (#8774)
Co-authored-by: Emerson Gomes emerson.gomes@thalesgroup.com
build(model_prices_and_context_window.json): add anthropic 3-7 models on vertex ai and bedrock
Support video_url (#8743)
Support video_url
Support VLMs that works with video. Example implemenation in vllm: vllm-project/vllm#10020
- llms openai.py: Add ChatCompletionVideoObject
Add data structures to support video_url in chat completion
test test_completion.py: add test for video_url
Arize Phoenix - ensure correct endpoint/protocol are used; and default to phoenix cloud (#8750)
minor fixes to default to http and to ensure that the correct endpoint is used
Update test_arize_phoenix.py
prioritize http over grpc
Co-authored-by: Emerson Gomes emerson.gomes@gmail.com Co-authored-by: Emerson Gomes emerson.gomes@thalesgroup.com Co-authored-by: Pang Wu 104795337+pang-wu@users.noreply.github.com Co-authored-by: Nate Mar 67926244+nate-mar@users.noreply.github.com
ghost pushed a commit to BerriAI/litellm that referenced this pull request
- Adding VertexAI Claude 3.7 Sonnet (#8774)
Co-authored-by: Emerson Gomes emerson.gomes@thalesgroup.com
build(model_prices_and_context_window.json): add anthropic 3-7 models on vertex ai and bedrock
Support video_url (#8743)
Support video_url
Support VLMs that works with video. Example implemenation in vllm: vllm-project/vllm#10020
- llms openai.py: Add ChatCompletionVideoObject
Add data structures to support video_url in chat completion
test test_completion.py: add test for video_url
Arize Phoenix - ensure correct endpoint/protocol are used; and default to phoenix cloud (#8750)
minor fixes to default to http and to ensure that the correct endpoint is used
Update test_arize_phoenix.py
prioritize http over grpc
Co-authored-by: Emerson Gomes emerson.gomes@gmail.com Co-authored-by: Emerson Gomes emerson.gomes@thalesgroup.com Co-authored-by: Pang Wu 104795337+pang-wu@users.noreply.github.com Co-authored-by: Nate Mar 67926244+nate-mar@users.noreply.github.com
ghost pushed a commit to BerriAI/litellm that referenced this pull request
- Adding VertexAI Claude 3.7 Sonnet (#8774)
Co-authored-by: Emerson Gomes emerson.gomes@thalesgroup.com
build(model_prices_and_context_window.json): add anthropic 3-7 models on vertex ai and bedrock
Support video_url (#8743)
Support video_url
Support VLMs that works with video. Example implemenation in vllm: vllm-project/vllm#10020
- llms openai.py: Add ChatCompletionVideoObject
Add data structures to support video_url in chat completion
test test_completion.py: add test for video_url
Arize Phoenix - ensure correct endpoint/protocol are used; and default to phoenix cloud (#8750)
minor fixes to default to http and to ensure that the correct endpoint is used
Update test_arize_phoenix.py
prioritize http over grpc
Co-authored-by: Emerson Gomes emerson.gomes@gmail.com Co-authored-by: Emerson Gomes emerson.gomes@thalesgroup.com Co-authored-by: Pang Wu 104795337+pang-wu@users.noreply.github.com Co-authored-by: Nate Mar 67926244+nate-mar@users.noreply.github.com
ghost pushed a commit to BerriAI/litellm that referenced this pull request
- Adding VertexAI Claude 3.7 Sonnet (#8774)
Co-authored-by: Emerson Gomes emerson.gomes@thalesgroup.com
build(model_prices_and_context_window.json): add anthropic 3-7 models on vertex ai and bedrock
Support video_url (#8743)
Support video_url
Support VLMs that works with video. Example implemenation in vllm: vllm-project/vllm#10020
- llms openai.py: Add ChatCompletionVideoObject
Add data structures to support video_url in chat completion
test test_completion.py: add test for video_url
Arize Phoenix - ensure correct endpoint/protocol are used; and default to phoenix cloud (#8750)
minor fixes to default to http and to ensure that the correct endpoint is used
Update test_arize_phoenix.py
prioritize http over grpc
update sentry_sdk to latest version (#8588)
Add anthropic thinking + reasoning content support (#8778)
feat(anthropic/chat/transformation.py): add anthropic thinking param support
feat(anthropic/chat/transformation.py): support returning thinking content for anthropic on streaming responses
feat(anthropic/chat/transformation.py): return list of thinking blocks (include block signature)
allows usage in tool call responses
fix(types/utils.py): extract and map reasoning_content from anthropic as content str
test: add testing to ensure thinking_blocks are returned at the root
fix(anthropic/chat/handler.py): return thinking blocks on streaming - include signature
feat(factory.py): handle anthropic thinking blocks translation if in assistant response
test: handle openai internal instability
test: handle openai audio instability
ci: pin anthropic dep
test: handle openai audio instability
fix: fix linting error
refactor(anthropic/chat/transformation.py): refactor function to remain <50 LOC
fix: fix linting error
fix: fix linting error
fix: fix linting error
fix: fix linting error
test: handle index error
bump: version 1.61.15 → 1.61.16
Co-authored-by: Emerson Gomes emerson.gomes@gmail.com Co-authored-by: Emerson Gomes emerson.gomes@thalesgroup.com Co-authored-by: Pang Wu 104795337+pang-wu@users.noreply.github.com Co-authored-by: Nate Mar 67926244+nate-mar@users.noreply.github.com Co-authored-by: stephaneminisini stephane.minisini@gmail.com
LeiWang1999 pushed a commit to LeiWang1999/vllm-bitblas that referenced this pull request
Signed-off-by: DarkLight1337 tlleungac@connect.ust.hk Co-authored-by: litianjian litianjian@bytedance.com Co-authored-by: DarkLight1337 tlleungac@connect.ust.hk Signed-off-by: LeiWang1999 leiwang1999@outlook.com
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})