[Misc] reuse num_tokens_across_dp of get_dp_padding to avoid unnecessary dp all reduce in set_forward_context by izhuhaoran · Pull Request #18935 · vllm-project/vllm (original) (raw)
…all reduce in set_forward_context
Signed-off-by: Tyler Michael Smith tysmith@redhat.com
ONLY add when PR is ready to merge/full CI is needed
label
0826joyce pushed a commit to 0826joyce/vllm-serving-optimization that referenced this pull request
…ary dp all reduce in set_forward_context (vllm-project#18935)
Signed-off-by: Tyler Michael Smith tysmith@redhat.com Co-authored-by: zhuhaoran zhuhaoran.zhr@alibaba-inc.com Co-authored-by: Tyler Michael Smith tysmith@redhat.com
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})