[Frontend] Make use_tqdm accept a callable for custom progress bars by reidliu41 · Pull Request #19357 · vllm-project/vllm (original) (raw)

Hello @reidliu41, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

Hello team, gemini-code-assist here to provide a summary of this pull request. This PR introduces a new parameter tqdm_leave_pbar to several public methods within the LLM class (generate, chat, encode, embed, classify, score). The purpose of this parameter is to control the behavior of the tqdm progress bars used during these operations, specifically whether they remain visible on the console after the operation completes. By default, the progress bars will still be left visible (tqdm_leave_pbar=True), maintaining the current behavior, but users can now set this to False to automatically remove the bars upon completion. This change addresses issue #18898, which requested this functionality.

Highlights

• New Parameter for tqdm Control: A new boolean parameter tqdm_leave_pbar is added to the generate, chat, encode, embed, classify, and score methods of the LLM class. This parameter defaults to True.
• Control Progress Bar Visibility: The tqdm_leave_pbar parameter is passed down to the internal _validate_and_add_requests and _run_engine methods, where it is used to set the leave argument of the tqdm progress bar constructors. Setting tqdm_leave_pbar=False will cause the progress bars to be removed from the console after completion.
• Documentation Updates: Docstrings for the affected methods have been updated to include the description and default value of the new tqdm_leave_pbar parameter.
• Fixes Issue #18898: This change directly implements the feature requested in issue #18898, allowing users more control over the console output when using vLLM's batch processing methods.

Changelog

• vllm/entrypoints/llm.py
  • Added tqdm_leave_pbar: bool = True parameter to generate method signature and docstring.
  • Added tqdm_leave_pbar: bool = True parameter to chat method signature and docstring.
  • Added tqdm_leave_pbar: bool = True parameter to encode method signature and docstring.
  • Added tqdm_leave_pbar: bool = True parameter to embed method signature and docstring.
  • Added tqdm_leave_pbar: bool = True parameter to classify method signature and docstring.
  • Added tqdm_leave_pbar: bool = True parameter to score method signature and docstring.
  • Added tqdm_leave_pbar: bool parameter to internal methods _validate_and_add_requests and _run_engine signatures.
  • Passed tqdm_leave_pbar parameter through calls from public methods to internal methods.
  • Used the tqdm_leave_pbar parameter to set the leave argument in tqdm constructor calls within _validate_and_add_requests and _run_engine. Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩