Fix when response.usage is None for some OpenRouter models by sentienthouseplant · Pull Request #1402 · huggingface/smolagents (original) (raw)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note that we support streaming with our stream_outputs parameter:

• if True, we call generate_stream and call the model with stream=True
• if False, we call generate, and the model is not called with stream=True

And according to OpenRouter docs, response.usage can only be None when passing stream=True, that is, when stream_outputs is True, and therefore when we call generate_stream. We don't call generate when streaming outputs.

Could you please provide an example raising the error in the reported issue:

• #1401