txtai - Caption (original) (raw)
The caption pipeline reads a list of images and returns a list of captions for those images.
Example
The following shows a simple example using this pipeline.
`from txtai.pipeline import Caption
Create and run pipeline
caption = Caption() caption("path to image file") `
See the link below for a more detailed example.
| Notebook | Description | |
|---|---|---|
| Generate image captions and detect objects | Captions and object detection for images |  |
Configuration-driven example
Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.
config.yml
`# Create pipeline using lower case class name caption:
Run pipeline with workflow
workflow: caption: tasks: - action: caption `
Run with Workflows
`from txtai import Application
Create and run pipeline with workflow
app = Application("config.yml") list(app.workflow("caption", ["path to image file"])) `
Run with API
`CONFIG=config.yml uvicorn "txtai.api:app" &
curl
-X POST "http://localhost:8000/workflow"
-H "Content-Type: application/json"
-d '{"name":"caption", "elements":["path to image file"]}'
`
Methods
Python documentation for the pipeline.
__init__(path=None, quantize=False, gpu=True, model=None, **kwargs)
Source code in txtai/pipeline/image/caption.py
| def __init__(self, path=None, quantize=False, gpu=True, model=None, **kwargs): if not PIL: raise ImportError('Captions pipeline is not available - install "pipeline" extra to enable') # Call parent constructor super().__init__("image-to-text", path, quantize, gpu, model, **kwargs) |
|---|
__call__(images)
Builds captions for images.
This method supports a single image or a list of images. If the input is an image, the return type is a string. If text is a list, a list of strings is returned
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
| images | image|list | required |
Returns:
| Type | Description |
|---|---|
| list of captions |
Source code in txtai/pipeline/image/caption.py
| 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 | def __call__(self, images): """ Builds captions for images. This method supports a single image or a list of images. If the input is an image, the return type is a string. If text is a list, a list of strings is returned Args: images: image|list Returns: list of captions """ # Convert single element to list values = [images] if not isinstance(images, list) else images # Open images if file strings values = [Image.open(image) if isinstance(image, str) else image for image in values] # Get and clean captions captions = [] for result in self.pipeline(values): text = " ".join([r["generated_text"] for r in result]).strip() captions.append(text) # Return single element if single element passed in return captions[0] if not isinstance(images, list) else captions |
|---|