pipelines (original) (raw)

Pipelines provide a high-level, easy to use, API for running machine learning models.

Example: Instantiate pipeline using the pipeline function.

import { pipeline } from '@huggingface/transformers';

const classifier = await pipeline('sentiment-analysis'); const output = await classifier('I love transformers!');

pipelines.Pipeline

The Pipeline class is the class from which all pipelines inherit. Refer to this class for methods shared across different pipelines.

Kind: static class of pipelines

new Pipeline(options)

Create a new Pipeline.

Param Type Default Description
options Object An object containing the following properties:
[options.task] string The task of the pipeline. Useful for specifying subtasks.
[options.model] PreTrainedModel The model used by the pipeline.
[options.tokenizer] PreTrainedTokenizer The tokenizer used by the pipeline (if any).
[options.processor] Processor The processor used by the pipeline (if any).

pipeline.dispose() : DisposeType

Kind: instance method of Pipeline

pipelines.TextClassificationPipeline

Text classification pipeline using any ModelForSequenceClassification.

Example: Sentiment-analysis w/ Xenova/distilbert-base-uncased-finetuned-sst-2-english.

const classifier = await pipeline('sentiment-analysis', 'Xenova/distilbert-base-uncased-finetuned-sst-2-english'); const output = await classifier('I love transformers!');

Example: Multilingual sentiment-analysis w/ Xenova/bert-base-multilingual-uncased-sentiment (and return top 5 classes).

const classifier = await pipeline('sentiment-analysis', 'Xenova/bert-base-multilingual-uncased-sentiment'); const output = await classifier('Le meilleur film de tous les temps.', { top_k: 5 });

Example: Toxic comment classification w/ Xenova/toxic-bert (and return all classes).

const classifier = await pipeline('text-classification', 'Xenova/toxic-bert'); const output = await classifier('I hate you!', { top_k: null });

Kind: static class of pipelines

new TextClassificationPipeline(options)

Create a new TextClassificationPipeline.

Param Type Description
options TextPipelineConstructorArgs An object used to instantiate the pipeline.

textClassificationPipeline._call() : TextClassificationPipelineCallback

Kind: instance method of TextClassificationPipeline

pipelines.TokenClassificationPipeline

Named Entity Recognition pipeline using any ModelForTokenClassification.

Example: Perform named entity recognition with Xenova/bert-base-NER.

const classifier = await pipeline('token-classification', 'Xenova/bert-base-NER'); const output = await classifier('My name is Sarah and I live in London');

Example: Perform named entity recognition with Xenova/bert-base-NER (and return all labels).

const classifier = await pipeline('token-classification', 'Xenova/bert-base-NER'); const output = await classifier('Sarah lives in the United States of America', { ignore_labels: [] });

Kind: static class of pipelines

new TokenClassificationPipeline(options)

Create a new TokenClassificationPipeline.

Param Type Description
options TextPipelineConstructorArgs An object used to instantiate the pipeline.

tokenClassificationPipeline._call() : TokenClassificationPipelineCallback

Kind: instance method of TokenClassificationPipeline

pipelines.QuestionAnsweringPipeline

Question Answering pipeline using any ModelForQuestionAnswering.

Example: Run question answering with Xenova/distilbert-base-uncased-distilled-squad.

const answerer = await pipeline('question-answering', 'Xenova/distilbert-base-uncased-distilled-squad'); const question = 'Who was Jim Henson?'; const context = 'Jim Henson was a nice puppet.'; const output = await answerer(question, context);

Kind: static class of pipelines

new QuestionAnsweringPipeline(options)

Create a new QuestionAnsweringPipeline.

Param Type Description
options TextPipelineConstructorArgs An object used to instantiate the pipeline.

questionAnsweringPipeline._call() : QuestionAnsweringPipelineCallback

Kind: instance method of QuestionAnsweringPipeline

pipelines.FillMaskPipeline

Masked language modeling prediction pipeline using any ModelWithLMHead.

Example: Perform masked language modelling (a.k.a. “fill-mask”) with Xenova/bert-base-uncased.

const unmasker = await pipeline('fill-mask', 'Xenova/bert-base-cased'); const output = await unmasker('The goal of life is [MASK].');

Example: Perform masked language modelling (a.k.a. “fill-mask”) with Xenova/bert-base-cased (and return top result).

const unmasker = await pipeline('fill-mask', 'Xenova/bert-base-cased'); const output = await unmasker('The Milky Way is a [MASK] galaxy.', { top_k: 1 });

Kind: static class of pipelines

new FillMaskPipeline(options)

Create a new FillMaskPipeline.

Param Type Description
options TextPipelineConstructorArgs An object used to instantiate the pipeline.

fillMaskPipeline._call() : FillMaskPipelineCallback

Kind: instance method of FillMaskPipeline

pipelines.Text2TextGenerationPipeline

Text2TextGenerationPipeline class for generating text using a model that performs text-to-text generation tasks.

Example: Text-to-text generation w/ Xenova/LaMini-Flan-T5-783M.

const generator = await pipeline('text2text-generation', 'Xenova/LaMini-Flan-T5-783M'); const output = await generator('how can I become more healthy?', { max_new_tokens: 100, });

Kind: static class of pipelines

new Text2TextGenerationPipeline(options)

Create a new Text2TextGenerationPipeline.

Param Type Description
options TextPipelineConstructorArgs An object used to instantiate the pipeline.

text2TextGenerationPipeline._key : ’ generated_text ’

Kind: instance property of Text2TextGenerationPipeline

text2TextGenerationPipeline._call() : Text2TextGenerationPipelineCallback

Kind: instance method of Text2TextGenerationPipeline

pipelines.SummarizationPipeline

A pipeline for summarization tasks, inheriting from Text2TextGenerationPipeline.

Example: Summarization w/ Xenova/distilbart-cnn-6-6.

const generator = await pipeline('summarization', 'Xenova/distilbart-cnn-6-6'); const text = 'The tower is 324 metres (1,063 ft) tall, about the same height as an 81-storey building, ' + 'and the tallest structure in Paris. Its base is square, measuring 125 metres (410 ft) on each side. ' + 'During its construction, the Eiffel Tower surpassed the Washington Monument to become the tallest ' + 'man-made structure in the world, a title it held for 41 years until the Chrysler Building in New ' + 'York City was finished in 1930. It was the first structure to reach a height of 300 metres. Due to ' + 'the addition of a broadcasting aerial at the top of the tower in 1957, it is now taller than the ' + 'Chrysler Building by 5.2 metres (17 ft). Excluding transmitters, the Eiffel Tower is the second ' + 'tallest free-standing structure in France after the Millau Viaduct.'; const output = await generator(text, { max_new_tokens: 100, });

Kind: static class of pipelines

new SummarizationPipeline(options)

Create a new SummarizationPipeline.

Param Type Description
options TextPipelineConstructorArgs An object used to instantiate the pipeline.

summarizationPipeline._key : ’ summary_text ’

Kind: instance property of SummarizationPipeline

pipelines.TranslationPipeline

Translates text from one language to another.

Example: Multilingual translation w/ Xenova/nllb-200-distilled-600M.

See herefor the full list of languages and their corresponding codes.

const translator = await pipeline('translation', 'Xenova/nllb-200-distilled-600M'); const output = await translator('जीवन एक चॉकलेट बॉक्स की तरह है।', { src_lang: 'hin_Deva', tgt_lang: 'fra_Latn', });

Example: Multilingual translation w/ Xenova/m2m100_418M.

See herefor the full list of languages and their corresponding codes.

const translator = await pipeline('translation', 'Xenova/m2m100_418M'); const output = await translator('生活就像一盒巧克力。', { src_lang: 'zh', tgt_lang: 'en', });

Example: Multilingual translation w/ Xenova/mbart-large-50-many-to-many-mmt.

See herefor the full list of languages and their corresponding codes.

const translator = await pipeline('translation', 'Xenova/mbart-large-50-many-to-many-mmt'); const output = await translator('संयुक्त राष्ट्र के प्रमुख का कहना है कि सीरिया में कोई सैन्य समाधान नहीं है', { src_lang: 'hi_IN', tgt_lang: 'fr_XX', });

Kind: static class of pipelines

new TranslationPipeline(options)

Create a new TranslationPipeline.

Param Type Description
options TextPipelineConstructorArgs An object used to instantiate the pipeline.

translationPipeline._key : ’ translation_text ’

Kind: instance property of TranslationPipeline

pipelines.TextGenerationPipeline

Language generation pipeline using any ModelWithLMHead or ModelForCausalLM. This pipeline predicts the words that will follow a specified text prompt. NOTE: For the full list of generation parameters, see GenerationConfig.

Example: Text generation with Xenova/distilgpt2 (default settings).

const generator = await pipeline('text-generation', 'Xenova/distilgpt2'); const text = 'I enjoy walking with my cute dog,'; const output = await generator(text);

Example: Text generation with Xenova/distilgpt2 (custom settings).

const generator = await pipeline('text-generation', 'Xenova/distilgpt2'); const text = 'Once upon a time, there was'; const output = await generator(text, { temperature: 2, max_new_tokens: 10, repetition_penalty: 1.5, no_repeat_ngram_size: 2, num_beams: 2, num_return_sequences: 2, });

Example: Run code generation with Xenova/codegen-350M-mono.

const generator = await pipeline('text-generation', 'Xenova/codegen-350M-mono'); const text = 'def fib(n):'; const output = await generator(text, { max_new_tokens: 44, });

Kind: static class of pipelines

new TextGenerationPipeline(options)

Create a new TextGenerationPipeline.

Param Type Description
options TextPipelineConstructorArgs An object used to instantiate the pipeline.

textGenerationPipeline._call() : TextGenerationPipelineCallback

Kind: instance method of TextGenerationPipeline

pipelines.ZeroShotClassificationPipeline

NLI-based zero-shot classification pipeline using a ModelForSequenceClassificationtrained on NLI (natural language inference) tasks. Equivalent of text-classificationpipelines, but these models don’t require a hardcoded number of potential classes, they can be chosen at runtime. It usually means it’s slower but it is much more flexible.

Example: Zero shot classification with Xenova/mobilebert-uncased-mnli.

const classifier = await pipeline('zero-shot-classification', 'Xenova/mobilebert-uncased-mnli'); const text = 'Last week I upgraded my iOS version and ever since then my phone has been overheating whenever I use your app.'; const labels = [ 'mobile', 'billing', 'website', 'account access' ]; const output = await classifier(text, labels);

Example: Zero shot classification with Xenova/nli-deberta-v3-xsmall (multi-label).

const classifier = await pipeline('zero-shot-classification', 'Xenova/nli-deberta-v3-xsmall'); const text = 'I have a problem with my iphone that needs to be resolved asap!'; const labels = [ 'urgent', 'not urgent', 'phone', 'tablet', 'computer' ]; const output = await classifier(text, labels, { multi_label: true });

Kind: static class of pipelines

new ZeroShotClassificationPipeline(options)

Create a new ZeroShotClassificationPipeline.

Param Type Description
options TextPipelineConstructorArgs An object used to instantiate the pipeline.

zeroShotClassificationPipeline.model : any

Kind: instance property of ZeroShotClassificationPipeline

zeroShotClassificationPipeline._call() : ZeroShotClassificationPipelineCallback

Kind: instance method of ZeroShotClassificationPipeline

pipelines.FeatureExtractionPipeline

Feature extraction pipeline using no model head. This pipeline extracts the hidden states from the base transformer, which can be used as features in downstream tasks.

Example: Run feature extraction with bert-base-uncased (without pooling/normalization).

const extractor = await pipeline('feature-extraction', 'Xenova/bert-base-uncased', { revision: 'default' }); const output = await extractor('This is a simple test.');

Example: Run feature extraction with bert-base-uncased (with pooling/normalization).

const extractor = await pipeline('feature-extraction', 'Xenova/bert-base-uncased', { revision: 'default' }); const output = await extractor('This is a simple test.', { pooling: 'mean', normalize: true });

Example: Calculating embeddings with sentence-transformers models.

const extractor = await pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2'); const output = await extractor('This is a simple test.', { pooling: 'mean', normalize: true });

Example: Calculating binary embeddings with sentence-transformers models.

const extractor = await pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2'); const output = await extractor('This is a simple test.', { pooling: 'mean', quantize: true, precision: 'binary' });

Kind: static class of pipelines

new FeatureExtractionPipeline(options)

Create a new FeatureExtractionPipeline.

Param Type Description
options TextPipelineConstructorArgs An object used to instantiate the pipeline.

featureExtractionPipeline._call() : FeatureExtractionPipelineCallback

Kind: instance method of FeatureExtractionPipeline

pipelines.ImageFeatureExtractionPipeline

Image feature extraction pipeline using no model head. This pipeline extracts the hidden states from the base transformer, which can be used as features in downstream tasks.

Example: Perform image feature extraction with Xenova/vit-base-patch16-224-in21k.

const image_feature_extractor = await pipeline('image-feature-extraction', 'Xenova/vit-base-patch16-224-in21k'); const url = 'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/cats.png'; const features = await image_feature_extractor(url);

Example: Compute image embeddings with Xenova/clip-vit-base-patch32.

const image_feature_extractor = await pipeline('image-feature-extraction', 'Xenova/clip-vit-base-patch32'); const url = 'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/cats.png'; const features = await image_feature_extractor(url);

Kind: static class of pipelines

new ImageFeatureExtractionPipeline(options)

Create a new ImageFeatureExtractionPipeline.

Param Type Description
options ImagePipelineConstructorArgs An object used to instantiate the pipeline.

imageFeatureExtractionPipeline._call() : ImageFeatureExtractionPipelineCallback

Kind: instance method of ImageFeatureExtractionPipeline

pipelines.AudioClassificationPipeline

Audio classification pipeline using any AutoModelForAudioClassification. This pipeline predicts the class of a raw waveform or an audio file.

Example: Perform audio classification with Xenova/wav2vec2-large-xlsr-53-gender-recognition-librispeech.

const classifier = await pipeline('audio-classification', 'Xenova/wav2vec2-large-xlsr-53-gender-recognition-librispeech'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/jfk.wav'; const output = await classifier(url);

Example: Perform audio classification with Xenova/ast-finetuned-audioset-10-10-0.4593 and return top 4 results.

const classifier = await pipeline('audio-classification', 'Xenova/ast-finetuned-audioset-10-10-0.4593'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/cat_meow.wav'; const output = await classifier(url, { top_k: 4 });

Kind: static class of pipelines

new AudioClassificationPipeline(options)

Create a new AudioClassificationPipeline.

Param Type Description
options AudioPipelineConstructorArgs An object used to instantiate the pipeline.

audioClassificationPipeline._call() : AudioClassificationPipelineCallback

Kind: instance method of AudioClassificationPipeline

pipelines.ZeroShotAudioClassificationPipeline

Zero shot audio classification pipeline using ClapModel. This pipeline predicts the class of an audio when you provide an audio and a set of candidate_labels.

Example: Perform zero-shot audio classification with Xenova/clap-htsat-unfused.

const classifier = await pipeline('zero-shot-audio-classification', 'Xenova/clap-htsat-unfused'); const audio = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/dog_barking.wav'; const candidate_labels = ['dog', 'vaccum cleaner']; const scores = await classifier(audio, candidate_labels);

Kind: static class of pipelines

new ZeroShotAudioClassificationPipeline(options)

Create a new ZeroShotAudioClassificationPipeline.

Param Type Description
options TextAudioPipelineConstructorArgs An object used to instantiate the pipeline.

zeroShotAudioClassificationPipeline._call() : ZeroShotAudioClassificationPipelineCallback

Kind: instance method of ZeroShotAudioClassificationPipeline

pipelines.AutomaticSpeechRecognitionPipeline

Pipeline that aims at extracting spoken text contained within some audio.

Example: Transcribe English.

const transcriber = await pipeline('automatic-speech-recognition', 'Xenova/whisper-tiny.en'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/jfk.wav'; const output = await transcriber(url);

Example: Transcribe English w/ timestamps.

const transcriber = await pipeline('automatic-speech-recognition', 'Xenova/whisper-tiny.en'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/jfk.wav'; const output = await transcriber(url, { return_timestamps: true });

Example: Transcribe English w/ word-level timestamps.

const transcriber = await pipeline('automatic-speech-recognition', 'Xenova/whisper-tiny.en'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/jfk.wav'; const output = await transcriber(url, { return_timestamps: 'word' });

Example: Transcribe French.

const transcriber = await pipeline('automatic-speech-recognition', 'Xenova/whisper-small'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/french-audio.mp3'; const output = await transcriber(url, { language: 'french', task: 'transcribe' });

Example: Translate French to English.

const transcriber = await pipeline('automatic-speech-recognition', 'Xenova/whisper-small'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/french-audio.mp3'; const output = await transcriber(url, { language: 'french', task: 'translate' });

Example: Transcribe/translate audio longer than 30 seconds.

const transcriber = await pipeline('automatic-speech-recognition', 'Xenova/whisper-tiny.en'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/ted_60.wav'; const output = await transcriber(url, { chunk_length_s: 30, stride_length_s: 5 });

Kind: static class of pipelines

new AutomaticSpeechRecognitionPipeline(options)

Create a new AutomaticSpeechRecognitionPipeline.

Param Type Description
options TextAudioPipelineConstructorArgs An object used to instantiate the pipeline.

automaticSpeechRecognitionPipeline._call() : AutomaticSpeechRecognitionPipelineCallback

Kind: instance method of AutomaticSpeechRecognitionPipeline

pipelines.ImageToTextPipeline

Image To Text pipeline using a AutoModelForVision2Seq. This pipeline predicts a caption for a given image.

Example: Generate a caption for an image w/ Xenova/vit-gpt2-image-captioning.

const captioner = await pipeline('image-to-text', 'Xenova/vit-gpt2-image-captioning'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/cats.jpg'; const output = await captioner(url);

Example: Optical Character Recognition (OCR) w/ Xenova/trocr-small-handwritten.

const captioner = await pipeline('image-to-text', 'Xenova/trocr-small-handwritten'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/handwriting.jpg'; const output = await captioner(url);

Kind: static class of pipelines

new ImageToTextPipeline(options)

Create a new ImageToTextPipeline.

Param Type Description
options TextImagePipelineConstructorArgs An object used to instantiate the pipeline.

imageToTextPipeline._call() : ImageToTextPipelineCallback

Kind: instance method of ImageToTextPipeline

pipelines.ImageClassificationPipeline

Image classification pipeline using any AutoModelForImageClassification. This pipeline predicts the class of an image.

Example: Classify an image.

const classifier = await pipeline('image-classification', 'Xenova/vit-base-patch16-224'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/tiger.jpg'; const output = await classifier(url);

Example: Classify an image and return top n classes.

const classifier = await pipeline('image-classification', 'Xenova/vit-base-patch16-224'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/tiger.jpg'; const output = await classifier(url, { top_k: 3 });

Example: Classify an image and return all classes.

const classifier = await pipeline('image-classification', 'Xenova/vit-base-patch16-224'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/tiger.jpg'; const output = await classifier(url, { top_k: 0 });

Kind: static class of pipelines

new ImageClassificationPipeline(options)

Create a new ImageClassificationPipeline.

Param Type Description
options ImagePipelineConstructorArgs An object used to instantiate the pipeline.

imageClassificationPipeline._call() : ImageClassificationPipelineCallback

Kind: instance method of ImageClassificationPipeline

pipelines.ImageSegmentationPipeline

Image segmentation pipeline using any AutoModelForXXXSegmentation. This pipeline predicts masks of objects and their classes.

Example: Perform image segmentation with Xenova/detr-resnet-50-panoptic.

const segmenter = await pipeline('image-segmentation', 'Xenova/detr-resnet-50-panoptic'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/cats.jpg'; const output = await segmenter(url);

Kind: static class of pipelines

new ImageSegmentationPipeline(options)

Create a new ImageSegmentationPipeline.

Param Type Description
options ImagePipelineConstructorArgs An object used to instantiate the pipeline.

imageSegmentationPipeline._call() : ImageSegmentationPipelineCallback

Kind: instance method of ImageSegmentationPipeline

pipelines.BackgroundRemovalPipeline

Background removal pipeline using certain AutoModelForXXXSegmentation. This pipeline removes the backgrounds of images.

Example: Perform background removal with Xenova/modnet.

const segmenter = await pipeline('background-removal', 'Xenova/modnet'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/portrait-of-woman_small.jpg'; const output = await segmenter(url);

Kind: static class of pipelines

new BackgroundRemovalPipeline(options)

Create a new BackgroundRemovalPipeline.

Param Type Description
options ImagePipelineConstructorArgs An object used to instantiate the pipeline.

backgroundRemovalPipeline._call() : BackgroundRemovalPipelineCallback

Kind: instance method of BackgroundRemovalPipeline

pipelines.ZeroShotImageClassificationPipeline

Zero shot image classification pipeline. This pipeline predicts the class of an image when you provide an image and a set of candidate_labels.

Example: Zero shot image classification w/ Xenova/clip-vit-base-patch32.

const classifier = await pipeline('zero-shot-image-classification', 'Xenova/clip-vit-base-patch32'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/tiger.jpg'; const output = await classifier(url, ['tiger', 'horse', 'dog']);

Kind: static class of pipelines

new ZeroShotImageClassificationPipeline(options)

Create a new ZeroShotImageClassificationPipeline.

Param Type Description
options TextImagePipelineConstructorArgs An object used to instantiate the pipeline.

zeroShotImageClassificationPipeline._call() : ZeroShotImageClassificationPipelineCallback

Kind: instance method of ZeroShotImageClassificationPipeline

pipelines.ObjectDetectionPipeline

Object detection pipeline using any AutoModelForObjectDetection. This pipeline predicts bounding boxes of objects and their classes.

Example: Run object-detection with Xenova/detr-resnet-50.

const detector = await pipeline('object-detection', 'Xenova/detr-resnet-50'); const img = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/cats.jpg'; const output = await detector(img, { threshold: 0.9 });

Kind: static class of pipelines

new ObjectDetectionPipeline(options)

Create a new ObjectDetectionPipeline.

Param Type Description
options ImagePipelineConstructorArgs An object used to instantiate the pipeline.

objectDetectionPipeline._call() : ObjectDetectionPipelineCallback

Kind: instance method of ObjectDetectionPipeline

pipelines.ZeroShotObjectDetectionPipeline

Zero-shot object detection pipeline. This pipeline predicts bounding boxes of objects when you provide an image and a set of candidate_labels.

Example: Zero-shot object detection w/ Xenova/owlvit-base-patch32.

const detector = await pipeline('zero-shot-object-detection', 'Xenova/owlvit-base-patch32'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/astronaut.png'; const candidate_labels = ['human face', 'rocket', 'helmet', 'american flag']; const output = await detector(url, candidate_labels);

Example: Zero-shot object detection w/ Xenova/owlvit-base-patch32 (returning top 4 matches and setting a threshold).

const detector = await pipeline('zero-shot-object-detection', 'Xenova/owlvit-base-patch32'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/beach.png'; const candidate_labels = ['hat', 'book', 'sunglasses', 'camera']; const output = await detector(url, candidate_labels, { top_k: 4, threshold: 0.05 });

Kind: static class of pipelines

new ZeroShotObjectDetectionPipeline(options)

Create a new ZeroShotObjectDetectionPipeline.

Param Type Description
options TextImagePipelineConstructorArgs An object used to instantiate the pipeline.

zeroShotObjectDetectionPipeline._call() : ZeroShotObjectDetectionPipelineCallback

Kind: instance method of ZeroShotObjectDetectionPipeline

pipelines.DocumentQuestionAnsweringPipeline

Document Question Answering pipeline using any AutoModelForDocumentQuestionAnswering. The inputs/outputs are similar to the (extractive) question answering pipeline; however, the pipeline takes an image (and optional OCR’d words/boxes) as input instead of text context.

Example: Answer questions about a document with Xenova/donut-base-finetuned-docvqa.

const qa_pipeline = await pipeline('document-question-answering', 'Xenova/donut-base-finetuned-docvqa'); const image = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/invoice.png'; const question = 'What is the invoice number?'; const output = await qa_pipeline(image, question);

Kind: static class of pipelines

new DocumentQuestionAnsweringPipeline(options)

Create a new DocumentQuestionAnsweringPipeline.

Param Type Description
options TextImagePipelineConstructorArgs An object used to instantiate the pipeline.

documentQuestionAnsweringPipeline._call() : DocumentQuestionAnsweringPipelineCallback

Kind: instance method of DocumentQuestionAnsweringPipeline

pipelines.TextToAudioPipeline

Text-to-audio generation pipeline using any AutoModelForTextToWaveform or AutoModelForTextToSpectrogram. This pipeline generates an audio file from an input text and optional other conditional inputs.

Example: Generate audio from text with Xenova/speecht5_tts.

const synthesizer = await pipeline('text-to-speech', 'Xenova/speecht5_tts', { quantized: false }); const speaker_embeddings = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/speaker_embeddings.bin'; const out = await synthesizer('Hello, my dog is cute', { speaker_embeddings });

You can then save the audio to a .wav file with the wavefile package:

import wavefile from 'wavefile'; import fs from 'fs';

const wav = new wavefile.WaveFile(); wav.fromScratch(1, out.sampling_rate, '32f', out.audio); fs.writeFileSync('out.wav', wav.toBuffer());

Example: Multilingual speech generation with Xenova/mms-tts-fra. See here for the full list of available languages (1107).

const synthesizer = await pipeline('text-to-speech', 'Xenova/mms-tts-fra'); const out = await synthesizer('Bonjour');

Kind: static class of pipelines

new TextToAudioPipeline(options)

Create a new TextToAudioPipeline.

Param Type Description
options TextToAudioPipelineConstructorArgs An object used to instantiate the pipeline.

textToAudioPipeline._call() : TextToAudioPipelineCallback

Kind: instance method of TextToAudioPipeline

pipelines.ImageToImagePipeline

Image to Image pipeline using any AutoModelForImageToImage. This pipeline generates an image based on a previous image input.

Example: Super-resolution w/ Xenova/swin2SR-classical-sr-x2-64

const upscaler = await pipeline('image-to-image', 'Xenova/swin2SR-classical-sr-x2-64'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/butterfly.jpg'; const output = await upscaler(url);

Kind: static class of pipelines

new ImageToImagePipeline(options)

Create a new ImageToImagePipeline.

Param Type Description
options ImagePipelineConstructorArgs An object used to instantiate the pipeline.

imageToImagePipeline._call() : ImageToImagePipelineCallback

Kind: instance method of ImageToImagePipeline

pipelines.DepthEstimationPipeline

Depth estimation pipeline using any AutoModelForDepthEstimation. This pipeline predicts the depth of an image.

Example: Depth estimation w/ Xenova/dpt-hybrid-midas

const depth_estimator = await pipeline('depth-estimation', 'Xenova/dpt-hybrid-midas'); const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/cats.jpg'; const out = await depth_estimator(url);

Kind: static class of pipelines

new DepthEstimationPipeline(options)

Create a new DepthEstimationPipeline.

Param Type Description
options ImagePipelineConstructorArgs An object used to instantiate the pipeline.

depthEstimationPipeline._call() : DepthEstimationPipelineCallback

Kind: instance method of DepthEstimationPipeline

pipelines.pipeline(task, [model], [options]) ⇒ *

Utility factory method to build a Pipeline object.

Kind: static method of pipelines
Returns: * - A Pipeline object for the specified task.
Throws:

Param Type Default Description
task T The task defining which pipeline will be returned. Currently accepted tasks are: "audio-classification": will return a AudioClassificationPipeline. "automatic-speech-recognition": will return a AutomaticSpeechRecognitionPipeline. "depth-estimation": will return a DepthEstimationPipeline. "document-question-answering": will return a DocumentQuestionAnsweringPipeline. "feature-extraction": will return a FeatureExtractionPipeline. "fill-mask": will return a FillMaskPipeline. "image-classification": will return a ImageClassificationPipeline. "image-segmentation": will return a ImageSegmentationPipeline. "image-to-text": will return a ImageToTextPipeline. "object-detection": will return a ObjectDetectionPipeline. "question-answering": will return a QuestionAnsweringPipeline. "summarization": will return a SummarizationPipeline. "text2text-generation": will return a Text2TextGenerationPipeline. "text-classification" (alias "sentiment-analysis" available): will return a TextClassificationPipeline. "text-generation": will return a TextGenerationPipeline. "token-classification" (alias "ner" available): will return a TokenClassificationPipeline. "translation": will return a TranslationPipeline. "translation_xx_to_yy": will return a TranslationPipeline. "zero-shot-classification": will return a ZeroShotClassificationPipeline. "zero-shot-audio-classification": will return a ZeroShotAudioClassificationPipeline. "zero-shot-image-classification": will return a ZeroShotImageClassificationPipeline. "zero-shot-object-detection": will return a ZeroShotObjectDetectionPipeline.
[model] string null The name of the pre-trained model to use. If not specified, the default model for the task will be used.
[options] * Optional parameters for the pipeline.

pipelines~ImagePipelineInputs : string | RawImage | URL | Blob | HTMLCanvasElement | OffscreenCanvas

Kind: inner typedef of pipelines

pipelines~AudioPipelineInputs : string | URL | Float32Array | Float64Array

Kind: inner typedef of pipelines

pipelines~BoundingBox : Object

Kind: inner typedef of pipelines
Properties

Name Type Description
xmin number The minimum x coordinate of the bounding box.
ymin number The minimum y coordinate of the bounding box.
xmax number The maximum x coordinate of the bounding box.
ymax number The maximum y coordinate of the bounding box.

pipelines~Disposable ⇒ Promise. < void >

Kind: inner typedef of pipelines
Returns: Promise.<void> - A promise that resolves when the item has been disposed.
Properties

Name Type Description
dispose DisposeType A promise that resolves when the pipeline has been disposed.

pipelines~TextPipelineConstructorArgs : Object

An object used to instantiate a text-based pipeline.

Kind: inner typedef of pipelines
Properties

Name Type Description
task string The task of the pipeline. Useful for specifying subtasks.
model PreTrainedModel The model used by the pipeline.
tokenizer PreTrainedTokenizer The tokenizer used by the pipeline.

pipelines~ImagePipelineConstructorArgs : Object

An object used to instantiate an audio-based pipeline.

Kind: inner typedef of pipelines
Properties

Name Type Description
task string The task of the pipeline. Useful for specifying subtasks.
model PreTrainedModel The model used by the pipeline.
processor Processor The processor used by the pipeline.

pipelines~TextImagePipelineConstructorArgs : Object

An object used to instantiate a text- and audio-based pipeline.

Kind: inner typedef of pipelines
Properties

Name Type Description
task string The task of the pipeline. Useful for specifying subtasks.
model PreTrainedModel The model used by the pipeline.
tokenizer PreTrainedTokenizer The tokenizer used by the pipeline.
processor Processor The processor used by the pipeline.

pipelines~TextClassificationPipelineType ⇒ Promise. < (TextClassificationOutput|Array < TextClassificationOutput > ) >

Parameters specific to text classification pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<(TextClassificationOutput|Array<TextClassificationOutput>)> - An array or object containing the predicted labels and scores.

Param Type Description
texts string | Array The input text(s) to be classified.
[options] TextClassificationPipelineOptions The options to use for text classification.

Properties

Name Type Default Description
label string The label predicted.
score number The corresponding probability.
[top_k] number 1 The number of top predictions to be returned.

pipelines~TokenClassificationPipelineType ⇒ Promise. < (TokenClassificationOutput|Array < TokenClassificationOutput > ) >

Parameters specific to token classification pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<(TokenClassificationOutput|Array<TokenClassificationOutput>)> - The result.

Param Type Description
texts string | Array One or several texts (or one list of texts) for token classification.
[options] TokenClassificationPipelineOptions The options to use for token classification.

Properties

Name Type Description
word string The token/word classified. This is obtained by decoding the selected tokens.
score number The corresponding probability for entity.
entity string The entity predicted for that token/word.
index number The index of the corresponding token in the sentence.
[start] number The index of the start of the corresponding entity in the sentence.
[end] number The index of the end of the corresponding entity in the sentence.
[ignore_labels] Array. A list of labels to ignore.

pipelines~QuestionAnsweringPipelineType ⇒ Promise. < (QuestionAnsweringOutput|Array < QuestionAnsweringOutput > ) >

Parameters specific to question answering pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<(QuestionAnsweringOutput|Array<QuestionAnsweringOutput>)> - An array or object containing the predicted answers and scores.

Param Type Description
question string | Array One or several question(s) (must be used in conjunction with the context argument).
context string | Array One or several context(s) associated with the question(s) (must be used in conjunction with the question argument).
[options] QuestionAnsweringPipelineOptions The options to use for question answering.

Properties

Name Type Default Description
score number The probability associated to the answer.
[start] number The character start index of the answer (in the tokenized version of the input).
[end] number The character end index of the answer (in the tokenized version of the input).
answer string The answer to the question.
[top_k] number 1 The number of top answer predictions to be returned.

pipelines~FillMaskPipelineType ⇒ Promise. < (FillMaskOutput|Array < FillMaskOutput > ) >

Parameters specific to fill mask pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<(FillMaskOutput|Array<FillMaskOutput>)> - An array of objects containing the score, predicted token, predicted token string, and the sequence with the predicted token filled in, or an array of such arrays (one for each input text). If only one input text is given, the output will be an array of objects.
Throws:

Param Type Description
texts string | Array One or several texts (or one list of prompts) with masked tokens.
[options] FillMaskPipelineOptions The options to use for masked language modelling.

Properties

Name Type Default Description
sequence string The corresponding input with the mask token prediction.
score number The corresponding probability.
token number The predicted token id (to replace the masked one).
token_str string The predicted token (to replace the masked one).
[top_k] number 5 When passed, overrides the number of predictions to return.

pipelines~Text2TextGenerationPipelineType ⇒ Promise. < (Text2TextGenerationOutput|Array < Text2TextGenerationOutput > ) >

Kind: inner typedef of pipelines

Param Type Description
texts string | Array Input text for the encoder.
[options] * Additional keyword arguments to pass along to the generate method of the model.

Properties

Name Type Description
generated_text string The generated text.

pipelines~SummarizationPipelineType ⇒ Promise. < (SummarizationOutput|Array < SummarizationOutput > ) >

Kind: inner typedef of pipelines

Param Type Description
texts string | Array One or several articles (or one list of articles) to summarize.
[options] * Additional keyword arguments to pass along to the generate method of the model.

Properties

Name Type Description
summary_text string The summary text.

pipelines~TranslationPipelineType ⇒ Promise. < (TranslationOutput|Array < TranslationOutput > ) >

Kind: inner typedef of pipelines

Param Type Description
texts string | Array Texts to be translated.
[options] * Additional keyword arguments to pass along to the generate method of the model.

Properties

Name Type Description
translation_text string The translated text.

pipelines~TextGenerationPipelineType ⇒ Promise. < (TextGenerationOutput|Array < TextGenerationOutput > ) >

Parameters specific to text-generation pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<(TextGenerationOutput|Array<TextGenerationOutput>)> - An array or object containing the generated texts.

Param Type Description
texts string | Array Chat Array One or several prompts (or one list of prompts) to complete.
[options] Partial. Additional keyword arguments to pass along to the generate method of the model.

Properties

Name Type Default Description
generated_text string | Chat The generated text.
[add_special_tokens] boolean Whether or not to add special tokens when tokenizing the sequences.
[return_full_text] boolean true If set to false only added text is returned, otherwise the full text is returned.

pipelines~ZeroShotClassificationPipelineType ⇒ Promise. < (ZeroShotClassificationOutput|Array < ZeroShotClassificationOutput > ) >

Parameters specific to zero-shot classification pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<(ZeroShotClassificationOutput|Array<ZeroShotClassificationOutput>)> - An array or object containing the predicted labels and scores.

Param Type Description
texts string | Array The sequence(s) to classify, will be truncated if the model input is too large.
candidate_labels string | Array The set of possible class labels to classify each sequence into. Can be a single label, a string of comma-separated labels, or a list of labels.
[options] ZeroShotClassificationPipelineOptions The options to use for zero-shot classification.

Properties

Name Type Default Description
sequence string The sequence for which this is the output.
labels Array. The labels sorted by order of likelihood.
scores Array. The probabilities for each of the labels.
[hypothesis_template] string ""This example is {}."" The template used to turn each candidate label into an NLI-style hypothesis. The candidate label will replace the {} placeholder.
[multi_label] boolean false Whether or not multiple candidate labels can be true. If false, the scores are normalized such that the sum of the label likelihoods for each sequence is 1. If true, the labels are considered independent and probabilities are normalized for each candidate by doing a softmax of the entailment score vs. the contradiction score.

pipelines~FeatureExtractionPipelineType ⇒ Promise. < Tensor >

Parameters specific to feature extraction pipelines.

Kind: inner typedef of pipelines
Returns: Promise. - The features computed by the model.

Param Type Description
texts string | Array One or several texts (or one list of texts) to get the features of.
[options] FeatureExtractionPipelineOptions The options to use for feature extraction.

Properties

Name Type Default Description
[pooling] 'none' | 'mean' 'cls' "none" The pooling method to use.
[normalize] boolean false Whether or not to normalize the embeddings in the last dimension.
[quantize] boolean false Whether or not to quantize the embeddings.
[precision] 'binary' | 'ubinary' 'binary' The precision to use for quantization.

pipelines~ImageFeatureExtractionPipelineType ⇒ Promise. < Tensor >

Parameters specific to image feature extraction pipelines.

Kind: inner typedef of pipelines
Returns: Promise. - The image features computed by the model.

Param Type Description
images ImagePipelineInputs One or several images (or one list of images) to get the features of.
[options] ImageFeatureExtractionPipelineOptions The options to use for image feature extraction.

Properties

Name Type Default Description
[pool] boolean Whether or not to return the pooled output. If set to false, the model will return the raw hidden states.

pipelines~AudioClassificationPipelineType ⇒ Promise. < (AudioClassificationOutput|Array < AudioClassificationOutput > ) >

Parameters specific to audio classification pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<(AudioClassificationOutput|Array<AudioClassificationOutput>)> - An array or object containing the predicted labels and scores.

Param Type Description
audio AudioPipelineInputs The input audio file(s) to be classified. The input is either: string or URL that is the filename/URL of the audio file, the file will be read at the processor's sampling rate to get the waveform using the AudioContext API. If AudioContext is not available, you should pass the raw waveform in as a Float32Array of shape (n, ). Float32Array or Float64Array of shape (n, ), representing the raw audio at the correct sampling rate (no further check will be done).
[options] AudioClassificationPipelineOptions The options to use for audio classification.

Properties

Name Type Default Description
label string The label predicted.
score number The corresponding probability.
[top_k] number 5 The number of top labels that will be returned by the pipeline. If the provided number is null or higher than the number of labels available in the model configuration, it will default to the number of labels.

pipelines~ZeroShotAudioClassificationPipelineType ⇒ Promise. < (Array < ZeroShotAudioClassificationOutput > |Array < Array < ZeroShotAudioClassificationOutput > > ) >

Parameters specific to zero-shot audio classification pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<(Array<ZeroShotAudioClassificationOutput>|Array<Array<ZeroShotAudioClassificationOutput>>)> - An array of objects containing the predicted labels and scores.

Param Type Description
audio AudioPipelineInputs The input audio file(s) to be classified. The input is either: string or URL that is the filename/URL of the audio file, the file will be read at the processor's sampling rate to get the waveform using the AudioContext API. If AudioContext is not available, you should pass the raw waveform in as a Float32Array of shape (n, ). Float32Array or Float64Array of shape (n, ), representing the raw audio at the correct sampling rate (no further check will be done).
candidate_labels Array. The candidate labels for this audio.
[options] ZeroShotAudioClassificationPipelineOptions The options to use for zero-shot audio classification.

Properties

Name Type Default Description
label string The label identified by the model. It is one of the suggested candidate_label.
score number The score attributed by the model for that label (between 0 and 1).
[hypothesis_template] string ""This is a sound of {}."" The sentence used in conjunction with candidate_labelsto attempt the audio classification by replacing the placeholder with the candidate_labels. Then likelihood is estimated by using logits_per_audio.

pipelines~Chunk : Object

Kind: inner typedef of pipelines
Properties

Name Type Description
timestamp * The start and end timestamp of the chunk in seconds.
text string The recognized text.

pipelines~AutomaticSpeechRecognitionPipelineType ⇒ Promise. < (AutomaticSpeechRecognitionOutput|Array < AutomaticSpeechRecognitionOutput > ) >

Parameters specific to automatic-speech-recognition pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<(AutomaticSpeechRecognitionOutput|Array<AutomaticSpeechRecognitionOutput>)> - An object containing the transcription text and optionally timestamps if return_timestamps is true.

Param Type Description
audio AudioPipelineInputs The input audio file(s) to be transcribed. The input is either: string or URL that is the filename/URL of the audio file, the file will be read at the processor's sampling rate to get the waveform using the AudioContext API. If AudioContext is not available, you should pass the raw waveform in as a Float32Array of shape (n, ). Float32Array or Float64Array of shape (n, ), representing the raw audio at the correct sampling rate (no further check will be done).
[options] Partial. Additional keyword arguments to pass along to the generate method of the model.

Properties

Name Type Description
text string The recognized text.
[chunks] Array. When using return_timestamps, the chunks will become a list containing all the various text chunks identified by the model.
[return_timestamps] boolean | 'word' Whether to return timestamps or not. Default is false.
[chunk_length_s] number The length of audio chunks to process in seconds. Default is 0 (no chunking).
[stride_length_s] number The length of overlap between consecutive audio chunks in seconds. If not provided, defaults to chunk_length_s / 6.
[force_full_sequences] boolean Whether to force outputting full sequences or not. Default is false.
[language] string The source language. Default is null, meaning it should be auto-detected. Use this to potentially improve performance if the source language is known.
[task] string The task to perform. Default is null, meaning it should be auto-detected.
[num_frames] number The number of frames in the input audio.

pipelines~ImageToTextPipelineType ⇒ Promise. < (ImageToTextOutput|Array < ImageToTextOutput > ) >

Kind: inner typedef of pipelines
Returns: Promise.<(ImageToTextOutput|Array<ImageToTextOutput>)> - An object (or array of objects) containing the generated text(s).

Param Type Description
texts ImagePipelineInputs The images to be captioned.
[options] * Additional keyword arguments to pass along to the generate method of the model.

Properties

Name Type Description
generated_text string The generated text.

pipelines~ImageClassificationPipelineType ⇒ Promise. < (ImageClassificationOutput|Array < ImageClassificationOutput > ) >

Parameters specific to image classification pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<(ImageClassificationOutput|Array<ImageClassificationOutput>)> - An array or object containing the predicted labels and scores.

Param Type Description
images ImagePipelineInputs The input images(s) to be classified.
[options] ImageClassificationPipelineOptions The options to use for image classification.

Properties

Name Type Default Description
label string The label identified by the model.
score number The score attributed by the model for that label.
[top_k] number 1 The number of top labels that will be returned by the pipeline.

pipelines~ImageSegmentationPipelineType ⇒ Promise. < Array < ImageSegmentationPipelineOutput > >

Parameters specific to image segmentation pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<Array<ImageSegmentationPipelineOutput>> - The annotated segments.

Param Type Description
images ImagePipelineInputs The input images.
[options] ImageSegmentationPipelineOptions The options to use for image segmentation.

Properties

Name Type Default Description
label string | null The label of the segment.
score number | null The score of the segment.
mask RawImage The mask of the segment.
[threshold] number 0.5 Probability threshold to filter out predicted masks.
[mask_threshold] number 0.5 Threshold to use when turning the predicted masks into binary values.
[overlap_mask_area_threshold] number 0.8 Mask overlap threshold to eliminate small, disconnected segments.
[subtask] null | string Segmentation task to be performed. One of [panoptic, instance, and semantic], depending on model capabilities. If not set, the pipeline will attempt to resolve (in that order).
[label_ids_to_fuse] Array. List of label ids to fuse. If not set, do not fuse any labels.
[target_sizes] Array.<Array> List of target sizes for the input images. If not set, use the original image sizes.

pipelines~BackgroundRemovalPipelineType ⇒ Promise. < Array < RawImage > >

Parameters specific to image segmentation pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<Array<RawImage>> - The images with the background removed.

Param Type Description
images ImagePipelineInputs The input images.
[options] BackgroundRemovalPipelineOptions The options to use for image segmentation.

pipelines~ZeroShotImageClassificationPipelineType ⇒ Promise. < (Array < ZeroShotImageClassificationOutput > |Array < Array < ZeroShotImageClassificationOutput > > ) >

Parameters specific to zero-shot image classification pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<(Array<ZeroShotImageClassificationOutput>|Array<Array<ZeroShotImageClassificationOutput>>)> - An array of objects containing the predicted labels and scores.

Param Type Description
images ImagePipelineInputs The input images.
candidate_labels Array. The candidate labels for this image.
[options] ZeroShotImageClassificationPipelineOptions The options to use for zero-shot image classification.

Properties

Name Type Default Description
label string The label identified by the model. It is one of the suggested candidate_label.
score number The score attributed by the model for that label (between 0 and 1).
[hypothesis_template] string ""This is a photo of {}"" The sentence used in conjunction with candidate_labelsto attempt the image classification by replacing the placeholder with the candidate_labels. Then likelihood is estimated by using logits_per_image.

pipelines~ObjectDetectionPipelineType ⇒ Promise. < (ObjectDetectionPipelineOutput|Array < ObjectDetectionPipelineOutput > ) >

Parameters specific to object detection pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<(ObjectDetectionPipelineOutput|Array<ObjectDetectionPipelineOutput>)> - A list of objects or a list of list of objects.

Param Type Description
images ImagePipelineInputs The input images.
[options] ObjectDetectionPipelineOptions The options to use for object detection.

Properties

Name Type Default Description
label string The class label identified by the model.
score number The score attributed by the model for that label.
box BoundingBox The bounding box of detected object in image's original size, or as a percentage if percentage is set to true.
[threshold] number 0.9 The threshold used to filter boxes by score.
[percentage] boolean false Whether to return the boxes coordinates in percentage (true) or in pixels (false).

pipelines~ZeroShotObjectDetectionPipelineType ⇒ Promise. < (Array < ZeroShotObjectDetectionOutput > |Array < Array < ZeroShotObjectDetectionOutput > > ) >

Parameters specific to zero-shot object detection pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<(Array<ZeroShotObjectDetectionOutput>|Array<Array<ZeroShotObjectDetectionOutput>>)> - An array of objects containing the predicted labels, scores, and bounding boxes.

Param Type Description
images ImagePipelineInputs The input images.
candidate_labels Array. What the model should recognize in the image.
[options] ZeroShotObjectDetectionPipelineOptions The options to use for zero-shot object detection.

Properties

Name Type Default Description
label string Text query corresponding to the found object.
score number Score corresponding to the object (between 0 and 1).
box BoundingBox Bounding box of the detected object in image's original size, or as a percentage if percentage is set to true.
[threshold] number 0.1 The probability necessary to make a prediction.
[top_k] number The number of top predictions that will be returned by the pipeline. If the provided number is null or higher than the number of predictions available, it will default to the number of predictions.
[percentage] boolean false Whether to return the boxes coordinates in percentage (true) or in pixels (false).

pipelines~DocumentQuestionAnsweringPipelineType ⇒ Promise. < (DocumentQuestionAnsweringOutput|Array < DocumentQuestionAnsweringOutput > ) >

Kind: inner typedef of pipelines
Returns: Promise.<(DocumentQuestionAnsweringOutput|Array<DocumentQuestionAnsweringOutput>)> - An object (or array of objects) containing the answer(s).

Param Type Description
image ImageInput The image of the document to use.
question string A question to ask of the document.
[options] * Additional keyword arguments to pass along to the generate method of the model.

Properties

Name Type Description
answer string The generated text.

pipelines~TextToAudioPipelineConstructorArgs : Object

Kind: inner typedef of pipelines
Properties

Name Type Description
[vocoder] PreTrainedModel The vocoder used by the pipeline (if the model uses one). If not provided, use the default HifiGan vocoder.

pipelines~TextToAudioPipelineType ⇒ Promise. < TextToAudioOutput >

Parameters specific to text-to-audio pipelines.

Kind: inner typedef of pipelines
Returns: Promise.<TextToAudioOutput> - An object containing the generated audio and sampling rate.

Param Type Description
texts string | Array The text(s) to generate.
options TextToAudioPipelineOptions Parameters passed to the model generation/forward method.

Properties

| Name | Type | Default | Description | | | | ----------------------- | --------------------------------- | -------------------------------------------------- | ----------- | | -------------------------------------------------- | | audio | Float32Array | The generated audio waveform. | | | | | sampling_rate | number | The sampling rate of the generated audio waveform. | | | | | [speaker_embeddings] | Tensor | Float32Array | string | URL | | The speaker embeddings (if the model requires it). |

pipelines~ImageToImagePipelineType ⇒ Promise. < (RawImage|Array < RawImage > ) >

Kind: inner typedef of pipelines
Returns: Promise.<(RawImage|Array<RawImage>)> - The transformed image or list of images.

Param Type Description
images ImagePipelineInputs The images to transform.

pipelines~DepthEstimationPipelineType ⇒ Promise. < (DepthEstimationPipelineOutput|Array < DepthEstimationPipelineOutput > ) >

Kind: inner typedef of pipelines
Returns: Promise.<(DepthEstimationPipelineOutput|Array<DepthEstimationPipelineOutput>)> - An image or a list of images containing result(s).

Param Type Description
images ImagePipelineInputs The images to compute depth for.

Properties

Name Type Description
predicted_depth Tensor The raw depth map predicted by the model.
depth RawImage The processed depth map as an image (with the same size as the input image).

pipelines~AllTasks : *

All possible pipeline types.

Kind: inner typedef of pipelines

< > Update on GitHub