feat: Opensearch update and parameterise timeouts by edwinjosechittilappilly · Pull Request #11881 · langflow-ai/langflow (original) (raw)
added 2 commits
Register the component as a VectorStore and extend the OpenSearch multimodal component to handle multi-model embeddings and robust ingestion/search. Updates include adding VectorStore base class, new inputs (request_timeout, max_retries), a vector store connection tool output, and an updated code_hash. The OpenSearch component code was enhanced with model name normalization, dynamic per-model embedding field names, multi-model detection/mapping, parallelized and rate-aware embedding generation (with retries), improved index mapping management, AOSS engine validation, and other ingestion/search robustness improvements.
[![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4)
Change the default 'engine' option from 'nmslib' to 'jvector' in OpenSearch vector store components. Updated in src/lfx/src/lfx/components/elastic/opensearch.py and opensearch_multimodal.py. Note: 'jvector' requires OpenSearch 2.9+, so ensure cluster compatibility before deploying.
Add multi-model multimodal support and robustness to the OpenSearch vector store component: introduce normalize_model_name and get_embedding_field_name to create dynamic fields (chunk_embedding_{model}), track embedding_model and embedding_dimensions in mappings, auto-detect available embedding models, select embeddings by deployment/model identifiers (including available_models dict), and use parallel or sequential embedding generation with rate-limit-aware retries (tenacity) and IBM-specific throttling. Improve index mapping handling, lazy addition of knn_vector fields, AOSS/engine validation, and bulk ingestion logic. Also update component index code_hash to reflect these changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})