Serverless Open-Source Search Engine – Amazon OpenSearch Serverless – AWS (original) (raw)

Deliver search and log analytics without provisioning and adjusting resources

Why Amazon OpenSearch Serverless?

Amazon OpenSearch Serverless is a serverless option in Amazon OpenSearch Service. As a developer, you can use OpenSearch Serverless to run petabyte-scale workloads without configuring, managing, and scaling OpenSearch clusters. You get the same interactive millisecond response times as OpenSearch Service with the simplicity of a serverless environment.

Get started in seconds

Use familiar open-source ingestion and pipelines without having to change your applications.

Scale on demand

OpenSearch Serverless automatically provisions and continually adjusts to get fast data ingestion rates and millisecond response times during changing usage patterns and demand.

Improve costs

Pay only for what you use by automatically scaling resources to provide the right amount of capacity for your application—without impacting data ingestion.

Store and search vector embeddings

Power your generative artificial intelligence (AI) applications with simple, scalable, and high-performing vector search.

Flex with variable workloads

Scale application resources seamlessly without having to preconfigure needed compute power and memory.

Meet sensitive service-level agreements (SLAs)

Pre-initialize application resources and achieve response times in seconds.

Create development and test environments

Quickly create development and test environments, automatically scale with unpredictable usage, and deliver products to market faster.

Build ML-augmented search experiences

Generate more precise and accurate search results by colocating vector and text search to power your generative AI applications.

Serverless FAQs

Amazon OpenSearch Serverless is a new serverless option for OpenSearch Service. OpenSearch Serverless provides a streamlined experience that removes the need for you to provision, configure, and tune clusters. The benefits of OpenSearch Serverless are as follows:

• Automatic provisioning and scaling to provide consistently fast data ingestion rates and millisecond response times during changing usage patterns and application demand.
• Support for production workloads with redundancy for AZ outages and infrastructure failures.
• Same data durability as Amazon S3.
• Pay only for the resources consumed by your workload

The OpenSearch Serverless core components are powered by the open-source OpenSearch project that includes a search engine, OpenSearch, and a visualization interface, OpenSearch Dashboards.

OpenSearch Serverless helps you run various log analytics and search workloads. You might, however, prefer to use the OpenSearch Service managed clusters when you need tight control over cluster configuration or specific customizations. With managed clusters, you can choose your preferred instances and have more control of configuration such as data-sharding strategy. This might be critical for use cases that fall outside the typical patterns supported by OpenSearch Serverless. Also, OpenSearch Serverless currently does not support advanced features or plugins such as alerting, anomaly detection, or kNN. You can use the managed clusters for these features until OpenSearch Serverless adds support for them.

To get started, select the OpenSearch Serverless option under Amazon OpenSearch Service and create new collections, a logical grouping of indexes that work together to support a workload. You can use the command line interface (CLI), AWS SDK, or the AWS Management Console to create collections. OpenSearch Serverless supports the same ingest and query APIs as the OpenSearch open-source suite. So you can continue using your existing clients and streaming ingestion pipelines such as Amazon Kinesis Data Firehose, Kafka, Logstash, Fluent Bit, and Fluentd. When your data is in OpenSearch Serverless, you can interactively analyze and visualize your data using the serverless OpenSearch Dashboards.

OpenSearch Serverless integrates seamlessly with AWS KMS, IAM, AWS billing, CloudWatch, CloudFormation, and CloudTrail.

OpenSearch Serverless is an enhanced security feature by default. All data is encrypted at rest with a collection-level option for you to use either a service managed key or assign your own key through AWS KMS. Access to the collections is controlled through IAM, VPC security group, and SAML 2.0. OpenSearch Serverless supports hierarchical data access policies where you can configure policies at the account, collection, and index levels. You can also configure role-based access control for your collections and indexes.

When the system resources such as CPU, memory, and disk limits in the ingestion or search nodes are breached or it notices hot shards processing large amounts of read or write requests, OpenSearch Serverless horizontally scales out nodes in response to increased workload demand. Similarly, when the resource utilization falls below a certain threshold, OpenSearch Serverless will automatically and gradually scale in the resources without impacting the performance.

AWS support for Internet Explorer ends on 07/31/2022. Supported browsers are Chrome, Firefox, Edge, and Safari. Learn more »