Introduction

To store and search over unstructured data, a widely adopted approach is embedding data into vectors, stored and indexed in vector databases. The promptflow-vectordb SDK is designed for PromptFlow, provides essential tools for vector similarity search within popular vector databases, including FAISS, Qdrant, Azure Congnitive Search, and more.

0.2.10

• Add support for bring-your-own Elasticsearch index.
• Serverless Deployments can now be used directly for embedding, without requiring the creation of a Serverless Connection.
• Rename Serverless Endpoints to Serverless Deployments.
• Remove preview tag from Index Lookup.

0.2.9

• Fix compatibility issue with langchain 0.1 that broke Azure AI Search semantic searches.
• Refactor metadata retrieval in Index Lookup. Metadata fields that are specifically requested are now present in the metadata property of a retrieval result, and all other retrieved fields have been moved to additional_fields, instead of being discarded.
• Add support for bring-your-own Azure CosmosDB for MongoDB vCore index.

0.2.8

• Add support for langchain 0.1
• Replace FAISS Index Lookup, Vector Index Lookup and Vector DB Lookup internals with Index Lookup internals.
• Use azureml.rag logger and promptflow.tool logger in Index Lookup.

0.2.7

• Add support for Serverless Deployment connections for embeddings in Index Lookup.
• Add support for multiple instances of Index Lookup running in the same process without conflicts.
• Auto-detect embedding vector length for supported embedding models.

0.2.6

• Emit granular trace information from Index Lookup for use by Action Analyzer.

0.2.5

• Introduce improved error messaging when input queries are of an unexpected type.
• Mark FAISS Index Lookup, Vector Index Lookup and Vector DB Lookup as archived.
• Add support for text-embedding-3-small and text-embedding-3-large embedding models.

0.2.4

• Mark FAISS Index Lookup, Vector Index Lookup and Vector DB Lookup as deprecated.
• Introduced a self section in the mlindex_content YAML, to carry information about the asset ID and path from which the MLIndex was retrieved.
• Index Lookup now caches vectorstore build steps for better runtime performance.
• Use functools.lru_cache instead of functools.cache for compatibility with python < 3.9
• Use ruamel.yaml instead of pyyaml, so that yaml 1.2 is supported.

0.2.3

• Implement HTTP caching to improve callback performance.
• Not specifying a value for embedding_type produces the same behavior as selecting None.
• Index Lookup honors log levels set via the PF_LOGGING_LEVEL environment variable.

0.2.2

• Introduced new tool - Index Lookup, to serve as a single tool to perform lookups against supported index types.
• Marked Index Lookup as preview.