Feature Proposal: VectorSearch enabled SQLChain? #5122
Description
Feature request
A VectorSearch enabled SQLChain that is capable of doing JOIN, WHERE filters and vector search at the same time.
It should be agnostic to any SQL database backend that supports common SQL and vector search, with customizable distance function composer.
Motivation
Hello from MyScale AI team! 😊👋
We have been working on features to fill up the gap among SQL, vector search and LLM applications. Some inspiring works like self-query retrievers for VectorStores (for example Weaviate and others) really turn those vector search databases into a powerful knowledge base! 🚀🚀
We are thinking if we can merge all in one, like SQL and vector search and LLMChains, making this SQL vector database memory as the only source of your data. Here are some benefits we can think of for now, maybe you have more 👀:
- With ALL data you have: since you store all your pasta in the database, you don't need to worry about the foreign keys or links between names from other data source.
- Flexible data structure: Even if you have changed your schema, for example added a table, the LLM will know how to
JOINthose tables and use those as filters. - SQL compatibility: We found that vector databases that supports SQL in the marketplace have similar interfaces, which means you can change your backend with no pain, just change the name of the distance function in your DB solution and you are ready to go!
We would like to consider PGVector and MyScale for now, but if you want more, just comment below and we will push hard to ship it! 🏃🏃
Your contribution
A PR contains the VectorSQLChain