Refactor async client connection-handling (#280)

Refactor interfaces related to the Redis client in AsyncRedisIndex (and
possibly others).

## Constraints
- We want to initialize the Redis connection lazily, not at object
instantiation or module import.
- Python doesn't have async properties, so properties are limited to
sync function calls
    - Therefore, we don't want properties to be part of the interface
- For functions we "remove," we'll leave them in place with a
deprecation warning
    - Remaining backwards-compatible until the next major release

## The Design Today
- Can create a client and set it using `index.connect()`
    - Pass in a `redis_url`
    - Calls `index.set_client()`
- Can pass in a client and set it with `index.set_client()`
    - Pass in a client instance
- **Can't** pass in a client instance with `__init__()`
    - Or anything URL, etc.
- Can create a client with `index.from_existing()`
    - Pass it a `redis_url`
    - Calls `index.set_client()`
- **Can't** use index as an async context manager
    - Requires explicit resource handling, `atexit` handler
    - But this is broken
- The `setup_async_redis()` decorator creates a function wrapper that
calls `validate_async_redis()` with every `set_client()` call
- The `RedisConnectionFactory.connect()` returns incompatible types
(sync, async `Redis`)
    - Sync vs. async depends on a parameter

## The Design After Refactor
- **Can** use as an async context manager
- Either disconnect the index manually with `disconnect()` or use it as
a context manager
- `async with Index()...` will `disconnect()` after you exit the context
block
- Lazily instantiate client with `self._get_client()` method ("private")
- Remove `set_client()` public interface if possible
    - Lazily instantiate client
- Remove `connect()` public interface if possible
    - Lazily instantiate client
- Leave `from_existing()`
- Leave `client()` property, now just returns `._redis_client` and can
be None
- Call `validate_async_redis()` when setting a new client instance for
the first time
- But make this an internal check rather than attached to public
interface
- Allow `redis_url`, `redis_kwargs`, or `redis_client` in `__init__()`

### Examples Post-Refactor
```python
#1: New instance with redis URL
index = AsyncRedisIndex(redis_url="...")
#2: New instance with existing client
index = AsyncRedisIndex(redis_client=client)
#3: New instance with `from_existing`
index = AsyncRedisIndex.from_existing(..., redis_url="...")

#4 Passing both a client and a URL is isallowed
try:
    index = AsyncRedisIndex(redis_client=client, redis_url="...")
except:
    pass
else:
    raise RuntimeError("Should have raised!")

# The client is lazily connected here, and we send telemetry.
await index.something_using_redis()
await index.something_else_using_redis()
# Close the client.
await index.close()

async with AsyncRedisIndex(redis_url="...") as index:
    # Same story: client is lazily created now
    await index.something_using_redis()
    await index.something_else_using_redis()
    # Client is automatically closed

# __enter__ is reentrant
async with index:
    # Lazily opens a new client connection.
    await index.a_third_thing_using_redis()
    # Client is automatically closed
```

---------

Co-authored-by: Tyler Hutcherson <[email protected]>

Author: abrookins

README.md

@@ -121,19 +121,18 @@ Choose from multiple Redis deployment options:
     })
     ```
 
-2. [Create a SearchIndex](https://docs.redisvl.com/en/stable/user_guide/01_getting_started.html#create-a-searchindex) class with an input schema and client connection in order to perform admin and search operations on your index in Redis:
+2. [Create a SearchIndex](https://docs.redisvl.com/en/stable/user_guide/01_getting_started.html#create-a-searchindex) class with an input schema to perform admin and search operations on your index in Redis:
     ```python
     from redis import Redis
     from redisvl.index import SearchIndex
 
-    # Establish Redis connection and define index
-    client = Redis.from_url("redis://localhost:6379")
-    index = SearchIndex(schema, client)
+    # Define the index
+    index = SearchIndex(schema, redis_url="redis://localhost:6379")
 
     # Create the index in Redis
     index.create()
     ```
-    > Async compliant search index class also available: [AsyncSearchIndex](https://docs.redisvl.com/en/stable/api/searchindex.html#redisvl.index.AsyncSearchIndex).
+    > An async-compatible index class also available: [AsyncSearchIndex](https://docs.redisvl.com/en/stable/api/searchindex.html#redisvl.index.AsyncSearchIndex).
 
 3. [Load](https://docs.redisvl.com/en/stable/user_guide/01_getting_started.html#load-data-to-searchindex)
 and [fetch](https://docs.redisvl.com/en/stable/user_guide/01_getting_started.html#fetch-an-object-from-redis) data to/from your Redis instance:
@@ -346,7 +345,7 @@ Commands:
         stats       Obtain statistics about an index
 ```
 
-> Read more about [using the CLI](https://docs.redisvl.com/en/stable/user_guide/cli.html).
+> Read more about [using the CLI](https://docs.redisvl.com/en/latest/overview/cli.html).
 
 ## 🚀 Why RedisVL?
 

docs/user_guide/01_getting_started.ipynb

@@ -81,7 +81,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -126,7 +126,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -178,7 +178,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -195,7 +195,7 @@
     "Now we also need to facilitate a Redis connection. There are a few ways to do this:\n",
     "\n",
     "- Create & manage your own client connection (recommended)\n",
-    "- Provide a simple Redis URL and let RedisVL connect on your behalf"
+    "- Provide a Redis URL and let RedisVL connect on your behalf (by default, it will connect to \"redis://localhost:6379\")"
    ]
   },
   {
@@ -209,7 +209,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 6,
    "metadata": {},
    "outputs": [
     {
@@ -227,9 +227,13 @@
     "from redis import Redis\n",
     "\n",
     "client = Redis.from_url(\"redis://localhost:6379\")\n",
+    "index = SearchIndex.from_dict(schema, redis_client=client)\n",
     "\n",
-    "index.set_client(client)\n",
-    "# optionally provide an async Redis client object to enable async index operations"
+    "# alternatively, provide an async Redis client object to enable async index operations\n",
+    "# from redis.asyncio import Redis\n",
+    "# from redisvl.index import AsyncSearchIndex\n",
+    "# client = Redis.from_url(\"redis://localhost:6379\")\n",
+    "# index = AsyncSearchIndex.from_dict(schema, redis_client=client)\n"
    ]
   },
   {
@@ -243,7 +247,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 7,
    "metadata": {},
    "outputs": [
     {
@@ -258,8 +262,10 @@
     }
    ],
    "source": [
-    "index.connect(\"redis://localhost:6379\")\n",
-    "# optionally use an async client by passing use_async=True"
+    "index = SearchIndex.from_dict(schema, redis_url=\"redis://localhost:6379\")\n",
+    "\n",
+    "# If you don't specify a client or Redis URL, the index will attempt to\n",
+    "# connect to Redis at the default address (\"redis://localhost:6379\")."
    ]
   },
   {
@@ -273,7 +279,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 8,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -297,7 +303,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 9,
    "metadata": {},
    "outputs": [
     {
@@ -315,7 +321,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 10,
    "metadata": {},
    "outputs": [
     {
@@ -358,7 +364,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 11,
    "metadata": {},
    "outputs": [
     {
@@ -392,7 +398,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 12,
    "metadata": {},
    "outputs": [
     {
@@ -429,7 +435,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 13,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -454,13 +460,20 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 14,
    "metadata": {},
    "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "*=>[KNN 3 @user_embedding $vector AS vector_distance] RETURN 6 user age job credit_score vector_distance vector_distance SORTBY vector_distance ASC DIALECT 2 LIMIT 0 3\n"
+     ]
+    },
     {
      "data": {
       "text/html": [
-       "<table><tr><th>vector_distance</th><th>user</th><th>age</th><th>job</th><th>credit_score</th></tr><tr><td>0</td><td>john</td><td>1</td><td>engineer</td><td>high</td></tr><tr><td>0</td><td>mary</td><td>2</td><td>doctor</td><td>low</td></tr><tr><td>0.0566299557686</td><td>tyler</td><td>9</td><td>engineer</td><td>high</td></tr></table>"
+       "table><tr><th>vector_distance</th><th>user</th><th>age</th><th>job</th><th>credit_score</th></tr><tr><td>0</td><td>john</td><td>1</td><td>engineer</td><td>high</td></tr><tr><td>0</td><td>mary</td><td>2</td><td>doctor</td><td>low</td></tr><tr><td>0.0566299557686</td><td>tyler</td><td>9</td><td>engineer</td><td>high</td></tr></table>"
       ],
       "text/plain": [
        "<IPython.core.display.HTML object>"
@@ -487,7 +500,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 15,
    "metadata": {},
    "outputs": [
     {
@@ -537,13 +550,12 @@
     "\n",
     "client = Redis.from_url(\"redis://localhost:6379\")\n",
     "\n",
-    "index = AsyncSearchIndex.from_dict(schema)\n",
-    "await index.set_client(client)"
+    "index = AsyncSearchIndex.from_dict(schema, redis_client=client)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 15,
+   "execution_count": 16,
    "metadata": {},
    "outputs": [
     {
@@ -584,7 +596,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 17,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -609,14 +621,14 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 18,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "11:53:25 redisvl.index.index INFO   Index already exists, overwriting.\n"
+      "11:28:32 redisvl.index.index INFO   Index already exists, overwriting.\n"
      ]
     }
    ],
@@ -627,13 +639,13 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 17,
+   "execution_count": 19,
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/html": [
-       "<table><tr><th>vector_distance</th><th>user</th><th>age</th><th>job</th><th>credit_score</th></tr><tr><td>0</td><td>john</td><td>1</td><td>engineer</td><td>high</td></tr><tr><td>0</td><td>mary</td><td>2</td><td>doctor</td><td>low</td></tr><tr><td>0.0566299557686</td><td>tyler</td><td>9</td><td>engineer</td><td>high</td></tr></table>"
+       "<table><tr><th>vector_distance</th><th>user</th><th>age</th><th>job</th><th>credit_score</th></tr><tr><td>0</td><td>mary</td><td>2</td><td>doctor</td><td>low</td></tr><tr><td>0</td><td>john</td><td>1</td><td>engineer</td><td>high</td></tr><tr><td>0.0566299557686</td><td>tyler</td><td>9</td><td>engineer</td><td>high</td></tr></table>"
       ],
       "text/plain": [
        "<IPython.core.display.HTML object>"
@@ -659,7 +671,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 20,
    "metadata": {},
    "outputs": [
     {
@@ -677,19 +689,19 @@
       "│ num_records                 │ 22          │\n",
       "│ percent_indexed             │ 1           │\n",
       "│ hash_indexing_failures      │ 0           │\n",
-      "│ number_of_uses              │ 5           │\n",
-      "│ bytes_per_record_avg        │ 50.9091     │\n",
+      "│ number_of_uses              │ 2           │\n",
+      "│ bytes_per_record_avg        │ 47.8        │\n",
       "│ doc_table_size_mb           │ 0.000423431 │\n",
-      "│ inverted_sz_mb              │ 0.00106812  │\n",
+      "│ inverted_sz_mb              │ 0.000911713 │\n",
       "│ key_table_size_mb           │ 0.000165939 │\n",
-      "│ offset_bits_per_record_avg  │ 8           │\n",
-      "│ offset_vectors_sz_mb        │ 5.72205e-06 │\n",
-      "│ offsets_per_term_avg        │ 0.272727    │\n",
-      "│ records_per_doc_avg         │ 5.5         │\n",
+      "│ offset_bits_per_record_avg  │ nan         │\n",
+      "│ offset_vectors_sz_mb        │ 0           │\n",
+      "│ offsets_per_term_avg        │ 0           │\n",
+      "│ records_per_doc_avg         │ 5           │\n",
       "│ sortable_values_size_mb     │ 0           │\n",
-      "│ total_indexing_time         │ 0.197       │\n",
-      "│ total_inverted_index_blocks │ 12          │\n",
-      "│ vector_index_sz_mb          │ 0.0201416   │\n",
+      "│ total_indexing_time         │ 0.239       │\n",
+      "│ total_inverted_index_blocks │ 11          │\n",
+      "│ vector_index_sz_mb          │ 0.235603    │\n",
       "╰─────────────────────────────┴─────────────╯\n"
      ]
     }
@@ -718,7 +730,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
+   "execution_count": 21,
    "metadata": {},
    "outputs": [
     {
@@ -727,7 +739,7 @@
        "4"
       ]
      },
-     "execution_count": 19,
+     "execution_count": 21,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -739,7 +751,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 20,
+   "execution_count": 22,
    "metadata": {},
    "outputs": [
     {
@@ -748,7 +760,7 @@
        "True"
       ]
      },
-     "execution_count": 20,
+     "execution_count": 22,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -760,7 +772,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 21,
+   "execution_count": 23,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -771,7 +783,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "env",
    "language": "python",
    "name": "python3"
   },