Skip to content

Add Cursor API Support to Aggregations #242

@elicore

Description

@elicore

The current version doesn't support search aggregations (FT.AGGREGATE) that use the WITHCURSOR modifier.

When benchmarking paginated searches, performance of a query using FT.AGGREGATE ... WITHCURSOR was more than 10x better than the equivalent query using FT.SEARCH, with the entire document being returned.

A workaround using the current version would be to construct the query manually with jedis and then deserialize the results to the document class:

// JSON serializer used by redis-om-spring, replace with a handle to an existing Gson instance
Gson gson = new Gson();
// create a jedis connection pool, replace this with a handle to your client
JedisPooled client = new JedisPooled("localhost", 6379);

// build the aggregation query
String query = "<the search query>";

// use .cursor() to load 50 results at a time, returning the cursor-id
AggregationBuilder ab = new AggregationBuilder(query)
        .cursor(50, Long.MAX_VALUE)
        .sortBy(1000, SortedField.desc("@my-sort-field"))
        .loadAll();

// execute the aggregation query
AggregationResult ar = client.ftAggregate("my-index", ab);

// get the results and deserialize the document payload
List<MyDocument> presults = ar.getResults().stream().map(
        d -> gson.fromJson(
                (String)d.get("$"),
                MyDocument.class
        ))
        .toList();
ArrayList<MyDocument> results = new ArrayList<>(presults);

// get the cursor id
long cursorId = ar.getCursorId();

// read the rest of the results
while (0 != cursorId) {
    ar = client.ftCursorRead("my-index", cursorId, 50);
    // parse the results
    presults = ar.getResults().stream().map(
                    d -> gson.fromJson(
                            (String)d.get("$"),
                            MyDocument.class
                    ))
            .toList();
    results.addAll(presults);
    cursorId = ar.getCursorId();
}

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions