diff --git a/next-env.d.ts b/next-env.d.ts index 52e831b43..254b73c16 100644 --- a/next-env.d.ts +++ b/next-env.d.ts @@ -1,5 +1,6 @@ /// /// +/// // NOTE: This file should not be edited // see https://nextjs.org/docs/pages/api-reference/config/typescript for more information. diff --git a/pages/advanced-algorithms.mdx b/pages/advanced-algorithms.mdx index 42374738e..78bc47c58 100644 --- a/pages/advanced-algorithms.mdx +++ b/pages/advanced-algorithms.mdx @@ -13,11 +13,11 @@ If you require procedures designed to solve specific graph problems, there is a number of advanced algorithms available in Memgraph. [BFS](/advanced-algorithms/deep-path-traversal#breadth-first-search), -[DFS](/advanced-algorithms/deep-path-traversal#depth-first-search), -[Weighted shortest -path](/advanced-algorithms/deep-path-traversal#weighted-shortest-path), +[DFS](/advanced-algorithms/deep-path-traversal#depth-first-search), [Weighted +shortest path](/advanced-algorithms/deep-path-traversal#weighted-shortest-path), [All shortest -paths](/advanced-algorithms/deep-path-traversal#all-shortest-paths) are +paths](/advanced-algorithms/deep-path-traversal#all-shortest-paths) and [K +shortest paths](/advanced-algorithms/deep-path-traversal#k-shortest-paths) are built-in deep path traversal algorithms you can run using their specific clauses. diff --git a/pages/advanced-algorithms/available-algorithms.mdx b/pages/advanced-algorithms/available-algorithms.mdx index 1518e04e2..f80acbb46 100644 --- a/pages/advanced-algorithms/available-algorithms.mdx +++ b/pages/advanced-algorithms/available-algorithms.mdx @@ -22,6 +22,7 @@ library](/advanced-algorithms/install-mage). | [Breadth-first search](/advanced-algorithms/deep-path-traversal#breadth-first-search) | C++ | An algorithm for traversing through a graph starting based on nodes' breadth (distance from the source node). | | [Weighted shortest path](/advanced-algorithms/deep-path-traversal#weighted-shortest-path) | C++ | The weighted shortest path problem is the problem of finding a path between two nodes in a graph such that the sum of the weights of relationships connecting nodes, or the sum of the weight of some node property on the path, is minimized. | | [All shortest paths](/advanced-algorithms/deep-path-traversal#all-shortest-paths) | C++ | Finding all shortest paths is an expansion of the weighted shortest paths problem. The goal of finding the shortest path is obtaining any minimum sum of weights on the path from one node to the other. | +| [K shortest paths](/advanced-algorithms/deep-path-traversal#k-shortest-paths) | C++ | Returning K shortest paths between 2 nodes in order of shortest to longest paths. | ## Traditional graph algorithms @@ -84,6 +85,7 @@ If you want to know more and learn how this affects you, read our [announcement] | Algorithms | Lang | Description | |------------------------------------------------------------------------------------------|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [collections](/advanced-algorithms/available-algorithms/collections) | C++ | The collections module is a collection manipulation module that offers functions to work with lists in Cypher queries, allowing operations like filtering, sorting, and modification for efficient data handling. | +| [convert](/advanced-algorithms/available-algorithms/convert) | C++ | The convert module is a data transformation module that offers functions to convert various data structures into different formats, allowing operations like data type transformation and structural modifications for efficient data handling. | | [create](/advanced-algorithms/available-algorithms/create) | C++ | The create module is a set of powerful tools that provide additional capabilities for creating nodes and relationships in the Memgraph graph database. | | [csv_utils](/advanced-algorithms/available-algorithms/csv_utils) | C++ | An utility module for creating and deleting CSV files. | | [date](/advanced-algorithms/available-algorithms/date) | Python | The `date` module provides various utilities to handle date and time operations within the Cypher query language. | @@ -123,3 +125,55 @@ If you want to know more and learn how this affects you, read our [announcement] | [elasticsearch](/advanced-algorithms/available-algorithms/elasticsearch_synchronization) | Python | A module used for synchronizing Memgraph and Elasticsearch. | | [igraph](/advanced-algorithms/available-algorithms/igraphalg) | Python | A module that provides igraph integration with Memgraph and implements many igraph algorithms. | | [nxalg](/advanced-algorithms/available-algorithms/nxalg) | Python | A module that provides NetworkX integration with Memgraph and implements many NetworkX algorithms. | + + +## APOC mappings + +This table shows the mapping between APOC functions/procedures and their Memgraph equivalents. Use these mappings if you're familiar with Neo4j's APOC library. + +| APOC | Description | Memgraph equivalent | +|-------------------------|-------------|---------------------| +| apoc.coll.union | Unites two lists into one, eliminating duplicates | [collections.union()](/advanced-algorithms/available-algorithms/collections#union) | +| apoc.coll.unionAll | Returns the union of two input lists, including duplicates | [collections.union_all()](/advanced-algorithms/available-algorithms/collections#union_all) | +| apoc.coll.removeAll | Removes defined elements from the input list | [collections.remove_all()](/advanced-algorithms/available-algorithms/collections#remove_all) | +| apoc.coll.contains | Verifies the existence of an input value in an input list | [collections.contains()](/advanced-algorithms/available-algorithms/collections#contains) | +| apoc.coll.flatten | Returns flattened list of inputs provided | [collections.flatten()](/advanced-algorithms/available-algorithms/collections#flatten) | +| apoc.coll.pairs | Creates pairs of neighbor elements within an input list | [collections.pairs()](/advanced-algorithms/available-algorithms/collections#pairs) | +| apoc.coll.toSet | Converts the input list to a set | [collections.to_set()](/advanced-algorithms/available-algorithms/collections#to_set) | +| apoc.coll.sum | Calculates the sum of listed elements | [collections.sum()](/advanced-algorithms/available-algorithms/collections#sum) | +| apoc.coll.partition | Partitions the input list into sub-lists of the specified size | [collections.partition()](/advanced-algorithms/available-algorithms/collections#partition) | +| apoc.convert.toTree | Converts values into tree structures | [convert_c.to_tree()](/advanced-algorithms/available-algorithms/convert_c#to_tree) | +| apoc.convert.fromJsonList | Converts a JSON string representation of a list into an actual list object | [json_util.from_json_list()](/advanced-algorithms/available-algorithms/json_util#from_json_list) | +| apoc.convert.toJson | Converts any value to its JSON string representation | [json_util.to_json()](/advanced-algorithms/available-algorithms/json_util#to_json) | +| apoc.create.node | Creates a single node with specified labels and properties | [create.node()](/advanced-algorithms/available-algorithms/create#node) | +| apoc.create.nodes | Creates multiple nodes with specified labels and properties | [create.nodes()](/advanced-algorithms/available-algorithms/create#nodes) | +| apoc.create.removeProperties | Removes properties from nodes | [create.remove_properties()](/advanced-algorithms/available-algorithms/create#remove_properties) | +| apoc.create.removeLabels | Removes labels from nodes | [create.remove_labels()](/advanced-algorithms/available-algorithms/create#remove_labels) | +| apoc.date.convertFormat | Converts date strings between different formats | [date.convert_format()](/advanced-algorithms/available-algorithms/date#convert_format) | +| apoc.label.exists | Checks if a label exists on a node | [label.exists()](/advanced-algorithms/available-algorithms/label#exists) | +| apoc.map.removeKeys | Removes specified keys from a map | [map.remove_keys()](/advanced-algorithms/available-algorithms/map#remove_keys) | +| apoc.map.merge | Merges multiple maps into one | [map.merge()](/advanced-algorithms/available-algorithms/map#merge) | +| apoc.map.fromLists | Creates a map from two lists (keys and values) | [map.from_lists()](/advanced-algorithms/available-algorithms/map#from_lists) | +| apoc.refactor.from | Redirects relationship to use a new start node | [refactor.from()](/advanced-algorithms/available-algorithms/refactor#from) | +| apoc.refactor.to | Redirects relationship to use a new end node | [refactor.to()](/advanced-algorithms/available-algorithms/refactor#to) | +| apoc.refactor.rename.label | Renames a label from old to new for all nodes | [refactor.rename_label()](/advanced-algorithms/available-algorithms/refactor#rename_label) | +| apoc.refactor.rename.nodeProperty | Renames a property from old to new for all nodes | [refactor.rename_node_property()](/advanced-algorithms/available-algorithms/refactor#rename_node_property) | +| apoc.refactor.cloneNodes | Clones specific nodes in the graph | [refactor.clone_nodes()](/advanced-algorithms/available-algorithms/refactor#clone_nodes) | +| apoc.refactor.cloneSubgraph | Clones the subgraph by cloning nodes and relationships | [refactor.clone_subgraph()](/advanced-algorithms/available-algorithms/refactor#clone_subgraph) | +| apoc.refactor.cloneSubgraphFromPaths | Clones subgraph specified by a list of paths | [refactor.clone_subgraph_from_paths()](/advanced-algorithms/available-algorithms/refactor#clone_subgraph_from_paths) | +| apoc.refactor.collapseNode | Collapses a node into a relationship | [refactor.collapse_node()](/advanced-algorithms/available-algorithms/refactor#collapse_node) | +| apoc.refactor.invert | Inverts the direction of a given relationship | [refactor.invert()](/advanced-algorithms/available-algorithms/refactor#invert) | +| apoc.refactor.normalizeAsBoolean | Normalizes properties to true/false values | [refactor.normalize_as_boolean()](/advanced-algorithms/available-algorithms/refactor#normalize_as_boolean) | +| apoc.refactor.extractNode | Creates a node from a relationship | [refactor.extract_node()](/advanced-algorithms/available-algorithms/refactor#extract_node) | +| apoc.refactor.deleteAndReconnect | Deletes nodes and reconnects remaining nodes | [refactor.delete_and_reconnect()](/advanced-algorithms/available-algorithms/refactor#delete_and_reconnect) | +| apoc.refactor.renameType | Changes the relationship type | [refactor.rename_type()](/advanced-algorithms/available-algorithms/refactor#rename_type) | +| apoc.refactor.rename.typeProperty | Renames the property of a relationship | [refactor.rename_type_property()](/advanced-algorithms/available-algorithms/refactor#rename_type_property) | +| apoc.refactor.mergeNodes | Merges properties, labels and relationships for source nodes to target node | [refactor.mergeNodes()](/advanced-algorithms/available-algorithms/refactor#mergenodes) | +| apoc.text.join | Joins all strings into a single one with given delimiter | [text.join()](/advanced-algorithms/available-algorithms/text#join) | +| apoc.text.indexOf | Finds the index of first occurrence of a substring within a string | [text.indexOf()](/advanced-algorithms/available-algorithms/text#indexof) | +| apoc.text.regexGroups | Returns all matched subexpressions of regex on provided text | [text.regexGroups()](/advanced-algorithms/available-algorithms/text#regexgroups) | +| apoc.text.format | Formats strings using the C++ fmt library | [text.format()](/advanced-algorithms/available-algorithms/text#format) | +| apoc.text.replace | Replaces substrings matching regex with replacement | [text.replace()](/advanced-algorithms/available-algorithms/text#replace) | +| apoc.text.regReplace | Replaces substrings matching regex with replacement | [text.regReplace()](/advanced-algorithms/available-algorithms/text#regreplace) | +| apoc.util.md5 | Gets MD5 hash of concatenated string representations | [util_module.md5()](/advanced-algorithms/available-algorithms/util_module#md5) | +| apoc.util.validatePredicate | Raises exception if predicate yields true with parameter interpolation | [mgps.validate_predicate()](/advanced-algorithms/available-algorithms/mgps#validate_predicate) | diff --git a/pages/advanced-algorithms/available-algorithms/_meta.ts b/pages/advanced-algorithms/available-algorithms/_meta.ts index f2ec94d59..d012cdc8a 100644 --- a/pages/advanced-algorithms/available-algorithms/_meta.ts +++ b/pages/advanced-algorithms/available-algorithms/_meta.ts @@ -9,6 +9,8 @@ export default { "csv_utils": "csv_utils", "community_detection_online": "community_detection_online", "community_detection": "community_detection", + "convert": "convert", + "convert_c": "convert_c", "do": "do", "create": "create", "cugraph": "cugraph", diff --git a/pages/advanced-algorithms/available-algorithms/collections.mdx b/pages/advanced-algorithms/available-algorithms/collections.mdx index 7c8cf9dd0..9e233c0a7 100644 --- a/pages/advanced-algorithms/available-algorithms/collections.mdx +++ b/pages/advanced-algorithms/available-algorithms/collections.mdx @@ -103,6 +103,10 @@ RETURN collections.contains_sorted([1, 2, 3.3, 4.4, 5], 2) AS contains; Unites two lists into one, eliminating duplicates. + +This function is equivalent to **apoc.coll.union**. + + {

Input:

} - `first: List[Any]` ➡ The first list of elements. @@ -132,6 +136,10 @@ RETURN collections.union([0, 1, 2, 3], [2, 2, 3, 4, 5]) AS union; Returns the union of two input lists, including duplicates. + +This function is equivalent to **apoc.coll.unionAll**. + + {

Input:

} - `first: List[Any]` ➡ The first list. @@ -162,6 +170,10 @@ RETURN collections.union_all([1,1,2,3],[3,"a","b","c"]) AS return_list; Removes defined elements from the input list. If a non-existent element is passed in the list, it will be ignored. + +This function is equivalent to **apoc.coll.removeAll**. + + {

Input:

} - `first: List[Any]` ➡ The list from which elements need to be removed. @@ -194,6 +206,10 @@ in the input list. Verifies the existence of an input value in an input list. + +This function is equivalent to **apoc.coll.contains**. + + {

Input:

} - `coll: List[Any]` ➡ The input list checked for a certain value. @@ -279,10 +295,50 @@ RETURN collections.intersection([1, 1, 2, 3, 4, 5], [1, 1, 3, 5, 7, 9]) AS inter +---------------------------------------------------------+ ``` +### `flatten()` + +Returns flattened list of inputs provided. + + +This function is equivalent to **apoc.coll.flatten**. + + +{

Input:

} + +- `list: List[Any]` ➡ The list used to flatten. + +{

Output:

} + +- `List[Any]` ➡ The list containing elements from the input list. + +{

Usage:

} + +The following query will flatten input list into same level: + +```cypher +WITH [1] as nums, + ["text", 2.5] as mixed, + [true, false] as bools +WITH COLLECT(nums) + [mixed] + [bools] + COLLECT(null) as input_list +RETURN collections_module.flatten(input_list) as result +``` + +```plaintext ++---------------------------------------------------------+ +| result | ++---------------------------------------------------------+ +| [1, "text", 2.5, True, False] | ++---------------------------------------------------------+ +``` + ### `pairs()` Creates pairs of neighbor elements within an input list. + +This function is equivalent to **apoc.coll.pairs**. + + {

Input:

} - `list: List[Any]` ➡ The list used to create pairs. @@ -311,6 +367,10 @@ RETURN collections.pairs([3, "s", 4.4, [1, 2]]) AS pairs; Converts the input list to a set. + +This function is equivalent to **apoc.coll.toSet**. + + {

Input:

} - `values: List[Any]` ➡ The list that will be converted into a set. @@ -341,6 +401,10 @@ Calculates the sum of listed elements if they are of the same type and can be summed (the elements need to be numerics). Listing elements of different data types, or data types that are impossible to sum, will throw an exception. + +This function is equivalent to **apoc.coll.sum**. + + {

Input:

} - `numbers: List[Any]` ➡ The list of elements that will be summed up. @@ -528,6 +592,10 @@ RETURN split; Partitions the input list into sub-lists of the specified `partition_size`. + +This procedure is equivalent to **apoc.coll.partition**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. diff --git a/pages/advanced-algorithms/available-algorithms/convert.mdx b/pages/advanced-algorithms/available-algorithms/convert.mdx new file mode 100644 index 000000000..35450a60b --- /dev/null +++ b/pages/advanced-algorithms/available-algorithms/convert.mdx @@ -0,0 +1,65 @@ +--- +title: convert +description: Harness Memgraph's convert module to transform data structures efficiently. Visualize the potential with detailed documentation and tutorials. +--- + +# convert + +import { Callout } from 'nextra/components' +import { Cards } from 'nextra/components' +import GitHub from '/components/icons/GitHub' + +The convert module is a data transformation module that offers functions to +convert various data structures into different formats, allowing operations like +data type transformation and structural modifications for efficient data +handling. + +Functions in the collection are called inline. + + + + } + title="Source code" + href="https://github.com/memgraph/mage/blob/main/cpp/convert_module/convert.cpp" + /> + + +| Trait | Value | +| ------------------- | ---------- | +| **Module type** | util | +| **Implementation** | C++ | +| **Parallelism** | sequential | + +## Functions + +### `str2object()` + +Converts a JSON string to an object representation. This function is useful for +parsing JSON data and converting it into a usable object format. + +{

Input:

} + +- `string: String` ➡ The JSON string to be converted to an object. + +{

Output:

} + +- `object: Any` ➡ The resulting object representation of the JSON string. + +{

Usage:

} + +Use the following query to convert a JSON string to an object: + +```cypher +RETURN convert.str2object('{"name": "Alice", "age": 30, "city": "New York"}') AS result; +``` + +The output shows the parsed object: + +```json +{ + "name": "Alice", + "age": 30, + "city": "New York" +} +``` diff --git a/pages/advanced-algorithms/available-algorithms/convert_c.mdx b/pages/advanced-algorithms/available-algorithms/convert_c.mdx new file mode 100644 index 000000000..ad30e9513 --- /dev/null +++ b/pages/advanced-algorithms/available-algorithms/convert_c.mdx @@ -0,0 +1,176 @@ +--- +title: convert_c +description: Harness Memgraph's convert_c module to transform data structures into tree representations efficiently. Visualize the potential with detailed documentation and tutorials. +--- + +# convert_c + +import { Callout } from 'nextra/components' +import { Cards } from 'nextra/components' +import GitHub from '/components/icons/GitHub' + +The `convert_c` module is a data transformation module that offers functions to +convert various data structures into different formats, allowing operations like +tree conversion, data type transformation, and structural modifications for +efficient data handling. + +Functions in the collection are called inline, while the procedure is called +using the `CALL` subclause. + + + + } + title="Source code" + href="https://github.com/memgraph/mage/blob/main/cpp/convert_module/convert.c" + /> + + +| Trait | Value | +| ------------------- | ---------- | +| **Module type** | util | +| **Implementation** | C | +| **Graph direction** | directed | +| **Edge weights** | unweighted | +| **Parallelism** | sequential | + +## Procedures + +### `to_tree()` + +Converts the provided value into a tree structure based on the specified +configuration. This procedure is useful for transforming hierarchical data, +nested objects, or graph structures into a standardized tree format. + + +This procedure is equivalent to **apoc.convert.toTree**. + + +{

Input:

} + +- `paths: Any` ➡ The input value that needs to be converted to a tree structure. + This can be a single path, a list of paths, or null. +- `lowerCaseRels: Boolean (default = true)` (**OPTIONAL**) ➡ If true, + relationship types will be converted to lowercase in the output tree + structure. +- `config: Map({})` (**OPTIONAL**) ➡ Configuration options for filtering node + and relationship properties. The config can contain: + - `nodes`: Map of node labels to property filter lists + - `rels`: Map of relationship types to property filter lists + + Property filters can be: +- `["*"]` - Include all properties (wildcard) +- `["prop1", "prop2"]` - Include only specified properties +- `["-prop1", "-prop2"]` - Exclude specified properties + +**Example filter configuration:** +```cypher +{ + nodes: { + Person: ["name", "age", "email"], + Company: ["*"], + Address: ["-internal_id", "-created_at"] + }, + rels: { + WORKS_FOR: ["since", "position"], + LIVES_AT: ["*"], + KNOWS: ["-confidence_score"] + } +} +``` + +This configuration: +- **Person nodes**: Include only `name`, `age`, and `email` properties +- **Company nodes**: Include all properties (wildcard) +- **Address nodes**: Include all properties except `internal_id` and `created_at` +- **WORKS_FOR relationships**: Include only `since` and `position` properties +- **LIVES_AT relationships**: Include all properties (wildcard) +- **KNOWS relationships**: Include all properties except `confidence_score` + + + **Limitations due to C implementation:** + - **Maximum 16 root nodes**: If processing multiple paths with different root + nodes, the procedure can handle at most 16 unique root nodes + - **Maximum 16 properties per filter**: Each property filter list can contain + at most 16 property names + - **Maximum 16 node labels**: The config can specify filters for at most 16 + different node labels + - **Maximum 16 relationship types**: The config can specify filters for at + most 16 different relationship types + - **String truncation**: Property names, node labels, and relationship types + - longer than 256 characters will be truncated + + +{

Output:

} + +- `value: Any` ➡ The resulting tree structure. + +{

Usage:

} + +Use the following query to convert graph paths to a tree structure: + +```cypher +CREATE (a:Student {name: 'Ana'}); +CREATE (b:Student {name: 'Bob'}); +CREATE (c:Student {name: 'Carol'}); +CREATE (d:Student {name: 'Dave'}); +CREATE (e:Student {name: 'Eve'}); +MATCH (a:Student {name: 'Ana'}), (b:Student {name: 'Bob'}) CREATE (a)-[:FRIEND]->(b); +MATCH (a:Student {name: 'Ana'}), (c:Student {name: 'Carol'}) CREATE (a)-[:FRIEND]->(c); +MATCH (b:Student {name: 'Bob'}), (d:Student {name: 'Dave'}) CREATE (b)-[:COLLEAGUE]->(d); +MATCH (c:Student {name: 'Carol'}), (e:Student {name: 'Eve'}) CREATE (c)-[:FRIEND]->(e); +MATCH (d:Student {name: 'Dave'}), (e:Student {name: 'Eve'}) CREATE (d)-[:FRIEND]->(e); + +MATCH p = (a:Student {name: 'Ana'})-[*]->(e:Student {name: 'Eve'}) +WITH collect(p) AS paths +CALL convert_c.to_tree(paths) YIELD value +RETURN value; +``` + +The output shows the nested tree structure with relationships and their properties: + +```json +{ + "_id": 0, + "_type": "Student", + "friend": [ + { + "_id": 1, + "_type": "Student", + "colleague": [ + { + "_id": 3, + "_type": "Student", + "colleague._id": 2, + "friend": [ + { + "_id": 4, + "_type": "Student", + "friend._id": 4, + "name": "Eve" + } + ], + "name": "Dave" + } + ], + "friend._id": 0, + "name": "Bob" + }, + { + "_id": 2, + "_type": "Student", + "friend": [ + { + "_id": 4, + "_type": "Student", + "friend._id": 3, + "name": "Eve" + } + ], + "friend._id": 1, + "name": "Carol" + } + ], + "name": "Ana" +} +``` diff --git a/pages/advanced-algorithms/available-algorithms/create.mdx b/pages/advanced-algorithms/available-algorithms/create.mdx index 98d50de70..77bd1637c 100644 --- a/pages/advanced-algorithms/available-algorithms/create.mdx +++ b/pages/advanced-algorithms/available-algorithms/create.mdx @@ -41,6 +41,10 @@ Provides a more flexible way of creating nodes than Cypher’s `CREATE` clause b allowing labels and properties to be extracted as results of other procedures and sent as a list or map. + +This procedure is equivalent to **apoc.create.node**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -86,6 +90,10 @@ RETURN node; Create nodes with given labels and properties. For each property map, a separate node is created. + +This procedure is equivalent to **apoc.create.nodes**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -267,6 +275,10 @@ Removes the properties from the given node(s). Input node(s) can be a single node, node ID, or a list of nodes and node IDs. Otherwise, an exception is thrown. + +This procedure is equivalent to **apoc.create.removeProperties**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -367,6 +379,10 @@ RETURN relationship; Removes the provided labels from the node(s). + +This procedure is equivalent to **apoc.create.removeLabels**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. diff --git a/pages/advanced-algorithms/available-algorithms/date.mdx b/pages/advanced-algorithms/available-algorithms/date.mdx index f99a7dfc1..ff4be425d 100644 --- a/pages/advanced-algorithms/available-algorithms/date.mdx +++ b/pages/advanced-algorithms/available-algorithms/date.mdx @@ -194,6 +194,10 @@ RETURN date.add(100, "day", 24, "h") AS sum; Converts a string containing a date or time in one specified format into a string of another specified date or time format. + +This function is equivalent to **apoc.date.convertFormat**. + + {

Input:

} - `temporal: string` ➡ The date to be converted. diff --git a/pages/advanced-algorithms/available-algorithms/json_util.mdx b/pages/advanced-algorithms/available-algorithms/json_util.mdx index 36c07a13e..ad2307870 100644 --- a/pages/advanced-algorithms/available-algorithms/json_util.mdx +++ b/pages/advanced-algorithms/available-algorithms/json_util.mdx @@ -28,11 +28,77 @@ and if it is a map, the module loads it as a single value. | **Implementation** | Python | | **Parallelism** | sequential | -## Procedures +## Functions + +### `from_json_list()` + +Converts a JSON string representation of a list into an actual list object. + + +This function is equivalent to **apoc.convert.fromJsonList**. + + +{

Input:

} + +- `json_str: string` ➡ The JSON string representation of a list that needs to be converted. + +{

Output:

} + +- `List[Any]` ➡ The converted list object from the JSON string. + +{

Usage:

} + +Use the following query to convert a JSON string to a list: + +```cypher +RETURN json_util.from_json_list('[1, 2, 3, "hello", true]') AS result; +``` + +Results: + +```plaintext ++---------------------------------------------------------+ +| result | ++---------------------------------------------------------+ +| [1, 2, 3, "hello", true] | ++---------------------------------------------------------+ +``` -### `load_from_path()` +### `to_json()` -The procedure loads data from a local JSON file. +Converts any value to its JSON string representation. + + +This function is equivalent to **apoc.convert.toJson**. + + +{

Input:

} + +- `value: Any` ➡ The value that needs to be converted to JSON string format. + +{

Output:

} + +- `string` ➡ The JSON string representation of the input value. + +{

Usage:

} + +Use the following query to convert a value to JSON string: + +```cypher +RETURN json_util.to_json([1, 2, 3, "hello", true]) AS result; +``` + +Results: + +```plaintext ++---------------------------------------------------------+ +| result | ++---------------------------------------------------------+ +| "[1, 2, 3, \"hello\", true]" | ++---------------------------------------------------------+ +``` + +## Procedures {

Input:

} diff --git a/pages/advanced-algorithms/available-algorithms/label.mdx b/pages/advanced-algorithms/available-algorithms/label.mdx index 79cf66bd7..048fe94da 100644 --- a/pages/advanced-algorithms/available-algorithms/label.mdx +++ b/pages/advanced-algorithms/available-algorithms/label.mdx @@ -35,6 +35,10 @@ to check the existence of a label within the node. Checks whether a specified node is labeled with the provided label. + +This function is equivalent to **apoc.label.exists**. + + {

Input:

} - `node: Any` ➡ The target node for label check. diff --git a/pages/advanced-algorithms/available-algorithms/map.mdx b/pages/advanced-algorithms/available-algorithms/map.mdx index 244cb0de4..4ae8f66f6 100644 --- a/pages/advanced-algorithms/available-algorithms/map.mdx +++ b/pages/advanced-algorithms/available-algorithms/map.mdx @@ -88,6 +88,10 @@ RETURN map.remove_key({c: "b", d: {e: "ba", c: "h", a: {c: "z"}}}, "c",{recursiv The following procedure removes keys from the input map. If recursive option is set to `true`, it will remove keys from maps nested inside the map. + +This function is equivalent to **apoc.map.removeKeys**. + + {

Input:

} - `map: Map[Any]` ➡ The input map. @@ -169,6 +173,10 @@ RETURN map.from_pairs([["b", 3], ["c", "c"]]) AS map; The procedure merges two maps into one. If the same key occurs twice, the later value will overwrite the previous one. + +This function is equivalent to **apoc.map.merge**. + + {

Input:

} - `first: Map` ➡ A map containing key-value pairs that need to be merged with another map. @@ -227,6 +235,10 @@ RETURN map.flatten({a: {b:3, d:4}},"/") AS result; The procedure makes a map from lists of keys and corresponding values. + +This function is equivalent to **apoc.map.fromLists**. + + {

Input:

} - `keys: List[string]` ➡ A list of keys. diff --git a/pages/advanced-algorithms/available-algorithms/mgps.mdx b/pages/advanced-algorithms/available-algorithms/mgps.mdx index 7143d01c8..a1e77e4d5 100644 --- a/pages/advanced-algorithms/available-algorithms/mgps.mdx +++ b/pages/advanced-algorithms/available-algorithms/mgps.mdx @@ -36,6 +36,10 @@ means you can filter out data that doesn't meet your validation criteria, and provide an error if the validation fails for a specific reason you want to highlight. + +This function is equivalent to **apoc.util.validatePredicate**. + + {

Input:

} - `predicate: boolean` ➡ Predicate condition to evaluate. - `message: string` ➡ The text of the error message raised if the predicate diff --git a/pages/advanced-algorithms/available-algorithms/refactor.mdx b/pages/advanced-algorithms/available-algorithms/refactor.mdx index b4f65d71e..4f16db965 100644 --- a/pages/advanced-algorithms/available-algorithms/refactor.mdx +++ b/pages/advanced-algorithms/available-algorithms/refactor.mdx @@ -35,6 +35,10 @@ You can execute this algorithm on [graph projections, subgraphs or portions of t Redirect the relationship to use a new start (from) node. + +This procedure is equivalent to **apoc.refactor.from**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -65,6 +69,10 @@ MATCH (:Person {name: "Ivan"})-[rel:Friends]->(:Person {name: "Matija"}) MATCH ( Redirect the relationship to use a new end (to) node. + +This procedure is equivalent to **apoc.refactor.to**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -101,6 +109,10 @@ Rename a label from `old_label` to `new_label` for all nodes. If `nodes` is provided renaming is applied only to the given nodes. If a node doesn't contain the `old_label` the procedure doesn't modify it. + +This procedure is equivalent to **apoc.refactor.rename.label**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -150,6 +162,10 @@ Rename a property from `old_property` to `new_property` for all nodes. If `nodes` is provided renaming is applied only to the given nodes. If a node doesn't contain the `old_property` the procedure doesn't modify it. + +This procedure is equivalent to **apoc.refactor.rename.nodeProperty**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -270,6 +286,10 @@ Clones specific nodes in the graph, preserving their original labels, properties and optionally relationships. Offers the flexibility to exclude specific node properties during the cloning process. + +This procedure is equivalent to **apoc.refactor.cloneNodes**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -351,6 +371,10 @@ Clones the subgraph by cloning the given list of nodes and relationships. If no relationships are provided, all existing relationships between the given nodes will be cloned. + +This procedure is equivalent to **apoc.refactor.cloneSubgraph**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -451,6 +475,10 @@ used as its "stand-in" node. Clones the subgraph specified by a specific list of paths. A path is a series of nodes connected by relationships. + +This procedure is equivalent to **apoc.refactor.cloneSubgraphFromPaths**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -550,6 +578,10 @@ exactly 1 in relationship, and exactly 1 out relationship. The node must not have any self relationships. Collapsing any node that doesn't satisfy these requirements results in an exception. + +This procedure is equivalent to **apoc.refactor.collapseNode**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -598,6 +630,10 @@ Result: Invert the direction of a given relationship. + +This procedure is equivalent to **apoc.refactor.invert**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -649,6 +685,10 @@ the property value is contained in `true_values` or `false_values`. If the property value is not contained in either list, the property is removed. Throws an exception if both lists contain the same value. + +This procedure is equivalent to **apoc.refactor.normalizeAsBoolean**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -691,6 +731,10 @@ The procedure creates a node from the given relationship (and deletes the relationship) with the given labels and connects the new node to the start and end node of the given relationship with the given types. + +This procedure is equivalent to **apoc.refactor.extractNode**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -742,6 +786,10 @@ The new state of the graph: The procedure deletes the given nodes from the given path (deleting them in the graph) and reconnects the remaining nodes. + +This procedure is equivalent to **apoc.refactor.deleteAndReconnect**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -857,6 +905,10 @@ relationships: [{"id": 28, "start": 35, "end": 37, "label": "Rel2_Rel1", "proper The procedure changes the relationship type. + +This procedure is equivalent to **apoc.refactor.renameType**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -902,6 +954,10 @@ The procedure returns the number of updated relationships: Renames the property of a relationship. + +This procedure is equivalent to **apoc.refactor.rename.typeProperty**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -971,6 +1027,10 @@ CALL refactor.rename_type_property("distance_in_km","distance",[r]) YIELD relati Merges the properties, labels and relationships for the source nodes to the target node. + +This procedure is equivalent to **apoc.refactor.mergeNodes**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. diff --git a/pages/advanced-algorithms/available-algorithms/text.mdx b/pages/advanced-algorithms/available-algorithms/text.mdx index cdacb0e7b..3a4bc88e3 100644 --- a/pages/advanced-algorithms/available-algorithms/text.mdx +++ b/pages/advanced-algorithms/available-algorithms/text.mdx @@ -36,6 +36,10 @@ The `text` module offers a toolkit for manipulating strings. Joins all the strings into a single one with the given delimiter between them. + +This procedure is equivalent to **apoc.text.join**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -67,11 +71,52 @@ Result: +----------------------------+ ``` +### `indexOf()` + +Finds the index of the first occurrence of a substring within a string, with optional start and end position parameters. + + +This function is equivalent to **apoc.text.indexOf**. + + +{

Input:

} + +- `text: string` ➡ The text string to search within. +- `lookup: string` ➡ The substring to search for. +- `from: integer` (default = 0) ➡ The starting position for the search (0-based index). +- `to: integer` (default = -1) ➡ The ending position for the search (-1 means search to the end of the string). + +{

Output:

} + +- `integer` ➡ The index of the first occurrence of the substring, or -1 if not found. + +{

Usage:

} + +Use the following query to find the position of a substring: + +```cypher +RETURN text.indexOf("Hello World!", "World", 0, -1) AS result; +``` + +Result: + +```plaintext ++--------+ +| result | ++--------+ +| 6 | ++--------+ +``` + ### `regexGroups()` The procedure returns all matched subexpressions of the regex on the provided text using the [C++ regex](https://en.cppreference.com/w/cpp/regex) library. + +This procedure is equivalent to **apoc.text.regexGroups**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -107,6 +152,10 @@ Result: The procedure formats strings using the [C++ fmt library](https://github.com/fmtlib/fmt). + +This procedure is equivalent to **apoc.text.format**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -142,6 +191,10 @@ Result: Replace each substring of the given string that matches the given regular expression with the given replacement. + +This function is equivalent to **apoc.text.replace**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -186,6 +239,10 @@ Result: Replace each substring of the given string that matches the given regular expression with the given replacement. + +This function is equivalent to **apoc.text.regReplace**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. diff --git a/pages/advanced-algorithms/available-algorithms/util_module.mdx b/pages/advanced-algorithms/available-algorithms/util_module.mdx index fea6c90c3..fb60fb315 100644 --- a/pages/advanced-algorithms/available-algorithms/util_module.mdx +++ b/pages/advanced-algorithms/available-algorithms/util_module.mdx @@ -3,6 +3,7 @@ title: util_module description: Streamline a variety of tasks related to database operations using procedures from the util_module. --- +import { Callout } from 'nextra/components' import { Cards } from 'nextra/components' import GitHub from '/components/icons/GitHub' @@ -38,6 +39,10 @@ concatenates it into a single string, and returns the md5 hash of that string. The format of string representations can be seen by checking `ToString` in [C++ API](/custom-query-modules/cpp/cpp-api) documentation for desired type or Value. + +This procedure is equivalent to **apoc.util.md5**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -82,6 +87,10 @@ The function gets the string representation of every element in the input list o concatenates it into a single string, and returns the md5 hash of that string. Does the same computation as the `md5()` procedure but can be called as a function + +This function is equivalent to **apoc.util.md5**. + + The format of string representations can be seen by checking `ToString` in [C++ API](/custom-query-modules/cpp/cpp-api) documentation for desired type or Value. diff --git a/pages/advanced-algorithms/deep-path-traversal.mdx b/pages/advanced-algorithms/deep-path-traversal.mdx index 27a129713..4702f04bf 100644 --- a/pages/advanced-algorithms/deep-path-traversal.mdx +++ b/pages/advanced-algorithms/deep-path-traversal.mdx @@ -18,6 +18,7 @@ algorithms are built into Memgraph and don't require any additional libraries: * [Breadth-first search (BFS)](#breadth-first-search) * [Weighted shortest path (WSP)](#weighted-shortest-path) * [All shortest paths (ASP)](#all-shortest-paths) + * [K shortest paths (KSP)](#k-shortest-paths) Below you can find examples of how to use these algorithms and you can try them out @@ -27,7 +28,7 @@ Europe backpacking dataset or adjust them to the dataset of your choice. -Memgraph has a lot more graph algorithms to offer besides these four, and they +Memgraph has a lot more graph algorithms to offer besides these five, and they are all a part of [MAGE](/advanced-algorithms/install-mage) - Memgraph Advanced Graph Extensions, an open-source repository that contains graph algorithms and modules written in the form of query modules that can be used to tackle the most @@ -795,4 +796,105 @@ RETURN path, total_weight; current traversal path p, allowing you to filter based on how the current node was reached. + +## K shortest paths + +The K shortest paths algorithm finds K shortest paths between two nodes in order +of increasing length. This algorithm is useful when you need to find alternative +routes or analyze path diversity in a graph. + +### Syntax + +The K shortest paths algorithm uses the `*KSHORTEST` syntax: + +```cypher +MATCH (source), (target) +WITH source, target +MATCH path=(source)-[*KSHORTEST]->(target) +RETURN path; +``` + +You can also limit the number of paths returned using the `|` syntax: + +```cypher +MATCH (source), (target) +WITH source, target +MATCH path=(source)-[*KSHORTEST|3]->(target) +RETURN path; +``` + +This will return at most 3 shortest paths between the source and target nodes. + +### Example + + + + + + ```cypher + MATCH (n) DETACH DELETE n; + CREATE (n1:Node {name: "A"}), (n2:Node {name: "B"}), (n3:Node {name: "C"}), (n4:Node {name: "D"}), (n5:Node {name: "E"}), + (n1)-[:ConnectedTo]->(n2), (n1)-[:ConnectedTo]->(n3), (n2)-[:ConnectedTo]->(n3), + (n2)-[:ConnectedTo]->(n4), (n4)-[:ConnectedTo]->(n3), (n3)-[:ConnectedTo]->(n5); + ``` + + + + ```cypher + MATCH (a:Node {name: "A"}), (e:Node {name: "E"}) + WITH a, e + MATCH path=(a)-[*KSHORTEST]->(e) + RETURN path; + ``` + + + + ```plaintext + +---------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | path | + +---------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | (:Node {name: "A"})-[:ConnectedTo]->(:Node {name: "C"})-[:ConnectedTo]->(:Node {name: "E"}) | + | (:Node {name: "A"})-[:ConnectedTo]->(:Node {name: "B"})-[:ConnectedTo]->(:Node {name: "C"})-[:ConnectedTo]->(:Node {name: "E"}) | + | (:Node {name: "A"})-[:ConnectedTo]->(:Node {name: "B"})-[:ConnectedTo]->(:Node {name: "D"})-[:ConnectedTo]->(:Node {name: "C"})-[:ConnectedTo]->(:Node {name: "E"}) | + +---------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + ``` + + + + + +### Path length constraints + +You can constrain the path length using the range syntax: + +```cypher +MATCH (a:Node {name: "A"}), (e:Node {name: "E"}) +WITH a, e +MATCH path=(a)-[*KSHORTEST 2..4]->(e) +RETURN path; +``` + +This will only return paths with a length between 2 and 4 hops (inclusive). + +### When to use K shortest paths? + +Use the K shortest paths algorithm when you need to: +- Find alternative routes between two nodes +- Analyze path diversity in a graph +- Identify backup paths or redundant connections +- Understand the network structure beyond just the shortest path + +### Interaction with other systems + +- **Fine-grained access control**: Supports enterprise access control features +- **Hops limit**: Supports query level hops limit + +### Important limitations + +- **Predefined nodes**: Both source and target nodes must be matched first using + a `WITH` clause +- **No filter lambdas**: K shortest paths does not support user-defined + filtering during expansion + + \ No newline at end of file diff --git a/pages/client-libraries/python.mdx b/pages/client-libraries/python.mdx index 75f668d74..251784b68 100644 --- a/pages/client-libraries/python.mdx +++ b/pages/client-libraries/python.mdx @@ -277,13 +277,19 @@ with GraphDatabase.driver(URI, auth=AUTH) as client: This is currently only supported for OIDC SSO. -To use SSO with the Python driver you need to get the access and id tokens yourself. +To use SSO with the Python driver, you need to get the access token and, optionally, the ID token yourself. One simple way to do it is to use the authlib library and follow the official [tutorial](https://docs.authlib.org/en/latest/client/oauth2.html). To connect to the Memgraph database you have to use the `custom_auth` class with the `scheme` parameter set as `oidc-entra-id`, `oidc-okta` or `oidc-custom` depending on which scheme you are using, -`credentials` parameter set to contain both access and id tokens in the format shown in the example below. Finally set `principal` and `realm` parameters to `None`. +`credentials` parameter set to contain the access token and optionally the ID token in the format shown in the example below. Finally, set `principal` and `realm` parameters to `None`. -Below is an example of connecting to the Memgraph database using OIDC SSO with custom auth scheme. + +The ID token is only required if your username configuration uses a field from the ID token (e.g., `id:sub`). If your username is configured to use a field from the access token (e.g., `access:preferred_username`), you can omit the ID token from the credentials string. + + +Below are examples of connecting to the Memgraph database using OIDC SSO with the custom auth scheme. + +**With both access and ID tokens:** ```python with neo4j.GraphDatabase.driver( "bolt://localhost:7687", @@ -296,6 +302,19 @@ with neo4j.GraphDatabase.driver( ) as driver: ``` +**With access token only (when username is configured to use access token fields):** +```python +with neo4j.GraphDatabase.driver( + "bolt://localhost:7687", + auth=neo4j.custom_auth( + scheme="oidc-custom", + credentials=f"access_token={token['access_token']}", + principal=None, + realm=None, + ) +) as driver: +``` + #### Impersonate a user diff --git a/pages/clustering/high-availability.mdx b/pages/clustering/high-availability.mdx index 3629f2c7a..fa0d39f1f 100644 --- a/pages/clustering/high-availability.mdx +++ b/pages/clustering/high-availability.mdx @@ -12,12 +12,12 @@ import {CommunityLinks} from '/components/social-card/CommunityLinks' A cluster is considered highly available if, at any point, there is some instance that can respond to a user query. Our high availability relies on replication. The cluster consists of: -- The MAIN instance on which the user can execute write queries -- REPLICA instances that can only respond to read queries +- The main instance on which the user can execute write queries +- replica instances that can only respond to read queries - COORDINATOR instances that manage the cluster state. Depending on how configuration flags are set, Memgraph can run as a data instance or coordinator instance. -The coordinator instance is a new addition to enable the high availability feature and orchestrates data instances to ensure that there is always one MAIN instance in the cluster. +The coordinator instance is a new addition to enable the high availability feature and orchestrates data instances to ensure that there is always one main instance in the cluster. ## Cluster management @@ -25,10 +25,10 @@ For achieving high availability, Memgraph uses Raft consensus protocol, which is a significant advantage that it is much easier to understand. It's important to say that Raft isn't a Byzantine fault-tolerant algorithm. You can learn more about Raft in the paper [In Search of an Understandable Consensus Algorithm](https://raft.github.io/raft.pdf). -Typical Memgraph's highly available cluster consists of 3 data instances (1 MAIN and 2 REPLICAS) and 3 coordinator instances backed up by Raft protocol. +Typical Memgraph's highly available cluster consists of 3 data instances (1 main and 2 replicaS) and 3 coordinator instances backed up by Raft protocol. Users can create more than 3 coordinators, but the replication factor (RF) of 3 is a de facto standard in distributed databases. -One coordinator instance is the leader whose job is to always ensure one writeable data instance (MAIN). The other two coordinator instances replicate +One coordinator instance is the leader whose job is to always ensure one writeable data instance (main). The other two coordinator instances replicate changes the leader coordinator did in its own Raft log. Operations saved into the Raft log are those that are related to cluster management. Memgraph doesn't have its implementation of the Raft protocol. For this task, Memgraph uses an industry-proven library [NuRaft](https://github.com/eBay/NuRaft). @@ -37,7 +37,7 @@ You can start the coordinator instance by specifying `--coordinator-id`, queries related to high availability, so you cannot execute any data-oriented query on it. The coordinator port is used for the Raft protocol, which all coordinators use to ensure the consistency of the cluster's state. Data instances are distinguished from coordinator instances by specifying only `--management-port` flag. This port is used for RPC network communication between the coordinator and data -instances. When started by default, the data instance is MAIN. The coordinator will ensure that no data inconsistency can happen during and after the instance's +instances. When started by default, the data instance is main. The coordinator will ensure that no data inconsistency can happen during and after the instance's restart. Once all instances are started, the user can start adding data instances to the cluster. @@ -70,19 +70,19 @@ but from the availability perspective, it is better to separate them physically. ## Bolt+routing -Directly connecting to the MAIN instance isn't preferred in the HA cluster since the MAIN instance changes due to various failures. Because of that, users +Directly connecting to the main instance isn't preferred in the HA cluster since the main instance changes due to various failures. Because of that, users can use bolt+routing so that write queries can always be sent to the correct data instance. This will prevent a split-brain issue since clients, when writing, won't be routed to the old main but rather to the new main instance on which failover got performed. This protocol works in a way that the client first sends a ROUTE bolt message to any coordinator instance. The coordinator replies to the message by returning the routing table with three entries specifying -from which instance can the data be read, to which instance data can be written to and which instances behave as routers. In the Memgraph HA cluster, the MAIN -data instance is the only writeable instance, REPLICAs are readable instances, and COORDINATORs behave as routers. However, the cluster can be configured in such a way -that MAIN can also be used for reading. Check this [paragraph](#setting-config-for-highly-available-cluster) for more info. +from which instance can the data be read, to which instance data can be written to and which instances behave as routers. In the Memgraph HA cluster, the main +data instance is the only writeable instance, replicas are readable instances, and COORDINATORs behave as routers. However, the cluster can be configured in such a way +that main can also be used for reading. Check this [paragraph](#setting-config-for-highly-available-cluster) for more info. Bolt+routing is the client-side routing protocol, meaning network endpoint resolution happens inside drivers. For more details about the Bolt messages involved in the communication, check [the following link](https://neo4j.com/docs/bolt/current/bolt/message/#messages-route). Users only need to change the scheme they use for connecting to coordinators. This means instead of using `bolt://,` you should -use `neo4j://` to get an active connection to the current MAIN instance in the cluster. You can find examples of how to +use `neo4j://` to get an active connection to the current main instance in the cluster. You can find examples of how to use bolt+routing in different programming languages [here](https://github.com/memgraph/memgraph/tree/master/tests/drivers). It is important to note that setting up the cluster on one coordinator (registration of data instances and coordinators, setting main) must be done using bolt connection @@ -217,17 +217,18 @@ Registering instances should be done on a single coordinator. The chosen coordin Register instance query will result in several actions: 1. The coordinator instance will connect to the data instance on the `management_server` network address. 2. The coordinator instance will start pinging the data instance every `--instance-health-check-frequency-sec` seconds to check its status. -3. Data instance will be demoted from MAIN to REPLICA. +3. Data instance will be demoted from main to replica. 4. Data instance will start the replication server on `replication_server`. ```plaintext -REGISTER INSTANCE instanceName ( AS ASYNC ) WITH CONFIG {"bolt_server": boltServer, "management_server": managementServer, "replication_server": replicationServer}; +REGISTER INSTANCE instanceName ( AS ASYNC | AS STRICT_SYNC ) ? WITH CONFIG {"bolt_server": boltServer, "management_server": managementServer, "replication_server": replicationServer}; ``` This operation will result in writing to the Raft log. -In case the MAIN instance already exists in the cluster, a replica instance will be automatically connected to the MAIN. You can specify whether the replica should behave -synchronously or asynchronously by using `AS ASYNC` construct after `instanceName`. +In case the main instance already exists in the cluster, a replica instance will be automatically connected to the main. Constructs ( AS ASYNC | AS STRICT_SYNC ) serve to specify +instance's replication mode when the instance behaves as replica. You can only have `STRICT_SYNC` and `ASYNC` or `SYNC` and `ASYNC` replicas together in the cluster. Combining `STRICT_SYNC` +and `SYNC` replicas together doesn't have proper semantic meaning so it is forbidden. ### Add coordinator instance @@ -262,23 +263,23 @@ REMOVE COORDINATOR ; ``` -### Set instance to MAIN +### Set instance to main -Once all data instances are registered, one data instance should be promoted to MAIN. This can be achieved by using the following query: +Once all data instances are registered, one data instance should be promoted to main. This can be achieved by using the following query: ```plaintext -SET INSTANCE instanceName to MAIN; +SET INSTANCE instanceName to main; ``` -This query will register all other instances as REPLICAs to the new MAIN. If one of the instances is unavailable, setting the instance to MAIN will not succeed. -If there is already a MAIN instance in the cluster, this query will fail. +This query will register all other instances as replicas to the new main. If one of the instances is unavailable, setting the instance to main will not succeed. +If there is already a main instance in the cluster, this query will fail. This operation will result in writing to the Raft log. ### Demote instance Demote instance query can be used by an admin to demote the current main to replica. In this case, the leader coordinator won't perform a failover, but as a user, -you should choose promote one of the data instances to MAIN using the `SET INSTANCE `instance` TO MAIN` query. +you should choose promote one of the data instances to main using the `SET INSTANCE `instance` TO main` query. ```plaintext DEMOTE INSTANCE instanceName; @@ -288,14 +289,12 @@ This operation will result in writing to the Raft log. -By combining the functionalities of queries `DEMOTE INSTANCE instanceName` and `SET INSTANCE instanceName TO MAIN` you get the manual failover capability. This can be useful -e.g during a maintenance work on the instance where the current MAIN is deployed. +By combining the functionalities of queries `DEMOTE INSTANCE instanceName` and `SET INSTANCE instanceName TO main` you get the manual failover capability. This can be useful +e.g during a maintenance work on the instance where the current main is deployed. - - ### Unregister instance There are various reasons which could lead to the decision that an instance needs to be removed from the cluster. The hardware can be broken, @@ -306,21 +305,21 @@ UNREGISTER INSTANCE instanceName; ``` When unregistering an instance, ensure that the instance being unregistered is -**not** the MAIN instance. Unregistering MAIN can lead to an inconsistent -cluster state. Additionally, the cluster must have an **alive** MAIN instance -during the unregistration process. If no MAIN instance is available, the +**not** the main instance. Unregistering main can lead to an inconsistent +cluster state. Additionally, the cluster must have an **alive** main instance +during the unregistration process. If no main instance is available, the operation cannot be guaranteed to succeed. -The instance requested to be unregistered will also be unregistered from the current MAIN's REPLICA set. +The instance requested to be unregistered will also be unregistered from the current main's replica set. ### Force reset cluster state In case the cluster gets stuck there is an option to do the force reset of the cluster. You need to execute a command on the leader coordinator. This command will result in the following actions: -1. The coordinator instance will demote each alive instance to REPLICA. -2. From the alive instance it will choose a new MAIN instance. -3. Instances that are down will be demoted to REPLICAs once they come back up. +1. The coordinator instance will demote each alive instance to replica. +2. From the alive instance it will choose a new main instance. +3. Instances that are down will be demoted to replicas once they come back up. ```plaintext FORCE RESET CLUSTER STATE; @@ -334,7 +333,7 @@ You can check the state of the whole cluster using the `SHOW INSTANCES` query. T each server you can see the following information: 1. Network endpoints they are using for managing cluster state 2. Health state of server - 3. Role - MAIN, REPLICA, LEADER, FOLLOWER or unknown if not alive + 3. Role - main, replica, LEADER, FOLLOWER or unknown if not alive 4. The time passed since the last response time to the leader's health ping This query can be run on either the leader or followers. Since only the leader knows the exact status of the health state and last response time, @@ -366,6 +365,16 @@ This query will return the information about: If the query `ADD COORDINATOR` wasn't run for the current instance, the value of the bolt server will be "". +### Show replication lag + +The user can find the current replication lag on each instance by running `SHOW REPLICATION LAG` on the cluster's leader. The replication lag is expressed with +the number of committed transactions. Such an info is made durable through snapshots and WALs so restarts won't cause the information loss. The information +about the replication lag can be useful when manually performing a failover to check whether there is a risk of a data loss. + +```plaintext +SHOW REPLICATION LAG; +``` + ## Setting config for highly-available cluster @@ -439,13 +448,14 @@ for which the timeout is used is the following: - TimestampReq -> main sending to replica - SystemHeartbeatReq -> main sending to replica - ForceResetStorageReq -> main sending to replica. The timeout is set to 60s. -- SystemRecoveryReq -> main sending to replica. The timeout set to 5s. +- SystemRecoveryReq -> main sending to replica. The timeout is set to 5s. +- FinalizeCommitReq -> main sending to replica. The timeout is set to 10s. -For replication-related RPC messages — AppendDeltasRpc, CurrentWalRpc, and +For RPC messages which are sending the variable number of storage deltas — PrepareCommitRpc, CurrentWalRpc, and WalFilesRpc — it is not practical to set a strict execution timeout. The -processing time on the replica side is directly proportional to the amount of -data being transferred. To handle this, the replica sends periodic progress +processing time on the replica side is directly proportional to the number of +deltas being transferred. To handle this, the replica sends periodic progress updates to the main instance after processing every 100,000 deltas. Since processing 100,000 deltas is expected to take a relatively consistent amount of time, we can enforce a timeout based on this interval. The default timeout for @@ -453,7 +463,7 @@ these RPC messages is 30 seconds, though in practice, processing 100,000 deltas typically takes less than 3 seconds. SnapshotRpc is also a replication-related RPC message, but its execution time -is tracked differently. The replica sends an update to the main instance after +is tracked a bit differently from RPC messages shipping deltas. The replica sends an update to the main instance after completing 1,000,000 units of work. The work units are assigned as follows: - Processing nodes, edges, or indexed entities (label index, label-property index, @@ -483,93 +493,93 @@ a multiplier of `--instance-health-check-frequency-sec`. Set the multiplier coef For example, set `--instance-down-timeout-sec=5` and `--instance-health-check-frequency-sec=1` which will result in coordinator contacting each instance every second and the instance is considered dead after it doesn't respond 5 times (5 seconds / 1 second). -In case a REPLICA doesn't respond to a health check, the leader coordinator will try to contact it again every `--instance-health-check-frequency-sec`. -When the REPLICA instance rejoins the cluster (comes back up), it always rejoins as REPLICA. For MAIN instance, there are two options. -If it is down for less than `--instance-down-timeout-sec`, it will rejoin as MAIN because it is still considered alive. If it is down for more than `--instance-down-timeout-sec`, -the failover procedure is initiated. Whether MAIN will rejoin as MAIN depends on the success of the failover procedure. If the failover procedure succeeds, now old MAIN -will rejoin as REPLICA. If failover doesn't succeed, MAIN will rejoin as MAIN once it comes back up. +In case a replica doesn't respond to a health check, the leader coordinator will try to contact it again every `--instance-health-check-frequency-sec`. +When the replica instance rejoins the cluster (comes back up), it always rejoins as replica. For main instance, there are two options. +If it is down for less than `--instance-down-timeout-sec`, it will rejoin as main because it is still considered alive. If it is down for more than `--instance-down-timeout-sec`, +the failover procedure is initiated. Whether main will rejoin as main depends on the success of the failover procedure. If the failover procedure succeeds, now old main +will rejoin as replica. If failover doesn't succeed, main will rejoin as main once it comes back up. ### Failover procedure - high level description -From alive REPLICAs coordinator chooses a new potential MAIN. -This instance is only potentially new MAIN as the failover procedure can still fail due to various factors (networking issues, promote to MAIN fails, any alive REPLICA failing to -accept an RPC message, etc). The coordinator sends an RPC request to the potential new MAIN, which is still in REPLICA state, to promote itself to the MAIN instance with info -about other REPLICAs to which it will replicate data. Once that request succeeds, the new MAIN can start replication to the other instances and accept write queries. - +From alive replicas coordinator chooses a new potential main and writes a log to the Raft storage about the new main. On the next leader's ping to the instance, +it will send to the instance an RPC request to the new main, which is still in replica state, to promote itself to the main instance with info +about other replicas to which it will replicate data. Once that request succeeds, the new main can start replication to the other instances and accept write queries. -### Choosing new MAIN from available REPLICAs +### Choosing new main from available replicas -When failover is happening, some REPLICAs can also be down. From the list of alive REPLICAs, a new MAIN is chosen. First, the leader coordinator contacts each alive REPLICA +When failover is happening, some replicas can also be down. From the list of alive replicas, a new main is chosen. First, the leader coordinator contacts each alive replica to get info about each database's last commit timestamp. In the case of enabled multi-tenancy, from each instance coordinator will get info on all databases and their last commit -timestamp. Currently, the coordinator chooses an instance to become a new MAIN by comparing the latest commit timestamps of all databases. The instance which is newest on most -databases is considered the best candidate for the new MAIN. If there are multiple instances which have the same number of newest databases, we sum timestamps of all databases +timestamp. Currently, the coordinator chooses an instance to become a new main by comparing the latest commit timestamps of all databases. The instance which is newest on most +databases is considered the best candidate for the new main. If there are multiple instances which have the same number of newest databases, we sum timestamps of all databases and consider instance with a larger sum as the better candidate. -### Old MAIN rejoining to the cluster +### Old main rejoining to the cluster -Once the old MAIN gets back up, the coordinator sends an RPC request to demote the old MAIN to REPLICA. The coordinator tracks at all times which instance was the last MAIN. +Once the old main gets back up, the coordinator sends an RPC request to demote the old main to replica. The coordinator tracks at all times which instance was the last main. -The leader coordinator sends two RPC requests in the given order to demote old MAIN to REPLICA: -1. Demote MAIN to REPLICA RPC request -2. A request to store the UUID of the current MAIN, which the old MAIN, now acting as a REPLICA instance, must listen to. +The leader coordinator sends two RPC requests in the given order to demote old main to replica: +1. Demote main to replica RPC request +2. A request to store the UUID of the current main, which the old main, now acting as a replica instance, must listen to. -### How REPLICA knows which MAIN to listen +### How replica knows which main to listen -Each REPLICA has a UUID of MAIN it listens to. If a network partition happens where MAIN can talk to a REPLICA but the coordinator can't talk to the MAIN, from the coordinator's -point of view that MAIN is down. From REPLICA's point of view, the MAIN instance is still alive. The coordinator will start the failover procedure, and we can end up with multiple MAINs -where REPLICAs can listen to both MAINs. To prevent such an issue, each REPLICA gets a new UUID that no current MAIN has. The coordinator generates the new UUID, -which the new MAIN will get to use on its promotion to MAIN. +Each replica has a UUID of main it listens to. If a network partition happens where main can talk to a replica but the coordinator can't talk to the main, from the coordinator's +point of view that main is down. From replica's point of view, the main instance is still alive. The coordinator will start the failover procedure, and we can end up with multiple mains +where replicas can listen to both mains. To prevent such an issue, each replica gets a new UUID that no current main has. The coordinator generates the new UUID, +which the new main will get to use on its promotion to main. -If REPLICA was down at one point, MAIN could have changed. When REPLICA gets back up, it doesn't listen to any MAIN until the coordinator sends an RPC request to REPLICA to start -listening to MAIN with the given UUID. +If replica was down at one point, main could have changed. When replica gets back up, it doesn't listen to any main until the coordinator sends an RPC request to replica to start +listening to main with the given UUID. ### Replication concerns #### Force sync of data During a failover event, Memgraph selects the most up-to-date, alive instance to -become the new MAIN. The selection process works as follows: -1. From the list of available REPLICA instances, Memgraph chooses the one with +become the new main. The selection process works as follows: +1. From the list of available replica instances, Memgraph chooses the one with the latest commit timestamp for the default database. 2. If an instance that had more recent data was down during this selection -process, it will not be considered for promotion to MAIN. +process, it will not be considered for promotion to main. If a previously down instance had more up-to-date data but was unavailable during failover, it will go through a specific recovery process upon rejoining the cluster: -- The new MAIN will clear the returning replica’s storage. -- The returning replica will then receive all commits from the new MAIN to +- The replica will reset its storage. +- The replica will receive all commits from the new main to synchronize its state. - The replica's old durability files will be preserved in a `.old` directory in `data_directory/snapshots` and `data_directory/wal` folders, allowing admins to manually recover data if needed. -Memgraph prioritizes availability over strict consistency (leaning towards AP in -the CAP theorem). While it aims to maintain consistency as much as possible, the -current failover logic can result in a non-zero Recovery Point Objective (RPO), -that is, the loss of committed data, because: -- The promoted MAIN might not have received all commits from the previous MAIN +Depending on the replication mode used, there are different levels of data loss +that can happen upon the failover. With the default `SYNC` replication mode, +Memgraph prioritizes availability over strict consistency and can result in +a non-zero Recovery Point Objective (RPO), that is, the loss of committed data, because: +- The promoted main might not have received all commits from the previous main before the failure. -- This design ensures that the MAIN remains writable for the maximum possible +- This design ensures that the main remains writable for the maximum possible time. -If your environment requires strong consistency and can tolerate write -unavailability, [reach out to -us](https://github.com/memgraph/memgraph/discussions). We are actively exploring -support for a fully synchronous mode. +With `ASYNC` replication mode, you also risk losing some data upon the failover because +main can freely continue commiting no matter the status of ASYNC replicas. + +The `STRICT_SYNC` replication mode allows users experiencing a failover without any data loss +in all situations. It comes with reduced throughput because of the cost of running two-phase commit protocol. ## Actions on follower coordinators -From follower coordinators you can only execute `SHOW INSTANCES`. Registration of data instance, unregistration of data instances, demoting instance, setting instance to MAIN and +From follower coordinators you can only execute `SHOW INSTANCES`. Registration of data instance, unregistration of data instances, demoting instance, setting instance to main and force resetting cluster state are all disabled. ## Instances' restart ### Data instances' restart -Data instances can fail both as MAIN and as REPLICA. When an instance that was REPLICA comes back, it won't accept updates from any instance until the coordinator updates its -responsible peer. This should happen automatically when the coordinator's ping to the instance passes. When the MAIN instance comes back, any writing to the MAIN instance will be + +Data instances can fail both as main and as replica. When an instance that was replica comes back, it won't accept updates from any instance until the coordinator updates its +responsible peer. This should happen automatically when the coordinator's ping to the instance passes. When the main instance comes back, any writing to the main instance will be forbidden until a ping from the coordinator passes. ### Coordinator instances restart @@ -612,7 +622,7 @@ It will also recover the following server config information: The following information will be recovered from a common RocksDB `logs` instance: - current version of `logs` durability store - snapshots found with `snapshot_id_` prefix in database: - - coordinator cluster state - all data instances with their role (MAIN or REPLICA), all coordinator instances and UUID of MAIN instance which REPLICA is listening to + - coordinator cluster state - all data instances with their role (main or replica), all coordinator instances and UUID of main instance which replica is listening to - last log idx - last log term - last cluster config @@ -645,12 +655,16 @@ Raft is a quorum-based protocol and it needs a majority of instances alive in or the cluster stays available. With 2+ coordinator instances down (in a cluster with RF = 3), the RTO depends on the time needed for instances to come back. -Failure of REPLICA data instance isn't very harmful since users can continue writing to MAIN data instance while reading from MAIN or other -REPLICAs. The most important thing to analyze is what happens when MAIN gets down. In that case, the leader coordinator uses +Depending on the replica's replication mode, its failure can lead to different situations. If the replica was registered with STRICT_SYNC mode, then on its failure, writing +on main will be disabled. On the other hand, if replica was registered as ASYNC or SYNC, further writes on main are still allowed. In both cases, reads are still allowed from +main and other replicas. + + +The most important thing to analyze is what happens when main gets down. In that case, the leader coordinator uses user-controllable parameters related to the frequency of health checks from the leader to replication instances (`--instance-health-check-frequency-sec`) -and the time needed to realize the instance is down (`--instance-down-timeout-sec`). After collecting enough evidence, the leader concludes the MAIN is down and performs failover +and the time needed to realize the instance is down (`--instance-down-timeout-sec`). After collecting enough evidence, the leader concludes the main is down and performs failover using just a handful of RPC messages (correct time depends on the distance between instances). It is important to mention that the whole failover is performed without the loss of committed data -if the newly chosen MAIN (previously REPLICA) had all up-to-date data. +if the newly chosen main (previously replica) had all up-to-date data. ## Raft configuration parameters @@ -695,7 +709,7 @@ ADD COORDINATOR 3 WITH CONFIG {"bolt_server": "localhost:7693", "coordinator_ser REGISTER INSTANCE instance_1 WITH CONFIG {"bolt_server": "localhost:7687", "management_server": "instance1:13011", "replication_server": "instance1:10001"}; REGISTER INSTANCE instance_2 WITH CONFIG {"bolt_server": "localhost:7688", "management_server": "instance2:13012", "replication_server": "instance2:10002"}; REGISTER INSTANCE instance_3 WITH CONFIG {"bolt_server": "localhost:7689", "management_server": "instance3:13013", "replication_server": "instance3:10003"}; -SET INSTANCE instance_3 TO MAIN; +SET INSTANCE instance_3 TO main; ``` @@ -898,10 +912,10 @@ REGISTER INSTANCE instance_2 WITH CONFIG {"bolt_server": "localhost:7688", "mana REGISTER INSTANCE instance_3 WITH CONFIG {"bolt_server": "localhost:7689", "management_server": "localhost:13013", "replication_server": "localhost:10003"}; ``` -4. Set instance_3 as MAIN: +4. Set instance_3 as main: ```plaintext -SET INSTANCE instance_3 TO MAIN; +SET INSTANCE instance_3 TO main; ``` 5. Connect to the leader coordinator and check cluster state with `SHOW INSTANCES`; @@ -917,8 +931,8 @@ SET INSTANCE instance_3 TO MAIN; ### Check automatic failover -Let's say that the current MAIN instance is down for some reason. After `--instance-down-timeout-sec` seconds, the coordinator will realize -that and automatically promote the first alive REPLICA to become the new MAIN. The output of running `SHOW INSTANCES` on the leader coordinator could then look like: +Let's say that the current main instance is down for some reason. After `--instance-down-timeout-sec` seconds, the coordinator will realize +that and automatically promote the first alive replica to become the new main. The output of running `SHOW INSTANCES` on the leader coordinator could then look like: | name | bolt_server | coordinator_server | management_server | health | role | last_succ_resp_ms | | ------------- | -------------- | ------------------ | ----------------- | ------ | -------- | ------------------| diff --git a/pages/clustering/replication.mdx b/pages/clustering/replication.mdx index efd1817bf..04332be02 100644 --- a/pages/clustering/replication.mdx +++ b/pages/clustering/replication.mdx @@ -10,15 +10,16 @@ import { Callout } from 'nextra/components' Instances need to remember their role and configuration details in a replication -cluster upon restart, and the `--replication-restore-state-on-startup` needs to be -set to `true` when first initializing the instances and remain `true` throughout -the instances' lifetime for replication to work correctly. If the flag is set to `false`, -MAIN can't communicate with instance, because each REPLICA has a UUID of MAIN which can communicate -with it, and it is set up only on instance registration. In case the flag is set to `false`, -the way to go forward is first to unregister the instance on MAIN and register it again. +cluster upon restart, and the `--replication-restore-state-on-startup` needs to +be set to `true` when first initializing the instances and remain `true` +throughout the instances' lifetime for replication to work correctly. If the +flag is set to `false`, MAIN can't communicate with instance, because each +REPLICA has a UUID of MAIN which can communicate with it, and it is set up only +on instance registration. In case the flag is set to `false`, the way to go +forward is first to unregister the instance on MAIN and register it again. -When reinstating a cluster, it is advised first to initialize the MAIN -instance, then the REPLICA instances. +When reinstating a cluster, it is advised first to initialize the MAIN instance, +then the REPLICA instances. Data replication currently **works only in the in-memory transactional [storage mode](/fundamentals/storage-memory-usage)**. @@ -26,14 +27,15 @@ mode](/fundamentals/storage-memory-usage)**. When distributing data across several instances, Memgraph uses replication to -provide a satisfying ratio of the following properties, known from the CAP theorem: +provide a satisfying ratio of the following properties, known from the CAP +theorem: -1. **Consistency** (C) - every node has the same view of data at a given point in - time -2. **Availability** (A) - all clients can find a replica of the data, even in the - case of a partial node failure -3. **Partition tolerance** (P) - the system continues to work as expected despite a - partial network failure +1. **Consistency** (C) - every node has the same view of data at a given point + in time +2. **Availability** (A) - all clients can find a replica of the data, even in + the case of a partial node failure +3. **Partition tolerance** (P) - the system continues to work as expected + despite a partial network failure In the replication process, the data is replicated from one storage (MAIN instance) to another (REPLICA instances). @@ -71,34 +73,44 @@ cluster. Once demoted to REPLICA instances, they will no longer accept write queries. In order to start the replication, each REPLICA instance needs to be registered -from the MAIN instance by setting a replication mode (SYNC or ASYNC) and -specifying the REPLICA instance's socket address. +from the MAIN instance by setting a replication mode (SYNC, ASYNC or +STRICT_SYNC) and specifying the REPLICA instance's socket address. The replication mode defines the terms by which the MAIN instance can commit the changes to the database, thus modifying the system to prioritize either consistency or availability: -- **SYNC** - After committing a transaction, the MAIN instance will communicate -the changes to all REPLICA instances running in SYNC mode and wait until it -receives a response or information that a timeout is reached. SYNC mode ensures + +- **STRICT_SYNC** - After committing a transaction, the MAIN instance will +communicate the changes to all REPLICA instances and wait until it receives a +response or information that a timeout is reached. The STRICT_SYNC mode ensures consistency and partition tolerance (CP), but not availability for writes. If the primary database has multiple replicas, the system is highly available for reads. But, when a replica fails, the MAIN instance can't process the write due -to the nature of synchronous replication. +to the nature of synchronous replication. It is implemented as two-phase commit +protocol. + + +- **SYNC** - After committing a transaction, the MAIN instance will communicate +the changes to all REPLICA instances and wait until it receives a response or +information that a timeout is reached. It is different from **STRICT_SYNC** mode +because it the MAIN can continue committing even in situations when **SYNC** +replica is down. + - **ASYNC** - The MAIN instance will commit a transaction without receiving confirmation from REPLICA instances that they have received the same - transaction. ASYNC mode ensures system availability and partition tolerance (AP), - while data can only be eventually consistent. + transaction. ASYNC mode ensures system availability and partition tolerance + (AP), while data can only be eventually consistent. -Users are advised to use the same value for configuration -flag `--storage-wal-file-flush-every-n-txn` on MAIN and SYNC REPLICAs. Otherwise, -the situation could occur in which there is a data which is fsynced on REPLICA and not on MAIN. -In the case MAIN crashes, this could leave to conflicts in system that would need to be -manually resolved by users. +Users are advised to use the same value for configuration flag +`--storage-wal-file-flush-every-n-txn` on MAIN and SYNC REPLICAs. Otherwise, the +situation could occur in which there is a data which is fsynced on REPLICA and +not on MAIN. In the case MAIN crashes, this could leave to conflicts in system +that would need to be manually resolved by users. @@ -118,32 +130,78 @@ If the REPLICA is so far behind the MAIN instance that the synchronization using WAL files and deltas within it is impossible, Memgraph will use snapshots to synchronize the REPLICA to the state of the MAIN instance. -From Memgraph version 2.15, a REPLICA instance has integrated support to only listen to one MAIN. -This part is introduced to support the high availability but also reflects on the replication. -The mechanism that is used is a unique identifier that which MAIN instance sends to all REPLICAs -when REPLICA is first registered on a MAIN. A REPLICA stores the UUID of the MAIN instance it listens to. -The MAIN's UUID is also stored on a disk, in case of restart of an instance to continue listening to the correct -MAIN instance. When REPLICA restarts, `--replication-restore-state-on-startup` must be set to -`true` to continue getting updates from the MAIN. +From Memgraph version 2.15, a REPLICA instance has integrated support to only +listen to one MAIN. This part is introduced to support the high availability but +also reflects on the replication. The mechanism that is used is a unique +identifier that which MAIN instance sends to all REPLICAs when REPLICA is first +registered on a MAIN. A REPLICA stores the UUID of the MAIN instance it listens +to. The MAIN's UUID is also stored on a disk, in case of restart of an instance +to continue listening to the correct MAIN instance. When REPLICA restarts, +`--replication-restore-state-on-startup` must be set to `true` to continue +getting updates from the MAIN. ## Auth data replication (Enterprise) -If you are using a Memgraph Enterprise license, all authentication/authorization data, including users, roles, -and associated permissions, will be replicated. +If you are using a Memgraph Enterprise license, all authentication/authorization +data, including users, roles, and associated permissions, will be replicated. ## Auth modules replication (Enterprise) -Authentication modules are not replicated and must be configured manually by the administrator. +Authentication modules are not replicated and must be configured manually by the +administrator. ## Multi-tenant data replication (Enterprise) -When you are using a Memgraph Enterprise license, multi-tenant commands are replicated as any other data command. -Database manipulation is allowed only on MAIN. However, REPLICAs have the -ability to use databases and read data contained in them. +When you are using a Memgraph Enterprise license, multi-tenant commands are +replicated as any other data command. Database manipulation is allowed only on +MAIN. However, REPLICAs have the ability to use databases and read data +contained in them. When dropping a database used on a REPLICA, the REPLICA will receive the command -and will partially drop the database. It will hide the database and prevent any new -usage. Once all clients have released the database, it will be deleted entirely. +and will partially drop the database. It will hide the database and prevent any +new usage. Once all clients have released the database, it will be deleted +entirely. + + + +As of Memgraph v3.5 replication queries (such as `REGISTER REPLICA`, `SHOW +REPLICAS`, `DROP REPLICA`, etc.) target the default "memgraph" database and +require access to it. The recommendation is to use the default "memgraph" +database as an admin/system database and store graphs under other databases. + + + +### Requirements for replication queries + +To execute replication queries, users must have: +1. The `REPLICATION` privilege +2. **AND** access to the default "memgraph" database + +### Impact on multi-tenant environments + +In multi-tenant environments where users might not have access to the "memgraph" +database, replication management operations will fail. This reinforces the +recommendation to treat the "memgraph" database as an administrative/system +database. + +{

Example: Admin user with replication privileges

} + +```cypher +-- Create admin role with replication privileges +CREATE ROLE replication_admin; +GRANT REPLICATION TO replication_admin; +GRANT DATABASE memgraph TO replication_admin; + +-- Create user with replication admin role +CREATE USER repl_admin IDENTIFIED BY 'admin_password'; +SET ROLE FOR repl_admin TO replication_admin; +``` + +In this setup, `repl_admin` can: +- Execute all replication queries (`REGISTER REPLICA`, `SHOW REPLICAS`, etc.) +- Access the "memgraph" database for administrative operations +- Manage the replication cluster configuration + ## Running multiple instances @@ -162,9 +220,10 @@ cluster](#set-up-a-replication-cluster). Each Memgraph instance has the role of the MAIN instance when it is first started. -Also, by default, each crashed instance restarts with its previous role (MAIN as MAIN, REPLICA as REPLICA). -To change this behavior, set the `--replication-restore-state-on-startup` to `false` when -first initializing the instance. In this way, all instances will get restarted as MAIN. +Also, by default, each crashed instance restarts with its previous role (MAIN as +MAIN, REPLICA as REPLICA). To change this behavior, set the +`--replication-restore-state-on-startup` to `false` when first initializing the +instance. In this way, all instances will get restarted as MAIN. ### Assigning the REPLICA role @@ -177,8 +236,8 @@ SET REPLICATION ROLE TO REPLICA WITH PORT ; ``` If you set the port of each REPLICA instance to `10000`, it will be easier to -register replicas later on because the query for registering replicas uses a port -10000 as the default one. +register replicas later on because the query for registering replicas uses a +port 10000 as the default one. Otherwise, you can use any unassigned port between 1000 and 10000. @@ -201,8 +260,8 @@ retrieve its original function. You need to [drop it](#dropping-a-replica-instance) from the MAIN and register it again. If the crashed MAIN instance goes back online once a new MAIN is already -assigned, it cannot reclaim its previous role. It can be cleaned and -demoted to become a REPLICA instance of the new MAIN instance. +assigned, it cannot reclaim its previous role. It can be cleaned and demoted to +become a REPLICA instance of the new MAIN instance. ### Checking the assigned role @@ -216,12 +275,12 @@ SHOW REPLICATION ROLE; Once all the nodes in the cluster are assigned with appropriate roles, you can enable replication in the MAIN instance by registering REPLICA instances, -setting a replication mode (SYNC and ASYNC), and specifying -the REPLICA instance's socket address. Memgraph doesn't support chaining REPLICA -instances, that is, a REPLICA instance cannot be replicated on another REPLICA -instance. +setting a replication mode (SYNC and ASYNC), and specifying the REPLICA +instance's socket address. Memgraph doesn't support chaining REPLICA instances, +that is, a REPLICA instance cannot be replicated on another REPLICA instance. -If you want to register a REPLICA instance with a SYNC replication mode, run the following query: +If you want to register a REPLICA instance with a SYNC replication mode, run the +following query: ```plaintext REGISTER REPLICA name SYNC TO ; @@ -234,6 +293,14 @@ the following query: REGISTER REPLICA name ASYNC TO ; ``` + +If you want to register a REPLICA instance with an STRICT_SYNC replication mode, +run the following query: + +```plaintext +REGISTER REPLICA name STRICT_SYNC TO ; +``` + The socket address must be a string value as follows: ```plaintext @@ -261,7 +328,8 @@ Example of a `` using only `IP_ADDRESS`: "172.17.0.5" ``` -Also, you can register REPLICA instances using DNS names. In that case, the socket address must be a string value as follows: +Also, you can register REPLICA instances using DNS names. In that case, the +socket address must be a string value as follows: ```plaintext "DOMAIN_NAME:PORT_NUMBER" @@ -274,7 +342,8 @@ number, for example: "memgraph-replica.memgraph.net:10050" ``` -If you set REPLICA roles using port `10000`, you can define the socket address specifying only the valid domain name, for example: +If you set REPLICA roles using port `10000`, you can define the socket address +specifying only the valid domain name, for example: ```plaintext "memgraph-replica.memgraph.net" @@ -282,8 +351,8 @@ If you set REPLICA roles using port `10000`, you can define the socket address s When a REPLICA instance is registered, it will start replication in ASYNC mode until it synchronizes to the current state of the database. Upon -synchronization, REPLICA instances will either continue working in the ASYNC -mode or reset to SYNC mode. +synchronization, REPLICA instances will either continue working in the ASYNC, +STRICT_SYNC or SYNC mode. ### Listing all registered REPLICA instances @@ -312,9 +381,9 @@ with the MAIN instance#synchronizing-instances. The missing data changes can be sent as snapshots or WAL files. Snapshot files represent an image of the current state of the database and are much larger than the WAL files, which only contain the changes, deltas. Because of the difference -in file size, Memgraph favors the WAL files. It is important to note that replicas -receive only changes which are made durable on the MAIN instance, in other words changes which -are already fsynced. +in file size, Memgraph favors the WAL files. It is important to note that +replicas receive only changes which are made durable on the MAIN instance, in +other words changes which are already fsynced. While the REPLICA instance is in the RECOVERY state, the MAIN instance calculates the optimal synchronization path based on the REPLICA instance's @@ -493,10 +562,12 @@ accepts read and write queries to the database and REPLICA instances accept only read queries. The changes or state of the MAIN instance are replicated to the REPLICA -instances in a SYNC or ASYNC mode. The SYNC mode ensures consistency and +instances in a SYNC, STRICT_SYNC or ASYNC mode. The STRICT_SYNC mode ensures consistency and partition tolerance (CP), but not availability for writes. The ASYNC mode ensures system availability and partition tolerance (AP), while data can only be -eventually consistent. +eventually consistent. The SYNC mode is something in between because it waits +for writes to be accepted on replicas but MAIN can still commit even in situations +when one of REPLICAs is down. By using the timestamp, the MAIN instance knows the current state of the REPLICA. If the REPLICA is not synchronized with the MAIN instance, the MAIN @@ -552,6 +623,17 @@ SYNC REPLICA doesn't answer within the expected timeout. ![](/pages/clustering/replication/workflow_diagram_data_manipulation.drawio.png) + +#### STRICT_SYNC replication mode + +The STRICT_SYNC replication mode behaves very similarly to a +SYNC mode except that MAIN won't commit a transaction locally in a situation in +which one of STRICT_SYNC replicas is down. To achieve that, all instances run +together a two-commit protocol which allows you such a synchronization. This +reduces the throughout but such a mode is super useful in a high-availability +scenario in which a failover is the most operation to support. Such a mode then +allows you a failover without the fear of experiencing a data loss. + #### ASYN replication mode In the ASYNC replication mode, the MAIN instance will commit a transaction @@ -571,6 +653,7 @@ instance. ASYNC mode ensures system availability and partition tolerance. + ### Synchronizing instances By comparing timestamps, the MAIN instance knows when a REPLICA instance is not diff --git a/pages/database-management/authentication-and-authorization.mdx b/pages/database-management/authentication-and-authorization.mdx index 9fb2be18e..9017ce078 100644 --- a/pages/database-management/authentication-and-authorization.mdx +++ b/pages/database-management/authentication-and-authorization.mdx @@ -3,12 +3,123 @@ title: Authentication and authorization description: Manage users and roles, implement role-based and fine-grained access control and learn how to integrate with other authentication systems. --- +import { Callout } from 'nextra/components' + # Authentication and authorization Learn how authentication and authorization works in Memgraph. Manage users and roles, secure the database with role-based and fine-grained access control and learn how to integrate with other authentication systems. +## Changes to authentication requirements + +**As of Memgraph v3.5** there are new requirements for authentication and +authorization operations, particularly affecting multi-tenant environments: + +### AUTH privilege requirement + +Authentication and authorization queries (such as `CREATE USER`, `CREATE ROLE`, +`GRANT`, `DENY`, `REVOKE`, etc.) now require the `AUTH` privilege **AND** access +to the "memgraph" database. Users must be explicitly granted this privilege to +perform user and role management operations. + + + +The recommendation is to use the default "memgraph" database as an admin/system +database and store graphs under other databases. + + + +### System queries in multi-tenant environment + +To execute system (auth, replication and multi-database) queries, users must +have: +- The appropriate privileges (`AUTH`, `REPLICATION`, `MULTI_DATABASE_EDIT`) +- **AND** access to the default "memgraph" database + + +## Multi-tenant environment query syntax changes + +### SHOW ROLE syntax in multi-tenant environments + +`SHOW ROLE FOR `: This command does not require database specification and +will show all roles assigned to the user across all databases. + +```cypher +-- Show all roles for a user (works in all environments) +SHOW ROLE FOR user_name; +SHOW ROLES FOR user_name; +``` + +If you need to see roles in a specific database context, you can optionally +specify: + +1. **Show roles for the user's main database:** +```cypher +SHOW ROLE FOR user_name ON MAIN; +``` + +2. **Show roles for the current database:** +```cypher +SHOW ROLE FOR user_name ON CURRENT; +``` + +3. **Show roles for a specific database:** +```cypher +SHOW ROLE FOR user_name ON DATABASE database_name; +``` + +### SHOW PRIVILEGES syntax in multi-tenant environments + +`SHOW PRIVILEGES FOR `: In multi-tenant environments, you must specify the +database context: + +1. **Show privileges for the user's main database:** +```cypher +SHOW PRIVILEGES FOR user_name ON MAIN; +``` + +2. **Show privileges for the current database:** +```cypher +SHOW PRIVILEGES FOR user_name ON CURRENT; +``` + +3. **Show privileges for a specific database:** +```cypher +SHOW PRIVILEGES FOR user_name ON DATABASE database_name; +``` + +These commands return the aggregated roles and privileges for the user in the +specified database context. The `ON MAIN` option shows information for the +user's main database, `ON CURRENT` shows information for whatever database is +currently active, and `ON DATABASE` shows information for the explicitly +specified database. + +`SHOW PRIVILEGES FOR `: This command does not require database +specification and will show all privileges for the role. + +```cypher +-- Show all privileges for a role (works in all environments) +SHOW PRIVILEGES FOR role_name; +``` + +### Multi-tenant recommendations + +For multi-tenant environments, we recommend: +- Treating the default "memgraph" database as an administrative/system database +- Restricting access to the "memgraph" database to privileged users only +- Storing application data in tenant-specific databases +- Ensuring users who need to perform authentication, replication, or + multi-database operations have appropriate access + +For detailed information about these requirements and best practices, see the +[Role-based access +control](/database-management/authentication-and-authorization/role-based-access-control#authentication-and-authorization-requirements), +[Multi-tenancy](/database-management/multi-tenancy#default-database-best-practices), +and +[Replication](/clustering/replication#replication-queries-and-the-memgraph-database) +documentation. + ## [Users](/database-management/authentication-and-authorization/users) Learn how to manage users in Memgraph. @@ -17,6 +128,10 @@ Learn how to manage users in Memgraph. Learn how to manage roles, set up their privileges and fine-grained access control. +## [Multi-role users and multi-tenant roles](/database-management/authentication-and-authorization/multiple-roles) (Enterprise) + +Learn how to assign multiple roles to users simultaneously and understand how permissions are combined from all roles. + ## [Auth system integrations](/database-management/authentication-and-authorization/auth-system-integrations) (Enterprise) Learn how to integrate with third-party auth systems and manage user @@ -25,4 +140,8 @@ authentication and access control using Memgraph's auth module. ## [Impersonate user](/database-management/authentication-and-authorization/impersonate-user) (Enterprise) Learn how the impersonate user feature enables authorized users to execute -queries with the full permissions and context of another user. \ No newline at end of file +queries with the full permissions and context of another user. + +## [User profiles](/database-management/authentication-and-authorization/user-profiles) (Enterprise) + +Learn how to manage user profiles and set resource limits for users to control resource consumption and prevent abuse. \ No newline at end of file diff --git a/pages/database-management/authentication-and-authorization/_meta.ts b/pages/database-management/authentication-and-authorization/_meta.ts index 52e2d13d8..65a49ee06 100644 --- a/pages/database-management/authentication-and-authorization/_meta.ts +++ b/pages/database-management/authentication-and-authorization/_meta.ts @@ -1,6 +1,8 @@ export default { "users": "Users", "role-based-access-control": "Role-based access control", + "multiple-roles": "Multiple roles per user and multi-tenant roles", "auth-system-integrations": "Auth system integrations", - "impersonate-user": "Impersonate user" + "impersonate-user": "Impersonate user", + "user-profiles": "User profiles" } diff --git a/pages/database-management/authentication-and-authorization/auth-system-integrations.mdx b/pages/database-management/authentication-and-authorization/auth-system-integrations.mdx index f5cd5e6c7..b5f22d743 100644 --- a/pages/database-management/authentication-and-authorization/auth-system-integrations.mdx +++ b/pages/database-management/authentication-and-authorization/auth-system-integrations.mdx @@ -27,7 +27,7 @@ privileges will still apply but you won't be able to manage them. ### Roles User roles must be defined in Memgraph before using auth modules because these -modules return the role associated with the user. +modules return the role(s) associated with the user. As of v3.5 Memgraph supports multiple roles per user, allowing auth modules to return either a single role or multiple roles. ### Flags @@ -85,8 +85,9 @@ The protocol used between Memgraph and the module is as follows: - Auth responses must be objects that contain the following fields: - `authenticated` - a `bool` indicating whether the user is allowed to log in to the database - - `role` - a `string` indicating which role the user should have (must be - supplied) + - `role` - a `string` indicating which role the user should have (backward compatible) + - `roles` - an array of strings indicating which roles the user should have (new format) + - `username` - the user's username (optional) - `errors` (optional) - if `authenticated` is false, Memgraph will put up a warning with the error message returned by the module @@ -95,6 +96,59 @@ Memgraph won't allow the user to log in to the database and will automatically restart the auth module for the next auth request. All crash logs will be seen in Memgraph's output (typically in `systemd` logs using `journalctl`). +### Multiple roles support + +**As of v3.5** Memgraph supports multiple roles per user in auth module +responses. Auth modules can return either a single role (backward compatible) or +multiple roles (new format). + +#### Single role response (backward compatible) + +```python +def authenticate(username, password): + return { + "authenticated": True, + "role": "moderator" # Single role as string + } +``` + +#### Multiple roles response (new format) + +```python +def authenticate(username, password): + return { + "authenticated": True, + "roles": ["admin", "user"] # Multiple roles as array + } +``` + +#### Single role in array format + +```python +def authenticate(username, password): + return { + "authenticated": True, + "roles": ["admin"] # Single role in array + } +``` + +The system will: +1. First check for a `roles` field in the response +2. If `roles` is an array, use all roles in the array +3. If `roles` is a string, use it as a single role +4. If no `roles` field is found, fall back to the `role` field for backward + compatibility +5. If no valid roles are found, authentication fails + +When a user has multiple roles, their permissions are combined using the +following rules: +- **Grants**: If any role grants a permission, the user has that permission +- **Denies**: If any role denies a permission, the user is denied that + permission +- **Database Access**: If any role grants and no role denies access to a + database, the user has access +- **Fine-grained Permissions**: Combined using the same grant/deny logic + ### Module example This very simple example auth module is written in Python, but any programming @@ -107,7 +161,15 @@ import io def authenticate(username, password): - return {"authenticated": True, "role": "moderator"} + # Example with multiple roles + if username == "admin_user" and password == "password": + return {"authenticated": True, "roles": ["admin", "user"]} + + # Example with single role (backward compatible) + if username == "moderator_user" and password == "password": + return {"authenticated": True, "role": "moderator"} + + return {"authenticated": False, "errors": "Invalid credentials"} if __name__ == "__main__": @@ -132,8 +194,8 @@ files. For example: #!/usr/bin/python3 import module -assert module.authenticate("sponge", "bob") == {"authenticated": True, "role": "analyst"} -assert module.authenticate("CHUCK", "NORRIS") == {"authenticated": True, "role": "admin"} +assert module.authenticate("admin_user", "password") == {"authenticated": True, "roles": ["admin", "user"]} +assert module.authenticate("moderator_user", "password") == {"authenticated": True, "role": "moderator"} ``` ## Single sign-on @@ -150,12 +212,21 @@ identity service roles correspond to Memgraph’s. The role mapping is defined as a string where individual mappings are separated by a semicolon `;`. Each mapping is structured as follows: -`{identity service role}:{Memgraph role}`. +`{identity service role}:{Memgraph role}, {another Memgraph role}, ...`. + +One identity service role can be mapped to one or more Memgraph roles. +When a user logs in and is assigned an identity service role that is mapped to an array of Memgraph roles, the user is assigned all of the mapped Memgraph roles. +For more information regarding how multi-role users are handled by Memgraph, please visit [Multi-role users and multi-tenant roles](/database-management/authentication-and-authorization/multiple-roles). For example, the `entra.admin:memadmin; entra.user:memuser` mapping defines that the identity service roles `entra.admin` and `entra.user` respectively map to Memgraph’s `memadmin` and `memuser` roles. +`entra.admin:memadmin; entra.user:memuser, memdev` maps `entra.user` to `memuser` and `memdev`. + +Different services use different parameters for defining roles. +Use `MEMGRAPH_SSO_{provider}_{protocol}_ROLE_FILED` to specify the token parameter that specifies the assigned roles. + For correct operation, the Memgraph roles defined in the mapping need to be @@ -163,6 +234,10 @@ created in the Memgraph DB beforehand. Additionally, you have to grant [label-ba + +SSO identity providers often return multiple roles for users. Memgraph now supports this natively - if your identity provider returns multiple roles, they will all be mapped to Memgraph roles and the user will have permissions from all assigned roles combined. + + ### SAML Memgraph has built-in support for single sign-on (SSO) over the SAML protocol @@ -320,9 +395,39 @@ One way to deduce the audience of the access and id tokens is to decode them usi Often time access and id token will the use the same audience. For example in MS Entra ID both tokens use the client ID as audience. +##### Connect via Neo4j drivers + +When connecting through a driver, you can choose to provide only the access token and skip the ID token. +In general, all connection methods follow the same approach: setting only the **scheme** and **credentials**. +- **Scheme**: Use the scheme that applies to your setup (`oidc-entra-id`, `oidc-okta` or `custom`) +- **Credentials**: Provide them as a string in this format: +```access_token=token-data;id_token=token-data``` +If you don't want to include the ID token, simply omit it: +```access_token=token-data``` + + +The OIDC module automatically determines whether to validate the ID token based on your username configuration. If your username is configured to use a field from the ID token (e.g., `id:sub`), the module will require and validate the ID token. If your username uses a field from the access token (e.g., `access:preferred_username`), the ID token validation is skipped. + + +Below is an example of connecting via the Neo4j Python driver. + +```python +from neo4j import GraphDatabase, Auth + +driver = GraphDatabase.driver(MEMGRAPH_URI, + auth = Auth( + scheme="oidc-entra-id", + credentials=`access_token=token-data;id_token=token-data`, + realm=None, + principal=None + ) +) +``` + ##### Username The username variable tells the OIDC module what to use as the username. It has the format `token-type:field`. Token type can be `id` or `access` depending on whether you want to use a field from the access or the ID token for the username. See the following to learn more about [access](https://www.okta.com/identity-101/access-token/) and [id](https://developer.okta.com/docs/guides/validate-id-tokens/main/#id-tokens-vs-access-tokens) tokens. + By default, it is set to `id:sub` as per the OIDC protocol it is recommended to use the `sub` field from the id token as it is non-mutable and globally unique for each application. For Okta one commonly used field is `access:sub` which is usually the email of the user. You can also configure [custom claims](https://developer.okta.com/docs/guides/customize-tokens-returned-from-okta/main/). diff --git a/pages/database-management/authentication-and-authorization/multiple-roles.mdx b/pages/database-management/authentication-and-authorization/multiple-roles.mdx new file mode 100644 index 000000000..648331be5 --- /dev/null +++ b/pages/database-management/authentication-and-authorization/multiple-roles.mdx @@ -0,0 +1,384 @@ +--- +title: Multi-role users and multi-tenant roles +description: Learn how users can have multiple roles and how to assign database-specific roles in multi-tenant environments with special permission filtering logic. +--- + +import { Callout } from 'nextra/components' + +# Multi-role users and multi-tenant roles (Enterprise) + +**As of v3.5** Memgraph allows users to have multiple roles assigned to them +simultaneously. Memgraph Enterprise supports multi-tenant roles, which allow +users to have different roles assigned to them for specific databases. This +feature enables proper tenant isolation and fine-grained access control in +multi-tenant environments. + +## Privileges with multiple roles + +When a user has multiple roles, their privileges are combined according to the +following rules: + +- **Grant**: If any assigned role with access to the database grants a + privilege, the user is granted that privilege. +- **Deny**: If any assigned role with access to the database denies a privilege, + the user is denied that privilege, even if another role grants it. +- **Effective privilege**: The user's effective privileges are the union of all + granted privileges, minus any that are denied by any role. + +This means that **deny always takes precedence over grant** when there is a +conflict.
+**Note:** The resulting user privileges contain user's privileges only +if the user also has access to the database. + +## Database access with users and roles + +### Basic database access + +In Memgraph Enterprise, both users and roles can have database access +permissions: + +```cypher +-- Grant database access to a role +GRANT DATABASE db_name TO user_or_role; + +-- Deny database access +DENY DATABASE db_name TO user_or_role; + +-- Revoke database access +REVOKE DATABASE db_name TO user_or_role; +``` + +### How database access works + +Database access follows these rules: +- **Grants**: If any role or user grants access to a database, database is + granted +- **Denies**: If any role or user denies access to a database, database is + denied +- **Access**: User or role has database access if it is granted and not denied + + +### Combining database access + +When a user has multiple roles, their database access is combined from all +roles: + +```cypher +-- Create roles with different database access +CREATE ROLE role1; +CREATE ROLE role2; + +-- Grant different database access to each role +GRANT DATABASE db1 TO role1; +GRANT DATABASE db2 TO role2; + +-- Create user and assign both roles +CREATE USER alice IDENTIFIED BY 'password'; +SET ROLE FOR alice TO role1, role2; + +-- Result: Alice has access to both db1 and db2 +``` + +### Database access conflicts + +When roles have conflicting database access, deny takes precedence: + +```cypher +-- Role1 grants access to db1 +GRANT DATABASE db1 TO role1; + +-- Role2 denies access to db1 +DENY DATABASE db1 TO role2; + +-- User with both roles +SET ROLE FOR user TO role1, role2; + +-- Result: User is denied access to db1 (deny takes precedence) +``` + +## Creating database-specific roles + +### Using database access for tenant isolation + +You can create roles that are specific to certain databases by controlling their +database access: + +```cypher +-- Create tenant-specific roles +CREATE ROLE tenant1_admin; +CREATE ROLE tenant2_user; + +-- Grant database access to specific tenants +GRANT DATABASE tenant1_db TO tenant1_admin; +GRANT DATABASE tenant2_db TO tenant2_user; + +-- Grant appropriate permissions +GRANT ALL PRIVILEGES TO tenant1_admin; +GRANT MATCH, CREATE, MERGE, SET TO tenant2_user; + +-- Create user with both roles +CREATE USER bob IDENTIFIED BY 'password'; +SET ROLE FOR bob TO tenant1_admin, tenant2_user; +``` + +### Limitations of this approach + +While this approach works, it has some limitations: +- Users get access to all databases their roles have access to +- No fine-grained control over which role applies to which database +- Permission filtering is based on database access, not role assignment +- Admin needs to create a set of role for each database + +## Better API: Database-specific role assignment + +### Setting roles ON a database + +Memgraph Enterprise provides a better API for database-specific role assignment +using the `ON database` clause: + +```cypher +-- Assign role to specific database +SET ROLE FOR user_name TO role_name ON database_name; + +-- Remove role from specific database +CLEAR ROLE FOR user_name ON database_name; + +-- Remove all roles +CLEAR ROLES; +``` + +**Note:** Multiple roles can be specified on a database. However, the list of +roles needs to be exhaustive. + +### How it works + +This API provides true database-specific role assignment: + +```cypher +-- Create roles with database access +CREATE ROLE tenant1_admin; +CREATE ROLE tenant2_user; + +-- Grant database access to roles +GRANT DATABASE tenant1_db TO tenant1_admin; +GRANT DATABASE tenant2_db TO tenant2_user; + +-- Grant permissions +GRANT ALL PRIVILEGES TO tenant1_admin; +GRANT MATCH, CREATE, MERGE, SET TO tenant2_user; + +-- Create user with database-specific roles +CREATE USER alice IDENTIFIED BY 'password'; +SET ROLE FOR alice TO tenant1_admin ON tenant1_db; +SET ROLE FOR alice TO tenant2_user ON tenant2_db; +``` + +**Note:** The list of roles defined in the SET ROLE query is an exhaustive list. + +### Special logic for database filtering + +When using `SET ROLE ... ON database`, the system applies special logic: + +1. **Database Access Check**: Only roles with access to the specified database + can be assigned +2. **Permission Filtering**: When accessing a database, only roles assigned to + that database are considered +3. **Isolation**: Users cannot access permissions from roles assigned to other + databases +4. **Automatic Filtering**: The system automatically filters permissions based + on the current database context + +## Multi-tenant role management + +### Adding database-specific roles + +To assign a role to a user for a specific database: + +```cypher +-- The role must have access to the database +GRANT DATABASE database_name TO role_name; +SET ROLE FOR user_name TO role_name ON database_name; +``` + +### Clearing database-specific roles + +To remove a role from a specific database: + +```cypher +CLEAR ROLE FOR user_name ON database_name; +``` + +### Viewing multi-tenant roles + +**Note**: The `SHOW ROLE FOR USER` command does not require database +specification, even in multi-tenant environments. It will show all roles +assigned to the user across all databases. + +```cypher +-- Show all roles for a user (works in all environments) +SHOW ROLE FOR user_name; +SHOW ROLES FOR user_name; +``` + +To see which roles a user has for a specific database: + +```cypher +SHOW ROLES FOR user_name ON database_name; +``` + +In multi-tenant environments, you can also use these additional options: + +1. **Show roles for the user's main database:** +```cypher +SHOW ROLES FOR user_name ON MAIN; +``` + +2. **Show roles for the current database:** +```cypher +SHOW ROLES FOR user_name ON CURRENT; +``` + +3. **Show roles for a specific database:** +```cypher +SHOW ROLES FOR user_name ON DATABASE database_name; +``` + +These commands return the aggregated roles for the user in the specified +database context. + +### Viewing permissions for a specific database + +To see what permissions a user has in a specific database: + +```cypher +SHOW PRIVILEGES FOR user_name ON database_name; +``` + +In multi-tenant environments, you can also use these additional options: + +1. **Show privileges for the user's main database:** +```cypher +SHOW PRIVILEGES FOR user_name ON MAIN; +``` + +2. **Show privileges for the current database:** +```cypher +SHOW PRIVILEGES FOR user_name ON CURRENT; +``` + +3. **Show privileges for a specific database:** +```cypher +SHOW PRIVILEGES FOR user_name ON DATABASE database_name; +``` + +These commands return the aggregated privileges for the user in the specified +database context. + +### SSO integration with multi-tenant roles + +When using external auth modules, users can be assigned multi-tenant roles based +on their identity provider roles: + +```python +def authenticate(username, password): + # Example: User has multiple roles from identity provider + if username == "cross_tenant_manager" and password == "password": + return { + "authenticated": True, + "roles": ["tenant1_admin", "tenant2_user"], + "role_databases": ["tenant1_db", "tenant2_db"], + "username": "cross_tenant_manager" + } + + return {"authenticated": False, "errors": "Invalid credentials"} +``` + +## Database access control + +### Main database selection + +When a user has multi-tenant roles with access to different databases, the +system determines the main database: + +```cypher +-- Create roles with different main databases +CREATE ROLE role1; +CREATE ROLE role2; +GRANT DATABASE db1 TO role1; +GRANT DATABASE db2 TO role2; +SET MAIN DATABASE db1 FOR role1; +SET MAIN DATABASE db2 FOR role2; + +-- User with both roles +CREATE USER user1 IDENTIFIED BY 'password'; +SET ROLE role1 FOR user1 ON db1; +SET ROLE role2 FOR user1 ON db2; + +``` + + + In case of an SSO connection, no user information is stored in Memgraph; + instead the auth module returns roles associated with the connection. In + this case, there are no guarantees which role's main database will be + selected. Use "database" in the session arguments to define the target + database. + + +### Database-specific permission filtering + +The special logic ensures that when accessing a specific database, only roles +assigned to that database are considered: + +```cypher +-- Role with access to multiple databases +CREATE ROLE multi_db_role; +GRANT DATABASE db1 TO multi_db_role; +GRANT DATABASE db2 TO multi_db_role; +GRANT MATCH, CREATE ON db1 TO multi_db_role; +GRANT MATCH ON db2 TO multi_db_role; + +-- User with this role +CREATE USER user1 IDENTIFIED BY 'password'; +SET ROLE multi_db_role FOR user1 ON db1; +SET ROLE multi_db_role FOR user1 ON db2; + +-- When accessing db1: has MATCH, CREATE +-- When accessing db2: has only MATCH +``` + +## Best practices for multi-tenant environments + +1. **Use descriptive role names**: Include tenant information in role names + (e.g., `tenant1_admin`, `tenant2_user`) +2. **Follow principle of least privilege**: Grant only necessary permissions to + each role +3. **Separate tenant-specific roles**: Create distinct roles for each tenant to + ensure proper isolation +4. **Test permission combinations**: Verify that multi-tenant permissions work + correctly in each database +5. **Document role assignments**: Keep track of which users have which roles for + which databases +6. **Use deny sparingly**: Remember that deny takes precedence over grant across + all databases +7. **Treat memgraph database as admin database**: In multi-tenant environments, + restrict access to the default "memgraph" database to privileged users only +8. **Ensure AUTH privilege access**: Users who need to perform + authentication/authorization operations must have both the `AUTH` privilege + and access to the "memgraph" database +9. **Ensure replication privilege access**: Users who need to perform + replication operations must have both the `REPLICATION` privilege and access + to the "memgraph" database +10. **Ensure multi-database privilege access**: Users who need to perform + multi-database operations must have the appropriate privileges + (`MULTI_DATABASE_USE`, `MULTI_DATABASE_EDIT`) and access to the "memgraph" + database +11. **Separate application data**: Store all application data in tenant-specific + databases, not in the default "memgraph" database +12. **Plan for administrative operations**: Design your role structure to ensure + that users who need to manage users, roles, replication, or multi-database + operations have appropriate access to the "memgraph" database +13. **Use explicit database context**: Use `ON MAIN` for the user's main database, + `ON CURRENT` for the currently active database, or `ON DATABASE` for a + specific database +14. **Verify permissions in context**: Always check roles and privileges in the + specific database context where they will be used diff --git a/pages/database-management/authentication-and-authorization/role-based-access-control.mdx b/pages/database-management/authentication-and-authorization/role-based-access-control.mdx index 7f1f93b87..7ae979e56 100644 --- a/pages/database-management/authentication-and-authorization/role-based-access-control.mdx +++ b/pages/database-management/authentication-and-authorization/role-based-access-control.mdx @@ -18,11 +18,15 @@ role, enhancing security and minimizing risks. With role-based access control, a database administrator can assign various privileges to roles, but for even more control over who can access certain data, Memgraph Enterprise offers [fine-grained access -control](#fine-grained-access-control). +control](#fine-grained-access-control). Additionally, you can use [user profiles](/database-management/authentication-and-authorization/user-profiles) to set resource limits for users. + + +[Multi-role users and multi-tenant roles](/database-management/authentication-and-authorization/multiple-roles) for more information regarding assigning multiple roles to users or assigning roles for a specific database. + ## User roles -Each user can be assigned at most one user role. User roles are abstractions +Users can be assigned multiple roles simultaneously, with permissions from all roles being combined. User roles are abstractions that capture the privilege levels of a set of users. For example, suppose that `Dominik` and `Marko` belong to the upper management of a @@ -37,6 +41,12 @@ to that user). Similarly, each privilege that is denied to a user role is automatically denied to all users with that role (even if it has been explicitly granted to that user). +When a user has multiple roles, their permissions are combined using the following rules: +- **Grants**: If any role grants a permission, the user has that permission +- **Denies**: If any role denies a permission, the user is denied that permission +- **Database Access**: If any role grants access to a database, the user has access +- **Fine-grained Permissions**: Combined using the same grant/deny logic + To create a user role, run the following query: ```cypher @@ -48,10 +58,10 @@ If a role already exists, you can use `IF NOT EXISTS` to only create new roles. To assign a user with a certain user role, run the following query: ```cypher -SET ROLE FOR user_name TO role_name; +SET ROLE FOR user_name TO role_name [, another_role, ...]; ``` -To remove the role from the user, run the following query: +To remove all roles from the user, run the following query: ```cypher CLEAR ROLE FOR user_name; @@ -63,18 +73,51 @@ To show all users with a certain role: SHOW USERS FOR role_name; ``` -To show what role a user has, run the following query: +To show what roles a user has, run the following query: ```cypher SHOW ROLE FOR user_name; ``` +**Note**: The `SHOW ROLE FOR USER` command does not require database specification, even in multi-tenant environments. It will show all roles assigned to the user across all databases. + +To show the current user's roles in the current session: + +```cypher +SHOW CURRENT ROLE; +``` + +In multi-tenant environments, you can optionally specify which database context to use when showing roles: + +1. **Show roles for the user's main database:** +```cypher +SHOW ROLE FOR user_name ON MAIN; +``` + +2. **Show roles for the current database:** +```cypher +SHOW ROLE FOR user_name ON CURRENT; +``` + +3. **Show roles for a specific database:** +```cypher +SHOW ROLE FOR user_name ON DATABASE database_name; +``` + +These commands return the aggregated roles for the user in the specified database context. The `ON MAIN` option shows roles for the user's main database, `ON CURRENT` shows roles for whatever database is currently active, and `ON DATABASE` shows roles for the explicitly specified database. + To list all defined user roles run: ```cypher SHOW ROLES; ``` +## User profiles + +User profiles allow you to set resource limits for individual users to control resource consumption and prevent system abuse. + +For detailed information about user profiles, including profile creation, management, and advanced features, see the [User profiles](/database-management/authentication-and-authorization/user-profiles) documentation. + ## Privileges At the moment, privileges are confined to users' abilities to perform certain @@ -106,9 +149,70 @@ of the following commands: | Privilege to change [storage mode](/fundamentals/storage-memory-usage#storage-modes). | `STORAGE_MODE` | | Privilege to manage [multi-tenant databases](/database-management/multi-tenancy). | `MULTI_DATABASE_EDIT` | | Privilege to use a database within the multi-tenant architecture. | `MULTI_DATABASE_USE` | +| Privilege to set limits and monitor resource usage per user. | `PROFILE_RESTRICTION` | | Privileges to specific labels. | `ALL LABELS` | | Privileges to specific relationships types. | `ALL EDGE TYPES` | +## Authentication and authorization requirements + + + +As of Memgraph v3.5 users can have different privileges on different databases. This is due to v3.5 introducing users with multiple roles and database specific roles. +All system queries (auth, replication and multi-database) now target the default "memgraph" database. Meaning that in order to execute one of these queries, a user must have the appropriate privilege AND access to "memgraph" database. +The recommendation is to use the default "memgraph" database as an admin/system database and store graphs under other databases. + + + +### System queries in multi-tenant environments + +To execute system queries (auth, replication and multi-database), users must have: +- The appropriate privileges (`AUTH`, `REPLICATION`, `MULTI_DATABASE_USE`, `MULTI_DATABASE_EDIT`) +- **AND** access to the default "memgraph" database + +### Recommended approach for multi-tenant environments + +In multi-tenant environments, we recommend treating the default "memgraph" database as an administrative/system database rather than storing application data in it. This approach provides better security and isolation: + +1. **Restrict access to the memgraph database**: Only grant access to privileged users who need to perform authentication, authorization, replication, or multi-database management operations +2. **Use tenant-specific databases**: Store application data in dedicated tenant databases rather than the default database +3. **Separate administrative functions**: Keep user management, system administration, replication management, and multi-database management separate from application data + +#### Example setup for multi-tenant environments + +```cypher +-- Create admin role with full privileges +CREATE ROLE admin; +GRANT ALL PRIVILEGES TO admin; +GRANT DATABASE memgraph TO admin; + +-- Create tenant-specific roles +CREATE ROLE tenant1_user; +CREATE ROLE tenant2_user; + +-- Grant appropriate permissions to tenant roles +GRANT MATCH, CREATE, MERGE, SET, DELETE TO tenant1_user; +GRANT MATCH, CREATE, MERGE, SET, DELETE TO tenant2_user; + +-- Grant access to tenant databases only +GRANT DATABASE tenant1_db TO tenant1_user; +GRANT DATABASE tenant2_db TO tenant2_user; + +-- Create users +CREATE USER admin_user IDENTIFIED BY 'admin_password'; +CREATE USER tenant1_user_account IDENTIFIED BY 'password1'; +CREATE USER tenant2_user_account IDENTIFIED BY 'password2'; + +-- Assign roles +SET ROLE FOR admin_user TO admin; +SET ROLE FOR tenant1_user_account TO tenant1_user; +SET ROLE FOR tenant2_user_account TO tenant2_user; +``` + +In this setup: +- `admin_user` has access to the "memgraph" database and can perform all authentication/authorization, replication, and multi-database operations +- `tenant1_user_account` and `tenant2_user_account` can only access their respective tenant databases +- Application data is stored in tenant-specific databases, not in the default "memgraph" database + After the first user is created, Memgraph will execute a query if and only if either a user or its role is granted that privilege and neither the user nor its role are denied that privilege. Otherwise, Memgraph will not execute that @@ -215,6 +319,30 @@ To check privilege for a certain user or role, run the following query: SHOW PRIVILEGES FOR user_or_role; ``` +In multi-tenant environments, privileges can differ depending on the target database. The SHOW PRIVILEGE query can be expanded to show privileges on specific databases as the following: + +1. **Show privileges for the user's main database:** +```cypher +SHOW PRIVILEGES FOR user_or_role ON MAIN; +``` + +2. **Show privileges for the current database:** +```cypher +SHOW PRIVILEGES FOR user_or_role ON CURRENT; +``` + +3. **Show privileges for a specific database:** +```cypher +SHOW PRIVILEGES FOR user_or_role ON DATABASE database_name; +``` + +These commands return the aggregated privileges (including label-based permissions) for the user or role in the specified database context. + +**Note**: +- For **users**: In multi-tenant environments, you must specify the database context. +- For **roles**: This command does not require database specification, even in multi-tenant environments. In which case, it will role's privileges without filtering for database. + + ## Fine-grained access control Sometimes, authorizing the database by granting and denying clause privileges is @@ -316,7 +444,28 @@ To check which privileges an existing user or role has in Memgraph, it is enough SHOW PRIVILEGES FOR user_or_role; ``` -and all the values of clause privileges, as well as label-based permissions will be displayed. +In multi-tenant environments, privileges can differ depending on the target database. The SHOW PRIVILEGE query can be expanded to show privileges on specific databases as the following: + +1. **Show privileges for the user's main database:** +```cypher +SHOW PRIVILEGES FOR user_or_role ON MAIN; +``` + +2. **Show privileges for the current database:** +```cypher +SHOW PRIVILEGES FOR user_or_role ON CURRENT; +``` + +3. **Show privileges for a specific database:** +```cypher +SHOW PRIVILEGES FOR user_or_role ON DATABASE database_name; +``` + +These commands return the aggregated privileges (including label-based permissions) for the user or role in the specified database context. + +**Note**: +- For **users**: In multi-tenant environments, you must specify the database context. +- For **roles**: This command does not require database specification, even in multi-tenant environments. In which case, it will role's privileges without filtering for database. ### Templates for granting privileges diff --git a/pages/database-management/authentication-and-authorization/user-profiles.mdx b/pages/database-management/authentication-and-authorization/user-profiles.mdx new file mode 100644 index 000000000..e582356ba --- /dev/null +++ b/pages/database-management/authentication-and-authorization/user-profiles.mdx @@ -0,0 +1,200 @@ +--- +title: User profiles +description: Learn how to manage user profiles and set resource limits for users in Memgraph Enterprise. +--- + +# User profiles + +User profiles allow you to set resource limits for users in Memgraph Enterprise. You can define limits on the number of sessions and memory usage to control resource consumption and prevent abuse. + +User profiles provide a way to: +- Set resource limits for individual users +- Control the number of concurrent sessions per user +- Limit query memory usage over all active +- Monitor resource consumption in real-time +- Enforce resource quotas to prevent system abuse + +## Prerequisites + +To use user profiles, you need: +- [Memgraph Enterprise Edition](/database-management/enabling-memgraph-enterprise) +- The [`PROFILE_RESTRICTION` privilege](/database-management/authentication-and-authorization/role-based-access-control#privileges) to manage profiles. + +## Creating profiles + +You can create a profile with default unlimited limits: + +```cypher +CREATE PROFILE profile_name; +``` + +Or create a profile with specific limits: + +```cypher +CREATE PROFILE profile_name LIMIT sessions 10, transactions_memory 100MB; +``` + +### Available limits + +- **sessions**: Maximum number of concurrent sessions (default: unlimited) +- **transactions_memory**: Maximum memory usage over all active transactions (default: unlimited) + +### Limit values + +You can specify limits in different formats: + +- **Unlimited**: `UNLIMITED` (default) +- **Quantity**: A positive number (e.g., `10`) +- **Memory**: A number with unit MB/KB (e.g., `100MB`, `512KB`) + +### Examples + +```cypher +-- Create a profile with session limit only +CREATE PROFILE session_limited LIMIT sessions 5; + +-- Create a profile with memory limit only +CREATE PROFILE memory_limited LIMIT transactions_memory 50MB; + +-- Create a profile with both limits +CREATE PROFILE strict_profile LIMIT sessions 3, transactions_memory 25MB; + +-- Create a profile with different memory units +CREATE PROFILE small_profile LIMIT transactions_memory 1KB; +``` + +## Managing profiles + +### Update a profile + +```cypher +UPDATE PROFILE profile_name LIMIT sessions 5, transactions_memory 50MB; +``` + +### Drop a profile + +```cypher +DROP PROFILE profile_name; +``` + +**Note**: When you drop a profile, all users assigned to that profile will have their limits reset and profile assignment cleared. + + +### Clear a profile assignment + +```cypher +CLEAR PROFILE FOR username; +``` + +This removes the profile assignment, returning the user to unlimited resources. + +## Viewing profile assignments + +### Show profile for a user + +```cypher +SHOW PROFILE FOR username; +``` + +### Show users assigned to a profile + +```cypher +SHOW USERS FOR PROFILE profile_name; +``` + +## Monitoring resource usage + +### Show resource usage for a user + +```cypher +SHOW RESOURCE USAGE FOR username; +``` + +This command shows the current resource consumption and imposed limits for the specified user, including: +- Number of active sessions +- Current memory usage over all active transactions + +## Profile management + +User profiles are assigned directly to users and provide resource limits for those specific users. Each user can have one profile assigned at a time. + +### Profile assignment behavior + +**Important**: Profile assignment is a simple mapping between profile names and usernames. This means: + +- **Users don't need to exist** when you assign a profile to them +- You can assign a profile to a username that hasn't been created yet +- You can assign a profile to a SSO user that will never exist in Memgraph +- The profile will be automatically applied when that user connects to the database +- Dropping a profile **does remove the mapping** + +## Error handling + +The system provides clear error messages for invalid operations: + +- **Duplicate profile creation**: Error when trying to create a profile with an existing name +- **Non-existent profile operations**: Error when trying to show, update, or drop non-existent profiles +- **Invalid limit values**: Error for negative numbers or invalid memory units +- **Invalid limit names**: Error for unsupported limit types + +**Note**: Assigning a profile to a non-existent user will **not** cause an error. The assignment will be stored and applied when the user connects to the database. + +## Best practices + +1. **Start with unlimited profiles**: Create profiles without limits first, then gradually add restrictions +2. **Monitor usage**: Regularly check resource usage to understand actual consumption patterns + +## Examples + +### Complete workflow example + +```cypher +-- 1. Create users +CREATE USER developer1; +CREATE USER developer2; + +-- 2. Create profiles with different restrictions +CREATE PROFILE basic_profile LIMIT sessions 10; +CREATE PROFILE strict_profile LIMIT sessions 3, transactions_memory 50MB; + +-- 3. Assign profiles +SET PROFILE FOR developer1 TO basic_profile; +SET PROFILE FOR developer2 TO strict_profile; + +-- 4. Verify assignments +SHOW PROFILE FOR developer1; +SHOW USERS FOR PROFILE basic_profile; + +-- 5. Monitor usage +SHOW RESOURCE USAGE FOR developer1; + +-- 6. Update limits based on usage patterns +UPDATE PROFILE strict_profile LIMIT sessions 5, transactions_memory 25MB; + +-- 7. Verify limits +SHOW RESOURCE USAGE FOR developer2; +``` + +## Syntax reference + +| Command | Description | +|---------|-------------| +| `CREATE PROFILE name [LIMIT limit_list]` | Create a new profile | +| `UPDATE PROFILE name LIMIT limit_list` | Update existing profile limits | +| `DROP PROFILE name` | Delete a profile | +| `SHOW PROFILES` | List all profiles | +| `SHOW PROFILE name` | Show specific profile details | +| `SET PROFILE FOR user TO profile` | Assign profile to user | +| `CLEAR PROFILE FOR user` | Remove profile assignment | +| `SHOW PROFILE FOR user` | Show profile assigned to user | +| `SHOW USERS FOR PROFILE name` | List users assigned to profile | +| `SHOW RESOURCE USAGE FOR user` | Show current resource usage | + +### Limit syntax + +``` +limit_list: limit_item [, limit_item]* +limit_item: sessions number | transactions_memory memory_value +memory_value: number (MB | KB) +number: positive integer +``` \ No newline at end of file diff --git a/pages/database-management/authentication-and-authorization/users.mdx b/pages/database-management/authentication-and-authorization/users.mdx index 41b8393ec..c9b8b5392 100644 --- a/pages/database-management/authentication-and-authorization/users.mdx +++ b/pages/database-management/authentication-and-authorization/users.mdx @@ -12,8 +12,17 @@ In Memgraph, users and their passwords can be created with a simple Cypher query. This level of security is supported within the Community version of Memgraph. For more advanced security features within Memgraph Enterprise, check out [role-based access -control](/database-management/authentication-and-authorization/role-based-access-control) -and [auth system integrations](/database-management/authentication-and-authorization/auth-system-integrations). +control](/database-management/authentication-and-authorization/role-based-access-control), +[auth system integrations](/database-management/authentication-and-authorization/auth-system-integrations), +and [user profiles](/database-management/authentication-and-authorization/user-profiles). + + +Memgraph Enterprise now supports database-specific roles for a user, allowing users to have different roles on specific databases. See the [role-based access control](/database-management/authentication-and-authorization/role-based-access-control) section for details on managing multiple roles. + + + +**Backward Compatibility**: Memgraph automatically migrates existing single-role users to the new multi-role format during startup. This migration is transparent and requires no user intervention. Existing users and their roles will continue to work without any changes. + ## Administer users @@ -96,6 +105,51 @@ SHOW USERS; If no users exist, `SHOW USERS` returns no results. +## User profiles (Enterprise) + +In Memgraph Enterprise, you can assign user profiles to control resource limits for users. User profiles allow you to set limits on: + +- **Number of concurrent sessions**: Control how many simultaneous connections a user can have +- **Transaction memory usage**: Limit the amount of query memory a user can consume over all active transactions + +### Basic profile operations + +To assign a profile to a user: + +```cypher +SET PROFILE FOR username TO profile_name; +``` + +To view the profile assigned to a user: + +```cypher +SHOW PROFILE FOR username; +``` + +To clear a user's profile (removes all limits): + +```cypher +CLEAR PROFILE FOR username; +``` + +To see all users assigned to a profile: + +```cypher +SHOW USERS FOR PROFILE profile_name; +``` + +To monitor current resource usage for a user: + +```cypher +SHOW RESOURCE USAGE FOR username; +``` + +### Profile assignment + +**Note**: You can assign a profile to a username even if the user doesn't exist yet. The profile will be automatically applied when the user connects to the database. + +For detailed information about user profiles, including profile creation, management, and advanced features, see the [User profiles](/database-management/authentication-and-authorization/user-profiles) documentation. + ### Password encryption algorithm Memgraph offers multiple password encryption algorithms: diff --git a/pages/database-management/backup-and-restore.mdx b/pages/database-management/backup-and-restore.mdx index 769b0d67f..658827587 100644 --- a/pages/database-management/backup-and-restore.mdx +++ b/pages/database-management/backup-and-restore.mdx @@ -111,9 +111,14 @@ SHOW SNAPSHOTS; ``` Its results contain the path to the file, the logical timestamp, the physical -timestamp and the file size. If the periodic snapshot background job is active, -the first element in the results will define the time at which the snapshots -will be created. +timestamp and the file size. + +As of Memgraph v3.5, the `SHOW SNAPSHOTS` query does not return information regarding the next scheduled snapshot. +A special query has been added: +``` +SHOW NEXT SNAPSHOT; +``` +If the periodic snapshot background job is active, the result will return the path and the time at which the snapshots will be created. If you are using Memgraph pre v2.22, follow these steps to restore data from a backup: @@ -262,3 +267,50 @@ keeps its own [WAL](https://github.com/facebook/rocksdb/wiki/Write-Ahead-Log-%28WAL%29) files. Memgraph persists the metadata used in the implementation of the on-disk storage. + +## Backup in multi-tenancy Enterprise + +When running Memgraph with multi-tenancy, every database other than the default database +(named `memgraph`) will have its own associated database UUID. Database UUID can be inspected +by running the `SHOW STORAGE INFO` command and reading the value under the `database_uuid` key. +The default data directory location for a specific database is `/var/lib/memgraph//`. +The default database `memgraph` does not follow this directory structure and the data files are directly located under `/var/lib/memgraph`. + +Manual snapshot backup flow should look like this: + + + +{

Create snapshots inside Memgraph

} + +Create snapshot for every database (or let it create automatically with periodic snapshot execution inside Memgraph) + +{

Perform backup

} + +Backup the snapshot for every database into a 3rd party location. Currently, you're encouraged to perform the backup mechanisms by +yourself with tools such as [rclone](https://rclone.org/). + +{

When performing recovery, copy the snapshot to Memgraph

} + +When recovering a specific database, copy the snapshot to any data location + - if the data directory the snapshot is being copied to is a location that's outside the database directory, ensure the snapshot + has the permissions to be copied to the database data directory + - if the data directory the snapshot is being copied to is a location that's inside the database directory, there should be no issues + with permissions as there is no copying being performed from source to target directory location + +{

Position to specific database

} + +Position the database driver interacting with Memgraph into the database using `USE DATABASE ` + +{

Recover the snapshot

} + +Execute `RECOVER SNAPSHOT FORCE` + + + + +## Best practices + +Memgraph can optimize restoring of snapshots in a multi-threaded manner. To enable multi-threaded restoration of a snapshot, you need to ensure +the following flags are present: +- `--storage-parallel-schema-recovery=true` +- `--storage-recovery-thread-count=` where `number_of_cores` is the amount of CPU cores to parallelize the restoration process \ No newline at end of file diff --git a/pages/database-management/enabling-memgraph-enterprise.mdx b/pages/database-management/enabling-memgraph-enterprise.mdx index 438a9776b..5ad2285be 100644 --- a/pages/database-management/enabling-memgraph-enterprise.mdx +++ b/pages/database-management/enabling-memgraph-enterprise.mdx @@ -9,7 +9,7 @@ The following Memgraph features are only available in Enterprise Edition: | **Category** | **Features** | |---------------|-------------| -| **Security** | [Role-based access control (RBAC)](#role-based-access-control)
[Label-based access control (LBAC)](#role-based-access-control)
[Auth system integrations](#authentication-system-integrations) (LDAP, SAML, OIDC)
[Impersonate user](#impersonate-user)
[Hiding sensitive information](#hiding-sensitive-information) | +| **Security** | [Role-based access control (RBAC)](#role-based-access-control)
[Label-based access control (LBAC)](#role-based-access-control)
[Multi-role users and multi-tenant roles](#multi-role-users-and-multi-tenant-roles)
[Auth system integrations](#authentication-system-integrations) (LDAP, SAML, OIDC)
[Impersonate user](#impersonate-user)
[Hiding sensitive information](#hiding-sensitive-information) | | **Logging & monitoring** | [Audit log](#audit-log)
[Metrics tracking via HTTP server](#metrics-tracking-via-http-server) (Prometheus integration) | | **Database management** | [High availability](#high-availability) with automatic failover
[Multi-tenancy](#multi-tenancy)
[CRON snapshot scheduling](#cron-snapshot-scheduling) | | **Querying** | [Dynamic graph algorithms](#dynamic-graph-algorithms)
[Time-to-live (TTL)](#time-to-live-ttl) for data expiration | @@ -99,6 +99,19 @@ citizens, a database administrator can now keep all the data in one database while keeping any private data secure from those who don’t have adequate permission. +### Multi-role users and multi-tenant roles + +[Multi-role users and multi-tenant roles](/database-management/authentication-and-authorization/multiple-roles) enable advanced user management and tenant isolation in Memgraph Enterprise. This feature allows users to have multiple roles assigned simultaneously, with permissions combined from all roles according to specific rules. + +**Key capabilities:** +- **Multiple roles per user**: Users can have multiple roles with combined permissions +- **Database-specific role assignment**: Roles can be assigned to specific databases using `SET ROLE ... ON database` syntax +- **Tenant isolation**: Users can have different roles for different databases, ensuring proper data isolation +- **SSO integration**: Support for external identity providers that return multiple roles +- **Permission filtering**: Automatic filtering of permissions based on the current database context + +This feature is particularly valuable for multi-tenant environments where organizations need to manage users with different access levels across multiple isolated databases while maintaining strict security boundaries. + ### Authentication system integrations Memgraph supports authentication and authorization using external auth modules. @@ -208,6 +221,10 @@ terms of graph algorithms: expiration time. Once a vertex has expired, the vertex and all associated edges will be deleted. +### User profiles +[User profiles](/database-management/authentication-and-authorization/user-profiles) allows administrators to monitor and limit +resources used by specific users. You can set limits on the number of concurrent sessions and transaction memory usage to control resource consumption and prevent system abuse. + ## Memgraph Lab Enterprise features ### Monitoring diff --git a/pages/database-management/multi-tenancy.md b/pages/database-management/multi-tenancy.md index f40366fef..21bfd9496 100644 --- a/pages/database-management/multi-tenancy.md +++ b/pages/database-management/multi-tenancy.md @@ -16,10 +16,69 @@ Instead, global limitations are imposed on Memgraph as a whole. ## Default database -A default database named 'memgraph' is automatically created during startup. New +A default database named `memgraph` is automatically created during startup. New users are granted access only to this default database. The default database name cannot be altered. +### Default database best practices + +In multi-tenant environments, we recommend treating the default "memgraph" database as an administrative/system database rather than storing application data in it. This approach provides better security and isolation, especially given recent changes to authentication and authorization requirements. + +#### Why treat memgraph as an admin database? + +As of Memgraph v3.5, users have to have both the `AUTH` privilege and access to the default "memgraph" database to execute authentication and authorization queries. Additionally, replication queries (such as `REGISTER REPLICA`, `SHOW REPLICAS`, etc.) and multi-database queries (such as `SHOW DATABASES`, `CREATE DATABASE`, etc.) also now target the "memgraph" database and require access to it. This requirement affects multi-tenant environments where users might have access to other databases but not the default one. + +#### Recommended setup + +1. **Restrict memgraph database access**: Only grant access to the "memgraph" database to privileged users who need to perform system administration tasks +2. **Use tenant-specific databases**: Store all application data in dedicated tenant databases +3. **Separate concerns**: Keep user management, role management, system administration, replication management, and multi-database management separate from application data + +#### Example configuration + +```cypher +-- Create admin role with full system privileges +CREATE ROLE system_admin; +GRANT ALL PRIVILEGES TO system_admin; +GRANT DATABASE memgraph TO system_admin; + +-- Create tenant-specific roles (no access to memgraph database) +CREATE ROLE tenant1_admin; +CREATE ROLE tenant1_user; +CREATE ROLE tenant2_admin; +CREATE ROLE tenant2_user; + +-- Grant appropriate permissions to tenant roles +GRANT MATCH, CREATE, MERGE, SET, DELETE, INDEX TO tenant1_admin; +GRANT MATCH, CREATE, MERGE, SET, DELETE TO tenant1_user; +GRANT MATCH, CREATE, MERGE, SET, DELETE, INDEX TO tenant2_admin; +GRANT MATCH, CREATE, MERGE, SET, DELETE TO tenant2_user; + +-- Grant access only to tenant databases +GRANT DATABASE tenant1_db TO tenant1_admin, tenant1_user; +GRANT DATABASE tenant2_db TO tenant2_admin, tenant2_user; + +-- Create users +CREATE USER system_admin_user IDENTIFIED BY 'admin_password'; +CREATE USER tenant1_admin_user IDENTIFIED BY 't1_admin_pass'; +CREATE USER tenant1_regular_user IDENTIFIED BY 't1_user_pass'; +CREATE USER tenant2_admin_user IDENTIFIED BY 't2_admin_pass'; +CREATE USER tenant2_regular_user IDENTIFIED BY 't2_user_pass'; + +-- Assign roles +SET ROLE FOR system_admin_user TO system_admin; +SET ROLE FOR tenant1_admin_user TO tenant1_admin; +SET ROLE FOR tenant1_regular_user TO tenant1_user; +SET ROLE FOR tenant2_admin_user TO tenant2_admin; +SET ROLE FOR tenant2_regular_user TO tenant2_user; +``` + +In this configuration: +- `system_admin_user` can perform all authentication/authorization, replication, and multi-database operations and has access to the "memgraph" database +- Tenant users can only access their respective tenant databases +- Application data is completely isolated in tenant-specific databases +- The "memgraph" database serves purely as an administrative database + ## Isolated databases Isolated databases within Memgraph function as distinct single-database Memgraph @@ -27,6 +86,9 @@ instances. Queries executed on a specific database should operate as if it were the sole database in the system, preventing cross-database contamination. Users interact with individual databases, and cross-database queries are prohibited. +Every database has its own database UUID, which can be read by running the `SHOW STORAGE INFO` +query on a particular database. + ## Database configuration and data directory At present, all isolated databases share identical configurations. There is no @@ -51,7 +113,8 @@ Users interact with multi-tenant features through specialized Cypher queries: 1. `CREATE DATABASE name`: Creates a new database. 2. `DROP DATABASE name`: Deletes a specified database. 3. `SHOW DATABASE`: Shows the current used database. It will return `NULL` if - there is not one. + no database is currently in use. You can also use `SHOW CURRENT DATABASE` for the same functionality. This command does not require any special privileges. + 4. `SHOW DATABASES`: Shows only the existing set of multitenant databases. 5. `USE DATABASE name`: Switches focus to a specific database (disabled during transactions). @@ -81,6 +144,88 @@ Access to all databases can be granted or revoked using wildcards: `GRANT DATABASE * TO user;`, `DENY DATABASE * TO user;` or `REVOKE DATABASE * FROM user;`. +### Multi-database queries and the memgraph database + +As of Memgraph v3.5 multi-database queries (such as `SHOW DATABASES`, `CREATE DATABASE`, `DROP DATABASE`, etc.) target the "memgraph" database and require access to it. + +To execute these queries, users must have: +- The appropriate privileges (`MULTI_DATABASE_USE`, `MULTI_DATABASE_EDIT`) +- **AND** access to the default "memgraph" database + +### Multi-tenant query syntax changes + +As of Memgraph v3.5 the syntax for certain queries in multi-tenant environments have changed. The `SHOW ROLE` and `SHOW PRIVILEGES` commands now require specifying the database context in some cases. + +**SHOW ROLE FOR USER**: This command does not require database specification and will show all roles assigned to the user across all databases. + +**SHOW PRIVILEGES FOR USER**: This command requires database specification in multi-tenant environments. + +**SHOW PRIVILEGES FOR ROLE**: This command does not require database specification and will show all privileges for the role. + +In multi-tenant environments, you must specify which database context to use when showing privileges for users: + +1. **Show roles for the user's main database:** +```cypher +SHOW ROLE FOR user_name ON MAIN; +``` + +2. **Show roles for the current database:** +```cypher +SHOW ROLE FOR user_name ON CURRENT; +``` + +3. **Show roles for a specific database:** +```cypher +SHOW ROLE FOR user_name ON DATABASE database_name; +``` + +#### SHOW PRIVILEGES syntax in multi-tenant environments + +Similarly, the `SHOW PRIVILEGES` command requires database context specification: + +1. **Show privileges for the user's main database:** +```cypher +SHOW PRIVILEGES FOR user_or_role ON MAIN; +``` + +2. **Show privileges for the current database:** +```cypher +SHOW PRIVILEGES FOR user_or_role ON CURRENT; +``` + +3. **Show privileges for a specific database:** +```cypher +SHOW PRIVILEGES FOR user_or_role ON DATABASE database_name; +``` + +These commands return the aggregated roles and privileges for the user in the specified database context. The `ON MAIN` option shows information for the user's main database, `ON CURRENT` shows information for whatever database is currently active, and `ON DATABASE` shows information for the explicitly specified database. + +#### Impact on multi-tenant environments + +In multi-tenant environments where users might not have access to the "memgraph" database, multi-database management operations will fail. This reinforces the recommendation to treat the "memgraph" database as an administrative/system database. + +#### Example: Admin user with multi-database privileges + +```cypher +-- Create admin role with multi-database privileges +CREATE ROLE multi_db_admin; +GRANT MULTI_DATABASE_USE, MULTI_DATABASE_EDIT TO multi_db_admin; +GRANT DATABASE memgraph TO multi_db_admin; + +-- Create user with multi-database admin role +CREATE USER db_admin IDENTIFIED BY 'admin_password'; +SET ROLE FOR db_admin TO multi_db_admin; +``` + +In this setup, `db_admin` can: +- Execute all multi-database queries (`SHOW DATABASES`, `CREATE DATABASE`, etc.) +- Access the "memgraph" database for administrative operations +- Manage the multi-tenant database configuration + +#### Best practice + +For multi-database management, ensure that users who need to perform multi-database operations have both the appropriate multi-database privileges and access to the "memgraph" database. This aligns with the overall recommendation to treat the "memgraph" database as an administrative database in multi-tenant environments. + ### Additional multi-tenant privileges Administrators manage multi-tenant privileges with: diff --git a/pages/database-management/server-stats.md b/pages/database-management/server-stats.md index ec145fda0..dfcd0d680 100644 --- a/pages/database-management/server-stats.md +++ b/pages/database-management/server-stats.md @@ -30,6 +30,7 @@ The result will contain the following fields: | Field | Description | |------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | name | Name of the current database. | +| database_uuid | Unique UUID of the database. | | vertex_count | The number of stored nodes (vertices). | | edge_count | The number of stored relationships (edges). | | average_degree | The average number of relationships of a single node. | diff --git a/pages/deployment/workloads/memgraph-in-cybersecurity.mdx b/pages/deployment/workloads/memgraph-in-cybersecurity.mdx index 88a591148..d95328aba 100644 --- a/pages/deployment/workloads/memgraph-in-cybersecurity.mdx +++ b/pages/deployment/workloads/memgraph-in-cybersecurity.mdx @@ -166,6 +166,7 @@ For security use cases involving attack path analysis and threat propagation, Me - **Weighted shortest paths**: Calculate the most likely attack paths based on security metrics (e.g., vulnerability scores, access levels) - **All shortest paths**: Identify all possible attack vectors between critical assets +- **K shortest paths**: Find alternative attack routes and analyze path diversity in security networks - **Path filtering**: Focus analysis on specific types of security relationships or nodes These algorithms are crucial for: @@ -267,7 +268,7 @@ possible for the **highest performance** in security event processing. When working with security event streams, sometimes your incoming payload contains **serialized JSON strings** that need to be transformed into property maps inside your graph. -Memgraph provides the [`convert.str2object` function](/querying/functions#conversion-functions) to easily handle this scenario. +Memgraph provides the [`convert.str2object` function](/advanced-algorithms/available-algorithms/convert#str2object) to easily handle this scenario. Example usage: diff --git a/pages/deployment/workloads/memgraph-in-high-throughput-workloads.mdx b/pages/deployment/workloads/memgraph-in-high-throughput-workloads.mdx index ee289d0f0..50a941b65 100644 --- a/pages/deployment/workloads/memgraph-in-high-throughput-workloads.mdx +++ b/pages/deployment/workloads/memgraph-in-high-throughput-workloads.mdx @@ -219,7 +219,7 @@ possible for the **highest performance**. When working with streamed data, sometimes your incoming payload contains **serialized JSON strings** that need to be transformed into property maps inside your graph. -Memgraph provides the [`convert.str2object` function](/querying/functions#conversion-functions) to easily handle this scenario. +Memgraph provides the [`convert.str2object` function](/advanced-algorithms/available-algorithms/convert#str2object) to easily handle this scenario. Example usage: diff --git a/pages/fundamentals/data-durability.mdx b/pages/fundamentals/data-durability.mdx index 933d56e90..ba90845d3 100644 --- a/pages/fundamentals/data-durability.mdx +++ b/pages/fundamentals/data-durability.mdx @@ -119,6 +119,7 @@ If another snapshot is already being created or no committed writes to the datab By default, snapshot files are saved inside the `var/lib/memgraph/snapshots` directory. +The `CREATE SNAPSHOT` query will return the path of the newly created snapshot file. To query which snapshots currently exist in the data directory, execute: ```opencypher diff --git a/pages/fundamentals/indexes.mdx b/pages/fundamentals/indexes.mdx index 9f60dc098..0bb9e964e 100644 --- a/pages/fundamentals/indexes.mdx +++ b/pages/fundamentals/indexes.mdx @@ -701,7 +701,11 @@ and [`storage-automatic-edge-type-index-creation-enabled`](/configuration/configuration-settings#storage) flags, it is possible to create label and edge-type indices automatically. Every time the database encounters a label or edge-type that is currently not indexed, -it will create an index for that construct. +it will queue an index creation request that runs asynchronously in the background. + +Auto-index creation operates with the following characteristics: +- **Asynchronous execution**: Index creation runs in dedicated background transaction +- **Concurrent creation**: Utilizes the same non-blocking mechanism as manual index creation, allowing user queries to continue uninterrupted ## Recovery @@ -894,9 +898,7 @@ extends how long write queries will retry before failing. The system maintains **full MVCC consistency** throughout the process, ensuring transactional integrity. Long-running index operations can be safely cancelled -if needed. Currently, some features like replication synchronization and TTL -indices still use blocking mode during operations, though these limitations will -be addressed in future releases. +if needed. For complete technical details about the implementation, consistency guarantees, and current limitations, please refer to the [Concurrent Index Creation diff --git a/pages/fundamentals/storage-memory-usage.mdx b/pages/fundamentals/storage-memory-usage.mdx index d40dd0183..da11d7d97 100644 --- a/pages/fundamentals/storage-memory-usage.mdx +++ b/pages/fundamentals/storage-memory-usage.mdx @@ -757,25 +757,26 @@ SHOW STORAGE INFO; ``` ```copy=false -+--------------------------------+--------------------------------+ -| storage info | value | -+--------------------------------+--------------------------------+ -| "name" | "memgraph" | -| "vertex_count" | 0 | -| "edge_count" | 0 | -| "average_degree" | 0 | -| "vm_max_map_count" | 453125 | -| "memory_res" | "43.16MiB" | -| "peak_memory_res" | "43.16MiB" | -| "unreleased_delta_objects" | 0 | -| "disk_usage" | "104.46KiB" | -| "memory_tracked" | "8.52MiB" | -| "allocation_limit" | "58.55GiB" | -| "global_isolation_level" | "SNAPSHOT_ISOLATION" | -| "session_isolation_level" | "" | -| "next_session_isolation_level" | "" | -| "storage_mode" | "IN_MEMORY_TRANSACTIONAL" | -+--------------------------------+--------------------------------+ ++--------------------------------+----------------------------------------+ +| storage info | value | ++--------------------------------+----------------------------------------+ +| "name" | "memgraph" | +| "database_uuid" | "99f743e8-5b98-4b78-8384-f96862b7f01b" | +| "vertex_count" | 0 | +| "edge_count" | 0 | +| "average_degree" | 0 | +| "vm_max_map_count" | 453125 | +| "memory_res" | "43.16MiB" | +| "peak_memory_res" | "43.16MiB" | +| "unreleased_delta_objects" | 0 | +| "disk_usage" | "104.46KiB" | +| "memory_tracked" | "8.52MiB" | +| "allocation_limit" | "58.55GiB" | +| "global_isolation_level" | "SNAPSHOT_ISOLATION" | +| "session_isolation_level" | "" | +| "next_session_isolation_level" | "" | +| "storage_mode" | "IN_MEMORY_TRANSACTIONAL" | ++--------------------------------+----------------------------------------+ ``` Find out more about `SHOW STORAGE INFO` query on [Server diff --git a/pages/getting-started.mdx b/pages/getting-started.mdx index 56ad5c9ed..c90f54cea 100644 --- a/pages/getting-started.mdx +++ b/pages/getting-started.mdx @@ -155,7 +155,7 @@ libraries](/client-libraries) and follow their getting started guide. Memgraph offers a range of procedures tailored to address specific graph problems. Built-in deep path traversal algorithms such as BFS, DFS, Weighted -shortest path, and All shortest paths can be executed using their specific +shortest path, All shortest paths, and K shortest paths can be executed using their specific clauses. Memgraph comes with expanded set of algorithms called [Memgraph Advanced Graph diff --git a/pages/getting-started/install-memgraph/kubernetes.mdx b/pages/getting-started/install-memgraph/kubernetes.mdx index 815171f10..688d2b95c 100644 --- a/pages/getting-started/install-memgraph/kubernetes.mdx +++ b/pages/getting-started/install-memgraph/kubernetes.mdx @@ -165,6 +165,35 @@ want to use. Using the latest tag can lead to issues, as a pod restart may pull a newer image, potentially causing unexpected changes or incompatibilities. +### Install Memgraph standalone chart with `minikube` + +If you are installing Memgraph standalone chart locally with `minikube`, we are strongly recommending to enable `csi-hostpath-driver` and use its storage class. Otherwise, +you could have problems with attaching PVCs to pods. + +1. Enable `csi-hostpath-driver` +``` +minikube addons disable storage-provisioner +minikube addons disable default-storageclass +minikube addons enable volumesnapshots +minikube addons enable csi-hostpath-driver +``` + +2. Create a storage class with `csi-hostpath-driver` as a provider (file sc.yaml) + +``` +apiVersion: storage.k8s.io/v1 +kind: StorageClass +metadata: + name: csi-hostpath-delayed +provisioner: hostpath.csi.k8s.io +volumeBindingMode: WaitForFirstConsumer +reclaimPolicy: Delete +``` + +3. `kubectl apply -f sc.yaml` + +4. Set `storageClassName` to `csi-hostpath-delayed` in `values.yaml` + #### Access Memgraph Once Memgraph is installed, you can access it using the provided services and @@ -177,71 +206,82 @@ Lab](/data-visualization). The following table lists the configurable parameters of the Memgraph chart and their default values. -| Parameter | Description | Default | -| ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | -| `image.repository` | Memgraph Docker image repository | `memgraph/memgraph` | -| `image.tag` | Specific tag for the Memgraph Docker image. Overrides the image tag whose default is chart version. | `""` (Defaults to chart's app version) | -| `image.pullPolicy` | Image pull policy | `IfNotPresent` | -| `useImagePullSecrets` | Override the default imagePullSecrets | `false` | -| `imagePullSecrets` | Specify image pull secrets | `- name: regcred` | -| `replicaCount` | Number of Memgraph instances to run. Note: no replication or HA support. | `1` | -| `affinity.nodeKey` | Key for node affinity (Preferred) | `""` | -| `affinity.nodeValue` | Value for node affinity (Preferred) | `""` | -| `nodeSelector` | Constrain which nodes your Memgraph pod is eligible to be scheduled on, based on the labels on the nodes. Left empty by default. | `{}` | -| `service.type` | Kubernetes service type | `ClusterIP` | -| `service.enableBolt` | Enable Bolt protocol | `true` | -| `service.boltPort` | Bolt protocol port | `7687` | -| `service.enableWebsocketMonitoring` | Enable WebSocket monitoring | `false` | -| `service.websocketPortMonitoring` | WebSocket monitoring port | `7444` | -| `service.enableHttpMonitoring` | Enable HTTP monitoring | `false` | -| `service.httpPortMonitoring` | HTTP monitoring port | `9091` | -| `service.annotations` | Annotations to add to the service | `{}` | -| `service.labels` | Labels to add to the service | `{}` | -| `persistentVolumeClaim.createStorageClaim` | Enable creation of a Persistent Volume Claim for storage | `true` | -| `persistentVolumeClaim.storageClassName` | Storage class name for the persistent volume claim | `""` | -| `persistentVolumeClaim.storageSize` | Size of the persistent volume claim for storage | `10Gi` | -| `persistentVolumeClaim.existingClaim` | Use an existing Persistent Volume Claim | `memgraph-0` | -| `persistentVolumeClaim.storageVolumeName` | Name of an existing Volume to create a PVC for | `""` | -| `persistentVolumeClaim.createLogStorage` | Enable creation of a Persistent Volume Claim for logs | `true` | -| `persistentVolumeClaim.logStorageClassName` | Storage class name for the persistent volume claim for logs | `""` | -| `persistentVolumeClaim.logStorageSize` | Size of the persistent volume claim for logs | `1Gi` | -| `memgraphConfig` | List of strings defining Memgraph configuration settings | `["--also-log-to-stderr=true"]` | -| `secrets.enabled` | Enable the use of Kubernetes secrets for Memgraph credentials | `false` | -| `secrets.name` | The name of the Kubernetes secret containing Memgraph credentials | `memgraph-secrets` | -| `secrets.userKey` | The key in the Kubernetes secret for the Memgraph user, the value is passed to the `MEMGRAPH_USER` env | `USER` | -| `secrets.passwordKey` | The key in the Kubernetes secret for the Memgraph password, the value is passed to the `MEMGRAPH_PASSWORD` | `PASSWORD` | -| `memgraphEnterpriseLicense` | Memgraph Enterprise License | `""` | -| `memgraphOrganizationName` | Organization name for Memgraph Enterprise License | `""` | -| `statefulSetAnnotations` | Annotations to add to the stateful set | `{}` | -| `podAnnotations` | Annotations to add to the pod | `{}` | -| `resources` | CPU/Memory resource requests/limits. Left empty by default. | `{}` | -| `tolerations` | A toleration is applied to a pod and allows the pod to be scheduled on nodes with matching taints. Left empty by default. | `[]` | -| `serviceAccount.create` | Specifies whether a service account should be created | `true` | -| `serviceAccount.annotations` | Annotations to add to the service account | `{}` | -| `serviceAccount.name` | The name of the service account to use. If not set and create is true, a name is generated. | `""` | -| `container.terminationGracePeriodSeconds` | Grace period for pod termination | `1800` | -| `container.livenessProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as bolt port. | `7687` | -| `container.livenessProbe.failureThreshold` | Failure threshold for liveness probe | `20` | -| `container.livenessProbe.timeoutSeconds` | Initial delay for readiness probe | `10` | -| `container.livenessProbe.periodSeconds` | Period seconds for readiness probe | `5` | -| `container.readinessProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as bolt port. | `7687` | -| `container.readinessProbe.failureThreshold` | Failure threshold for readiness probe | `20` | -| `container.readinessProbe.timeoutSeconds` | Initial delay for readiness probe | `10` | -| `container.readinessProbe.periodSeconds` | Period seconds for readiness probe | `5` | -| `container.startupProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as bolt port. | `7687` | -| `container.startupProbe.failureThreshold` | Failure threshold for startup probe | `1440` | -| `container.startupProbe.periodSeconds` | Period seconds for startup probe | `10` | -| `nodeSelectors` | Node selectors for pod. Left empty by default. | `{}` | -| `customQueryModules` | List of custom Query modules that should be mounted to Memgraph Pod | `[]` | -| `sysctlInitContainer.enabled` | Enable the init container to set sysctl parameters | `true` | -| `sysctlInitContainer.maxMapCount` | Value for `vm.max_map_count` to be set by the init container | `262144` | -| `storageClass.create` | If set to true, new StorageClass will be created. | `false` | -| `storageClass.name` | Name of the StorageClass | `"memgraph-generic-storage-class"` | -| `storageClass.provisioner` | Provisioner for the StorageClass | `""` | -| `storageClass.storageType` | Type of storage for the StorageClass | `""` | -| `storageClass.fsType` | Filesystem type for the StorageClass | `""` | -| `storageClass.reclaimPolicy` | Reclaim policy for the StorageClass | `Retain` | -| `storageClass.volumeBindingMode` | Volume binding mode for the StorageClass | `Immediate` | +| Parameter | Description | Default | +| --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | +| `image.repository` | Memgraph Docker image repository | `memgraph/memgraph` | +| `image.tag` | Specific tag for the Memgraph Docker image. Overrides the image tag whose default is chart version. | `""` (Defaults to chart's app version) | +| `image.pullPolicy` | Image pull policy | `IfNotPresent` | +| `memgraphUserId` | The user id that is hardcoded in Memgraph and Mage images | `101` | +| `memgraphGroupId` | The group id that is hardcoded in Memgraph and Mage images | `103` | +| `useImagePullSecrets` | Override the default imagePullSecrets | `false` | +| `imagePullSecrets` | Specify image pull secrets | `- name: regcred` | +| `replicaCount` | Number of Memgraph instances to run. Note: no replication or HA support. | `1` | +| `affinity.nodeKey` | Key for node affinity (Preferred) | `""` | +| `affinity.nodeValue` | Value for node affinity (Preferred) | `""` | +| `nodeSelector` | Constrain which nodes your Memgraph pod is eligible to be scheduled on, based on the labels on the nodes. Left empty by default. | `{}` | +| `service.type` | Kubernetes service type | `ClusterIP` | +| `service.enableBolt` | Enable Bolt protocol | `true` | +| `service.boltPort` | Bolt protocol port | `7687` | +| `service.enableWebsocketMonitoring` | Enable WebSocket monitoring | `false` | +| `service.websocketPortMonitoring` | WebSocket monitoring port | `7444` | +| `service.enableHttpMonitoring` | Enable HTTP monitoring | `false` | +| `service.httpPortMonitoring` | HTTP monitoring port | `9091` | +| `service.annotations` | Annotations to add to the service | `{}` | +| `service.labels` | Labels to add to the service | `{}` | +| `persistentVolumeClaim.createStorageClaim` | Enable creation of a Persistent Volume Claim for storage | `true` | +| `persistentVolumeClaim.storageClassName` | Storage class name for the persistent volume claim | `""` | +| `persistentVolumeClaim.storageSize` | Size of the persistent volume claim for storage | `10Gi` | +| `persistentVolumeClaim.existingClaim` | Use an existing Persistent Volume Claim | `memgraph-0` | +| `persistentVolumeClaim.storageVolumeName` | Name of an existing Volume to create a PVC for | `""` | +| `persistentVolumeClaim.createLogStorage` | Enable creation of a Persistent Volume Claim for logs | `true` | +| `persistentVolumeClaim.logStorageClassName` | Storage class name for the persistent volume claim for logs | `""` | +| `persistentVolumeClaim.logStorageSize` | Size of the persistent volume claim for logs | `1Gi` | +| `persistentVolumeClaim.createUserClaim` | Create a Dynamic Persistant Volume Claim for Configs, Certificates (e.g. Bolt cert ) and rest of User related files | `false` | +| `persistentVolumeClaim.userStorageClassName` | Storage class name for the persistent volume claim for user storage | `""` | +| `persistentVolumeClaim.userStorageSize` | Size of the persistent volume claim for user storage | `1Gi` | +| `persistentVolumeClaim.userStorageAccessMode` | Storage Class Access Mode. If you need a different pod to add data into Memgraph (e.g. CSV files) set this to "ReadWriteMany" | `ReadWriteOnce` | +| `persistentVolumeClaim.userMountPath` | Where to mount the `userStorageClass` you should set this variable if you are enabling the `UserClaim` | `""` | +| `memgraphConfig` | List of strings defining Memgraph configuration settings | `["--also-log-to-stderr=true"]` | +| `secrets.enabled` | Enable the use of Kubernetes secrets for Memgraph credentials | `false` | +| `secrets.name` | The name of the Kubernetes secret containing Memgraph credentials | `memgraph-secrets` | +| `secrets.userKey` | The key in the Kubernetes secret for the Memgraph user, the value is passed to the `MEMGRAPH_USER` env | `USER` | +| `secrets.passwordKey` | The key in the Kubernetes secret for the Memgraph password, the value is passed to the `MEMGRAPH_PASSWORD` | `PASSWORD` | +| `memgraphEnterpriseLicense` | Memgraph Enterprise License | `""` | +| `memgraphOrganizationName` | Organization name for Memgraph Enterprise License | `""` | +| `statefulSetAnnotations` | Annotations to add to the stateful set | `{}` | +| `podAnnotations` | Annotations to add to the pod | `{}` | +| `resources` | CPU/Memory resource requests/limits. Left empty by default. | `{}` | +| `tolerations` | A toleration is applied to a pod and allows the pod to be scheduled on nodes with matching taints. Left empty by default. | `[]` | +| `serviceAccount.create` | Specifies whether a service account should be created | `true` | +| `serviceAccount.annotations` | Annotations to add to the service account | `{}` | +| `serviceAccount.name` | The name of the service account to use. If not set and create is true, a name is generated. | `""` | +| `container.terminationGracePeriodSeconds` | Grace period for pod termination | `1800` | +| `container.livenessProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as bolt port. | `7687` | +| `container.livenessProbe.failureThreshold` | Failure threshold for liveness probe | `20` | +| `container.livenessProbe.timeoutSeconds` | Initial delay for readiness probe | `10` | +| `container.livenessProbe.periodSeconds` | Period seconds for readiness probe | `5` | +| `container.readinessProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as bolt port. | `7687` | +| `container.readinessProbe.failureThreshold` | Failure threshold for readiness probe | `20` | +| `container.readinessProbe.timeoutSeconds` | Initial delay for readiness probe | `10` | +| `container.readinessProbe.periodSeconds` | Period seconds for readiness probe | `5` | +| `container.startupProbe.tcpSocket.port` | Port used for TCP connection. Should be the same as bolt port. | `7687` | +| `container.startupProbe.failureThreshold` | Failure threshold for startup probe | `1440` | +| `container.startupProbe.periodSeconds` | Period seconds for startup probe | `10` | +| `nodeSelectors` | Node selectors for pod. Left empty by default. | `{}` | +| `customQueryModules` | List of custom Query modules that should be mounted to Memgraph Pod | `[]` | +| `storageClass.create` | If set to true, new StorageClass will be created. | `false` | +| `storageClass.name` | Name of the StorageClass | `"memgraph-generic-storage-class"` | +| `storageClass.provisioner` | Provisioner for the StorageClass | `""` | +| `storageClass.storageType` | Type of storage for the StorageClass | `""` | +| `storageClass.fsType` | Filesystem type for the StorageClass | `""` | +| `storageClass.reclaimPolicy` | Reclaim policy for the StorageClass | `Retain` | +| `storageClass.volumeBindingMode` | Volume binding mode for the StorageClass | `Immediate` | +| `initContainers` | User specific init containers | `[]` | +| `sysctlInitContainer.enabled` | Enable the init container to set sysctl parameters | `true` | +| `sysctlInitContainer.maxMapCount` | Value for `vm.max_map_count` to be set by the init container | `262144` | +| `sysctlInitContainer.image.repository` | Busybox image repository | `library/busybox` | +| `sysctlInitContainer.image.tag` | Specific tag for the Busybox Docker image | `latest` | +| `sysctlInitContainer.image.pullPolicy` | Image pull policy for busybox | `IfNotPresent` | To change the default chart values, provide your own `values.yaml` file during the installation: @@ -329,6 +369,36 @@ want to use. Using the latest tag can lead to issues, as a pod restart may pull a newer image, potentially causing unexpected changes or incompatibilities. +### Install Memgraph HA chart with `minikube` + +If you are installing Memgraph HA chart locally with `minikube`, we are strongly recommending to enable `csi-hostpath-driver` and use its storage class. Otherwise, +you could have problems with attaching PVCs to pods. + +1. Enable `csi-hostpath-driver` +``` +minikube addons disable storage-provisioner +minikube addons disable default-storageclass +minikube addons enable volumesnapshots +minikube addons enable csi-hostpath-driver +``` + +2. Create a storage class with `csi-hostpath-driver` as a provider (file sc.yaml) + +``` +apiVersion: storage.k8s.io/v1 +kind: StorageClass +metadata: + name: csi-hostpath-delayed +provisioner: hostpath.csi.k8s.io +volumeBindingMode: WaitForFirstConsumer +reclaimPolicy: Delete +``` + +3. `kubectl apply -f sc.yaml` + +4. Set `libStorageClassName` to `csi-hostpath-delayed` in `values.yaml` + + ### Changing the default chart values To change the default chart values, run the command with the specified set of @@ -369,8 +439,7 @@ Uninstalling the chart won't trigger deletion of persistent volume claims (PVCs) ### Security context -All instances are started as `StatefulSet` with one pod. The pod has two or three containers depending on whether the sysctlInitContainer.enabled is used. The **init** container -is used to set permissions on volume mounts. It is used as root user with `CHOWN` capability and without privileged access. The **memgraph-coordinator** container is the one which +All instances are started as `StatefulSet` with one pod. The pod has two or three containers depending on whether the sysctlInitContainer.enabled is used. The **memgraph-coordinator** container is the one which actually runs Memgraph image. The process is run by non-root **memgraph** user without any Linux capabilities. Privileges cannot escalate. ### High availability storage @@ -637,6 +706,8 @@ The following table lists the configurable parameters of the Memgraph HA chart a | `image.pullPolicy` | Image pull policy | `IfNotPresent` | | `env.MEMGRAPH_ENTERPRISE_LICENSE` | Memgraph enterprise license | `` | | `env.MEMGRAPH_ORGANIZATION_NAME` | Organization name | `` | +| `memgraphUserId` | The user id that is hardcoded in Memgraph and Mage images | `101` | +| `memgraphGroupId` | The group id that is hardcoded in Memgraph and Mage images | `103` | | `storage.libPVCSize` | Size of the storage PVC | `1Gi` | | `storage.libStorageClassName` | The name of the storage class used for storing data. | `""` | | `storage.libStorageAccessMode` | Access mode used for lib storage. | `ReadWriteOnce` | @@ -700,6 +771,10 @@ The following table lists the configurable parameters of the Memgraph HA chart a | `prometheus.memgraphExporter.tag` | The tag of Memgraph's Prometheus exporter image. | `0.2.1` | | `prometheus.serviceMonitor.kubePrometheusStackReleaseName` | The release name under which `kube-prometheus-stack` chart is installed. | `kube-prometheus-stack` | | `prometheus.serviceMonitor.interval` | How often will Prometheus pull data from Memgraph's Prometheus exporter. | `15s` | +| `labels.coordinators.podLabels` | Enables you to set labels on a pod level. | `{}` | +| `labels.coordinators.statefulSetLabels` | Enables you to set labels on a stateful set level. | `{}` | +| `labels.coordinators.serviceLabels` | Enables you to set labels on a service level. | `{}` | +| `updateStrategy.type` | Update strategy for StatefulSets. Possible values are `RollingUpdate` and `OnDelete` | `RollingUpdate` | For the `data` and `coordinators` sections, each item in the list has the following parameters: diff --git a/pages/help-center/errors/auth.mdx b/pages/help-center/errors/auth.mdx index 267972e82..106c38f0c 100644 --- a/pages/help-center/errors/auth.mdx +++ b/pages/help-center/errors/auth.mdx @@ -19,7 +19,8 @@ import {CommunityLinks} from '/components/social-card/CommunityLinks' details, visit: memgr.ph/auth.](#error-2) 6. [Couldn't authenticate user '{}'. For more details, visit: memgr.ph/auth.](#error-1) -7. ["No user nor role '{}' found."](#error-1) +7. [You are not authorized to execute this query on database "memgraph"!](#error-3) +8. [In a multi-tenant environment, SHOW PRIVILEGES query requires database specification. Use ON MAIN, ON CURRENT or ON DATABASE db_name.](#error-4) ## Couldn't authenticate user [#error-1] @@ -28,10 +29,69 @@ A user authentication can fail for many reasons. The user could be missing, the wrong password might be entered, the role defined by the auth module might be missing. -## Operation not permitted when using auth module [#error-4] +## Operation not permitted when using auth module [#error-2] Queries that modify a user's authentication data are forbidden while using an auth module. Users are handled by the module and local users are disabled. +## User doesn't have access to the memgraph database [#error-3] + +This error occurs in multi-tenant environments when a user attempts to execute authentication or authorization queries but doesn't have access to the default "memgraph" database. + +{

Solution

} + +Grant access to the "memgraph" database to the user or their role: + +```cypher +-- Grant access to memgraph database for a user +GRANT DATABASE memgraph TO username; + +-- Grant access to memgraph database for a role +GRANT DATABASE memgraph TO role_name; +``` + +{

Best practice

} + +In multi-tenant environments, we recommend treating the "memgraph" database as an administrative/system database and restricting access to privileged users only. See the [multi-tenancy documentation](/database-management/multi-tenancy#default-database-best-practices) for recommended setup patterns. + +## Database context must be specified for SHOW PRIVILEGES in multi-tenant environment [#error-4] + +This error occurs when attempting to use `SHOW PRIVILEGES` in a multi-tenant environment without specifying the database context. + +**Note**: This error only occurs for `SHOW PRIVILEGES FOR USER` commands. The `SHOW PRIVILEGES FOR ROLE` command does not require database specification and will show all privileges for the role. + +{

Solution

} + +For `SHOW PRIVILEGES FOR ROLE` commands, you can use them without database specification: + +```cypher +-- Show all privileges for a role (works in all environments) +SHOW PRIVILEGES FOR role_name; +``` + +For `SHOW PRIVILEGES FOR USER` commands in multi-tenant environments, you must specify the database context: + +```cypher +-- Show privileges for the user's main database +SHOW PRIVILEGES FOR user_name ON MAIN; + +-- Show privileges for the current database +SHOW PRIVILEGES FOR user_name ON CURRENT; + +-- Show privileges for a specific database +SHOW PRIVILEGES FOR user_name ON DATABASE database_name; +``` + +{

When this occurs

} + +This error typically occurs when: +- Running `SHOW PRIVILEGES FOR USER` in a multi-tenant environment +- The system detects multiple databases and requires explicit context specification for users +- The user is connected to a multi-tenant Memgraph instance + +{

Best practice

} + +- For roles: Use `SHOW PRIVILEGES FOR role_name` without database specification +- For users: Always specify the database context when working in multi-tenant environments to ensure you're viewing the correct privileges for the intended database \ No newline at end of file diff --git a/pages/help-center/faq.mdx b/pages/help-center/faq.mdx index 7e92527cd..943163c43 100644 --- a/pages/help-center/faq.mdx +++ b/pages/help-center/faq.mdx @@ -359,6 +359,10 @@ For comprehensive documentation spanning versions from 1.3.0 to 2.10.1, refer to the [archived GitHub repository](https://github.com/memgraph/docs/tree/master/memgraph_versioned_docs). +### Is Memgraph available on AWS? +Yes, Memgraph is available for access through the [AWS +Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=59fd38c6-06e9-447f-909d-b7c6beedb44c). + ## Cypher ### Are there any differences in Cypher implementation between Memgraph and Neo4j? diff --git a/pages/querying/best-practices.mdx b/pages/querying/best-practices.mdx index 1ba7ce612..077732a95 100644 --- a/pages/querying/best-practices.mdx +++ b/pages/querying/best-practices.mdx @@ -598,9 +598,9 @@ utilized whenever possible to achieve the best performance**. In contrast to other graph databases, Memgraph deep path traversals efficiently handle complex graph queries, as these algorithms have been built into Memgraph's core. This eliminates the need for the overhead of business logic on -the application side. There are four built-in deep path traversal algorithms: -Depth-first search (DFS), Breadth-first search (BFS), Weighted Shortest Path and -All Shortest Paths. +the application side. There are five built-in deep path traversal algorithms: +Depth-first search (DFS), Breadth-first search (BFS), Weighted Shortest Path, +All Shortest Paths, and K Shortest Paths. 2. [String matching](#2-string-matching)
3. [Regular Expressions](#3-regular-expressions) +4. [Existential subqueries](#4-existential-subqueries)
+ 4.1. [Basic EXISTS in WHERE](#41-basic-exists-in-where)
+ 4.2. [Negation with NOT EXISTS](#42-negation-with-not-exists)
+ 4.3. [When to use EXISTS instead of pattern expressions](#43-when-to-use-exists-instead-of-pattern-expressions)
+ 4.4. [RETURN in EXISTS subqueries](#44-return-in-exists-subqueries)
+ 4.5. [EXISTS with UNION](#45-exists-with-union)
+ 4.6. [Outer scope variables and WITH](#46-outer-scope-variables-and-with) ## Dataset @@ -238,6 +245,184 @@ expression grammar. The ECMAScript grammar can be found modifications are described in [this document](https://en.cppreference.com/w/cpp/regex/ecmascript). +## 4. Existential subqueries + +Existential subqueries allow you to use `EXISTS { ... }` within `WHERE` (or as a standalone expression) to test whether a subquery returns at least one row. The subquery can reference variables from the outer scope (correlated subquery), while variables created inside the subquery are not visible outside of it. + +### 4.1. Basic EXISTS in WHERE + +Return people who live in Germany: + +```cypher +MATCH (p:Person) +WHERE EXISTS { + MATCH (p)-[:LIVING_IN]->(:Country {name: 'Germany'}) +} +RETURN p.name +ORDER BY p.name; +``` + +Output: + +```nocopy ++---------+ +| p.name | ++---------+ +| Anna | +| John | ++---------+ +``` + +### 4.2. Negation with NOT EXISTS + +Return people who do not live in the United Kingdom: + +```cypher +MATCH (p:Person) +WHERE NOT EXISTS { + MATCH (p)-[:LIVING_IN]->(:Country {name: 'United Kingdom'}) +} +RETURN p.name +ORDER BY p.name; +``` + +Output: + +```nocopy ++---------+ +| p.name | ++---------+ +| John | ++---------+ +``` + +### 4.3. When to use EXISTS instead of pattern expressions + +Pattern expressions like `exists( (p)-[:FRIENDS_WITH]-() )` are convenient for simple existence checks, but they cannot contain additional clauses such as `WHERE`, `WITH`/aggregation, or multiple pattern parts. `EXISTS { ... }` supports this additional logic. + +For example, return people who have at least two friendships (uses aggregation inside the subquery): + +```cypher +MATCH (p:Person) +WHERE EXISTS { + MATCH (p)-[:FRIENDS_WITH]-(:Person) + WITH count(*) AS friendsCount + WHERE friendsCount >= 2 +} +RETURN p.name +ORDER BY p.name; +``` + +Output: + +```nocopy ++---------+ +| p.name | ++---------+ +| Anna | +| Harry | +| John | ++---------+ +``` + +You can also include property predicates on relationships inside the subquery. For example, people connected by a friendship that started before 2012: + +```cypher +MATCH (p:Person) +WHERE EXISTS { + MATCH (p)-[r:FRIENDS_WITH]-(:Person) + WHERE r.date_of_start < 2012 +} +RETURN p.name +ORDER BY p.name; +``` + +Output: + +```nocopy ++---------+ +| p.name | ++---------+ +| Harry | +| John | ++---------+ +``` + +### 4.4. RETURN in EXISTS subqueries + +EXISTS subqueries do not require a `RETURN` clause. If one is present, it does not need to be aliased (unlike `CALL` subqueries), and any variables returned within the `EXISTS` subquery are not available after the subquery finishes. + +```cypher +MATCH (person:Person) +WHERE EXISTS { + MATCH (person)-[:LIVING_IN]->(country:Country) + RETURN country.name +} +RETURN person.name AS name +``` + +Output: + +```nocopy ++---------+ +| name | ++---------+ +| Anna | +| Harry | +| John | ++---------+ +``` + +### 4.5. EXISTS with UNION + +EXISTS can be used with a `UNION` clause, and the `RETURN` clauses are not required. If one branch has a `RETURN` clause, then all branches require one. If any `UNION` branch returns at least one row, the entire `EXISTS` expression evaluates to `true`. + +```cypher +MATCH (person:Person) +WHERE EXISTS { + MATCH (person)-[:LIVING_IN]->(:Country {name: 'Germany'}) + UNION + MATCH (person)-[:WORKING_IN]->(:Country {name: 'United Kingdom'}) + } +RETURN person.name AS name +``` + +Output: + +```nocopy ++---------+ +| name | ++---------+ +| Anna | +| Harry | ++---------+ +``` + +### 4.6. Outer scope variables and WITH + +Variables from the outside scope are visible for the entire subquery, even when using a `WITH` clause. Shadowing these variables is not allowed. An outside scope variable is shadowed when a newly introduced variable within the inner scope is defined with the same name. The example below shadows the outer variable `countryName` and will therefore throw an error. + +```cypher +WITH 'United Kingdom' as countryName +MATCH (person:Person)-[:LIVING_IN]->(c:Country {name: countryName}) +WHERE EXISTS { + WITH "Germany" AS countryName + MATCH (person)-[:LIVING_IN]->(d:Country) + WHERE d.name = countryName +} +RETURN person.name AS name +``` + +```nocopy ++---------+ +| name | ++---------+ +| Anna | +| John | ++---------+ +``` + + ## Dataset queries We encourage you to try out the examples by yourself. diff --git a/pages/querying/differences-in-cypher-implementations.mdx b/pages/querying/differences-in-cypher-implementations.mdx index 1fc8a200e..b17f92dc9 100644 --- a/pages/querying/differences-in-cypher-implementations.mdx +++ b/pages/querying/differences-in-cypher-implementations.mdx @@ -105,13 +105,33 @@ MATCH p=shortestPath( RETURN p ``` -Memgraph offers fast deep path traversals as [built-in graph algorithms](/advanced-algorithms/deep-path-traversal), including BFS, DFS, WSP and ASP algorithms. That is a bit different from the `shortestPath` and `allShortestPaths` functions you might be used to, but with such algorithms being built in, Memgraph offers fast traversals. Here is an example of how you would rewrite the above query to work in Memgraph: +Memgraph offers fast deep path traversals as [built-in graph algorithms](/advanced-algorithms/deep-path-traversal), including BFS, DFS, WSP, ASP, and KSP algorithms. That is a bit different from the `shortestPath` and `allShortestPaths` functions you might be used to, but with such algorithms being built in, Memgraph offers fast traversals. Here is an example of how you would rewrite the above query to work in Memgraph: ```cypher MATCH p=(:Person {name:"Keanu Reeves"})-[*BFS]-(:Person {name:"Tom Hanks"}) RETURN p ``` +### K shortest paths + +In Neo4j, to find K shortest paths between two nodes, you would use the `SHORTEST` algorithm with a number: + +```cypher +MATCH p = SHORTEST 3 (start:A)-[:E]->(end:B) +RETURN p; +``` + +In Memgraph, you need to first match the source and target nodes, then use the `*KSHORTEST` syntax with a limit: + +```cypher +MATCH (start:A), (end:B) +WITH start, end +MATCH p=(start)-[:E *KSHORTEST | 3]->(end) +RETURN p; +``` + +Note that Memgraph requires both source and target nodes to be matched first using a `WITH` clause before applying the K-shortest paths algorithm. + ### NOT label expression In Neo4j, you can use the `NOT` label expression (`!`): ```cypher @@ -159,16 +179,6 @@ LIMIT 5 ### Unsupported constructs -{

EXISTS subqueries

} -Such clause is not supported in Memgraph, but you can use `exists()` [pattern function](/querying/functions#pattern-functions) with the `WHERE` clause to [filter with pattern expressions](/querying/clauses/where#17-filter-with-pattern-expressions). - -The following constructs are not yet supported in Memgraph: -- **`EXISTS` subquery with `WHERE` clause** -> Track progress on [GitHub](https://github.com/memgraph/memgraph/issues/1483) and add a comment if you require such a feature. -- **Nesting `EXISTS` subqueries** -> Track progress on [GitHub](https://github.com/memgraph/memgraph/issues/905) and add a comment if you require such a feature. -- **`EXISTS` subquery outside of a `WHERE` clause** -> Track progress on [GitHub](https://github.com/memgraph/memgraph/issues/1308) and add a comment if you require such a feature. - -For all other unsupported constructs that you require, please open an issue on our [GitHub repository](https://github.com/memgraph/memgraph/issues). Learn more in our Community Call on [Subqueries & Patterns in Filtering Clauses](https://www.youtube.com/watch?v=QYFU5d_xLIs&list=PL7Eotag2rRhZssS4f11PKAHuCykMCljg3). - {

COUNT subqueries

} Such a construct is not supported in Memgraph, but you can use `count()` [aggregation function](/querying/functions#aggregation-functions) to count the number of non-null values returned by the expression. diff --git a/pages/querying/expressions.mdx b/pages/querying/expressions.mdx index 2dcc90e8c..91e7c0538 100644 --- a/pages/querying/expressions.mdx +++ b/pages/querying/expressions.mdx @@ -202,6 +202,17 @@ predicates. In this case, anything that is not true is interpreted as being fals The [`exists()`](/querying/functions#pattern-functions) function cannot be used with the CASE clause. +## Pattern existence (exists(pattern)) + +For simple pattern existence checks inside `WHERE` using `exists(pattern)`, see +[Filter with pattern expressions](/querying/clauses/where#17-filter-with-pattern-expressions) for dataset-backed examples. + +## Existential subqueries (EXISTS) + +For a comprehensive guide to existential subqueries with `EXISTS` in the `WHERE` clause — including `NOT EXISTS`, +optional `RETURN` inside subqueries, usage with `UNION`, and +`WITH` variable visibility rules — see [Existential subqueries in WHERE](/querying/clauses/where#4-existential-subqueries). + ## Pattern comprehension Pattern comprehension is a syntactic construct available in Cypher for creating a list based on matchings of a pattern. diff --git a/pages/querying/functions.mdx b/pages/querying/functions.mdx index a1ad8c0cb..08cf8e118 100644 --- a/pages/querying/functions.mdx +++ b/pages/querying/functions.mdx @@ -180,4 +180,4 @@ All aggregation functions can be used with the `DISTINCT` operator to perform ca | Name | Signature | Description | | -------------------- | ------------------------------------------------------------------ | --------------------------------------------------- | - | `convert.str2object` | `convert.str2object(string: string) -> (object: any)` | Converts the input string to an object it presents. | + | [`convert.str2object`](/advanced-algorithms/available-algorithms/convert#str2object) | `convert.str2object(string: string) -> (object: any)` | Converts the input string to an object it presents. | diff --git a/pages/querying/text-search.mdx b/pages/querying/text-search.mdx index 386af74fc..e3c3f14e1 100644 --- a/pages/querying/text-search.mdx +++ b/pages/querying/text-search.mdx @@ -13,11 +13,6 @@ Text search is an [experimental feature](/database-management/experimental-features) introduced in Memgraph 2.15.1. To use it, start Memgraph with the `--experimental-enabled=text-search` flag. - -Make sure to start a fresh instance and then load your data. Snapshots and WALs -created with version `vX.Y.Z` without the experimental flag are currently -incompatible with the same version `vX.Y.Z` when the experimental flag is -enabled. Text search allows you to look up nodes with properties that contain specific content. @@ -33,18 +28,42 @@ Text indices and search are powered by the Text indices are created with the `CREATE TEXT INDEX` command. You need to give a name to the new index and specify which labels it should apply to. +### Index all properties + This statement creates a text index named `complianceDocuments` for nodes with -the `Report` label: +the `Report` label, indexing all text-indexable properties: ```cypher CREATE TEXT INDEX complianceDocuments ON :Report; ``` +### Index specific properties + +You can also create a text index on a subset of properties by specifying them explicitly: + +```cypher +CREATE TEXT INDEX index_name ON :Label(prop1, prop2, prop3); +``` + +For example, to create an index only on the `title` and `content` properties of `Report` nodes: + +```cypher +CREATE TEXT INDEX complianceDocuments ON :Report(title, content); +``` + If you attempt to create an index with an existing name, the statement will fail. ### What is indexed -For any given node, if a text index applies to it, all its properties with text-indexable types (`String`, `Integer`, `Float`, or `Boolean`) are stored. +For any given node, if a text index applies to it: +- When no specific properties are listed, all properties with text-indexable types (`String`, `Integer`, `Float`, or `Boolean`) are stored. +- When specific properties are listed, only those properties (if they have text-indexable types) are stored. + + + +Changes made within the same transaction are not visible to the index. To see your changes in text search results, you need to commit the transaction first. + + ## Show text indices @@ -199,20 +218,13 @@ fail. ## Compatibility -Being an experimental feature, text search only supports some usage modalities -that are available in Memgraph. Refer to the table below for an overview: - -| Feature | Support | -|-------------------------|-----------| -| Multitenancy | yes | -| Durability | yes | -| Storage modes | yes (all) | -| Replication | no | -| Concurrent transactions | no | - - - -Disclaimer: For now, text search is not guaranteed to work correctly in use -cases that involve concurrent transactions and replication. +Even though text search is an experimental feature, it supports most usage modalities +that are available in Memgraph from version 3.5. Refer to the table below for an overview: - \ No newline at end of file +| Feature | Support | +|-------------------------|--------------------------------------------- | +| Multitenancy | ✅ Yes | +| Durability | ✅ Yes | +| Storage modes | ❌ No (doesn't work in IN_MEMORY_ANALYTICAL) | +| Replication | ✅ Yes (from version 3.5) | +| Concurrent transactions | ✅ Yes (from version 3.5) | \ No newline at end of file diff --git a/pages/querying/time-to-live.mdx b/pages/querying/time-to-live.mdx index 829767493..e14ed7e1e 100644 --- a/pages/querying/time-to-live.mdx +++ b/pages/querying/time-to-live.mdx @@ -11,6 +11,12 @@ Time-to-live allows a user to tag vertices and edges with an expiration time. On +**Breaking change in v3.5.0**: TTL durability from versions before v3.5.0 is not compatible with the new implementation. If you are upgrading from an earlier version, you will need to reconfigure TTL after the upgrade. + + + + + The `TTL` label and `ttl` property are reserved names for TTL. See [Tagging objects](#tagging-objects) for more info. @@ -32,9 +38,15 @@ Once that is done, a background job will periodically delete expired vertices, i ### What is indexed -Time-to-live uses a label `TTL` and property `ttl` to tag vertices. A label+property value index is used to speed up query execution. +Time-to-live uses a label `TTL` and property `ttl` to tag vertices. A label+property value index is created using concurrent index creation to minimize blocking. Edges are tagged using only the `ttl` property and are scanned using the global edge property index. + + +TTL index creation now uses concurrent index creation. Hence, from v3.5.0, TTL query commands are no longer blocked waiting for index to be created. This significantly reduces the impact on database operation. + + + ### Executed query Time-to-live is implemented as a background job that execute the following queries: @@ -187,6 +199,14 @@ Time-to-live configuration is tenant based; meaning that the feature will need t ### Replication -Time-to-live background job will be execute only on MAIN and the changes will be replicated. -While the TTL effect is replicated, the configuration is not. TTL needs to be configured manually on every instance that can become MAIN. -If an instance is a REPLICA, the TTL background job will be paused until the instance becomes MAIN. +Time-to-live is fully integrated with replication: +- The TTL background job executes only on the MAIN instance +- All TTL deletions are automatically replicated to REPLICA instances +- TTL configuration is now properly replicated across instances +- If an instance is a REPLICA, the TTL background job will be paused until the instance becomes MAIN + + + +Starting from v3.5.0, TTL fully supports replication. The TTL operations are performed at the storage level and are automatically synchronized across all replicas. + + diff --git a/pages/release-notes.mdx b/pages/release-notes.mdx index e14cee6a4..966ea0d46 100644 --- a/pages/release-notes.mdx +++ b/pages/release-notes.mdx @@ -66,6 +66,219 @@ updated. ## 🚀 Latest release +### Memgraph v3.5.0 - August 27th, 2025 + +{

⚠️ Breaking changes

} + +- Time-to-live (TTL) durability from before v3.5.0 is ignored, and you will + have to set up TTL again. + [#3126](https://github.com/memgraph/memgraph/pull/3126) + +{

✨ New features

} + +- Added `USER PROFILE` to achieve per-user resource limits and real-time + monitoring. Admins can now control the number of active sessions, memory + usage per user and monitor resources in real time. + [2980](https://github.com/memgraph/memgraph/pull/2980) +- A new STRICT_SYNC replication mode is added. Its benefits are mostly + observable in high-availability deployments where users can, when registering + replicas as strictly sync, run a high-availability cluster without the fear + of experiencing data loss during failovers. + [#3026](https://github.com/memgraph/memgraph/pull/3026) +- Added Cypher support for existential/exists subqueries. Users can use the + following Cypher construct `... EXISTS { ... subquery ... } ...`. + [#3057](https://github.com/memgraph/memgraph/pull/3057) + [#3120](https://github.com/memgraph/memgraph/pull/3120) +- Added support for multiple roles per user, enabling database-specific + permissions and enhanced SSO integration. Users can now have different roles + for different databases, and SSO providers can assign multiple roles that + will all be recognized by the system. Existing deployments will + automatically migrate. New multi-role features are introduced, e.g., `SHOW + CURRENT ROLE` and enhanced `SET ROLE/CLEAR ROLE` commands. + [3086](https://github.com/memgraph/memgraph/pull/3086) +- Query modules now support timezone-aware datetime objects. This means your + custom procedures and functions can now accept and return datetime values + that include timezone information in either named IANA or numeric offset + formats. [#3121](https://github.com/memgraph/memgraph/pull/3121) + [#3200](https://github.com/memgraph/memgraph/pull/3200) +- Added methods `SHOW INDEXES`, `SHOW CONSTRAINTS`, `SHOW VECTOR INDEXES`, + `SHOW ACTIVE USERS`, `SHOW TRIGGER INFO` for better compatibility and + robustness when operating Memgraph. + [#3149](https://github.com/memgraph/memgraph/pull/3149) +- Added the ability to create a text index on a subset of properties using the + following syntax: `CREATE TEXT INDEX index_name ON :Label(prop1, prop2, + prop3);` [#3155](https://github.com/memgraph/memgraph/pull/3155) +- `CREATE SNAPSHOT` returns the path to the newly created file. The query will + not fail due to no new changes present (snapshot will be forced when the user + requests it). Users can capture the return value and create a more complex + snapshot handling logic on top of Memgraph. + [3166](https://github.com/memgraph/memgraph/pull/3166) +- A new query, `SHOW REPLICATION LAG;` has been added and can be executed on + the cluster's leader. Users can now see how many transactions were committed + on each instance and use this information to decide whether to perform a + failover or not. [#3167](https://github.com/memgraph/memgraph/pull/3167) +- Added [KShortest deep path + traversal](https://memgraph.com/docs/advanced-algorithms/deep-path-traversal) + using the `((n1)-[*KShortest]->(n2))` syntax. The implementation offers + shortest-path search between two nodes with path length filtering, hop + limits, and result limits. Users can now query multiple shortest paths, + returned in order of length, and restrict them by length, hops, or number of + results. Use `WITH n1, n2` to define the endpoints, and apply filters or + result limits directly in the KShortest expression as needed. + [3170](https://github.com/memgraph/memgraph/pull/3170) +- Added `convert_c.to_tree` procedure. The procedure transforms a collection of + `PATH` objects into a structured, nested document - a "tree" of maps - + representing one or more root entities and their connected relationships in + hierarchical form. `to_tree` is equivalent to Neo4j's `apoc.convert.toTree`. + [#3089](https://github.com/memgraph/memgraph/pull/3089) + [#3148](https://github.com/memgraph/memgraph/pull/3148) + [#3158](https://github.com/memgraph/memgraph/pull/3158) + [#3165](https://github.com/memgraph/memgraph/pull/3165) + [#3185](https://github.com/memgraph/memgraph/pull/3185) + +{

🛠️ Improvements

} + +- Several durability-related issues under text search have been fixed, and + replication support has been added. Text search queries now correctly handle + boolean logic and uppercase letters. Users can now safely recover data from + databases. Replication can be used if the text search flag is enabled on both + the main and replica. [#3095](https://github.com/memgraph/memgraph/pull/3095) +- Added missing query module mappings such as `algo`, `convert`, `coll`, + `export`, `math`, `node`, `path`, `refactor`, `text` and `util` under + `/etc/memgraph/apoc_compatibility_mappings.json`. Queries using those APOC + functionalities don't have to be refactored. That's especially useful when + GraphQL middleware/transpiler is used in front of Memgraph. + [#3109](https://github.com/memgraph/memgraph/pull/3109) + [#3128](https://github.com/memgraph/memgraph/pull/3128) + [#3143](https://github.com/memgraph/memgraph/pull/3143) + [#3152](https://github.com/memgraph/memgraph/pull/3152) + [#3159](https://github.com/memgraph/memgraph/pull/3159) + [#3162](https://github.com/memgraph/memgraph/pull/3162) +- Added information about database UUID under `SHOW STORAGE INFO`. + [#3111](https://github.com/memgraph/memgraph/pull/3111) +- Added more logs when authenticating via SSO for an easier debugging + experience. [#3113](https://github.com/memgraph/memgraph/pull/3113) +- `SHOW DATABASE` does not require any special privileges. Users can query + their current database regardless of whether they have `MULTI_DATABASE` + privileges. [3123](https://github.com/memgraph/memgraph/pull/3123) +- Removed the first line showing the next scheduler snapshot from `SHOW + SNAPSHOT;` instead added a special query `SHOW NEXT SNAPSHOT`. Users can now + explicitly query information regarding the next scheduled snapshot. + [3123](https://github.com/memgraph/memgraph/pull/3123) +- General improvements to `oidc.py` auth module. + [3123](https://github.com/memgraph/memgraph/pull/3123) +- Auto-index creation (when enabled) now runs asynchronously in dedicated + background transactions and takes advantage of the new concurrent index + creation. The commit stage is no longer blocked by auto-index creation. This + change improves database performance and reliability. + [#3124](https://github.com/memgraph/memgraph/pull/3124) +- Added time-to-live (TTL) improvements. TTL now supports replication. + Internally, we also now use concurrent index creation, meaning TTL query + commands block for less time. + [#3126](https://github.com/memgraph/memgraph/pull/3126) +- [All shortest paths + traversal](https://memgraph.com/docs/advanced-algorithms/deep-path-traversal#all-shortest-paths) + should be roughly 5% faster. + [#3127](https://github.com/memgraph/memgraph/pull/3127) +- Improved recovery under HA while loading the WAL file after the instance's + restart. The recovery will check whether the given WAL actually contains some + changes before saving the WAL's epoch to the history. Users could benefit + from it since they will see less frequent brute-force recovery of the replica + and more incremental recovery with only the data the replica needs. + [#3141](https://github.com/memgraph/memgraph/pull/3141) +- Pattern comprehension support was added under the `WHERE` clause. It's + possible to run queries like `MATCH (a) WHERE single(x in [(a)-[:R]->(b) + WHERE b IS NOT null | 1] WHERE true) RETURN a;`. It's also possible to nest + pattern comprehension under the `WHERE` clause. + [#3146](https://github.com/memgraph/memgraph/pull/3146) + [#3169](https://github.com/memgraph/memgraph/pull/3169) +- Reduced Docker image size by ~100MB, meaning more hard drive space for cat + memes. [#3151](https://github.com/memgraph/memgraph/pull/3151) +- Improved `date.convert_format` function when `null` value is provided by not + making an exception and returning null instead. + [#3157](https://github.com/memgraph/memgraph/pull/3157) +- The `mgcxx` version has been updated to `0.0.9`, bringing drop index + enhancements and text search performance improvements. For users, this means + faster index operations, and `SHOW INDEX INFO` now returns the number of + entries in a text index instead of null. + [#3171](https://github.com/memgraph/memgraph/pull/3171) +- Make OIDC SSO work without the ID token if it's not needed. Users + authenticating with OIDC SSO via a Neo4j driver don't need to pass an ID + token if they don't use it for the username field. + [#3175](https://github.com/memgraph/memgraph/pull/3175) + +{

🐞 Bug fixes

} + +- The query memory tracker has been moved to custom malloc/free + implementations, allowing more precise tracking. To ensure modules use the + correct implementation, libstd is dynamically linked. Users must ensure they + have the correct libstd version (in the case Memgraph is natively installed). + [3061](https://github.com/memgraph/memgraph/pull/3061) +- [All shortest paths + traversal](https://memgraph.com/docs/advanced-algorithms/deep-path-traversal#all-shortest-paths) + was missing some paths. It correctly returns all results now. + [#3119](https://github.com/memgraph/memgraph/pull/3119) +- Fixed a bug when the query module is called inside another query module. + Query modules can now be safely nested without causing execution errors. + Users can now structure complex logic using nested query modules like `CALL + do.when(...)` without workarounds. + [#3120](https://github.com/memgraph/memgraph/pull/3129) +- Fixed an issue where SSO users were missing from the `SHOW ACTIVE USERS INFO` + results. Admins can now see a complete list of logged-in users. + [3144](https://github.com/memgraph/memgraph/pull/3144) +- Fixed a bug in the `UNWIND` clause shadowing symbols in particular + subqueries. [#3154](https://github.com/memgraph/memgraph/pull/3154) +- Fixed a concurrency issue in text index updates. From the user's perspective, + text index operations will now run more reliably under concurrent workloads. + [#3177](https://github.com/memgraph/memgraph/pull/3177) +- Changing your own password does not require any particular privilege. Users + can now change their own passwords even if they do not have the AUTH + privilege. [3192](https://github.com/memgraph/memgraph/pull/3192) +- Resolved segmentation fault in `AllShortestPathCursor` due to incorrect + deletion of the last element from the vector (internal implementation + detail). Users should no longer encounter segfaults when executing all + shortest path queries. + [#3202](https://github.com/memgraph/memgraph/pull/3202) +- `MonotonicBufferResource` allocate method now checks for possible overflow. + Users can expect more accurate memory measurements. + [#3204](https://github.com/memgraph/memgraph/pull/3204) + +### MAGE v3.5.0 - August 27th, 2025 + +{

✨ New features

} + +- Added `collections_module.flatten` function for flattening nested + collections. Example usage: `RETURN collections_module.flatten([[1, 2], [3, + 4], 5]);`. [#624](https://github.com/memgraph/mage/pull/624) +- Added `text.indexOf` function. An example of usage is `RETURN + text.indexOf("string", "what", from, to);`. + [#645](https://github.com/memgraph/mage/pull/645) + [#648](https://github.com/memgraph/mage/pull/648) +- Added `math.round` function for rounding numbers with various modes. + [#652](https://github.com/memgraph/mage/pull/652) + +{

🛠️ Improvements

} + +- Added normalization of UTF8 characters in the `text.distance` module, making + the module work for inputs that are UTF8 strings. + [#640](https://github.com/memgraph/mage/pull/640) +- Added optimization for `text.regreplace` by caching the regex pattern, + resulting in faster execution. + [#641](https://github.com/memgraph/mage/pull/641) +- Improved the case when null is passed to the `json_util.from_json_list` + function by just returning null in that case. + [#655](https://github.com/memgraph/mage/pull/655) + +{

🐞 Bug fixes

} + +- Fixed handling the specific MySQL types (Blob, Geometry, etc.) so the + migration does not fail with the `Unsupported PyObject conversion`. + [#656](https://github.com/memgraph/mage/pull/656) + +### Lab v3.5.0 - August 27th, 2025 + +## Previous releases + ### Memgraph v3.4.0 - July 10th, 2025 {

⚠️ Breaking changes

} @@ -152,8 +365,6 @@ updated. -## Previous releases - ### Memgraph v3.3.0 - June 4th, 2025 {

✨ New features

}