diff --git a/docs/clusters/aws/logs.md b/docs/clusters/aws/logs.md index 3b80eab02b..2c01a6d87d 100644 --- a/docs/clusters/aws/logs.md +++ b/docs/clusters/aws/logs.md @@ -35,3 +35,7 @@ fields @timestamp, log ``` Please make sure to select the log group for your cluster and adjust the time range accordingly before running the queries. + +## Structured logging + +You can use Cortex's logger in your Python code to log in JSON, which will enrich your logs with Cortex's metadata, and enable you to add custom metadata to the logs. See the structured logging docs for [Realtime](../../workloads/realtime/predictors.md#structured-logging), [Batch](../../workloads/batch/predictors.md#structured-logging), and [Task](../../workloads/task/definitions.md#structured-logging) APIs. diff --git a/docs/clusters/gcp/logs.md b/docs/clusters/gcp/logs.md index bf772d5127..065b2bdd0f 100644 --- a/docs/clusters/gcp/logs.md +++ b/docs/clusters/gcp/logs.md @@ -22,3 +22,7 @@ jsonPayload.labels.jobID="" ``` Please make sure to navigate to the project containing your cluster and adjust the time range accordingly before running queries. + +## Structured logging + +You can use Cortex's logger in your Python code to log in JSON, which will enrich your logs with Cortex's metadata, and enable you to add custom metadata to the logs. See the structured logging docs for [Realtime](../../workloads/realtime/predictors.md#structured-logging) and [Task](../../workloads/task/definitions.md#structured-logging) APIs. diff --git a/docs/workloads/batch/predictors.md b/docs/workloads/batch/predictors.md index 1afa8a4c9f..22b8527a1f 100644 --- a/docs/workloads/batch/predictors.md +++ b/docs/workloads/batch/predictors.md @@ -207,3 +207,24 @@ class ONNXPredictor: Cortex provides an `onnx_client` to your Predictor's constructor. `onnx_client` is an instance of [ONNXClient](https://github.com/cortexlabs/cortex/tree/master/pkg/cortex/serve/cortex_internal/lib/client/onnx.py) that manages an ONNX Runtime session to make predictions using your model. It should be saved as an instance variable in your Predictor, and your `predict()` function should call `onnx_client.predict()` to make an inference with your exported ONNX model. Preprocessing of the JSON payload and postprocessing of predictions can be implemented in your `predict()` function as well. When multiple models are defined using the Predictor's `models` field, the `onnx_client.predict()` method expects a second argument `model_name` which must hold the name of the model that you want to use for inference (for example: `self.client.predict(model_input, "text-generator")`). + +## Structured logging + +You can use Cortex's logger in your predictor implemention to log in JSON. This will enrich your logs with Cortex's metadata, and you can add custom metadata to the logs by adding key value pairs to the `extra` key when using the logger. For example: + +```python +... +from cortex_internal.lib.log import logger as cortex_logger + +class PythonPredictor: + def predict(self, payload, batch_id): + cortex_logger.info("completed processing batch", extra={"batch_id": batch_id, "confidence": confidence}) +``` + +The dictionary passed in via the `extra` will be flattened by one level. e.g. + +```text +{"asctime": "2021-01-19 15:14:05,291", "levelname": "INFO", "message": "completed processing batch", "process": 235, "batch_id": "iuasyd8f7", "confidence": 0.97} +``` + +To avoid overriding essential Cortex metadata, please refrain from specifying the following extra keys: `asctime`, `levelname`, `message`, `labels`, and `process`. Log lines greater than 5 MB in size will be ignored. diff --git a/docs/workloads/realtime/predictors.md b/docs/workloads/realtime/predictors.md index 240cc9149d..66f3623e2b 100644 --- a/docs/workloads/realtime/predictors.md +++ b/docs/workloads/realtime/predictors.md @@ -455,3 +455,24 @@ class PythonPredictor: ``` Note that the autoscaling configuration (i.e. `target_replica_concurrency`) for the API that is making the request should be modified with the understanding that requests will still be considered "in-flight" with the first API as the request is being fulfilled in the second API (during which it will also be considered "in-flight" with the second API). + +## Structured logging + +You can use Cortex's logger in your predictor implemention to log in JSON. This will enrich your logs with Cortex's metadata, and you can add custom metadata to the logs by adding key value pairs to the `extra` key when using the logger. For example: + +```python +... +from cortex_internal.lib.log import logger as cortex_logger + +class PythonPredictor: + def predict(self, payload): + cortex_logger.info("received payload", extra={"payload": payload}) +``` + +The dictionary passed in via the `extra` will be flattened by one level. e.g. + +```text +{"asctime": "2021-01-19 15:14:05,291", "levelname": "INFO", "message": "received payload", "process": 235, "payload": "this movie is awesome"} +``` + +To avoid overriding essential Cortex metadata, please refrain from specifying the following extra keys: `asctime`, `levelname`, `message`, `labels`, and `process`. Log lines greater than 5 MB in size will be ignored. diff --git a/docs/workloads/task/definitions.md b/docs/workloads/task/definitions.md index 486f3cd1dd..78eb3c55de 100644 --- a/docs/workloads/task/definitions.md +++ b/docs/workloads/task/definitions.md @@ -51,3 +51,25 @@ class Task: """ pass ``` + +## Structured logging + +You can use Cortex's logger in your predictor implemention to log in JSON. This will enrich your logs with Cortex's metadata, and you can add custom metadata to the logs by adding key value pairs to the `extra` key when using the logger. For example: + +```python +... +from cortex_internal.lib.log import logger as cortex_logger + +class Task: + def __call__(self, config): + ... + cortex_logger.info("completed validations", extra={"accuracy": accuracy}) +``` + +The dictionary passed in via the `extra` will be flattened by one level. e.g. + +```text +{"asctime": "2021-01-19 15:14:05,291", "levelname": "INFO", "message": "completed validations", "process": 235, "accuracy": 0.97} +``` + +To avoid overriding essential Cortex metadata, please refrain from specifying the following extra keys: `asctime`, `levelname`, `message`, `labels`, and `process`. Log lines greater than 5 MB in size will be ignored.