System package docs (#250)

Author: vishalbollu

docs/apis/python-packages.md

@@ -1,6 +1,6 @@
 # Python Packages
 
-Cortex allows you to install additional Python packages to be used in any workload.
+Cortex allows you to install additional Python packages to be used in any workload. If your Python packages require system packages to be installed, see [system packages](system-packages.md) for more details.
 
 ## PyPI Packages
 

docs/apis/system-packages.md

@@ -0,0 +1,75 @@
+# System Packages
+
+Cortex uses Docker images to run workloads. These Docker images can be replaced with custom images based on Cortex images and augmented with your system packages and libraries. Your custom images need to be pushed to a container registry (e.g. Docker Hub, ECR, GCR) that can be accessed by your cluster.
+
+See `Image paths` section in [cortex config](../cluster/config.md) for all images that can be customized.
+
+The example below demonstrates how to create a custom Docker image and configure Cortex to use it.
+
+## Create Custom Image
+
+Create a Dockerfile to build your custom image:
+
+```
+mkdir my-api && cd my-api && touch Dockerfile
+```
+
+Specify the base image you want to override followed by your customizations. The sample Dockerfile below inherits from Cortex's ONNX Serving image and installs the `tree` system package.
+
+```
+# Dockerfile
+
+FROM cortexlabs/onnx-serve
+
+RUN apt-get update \
+    && apt-get install -y tree \
+    && apt-get clean && rm -rf /var/lib/apt/lists/*
+```
+
+## Build and Push to a Container Registry
+
+Create a repository to store your image:
+
+```
+# We create a repository in AWS ECR
+
+export AWS_ACCESS_KEY_ID="***"
+export AWS_SECRET_ACCESS_KEY="***"
+
+eval $(aws ecr get-login --no-include-email --region us-west-2)
+
+aws ecr create-repository --repository-name=org/my-api --region=us-west-2
+# take note of repository url
+```
+
+Build the image based on your Dockerfile and push to its repository in AWS ECR:
+
+```
+docker build . -t org/my-api:latest -t <repository_url>:latest
+
+docker push <repository_url>:latest
+```
+
+## Configure Cortex
+
+Set the environment variable of the image to your `my-api` image repository url:
+
+```
+export CORTEX_IMAGE_ONNX_SERVE="<repository_url>:latest"
+
+./cortex.sh update
+```
+
+## Use System Package in Workloads
+
+Cortex will use your image to launch ONNX serving workloads. You will have access to any customizations you made:
+
+```
+# request_handler.py
+
+import subprocess
+
+def pre_inference(sample, metadata):
+    subprocess.run(["tree"])
+    ...
+```

docs/summary.md

@@ -13,6 +13,7 @@
 * [Request Handlers](apis/request-handlers.md)
 * [Compute](apis/compute.md)
 * [Python Packages](apis/python-packages.md)
+* [System Packages](apis/system-packages.md)
 * [CLI Commands](cluster/cli.md)
 * [Resource Statuses](apis/statuses.md)