0.1.1 release preparation - cherry-pick #174
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In this commit, we are handling issues that arise from gRPC. Essentially, if we build gRPC artifacts into our Docker image, it causes the Docker image to only be compatible with applications built using the same Python version. To solve this, we are doing two things: 1) we are removing gRPC artifacts from the docker image and 2) we are changing the default OTLP protocol to be HTTP. If customers attempt to set the protocol as gRPC for ApplicationSignals, we will set the default endpoint correctly. Also we are changing Docker image to build with Python 3.11, which is what we were originally doing when we encountered this issue (reference: 5b3ed74). This is what the upstream does (see [autoinstrumentation/python/Dockerfile](https://github.com/open-telemetry/opentelemetry-operator/blob/b5bb0ae34720d4be2d229dafecb87b61b37699b0/autoinstrumentation/python/Dockerfile)), and having parity here is beneficial to us. Testing: * Create `app.py`: ``` from time import sleep import boto3 try: boto3.client('s3').list_buckets() except Exception: sleep(100) ``` * Run `./scripts/build_and_install_distro.sh` * Run: ``` export OTEL_PYTHON_DISTRO="aws_distro" export OTEL_PYTHON_CONFIGURATOR="aws_configurator" export OTEL_METRICS_EXPORTER="none" unset OTEL_EXPORTER_OTLP_PROTOCOL unset OTEL_AWS_APPLICATION_SIGNALS_ENABLED ``` * Run `opentelemetry-instrument python ./app.py` ``` urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4318): Max retries exceeded with url: /v1/traces (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc647cd5e50>: Failed to establish a new connection: [Errno 111] Connection refused')) ``` * Run `export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf; opentelemetry-instrument python ./app.py` ``` requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4318): Max retries exceeded with url: /v1/traces (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f20ac96e490>: Failed to establish a new connection: [Errno 111] Connection refused')) ``` * Run `export OTEL_EXPORTER_OTLP_PROTOCOL=grpc; opentelemetry-instrument python ./app.py` ``` Transient error StatusCode.UNAVAILABLE encountered while exporting traces to localhost:4317, retrying in 1s. ``` * Run ``` unset OTEL_EXPORTER_OTLP_PROTOCOL export OTEL_METRIC_EXPORT_INTERVAL=1000 export OTEL_AWS_APPLICATION_SIGNALS_ENABLED=True ``` * Run `opentelemetry-instrument python ./app.py` ``` urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4316): Max retries exceeded with url: /v1/metrics (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8f6c5aa5b0>: Failed to establish a new connection: [Errno 111] Connection refused')) urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4318): Max retries exceeded with url: /v1/traces (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8f6dd9c9d0>: Failed to establish a new connection: [Errno 111] Connection refused')) ``` * Run `export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf; opentelemetry-instrument python ./app.py` ``` urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4318): Max retries exceeded with url: /v1/traces (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff5868efdc0>: Failed to establish a new connection: [Errno 111] Connection refused')) requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4318): Max retries exceeded with url: /v1/traces (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff5868efdc0>: Failed to establish a new connection: [Errno 111] Connection refused')) ``` * Run `export OTEL_EXPORTER_OTLP_PROTOCOL=grpc; opentelemetry-instrument python ./app.py` ``` Transient error StatusCode.UNAVAILABLE encountered while exporting metrics to localhost:4315, retrying in 1s. Transient error StatusCode.UNAVAILABLE encountered while exporting traces to localhost:4317, retrying in 1s. ``` By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
We are installing using `pip install --target`. To fix this we must set PYTHONPATH so pip can find and uninstall these packages. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
The previous fix (#172) was not working as deleting the files after installation caused dangling dependencies - the instrumentation expected a dependency, but it was not found, causing instrumentation to fail. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
srprash
approved these changes
Apr 22, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In this PR, we are bringing in patches to resolve an issue introduced by depending on OTLP gRPC protocol. Please see #170 for more details.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.