Python "packaging" is required

[moserke]

When trying to run instrumentation for a python lambda, I get the following error. It implies I need to install the packaging package into my lambda, but why is this not in the otel layer if it is indeed required? What is leveraging this? Or am I missing a configuration? I was following https://opentelemetry.io/docs/faas/lambda-auto-instrument/

Instrumenting of sqlalchemy failed
Traceback (most recent call last):
File "/opt/python/opentelemetry/instrumentation/auto_instrumentation/_load.py", line 87, in _load_instrumentors
distro.load_instrumentor(entry_point, skip_dep_check=True)
File "/opt/python/opentelemetry/instrumentation/distro.py", line 62, in load_instrumentor
instrumentor: BaseInstrumentor = entry_point.load()
^^^^^^^^^^^^^^^^^^
File "/var/task/pkg_resources/__init__.py", line 2470, in load
self.require(*args, **kwargs)
File "/var/task/pkg_resources/__init__.py", line 2493, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/task/pkg_resources/__init__.py", line 795, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'packaging>=21.0' distribution was not found and is required by the application
Failed to auto initialize opentelemetry
Traceback (most recent call last):
File "/opt/python/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py", line 39, in initialize
_load_instrumentors(distro)
File "/opt/python/opentelemetry/instrumentation/auto_instrumentation/_load.py", line 91, in _load_instrumentors
raise exc
File "/opt/python/opentelemetry/instrumentation/auto_instrumentation/_load.py", line 87, in _load_instrumentors
distro.load_instrumentor(entry_point, skip_dep_check=True)
File "/opt/python/opentelemetry/instrumentation/distro.py", line 62, in load_instrumentor
instrumentor: BaseInstrumentor = entry_point.load()
^^^^^^^^^^^^^^^^^^
File "/var/task/pkg_resources/__init__.py", line 2470, in load
self.require(*args, **kwargs)
File "/var/task/pkg_resources/__init__.py", line 2493, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/task/pkg_resources/__init__.py", line 795, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'packaging>=21.0' distribution was not found and is required by the application

[rapphil]

What version of the layer are you using?

[rapphil]

I think this is happening because we install the instrumentation into the layer with the --no-deps option in pip. Some instrumentations require non-opentelemetry dependencies to be present. In your case https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/instrumentation/opentelemetry-instrumentation-sqlalchemy/pyproject.toml#L31

I don't have much context about the decision of using --no-deps, but I think we can start shipping the layers with all the dependencies included. We just need to make sure that all the instrumentation that we include is not pining a specific version of the dependencies.

[moserke]

Sorry yeah version 0.5.0 through 0.8.0 I tested with. I definitely could see both sides of the argument of trying to keep the layer slim without all of the does but it definitely makes it more whack a mole for the user to find all of the deps to put in the function layer when it’s supposed to be auto instrumentation.

thanks for the pointer though as that will help me track down the needed deps for now!

[moserke]

FWIW just did a local creation of the layer with and without the --no-deps and the only package that was added was packaging. Uploaded that layer into my account and worked.