Django app and MySQLInstrumentor not working

[DeWaRs1206]

Discussed in open-telemetry/opentelemetry-python#3082

^{Originally posted by DeWaRs1206 December 9, 2022}
Hello,

I'm not able to collect MySQL trace using MySQLInstrumentor on a django App.
I created a very simple sample app here: https://github.com/DeWaRs1206/django-opentelemetry to reproduce.
Basically DjangoInstrumentor().instrument() is showing data in the console, but MySQLInstrumentor().instrument() does not.

Did I missed something ?

Thanks for your help.

DeWaRs

[nabadger]

@DeWaRs1206 I think I hit the same problem which was a result of not using the python impl. of mysql connector.

https://github.com/DeWaRs1206/django-opentelemetry/blob/master/blog/settings.py#L78

Try setting ENGINE to mysql.connector.django

https://dev.mysql.com/doc/connector-python/en/connector-python-django-backend.html

This relies on the mysql-connector-python py-package which you have already by the looks of it.

More info here:https://docs.djangoproject.com/en/4.1/ref/databases/#mysql-db-api-drivers

Note, I would imagine switching to the native python impl. would have a performance hit....

---

I don't know if I've missed any docs/examples from open-telemetry...but this wasn't obvious to me and took a while to figure out :)

[nabadger]

After further reading..is it that MySQLInstrumentor is specific to the python mysql connector?

The original setup was using the underlying mysqlclient...the following might work instead, and would I guess avoid any performance hit from using the native python-db impl.

import MySQLdb
from opentelemetry.instrumentation.dbapi import trace_integration

trace_integration(MySQLdb, "connect", "mysql")

[DeWaRs1206]

Hello @nabadger,

Thanks for the details.
I tried in the past

import MySQLdb
from opentelemetry.instrumentation.dbapi import trace_integration

trace_integration(MySQLdb, "connect", "mysql")

But after loading my application, Django raise an OperationalError

Traceback (most recent call last):
  File "/virtualenv/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/virtualenv/lib/python3.7/site-packages/django/utils/deprecation.py", line 93, in __call__
    response = self.process_request(request)
  File "/virtualenv/lib/python3.7/site-packages/cms/middleware/utils.py", line 14, in process_request
    apphook_reload.ensure_urlconf_is_up_to_date()
  File "/virtualenv/lib/python3.7/site-packages/cms/utils/apphook_reload.py", line 25, in ensure_urlconf_is_up_to_date
    global_revision = get_global_revision()
  File "/virtualenv/lib/python3.7/site-packages/cms/utils/apphook_reload.py", line 68, in get_global_revision
    revision=str(uuid.uuid4()))
  File "/virtualenv/lib/python3.7/site-packages/cms/models/apphooks_reload.py", line 31, in get_or_create_revision
    pk=1, defaults=dict(revision=revision))
  File "/virtualenv/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/virtualenv/lib/python3.7/site-packages/django/db/models/query.py", line 538, in get_or_create
    return self.get(**kwargs), False
  File "/virtualenv/lib/python3.7/site-packages/django/db/models/query.py", line 402, in get
    num = len(clone)
  File "/virtualenv/lib/python3.7/site-packages/django/db/models/query.py", line 256, in __len__
    self._fetch_all()
  File "/virtualenv/lib/python3.7/site-packages/django/db/models/query.py", line 1242, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/virtualenv/lib/python3.7/site-packages/django/db/models/query.py", line 55, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/virtualenv/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1140, in execute_sql
    cursor = self.connection.cursor()
  File "/virtualenv/lib/python3.7/site-packages/django/db/backends/base/base.py", line 256, in cursor
    return self._cursor()
  File "/virtualenv/lib/python3.7/site-packages/django/db/backends/base/base.py", line 233, in _cursor
    self.ensure_connection()
  File "/virtualenv/lib/python3.7/site-packages/django/db/backends/base/base.py", line 217, in ensure_connection
    self.connect()
  File "/virtualenv/lib/python3.7/site-packages/django/db/backends/base/base.py", line 196, in connect
    self.set_autocommit(self.settings_dict['AUTOCOMMIT'])
  File "/virtualenv/lib/python3.7/site-packages/django/db/backends/base/base.py", line 405, in set_autocommit
    self._set_autocommit(autocommit)
  File "/virtualenv/lib/python3.7/site-packages/django/db/backends/mysql/base.py", line 257, in _set_autocommit
    self.connection.autocommit(autocommit)
  File "/virtualenv/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/virtualenv/lib/python3.7/site-packages/django/db/backends/mysql/base.py", line 257, in _set_autocommit
    self.connection.autocommit(autocommit)
  File "/virtualenv/lib/python3.7/site-packages/MySQLdb/connections.py", line 210, in autocommit
    if self.get_autocommit() != on:
django.db.utils.OperationalError: (2006, '')

Collection seems to work before the crash ... I wasn't able to find a solution for this ...

[nabadger]

@DeWaRs1206 I have the same issue - looks like it's #1319

The associated MR has recent activty, so fingers crossed :)

[DeWaRs1206]

@nabadger thanks for the update. I applied the proposed patch on my DEV instance, things are working better but I have a lot of DB errors now. Lock timeout, deadlock, which is pretty weird...