feature: add open telemetry integration#3
Conversation
1a9f857 to
fff33eb
Compare
caseylocker
left a comment
There was a problem hiding this comment.
You're defining randomString in api/tests/open_telemetry_instrumentation_tests.py but not using it:
class OpenTelemetryInstrumentationTest(APITestCase):
def randomString(self, str_len):
...
Wrapt was downgraded in the requirements.txt. Was that intentional?
wrapt==2.0.1 -> wrapt==1.17.3
@caseylocker I removed this code you are right, its not been used, this was to generate a random string for the access_token we use to authenticate the requests on the other endpoints but here that does not happen
@caseylocker it was on propuse because newer versions create dependency conflicts when we try to install the open telemetry instrumentation libraries |
caseylocker
left a comment
There was a problem hiding this comment.
LGTM. I've confirmed that the wrapt downgrade was a requirement by the Opentelementry instrumentation packages. They explicitly require wrapt<2.0.0. No other packages require wrapt 2.x
smarcet
left a comment
There was a problem hiding this comment.
@fcarreno-collab please review comments
|
@smarcet please can you review this. |
backend/settings.py
Outdated
|
|
||
| DEV_EMAIL = os.getenv('DEV_EMAIL') | ||
|
|
||
| from backend.env_var_eval import env_bool |
There was a problem hiding this comment.
@fcarreno-collab this import should be moved to the begging of the file
| # Exported spans should exist | ||
| spans = memory_exporter.get_finished_spans() | ||
| self.assertEqual(spans[0].resource.attributes.get('service.name'), 'marketing-api') | ||
| self.assertEqual(len(spans), 6) |
There was a problem hiding this comment.
@fcarreno-collab
The test doesn't care about how many spans exist , it cares that the Django HTTP span has the right attributes. So instead of indexing by position spans[5], find the span by name:
spans = memory_exporter.get_finished_spans()
self.assertGreaterEqual(len(spans), 1)
# Find the top-level Django HTTP span by name pattern
http_spans = [s for s in spans if s.name.startswith("GET ")]
self.assertEqual(len(http_spans), 1)
exported_span = http_spans[0]
# Now assert the actual behavior under test
self.assertEqual(exported_span.attributes.get("cf.ray_id"), "abc123")
This way the test validates the hook behavior (CF-Ray attribute is set, traceparent propagation works) without coupling to the exact internal span count, which is an implementation detail of the
instrumentation libraries and the request pipeline.
|
|
||
| # Verify a span was exported | ||
| spans = memory_exporter.get_finished_spans() | ||
| self.assertEqual(len(spans), 6) |
There was a problem hiding this comment.
| # And should also show up in span attributes (if your request_hook adds it) | ||
| self.assertEqual(exported_span.attributes.get("baggage.cf.ray_id"), "xyz") | ||
|
|
||
| def test_mysql_span_has_db_name(self): |
There was a problem hiding this comment.
@fcarreno-collab
The tests test_mysql_span_has_db_name and test_redis_span_has_key in OpenTelemetryInstrumentationTest (api/tests/open_telemetry_instrumentation_tests.py:138–170) don't actually call the hook
methods they're supposed to test.
the test creates a span, manually sets attributes on it, then reads those same attributes back and asserts they match. It's a round-trip test of the OTEL SDK itself, not of
DjangoTelemetry.mysql_hook.
The mysql_hook has actual logic worth testing:
- It reads DB_NAME from the environment (os.getenv('DB_NAME', 'db'))
- It adds db.system = "mysql" (not present in the test at all)
- It guards with span.is_recording()
- It swallows exceptions silently
None of that is exercised. The test would pass even if mysql_hook didn't exist.
smarcet
left a comment
There was a problem hiding this comment.
@fcarreno-collab please review
feature: move settings import and update tests
36aa1c2 to
5ed2630
Compare
|
@smarcet changes related your comments already on PR |
| urllib3==2.5.0 | ||
| Werkzeug==3.1.4 | ||
| wrapt==2.0.1 | ||
| wrapt==1.17.3 |
There was a problem hiding this comment.
@fcarreno-collab why this deps was downgraded ?
wrapt==2.0.1
There was a problem hiding this comment.
@smarcet the dependency downgrade was on propuse because newer versions create dependency conflicts when we try to install the open telemetry instrumentation libraries, I explain this to @caseylocker and he confirm it
backend/otel_instrumentation.py
Outdated
|
|
||
| @staticmethod | ||
| def response_hook(span, request, response): | ||
| if span.is_recording() and hasattr(response, "content"): |
There was a problem hiding this comment.
@fcarreno-collab please follow the same pattern as other hooks here
if not span.is_recording():
return
try:
if hasattr(response, "content"):
span.set_attribute("http.response.length", len(response.content))
except Exception:
pass
backend/otel_instrumentation.py
Outdated
| return | ||
|
|
||
| # Attach CF-Ray header | ||
| span.set_attribute("cf.ray_id", request.headers.get("Cf-Ray", "")) |
There was a problem hiding this comment.
@fcarreno-collab
When there's no Cf-Ray header, this sets cf.ray_id = "" on every span, adding noise. Should skip the attribute entirely
cf_ray = request.headers.get("Cf-Ray")
if cf_ray:
span.set_attribute("cf.ray_id", cf_ray)
smarcet
left a comment
There was a problem hiding this comment.
@fcarreno-collab please review comments
|
@smarcet branch updated with requested changes |
ref: https://app.clickup.com/t/86b7bdn9g