feat: add System Status page with Celery queue monitoring and purge#14349
Open
valentijnscholten wants to merge 2 commits intoDefectDojo:devfrom
Open
feat: add System Status page with Celery queue monitoring and purge#14349valentijnscholten wants to merge 2 commits intoDefectDojo:devfrom
valentijnscholten wants to merge 2 commits intoDefectDojo:devfrom
Conversation
- Add dedicated System Status page (/system_status) with superuser-only
access, accessible from the navigation menu alongside System Settings
- Display Celery worker liveness, pending queue length with human-readable
duration formatting, and active task timeout/expiry configuration
- Add Purge queue button that POSTs to the new API endpoint and reloads
the page on success
- Fix get_celery_worker_status() to use app.control.ping() via the
pidbox control channel, which works correctly even when the task queue
is clogged (previously dispatched a task that would never be picked up)
- Add purge_celery_queue() utility using a direct broker connection
- Add two new superuser-only REST API endpoints:
GET /api/v2/celery/status/ - worker status, queue length, config
POST /api/v2/celery/queue/purge/ - purge all pending tasks
Both use the same permission guards as SystemSettingsViewSet
(IsSuperUser + DjangoModelPermissions against System_Settings)
- Add DD_CELERY_TASK_TIME_LIMIT (default 12h), DD_CELERY_TASK_SOFT_TIME_LIMIT
(default disabled), and DD_CELERY_TASK_DEFAULT_EXPIRES (default 12h)
environment variables to settings.dist.py with explanatory comments
- Move celery status rendering from server-side Django view to client-side
AJAX so dojo-pro can consume the same API endpoints
valentijnscholten
requested review from
Maffooch and
mtesauro
as code owners
github-actions
bot
added
settings_changes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/system_status) accessible to superusers from the navigation menu, alongside System Settings. Displays Celery worker liveness, pending queue length, and active task timeout/expiry settings.python manage.py dedupe). Reloads the page after a successful purge.SystemSettingsViewSet):GET /api/v2/celery/status/— worker status, queue length, and task configPOST /api/v2/celery/queue/purge/— purge all pending tasks from the broker queueget_celery_worker_status()to useapp.control.ping()via Celery's pidbox control channel instead of dispatching a task. The old approach would hang indefinitely when the task queue was clogged.settings.dist.py:DD_CELERY_TASK_TIME_LIMIT(default: 43200 = 12 h) — hard kill via SIGKILL after this many secondsDD_CELERY_TASK_SOFT_TIME_LIMIT(default: disabled) — raisesSoftTimeLimitExceededfor graceful cleanupDD_CELERY_TASK_DEFAULT_EXPIRES(default: 43200 = 12 h) — silently discard tasks that have been waiting in the queue longer than this before any worker picks them up