Recently, we upgraded to Django 4.2.16, then one of our Django servers crashed with the below error:
gunicorn[1014232]: [18/Sep/2024 10:58:11][common.services.pandora_service._user_access_token:187][Root=1-66ea9623-5aee4eae0266bcd41a3a1159][DEBUG] Token retrieved in : 9.248 ms
gunicorn[1182738]: [2024-09-18 08:58:11 +0000] [1182738] [ERROR] Exception in worker process
gunicorn[1182738]: Traceback (most recent call last):
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 609, in spawn_worker
gunicorn[1182738]: worker.init_process()
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/workers/ggevent.py", line 147, in init_process
gunicorn[1182738]: super().init_process()
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 134, in init_process
gunicorn[1182738]: self.load_wsgi()
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
gunicorn[1182738]: self.wsgi = self.app.wsgi()
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
gunicorn[1182738]: self.callable = self.load()
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
gunicorn[1182738]: return self.load_wsgiapp()
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
gunicorn[1182738]: return util.import_app(self.app_uri)
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/util.py", line 371, in import_app
gunicorn[1182738]: mod = importlib.import_module(module)
gunicorn[1182738]: File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
gunicorn[1182738]: return _bootstrap._gcd_import(name[level:], package, level)
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 991, in _find_and_load
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 991, in _find_and_load
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
gunicorn[1182738]: File "<frozen importlib._bootstrap_external>", line 848, in exec_module
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
gunicorn[1182738]: File "/opt/cloud/current/pix4uav/pix4d/__init__.py", line 3, in <module>
gunicorn[1182738]: from .celery import app as celery_app # noqa
gunicorn[1182738]: File "/opt/cloud/current/pix4uav/pix4d/celery.py", line 16, in <module>
gunicorn[1182738]: from .settings.sentry import before_send
gunicorn[1182738]: File "/opt/cloud/current/pix4uav/pix4d/settings/sentry.py", line 5, in <module>
gunicorn[1182738]: from sentry_sdk.integrations.django import DjangoIntegration
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/sentry_sdk/integrations/django/__init__.py", line 61, in <module>
gunicorn[1182738]: from sentry_sdk.integrations.django.middleware import patch_django_middlewares
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/sentry_sdk/integrations/django/middleware.py", line 38, in <module>
gunicorn[1182738]: from .asgi import _asgi_middleware_mixin_factory
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/sentry_sdk/integrations/django/asgi.py", line 13, in <module>
gunicorn[1182738]: from django.core.handlers.wsgi import WSGIRequest
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/django/core/handlers/wsgi.py", line 5, in <module>
gunicorn[1182738]: from django.core.handlers import base
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 12, in <module>
gunicorn[1182738]: from django.utils.log import log_response
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/django/utils/log.py", line 6, in <module>
gunicorn[1182738]: from django.core import mail
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/django/core/mail/__init__.py", line 10, in <module>
gunicorn[1182738]: from django.core.mail.message import (
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/django/core/mail/message.py", line 4, in <module>
gunicorn[1182738]: from email import generator, message_from_string
gunicorn[1182738]: File "/usr/lib/python3.8/email/generator.py", line 17, in <module>
gunicorn[1182738]: from email.errors import HeaderWriteError
gunicorn[1182738]: ImportError: cannot import name 'HeaderWriteError' from 'email.errors' (/usr/lib/python3.8/email/errors.py)
gunicorn[1182738]: [2024-09-18 08:58:11 +0000] [1182738] [INFO] Worker exiting (pid: 1182738)
gunicorn[1014225]: [2024-09-18 08:58:11 +0000] [1014225] [ERROR] Worker (pid:1182738) exited with code 3
gunicorn[1014228]: [2024-09-18 10:58:12 +0200] [1014228] [INFO] Worker exiting (pid: 1014228)
gunicorn[1014235]: [2024-09-18 10:58:12 +0200] [1014235] [INFO] Worker exiting (pid: 1014235)
gunicorn[1014232]: [2024-09-18 10:58:12 +0200] [1014232] [INFO] Worker exiting (pid: 1014232)
gunicorn[1014229]: [2024-09-18 10:58:12 +0200] [1014229] [INFO] Worker exiting (pid: 1014229)
gunicorn[1014225]: [2024-09-18 08:58:17 +0000] [1014225] [ERROR] Shutting down: Master
gunicorn[1014225]: [2024-09-18 08:58:17 +0000] [1014225] [ERROR] Reason: Worker failed to boot.
systemd[1]: gunicorn.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
systemd[1]: gunicorn.service: Failed with result 'exit-code'.
gunicorn[1182738]: [2024-09-18 08:58:11 +0000] [1182738] [ERROR] Exception in worker process
gunicorn[1182738]: Traceback (most recent call last):
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 609, in spawn_worker
gunicorn[1182738]: worker.init_process()
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/workers/ggevent.py", line 147, in init_process
gunicorn[1182738]: super().init_process()
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 134, in init_process
gunicorn[1182738]: self.load_wsgi()
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
gunicorn[1182738]: self.wsgi = self.app.wsgi()
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
gunicorn[1182738]: self.callable = self.load()
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
gunicorn[1182738]: return self.load_wsgiapp()
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
gunicorn[1182738]: return util.import_app(self.app_uri)
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/gunicorn/util.py", line 371, in import_app
gunicorn[1182738]: mod = importlib.import_module(module)
gunicorn[1182738]: File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
gunicorn[1182738]: return _bootstrap._gcd_import(name[level:], package, level)
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 991, in _find_and_load
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 991, in _find_and_load
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
gunicorn[1182738]: File "<frozen importlib._bootstrap_external>", line 848, in exec_module
gunicorn[1182738]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
gunicorn[1182738]: File "/opt/cloud/current/pix4uav/pix4d/__init__.py", line 3, in <module>
gunicorn[1182738]: from .celery import app as celery_app # noqa
gunicorn[1182738]: File "/opt/cloud/current/pix4uav/pix4d/celery.py", line 16, in <module>
gunicorn[1182738]: from .settings.sentry import before_send
gunicorn[1182738]: File "/opt/cloud/current/pix4uav/pix4d/settings/sentry.py", line 5, in <module>
gunicorn[1182738]: from sentry_sdk.integrations.django import DjangoIntegration
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/sentry_sdk/integrations/django/__init__.py", line 61, in <module>
gunicorn[1182738]: from sentry_sdk.integrations.django.middleware import patch_django_middlewares
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/sentry_sdk/integrations/django/middleware.py", line 38, in <module>
gunicorn[1182738]: from .asgi import _asgi_middleware_mixin_factory
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/sentry_sdk/integrations/django/asgi.py", line 13, in <module>
gunicorn[1182738]: from django.core.handlers.wsgi import WSGIRequest
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/django/core/handlers/wsgi.py", line 5, in <module>
gunicorn[1182738]: from django.core.handlers import base
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 12, in <module>
gunicorn[1182738]: from django.utils.log import log_response
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/django/utils/log.py", line 6, in <module>
gunicorn[1182738]: from django.core import mail
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/django/core/mail/__init__.py", line 10, in <module>
gunicorn[1182738]: from django.core.mail.message import (
gunicorn[1182738]: File "/opt/cloud/releases/20240913T095509/.venv/lib/python3.8/site-packages/django/core/mail/message.py", line 4, in <module>
gunicorn[1182738]: from email import generator, message_from_string
gunicorn[1182738]: File "/usr/lib/python3.8/email/generator.py", line 17, in <module>
gunicorn[1182738]: from email.errors import HeaderWriteError
gunicorn[1182738]: ImportError: cannot import name 'HeaderWriteError' from 'email.errors' (/usr/lib/python3.8/email/errors.py)
gunicorn[1182738]: [2024-09-18 08:58:11 +0000] [1182738] [INFO] Worker exiting (pid: 1182738)
gunicorn[1014225]: [2024-09-18 08:58:11 +0000] [1014225] [ERROR] Worker (pid:1182738) exited with code 3
gunicorn[1014228]: [2024-09-18 10:58:12 +0200] [1014228] [INFO] Worker exiting (pid: 1014228)
gunicorn[1014235]: [2024-09-18 10:58:12 +0200] [1014235] [INFO] Worker exiting (pid: 1014235)
gunicorn[1014232]: [2024-09-18 10:58:12 +0200] [1014232] [INFO] Worker exiting (pid: 1014232)
gunicorn[1014229]: [2024-09-18 10:58:12 +0200] [1014229] [INFO] Worker exiting (pid: 1014229)
gunicorn[1014225]: [2024-09-18 08:58:17 +0000] [1014225] [ERROR] Shutting down: Master
gunicorn[1014225]: [2024-09-18 08:58:17 +0000] [1014225] [ERROR] Reason: Worker failed to boot.
systemd[1]: gunicorn.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
systemd[1]: gunicorn.service: Failed with result 'exit-code'.
It seems related to https://github.com/python/cpython/issues/124170 and https://github.com/django/django/commit/bf4888d317ba4506d091eeac6e8b4f1fcc731199.
Has anyone faced the same issue?
Thank you,
-- You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/6b750b2c-3b14-46f4-9541-a0ad678547aen%40googlegroups.com.
No comments:
Post a Comment