Re: Built-in logging module may lead to memory leak?

Hello,

Can you please print here a code snippet where you are using:

> logger = logging.getLogger(__name__)

Regards,

Constantine C.

On Thu, Jun 29, 2017 at 11:29 PM, Yp Xie <pengye91@gmail.com> wrote:

Hi guys,

I'm using the python built-in logging module in a django project. Using dictConfig, the configuration in settings.py is like this:

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

LOGGING = {

    'version': 1,

    'disable_existing_loggers': False,

    'handlers': {

        'mail_admins': {

            'level': 'ERROR',

            'class': 'django.utils.log.AdminEmailHandler'

        },

        'console': {

            'level': 'DEBUG',

            'class': 'logging.StreamHandler',

            'formatter': 'verbose'

        },

        'logfile': {

            'level': 'DEBUG',

            'class': 'logging.handlers.RotatingFileHandler',

            'filename': os.path.join(ROOT_DIR(), 'django.log'),

            'maxBytes': 1024 * 1024 * 10,

            'backupCount': 5,

            'formatter': 'verbose',

        },

    }

    'formatters': {

        'verbose': {

            'format': '######################################################################\n'

                      '%(levelname)s | %(asctime)s | %(process)s | %(module)s | %(name)s.%(funcName)s:%(lineno)d | \n%(message)s',

            'datefmt': '%Y-%m-%d %H:%M:%S',

        },

        'simple': {

            'format': '%(levename)s | %(asctime)s | %(message)s'

        },

    },

    'loggers': {

        'django': {

            'handlers': ['console', 'logfile'],

            'level': 'INFO',

        },

        'django.request': {

            'handlers': ['console', 'mail_admins'],

            'level': 'ERROR',

            'propagate': False

        },

        'project_name': {

            'handlers': ['console', 'logfile'],

            'level': 'DEBUG',

            'propagate': True,

        },

        'utility': {

            'handlers': ['console', 'logfile'],

            'level': 'DEBUG',

            'propagate': True,

        },

    }

}

LOGGING_CONFIG = None

import logging.config

logging.config.dictConfig(LOGGING)

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

And I use logger = logging.getLogger(__name__) in the code under 'project_name' module.

And I find it causes huge memory leak problem, the memory usage keeps growing and never goes down.

if I replace the logger.info() part with a simple print() function, the memory usage is small and stable, so I guess it's the logging module that should to blame. 

But it's a core python built-in module, I don't think there is a huge memory leak problem without others point it out.
The only useful infomation of google results I found is https://www.codeday.top/2017/02/10/12540.html

So I used objgraph to detect if the Logger type numbers kept growing, but the result suggested that no obvious leaking types.

I've been in this situation for almost a week, and still can't work it out.

Anyone can give me some hints on this?

Thanks!

YP

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/28172cbd-ce76-499e-a2c1-38eb9af6c0b4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Sincerely yours,

Constantine C

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAK52boV2JctahkCk97%3D_nYvtzWnjFa%2Bw76QKs87SufMPgVEBeA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.