Saturday, December 27, 2014

cx_Oracle - Django ORM - Reference Count Increase

All,

I have a Long Running Python Process that uses Django ORM against Oracle database.

The size of the process keeps on increasing steadily. 

I was profiling this process using mem_top and i find that the reference count of one particular data type <list> increases continuously with iterations.

Datatype is <type 'list'> [{u'time': u'0.004', u'sql': u'QUERY = u\'SELECT "RANGE_STATUS"."I

References increased from 534 to 53295

This list seems to have almost all the queries that were executed using the ORM

Does this mean that there is a memory leak in the cx_Oracle module or somewhere in Django ORM?

Here is the memtop result between two iterations 10 minutes apart.

WARNING : 27/12/2014 05:45:46 PM : 
refs:
9500 <type 'dict'> {'TAPE_DRIVE_FORMAT': -1610612736, 'SLE_ERROR': 1, 'IMAGE_REL_I386_SEC
8410 <type 'list'> [('200', '343045', 1321), ('200', '343046', 1322), ('200', '343047', 1
1578 <type 'dict'> {'logging.atexit': None, 'django.core.files.errno': None, 'django.test
1567 <type 'list'> ['mem_top', 'mem_top', 'datetime', 'datetime', 'sys', 'string', 'os', 
688 <type 'dict'> {'FILE_SYSTEM_ATTR': 2, 'GetDriveTypeW': <built-in function GetDriveTy
688 <type 'dict'> {'FILE_SYSTEM_ATTR': 2, 'GetDiskFreeSpaceEx': <built-in function GetDi
534 <type 'list'> [{u'time': u'0.004', u'sql': u'QUERY = u\'SELECT "RANGE_STATUS"."I
510 <type 'dict'> {'GetDiskFreeSpaceEx': <built-in function GetDiskFreeSpaceEx>, 'SetThr
510 <type 'dict'> {'GetDiskFreeSpaceEx': <built-in function GetDiskFreeSpaceEx>, 'SetThr
370 <type 'list'> [<weakref at 01CF5030; to 'type' at 1E228030 (type)>, <weakref at 01CF
types:
8625 <type 'function'>
3778 <type 'tuple'>
3128 <type 'dict'>
1672 <type 'list'>
1661 <type 'cell'>
1440 <type 'weakref'>
1351 <type 'wrapper_descriptor'>
1103 <type 'builtin_function_or_method'>
888 <type 'type'>
734 <type 'getset_descriptor'>


WARNING : 27/12/2014 05:54:37 PM : 
refs:
53295 <type 'list'> [{u'time': u'0.004', u'sql': u'QUERY = u\'SELECT "RANGE_STATUS"."I
9500 <type 'dict'> {'TAPE_DRIVE_FORMAT': -1610612736, 'SLE_ERROR': 1, 'IMAGE_REL_I386_SEC
8410 <type 'list'> [('200', '343045', 1321), ('200', '343046', 1322), ('200', '343047', 1
1578 <type 'dict'> {'logging.atexit': None, 'django.core.files.errno': None, 'django.test
1567 <type 'list'> ['mem_top', 'mem_top', 'datetime', 'datetime', 'sys', 'string', 'os', 
749 <type 'list'> ['A. HISTORY OF THE SOFTWARE', '==========================', '', 'Pyth
688 <type 'dict'> {'FILE_SYSTEM_ATTR': 2, 'GetDriveTypeW': <built-in function GetDriveTy
688 <type 'dict'> {'FILE_SYSTEM_ATTR': 2, 'GetDiskFreeSpaceEx': <built-in function GetDi
510 <type 'dict'> {'GetDiskFreeSpaceEx': <built-in function GetDiskFreeSpaceEx>, 'SetThr
510 <type 'dict'> {'GetDiskFreeSpaceEx': <built-in function GetDiskFreeSpaceEx>, 'SetThr
types:
8625 <type 'function'>
3778 <type 'tuple'>
3130 <type 'dict'>
1675 <type 'list'>
1661 <type 'cell'>
1440 <type 'weakref'>
1351 <type 'wrapper_descriptor'>
1103 <type 'builtin_function_or_method'>
888 <type 'type'>
734 <type 'getset_descriptor'>

Please guide.

Regards,
Guddu

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CANFgmFDYW_KbLwvk62DwpdinHi2eBJc_fJBUcz0frABnWBbC-g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

No comments:

Post a Comment