Re: Is Django logging multi-process safe?

So, there are some issues in managing the files.   It is not very 12-factor to have the application worry about such things.

The issues with using a FileHandler or TimedRotatingFileHandler from multiple processes/kernel threads are less important, but still exist.  If a file is append-only, then write automatically does a seek to the end before writing, and python logging works hard to make sure this is a single write.   Practically, that means that a log write to a file is atomic if:

• It's size is less than a page in the buffer cache (4k)
• You are not using NFS as a target for logs

In the later case (NFS), the kernel *simulates* the atomic seek to the end and write.  So, if there are multiple writers across multiple servers, you are going to be screwed, but if each single server has its own file, and multiple processes are writing, then it is mostly going to be OK.

There may be some other potential pitfalls using rotating files, such as logging.handlers.TimedRotatingFileHandler.  Haven't analyzed that closely.

So, anyway, the most likely problem then is if the log entries get larger than 4k, which may not be very likely but is possible.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/ebde5694-6494-497a-8873-a43f61872711%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.