Custom ManyRelatedManager?

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEE9graf/i1y59DvwpEQjjPmu0+JdwFAl1ISuMACgkQQjjPmu0+
Jdz02w//SbKjTgHkufAhlsevtNxk+uIHtvCcgsxQACGRAUw/9Z46FDOJ/sYlJ5z/
oUuslabHrDVfFGjDy35msIImKWZ3g+z+pwoZvczuGlmlGbClyPiqWc/bBZlYnSwJ
eeLPALE9l8wpI/1mFs9upId+fVQTlzxTAsV5XUPj0Wfzwp6OG4cb8gsc7YEnk/Cm
Fyw0JHUI+syhvHCjAcA/bOwmJbjUx4Hfd0bzVxXvh0u2kjpXLd3ZZWpjEgoTbYUE
oSvlzyOY6oo+Gu5x1LFNfz0aBtnkhTcJr2H0prJWUralIh+In27VxN0GUqmGkoxE
ncVgL9Iaev7gfevMyT6hgO3VuzIv8ibHDPBVQVdfR6Bv7qxYdCu4iwTxQ/4j341X
oZ2QWfKa4aTXYx+JDk245JvLYiV/y8ViGm0VxnzAdUVV4hLwbleo/H747ij+ABH0
OhVTqd2cOqMnd4NRJbnG6feefHOWKTsQA+uzMQ2ls+eVMw8BMiDFFr/cmS2R6is0
RroxzfbO/vU/Aaey0S7+APrZTl7NBdU0qlPc8JfFWrjT/cghOytHYrNSy00lOYrQ
XwVASVI41aKBJ4WtZlkZ+ecPX7WvD7gWuv5qzGdQ2yAqe94wp9ajg/vSOUTihleo
kptsmo+ZqbjRwe6w1C9CIYsyWdBPa0xSc98vl86qY8po0+yoJn4=
=Aq+c
-----END PGP SIGNATURE-----
Hi everyone,

on StackOverflow the question on how to overload the ManyRelatedManager
for ManyToMany relations has been asked multiple times, but none of the
answers seemed to be satisfying. So I would like to talk about it with you.

## My problem

The backend of my Django app gets triggered by events of a RabbitMQ
server. After pre-filtering the backend dispatches a Celery task to
process data from a remote system to which the event belongs.
Unfortunately events for the same object can arrive in bursts leading to
multiple Celery tasks processing the same objects almost simultaneously.

## My solution so far

In my custom ModelManager I explicitly use Postgres Advisory locks, e.g.
in `get_or_create`, which is working really well.

## Missing link

As my data model contains quite a few ManyToMany relations I would
prefer to define the usage of locks only once in the methods of a class
derived from ManyRelatedManager, e.g. in the `set` method.
So far I was unable to figure out how to do that. Any suggestions?

## Example

To give you a better understanding let's look at some code excerpts:

```
class ExtendedQuerySet(QuerySet):
def get_or_create(self, defaults=None, _deadlocks=5, **kwargs):
try:
with advisory_lock(self.model.__name__):
return super().get_or_create(defaults, **kwargs)
except DeadlockDetected as error:
# retry in a second...
...

class ExtendedManager(Manager.from_queryset(ExtendedQuerySet)):
pass

class Reference(models.Model):
# removed field definitions

objects = ExtendedManager()

class Request(models.Model):
# removed some field definitions
references = models.ManyToManyField(Reference)

objects = ExtendedManager()
```

If two Celery tasks try to call e.g.
`Reference.objects.get_o_create(...)` at the same time, the locks work
as expected. But e.g. calling `request.references.set(...)` can fail
(unique constraint violation) in the second Celery task due to a race
condition, that I would like to avoid using locks.


Thanks for your help.


Andreas

--
Andreas Hasenkopf
Phone: +49 151 11728439
Homepage: https://hasenkopf.xyz
GPG Pub Key: http://goo.gl/4mOsM

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/fa44eb44-4a05-7c86-cd72-22e03817e545%40hasenkopf2000.net.