Re: Atomic block termination question

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCAAGBQJVjDMHAAoJEC0ft5FqUuEhhh8P/jkJp5Vld9HC2kGY+pTD7JnS
c3MDN9bFbrX+fcuDH1TPnVuaLEBtiOmlj/G3PIvQXZhW4CAyU/5YATXOq81F2R8S
AR1MaUuIsO5LHTAN46j2vXF/s2CjsGXW/+y23R0rEFMHril+ltGvR+rDWkN8Dci/
1gIpa5f6UjsHxIFMGNc54c8kt0/a2DObcWAq2TeA/UDwOnsnYCFOegj3m0xpGuPy
Z6S546nU63iHqzZU4Yu5te9wdNx86lM/L7iFx+eemwEyqWLaXMELpJnpmrP2YeJ3
n//9Ji3eT14o7LgtnOeTnILspvro0r9Tl/ZP6dWhMSvwmF4gGlIfpXRLZfJvrcKy
GEaXp+zNlfo4shBRCMMlps1gRMgrWC1Z1Mnw72J4c2ghcAkwgciy2SrpK0Fxr+EX
s4Pi3Ho31rniecSOPShOqesLJDQTvpUhRsHXxXoHkMpIixT2G5b5DYvVqfQYBidJ
Rfl/2IQoMdwQHLajhsdGBEj7CdfuHtq94v7uQJu96dITWl7iC0VqxowTzHol+gek
MZkBmFs/VR4cP6P7pI7nJ2L6+joN6LOlTAM7aX97KB+QgagaEgK/vd33B9TXdNr6
cdzGmIVrkTEHujwhKRmmZcGvA1ia2ONEbLuaNA7JoZv1Wmr1qKKBpdBgVJO0yBqu
8NiXdEM4K/9yA2FJF4a4
=pkHI
-----END PGP SIGNATURE-----
Hi Mike,

On 06/25/2015 01:53 AM, Mike Dewhirst wrote:
> On 25/06/2015 2:34 AM, Carl Meyer wrote:
>> On 06/24/2015 02:16 AM, Mike Dewhirst wrote:
>>> On 24/06/2015 4:43 PM, Mike Dewhirst wrote:
>>>> When saving a model I'm getting a TransactionManagementError - You
>>>> can't
>>>> execute queries until the end of the 'atomic' block
>>>>
>>>> Ticket #21540 seems fairly explicit at least where Postgres is
>>>> concerned. TransactionManagementError prevents what I want to do and
>>>> I'm
>>>> not a nuclear expert.
>>>>
>>>> How do I terminate the save() method code in that atomic block and then
>>>> immediately execute my queries?
>>
>> I'm afraid this description of what you're trying to do is too vague to
>> be useful. Maybe some sample code would help?
>>
>> TransactionManagementError is a symptom, not a cause. It means that a
>> database error occurred inside a transaction, which leaves the
>> transaction in an unpredictable state, so Postgres wants you to roll
>> back the transaction (or roll back to a savepoint prior to the error)
>> before issuing any more database queries.
>
> Ok. I thought from reading the ticket that I was trying to do something
> illegal in Postgres - that is issuing a query within a transaction which
> needed to be finalised or rolled back. I took it as a symptom or signal
> and think I understand that Postgres is somewhat more rigorous in this
> regard than MySQL.

Yes, that's right. (Well, the transaction needs to be rolled back. Once
there's been an error, there is no other "finalizing" possible besides a
rollback.) My point was that the transaction gets into this state
because some other query causes an error, so your efforts should first
focus on figuring out what that error was and making it not happen, if
possible.

>>
>> Possible solutions include:
>>
>> a) Figuring out why there was a database error, and fixing it so it
>> doesn't occur.
>
> I separated out all the pre and post-save stuff without the offending
> queries and put them into ...
>
> def save(self, *args, **kwargs):
> self._pre_save() # nothing tricky here
> super(Substance, self).save(*args, **kwargs)
> self._post_save() # nothing tricky here
>
> ... which stopped the TransactionManagementError and everything worked
> on existing substances which already had the necessary 1:1 relations AND
> it kinda "worked" when I [saved as new] except obviously the 1:1
> relations were not created.
>
> ... then did a _post_post_save() with the offending queries ...
>
> def _post_post_save(self):
> if self.physical_state == SOLID:
> Solid.objects.get_or_create(substance=self)
> elif self.physical_state == LIQUID:
> Liquid.objects.get_or_create(substance=self)
> elif self.physical_state == GAS:
> Gas.objects.get_or_create(substance=self)
> elif self.physical_state == AEROSOL:
> Aerosol.objects.get_or_create(substance=self)
> Aquatic.objects.get_or_create(substance=self)
> Tox.objects.get_or_create(substance=self)
> Terrestrial.objects.get_or_create(substance=self)
> if self.terrestrial:
> # We can't do this in terrestrial.save() and it needs
> # to be recomputed on every save
> self.terrestrial.set_soil_m_factor()
> self.terrestrial.set_vertebrate_m_factor()
> self.terrestrial.set_invertebrate_m_factor()
>
> ... which as I said is now called from substance.clean(). I realise
> clean() is called before save() but that's all I can think of at the
> moment. Those m_factors are unlikely to change once the concentration
> values (EC50, LD50 etc) upon which they are based are set.

I don't understand why you want to call this from clean() instead of
from save(), but it should work OK as long as you wrap it in an `if
self.pk:` so it doesn't try to run on an unsaved object. Of course that
means it will never run at all when saving a new object.

>> b) Wrapping the code that might cause a database error in its own
>> `transaction.atomic` block, so on error that bit of code is rolled back
>> and later queries within the same transaction can go forward.
>
> That sounds like nuclear physics to me. I could probably follow a recipe
> but might have trouble figuring out when to use it.

Here's the general recipe. This code is problematic:

@transaction.atomic
def do_something():
do_the_first_thing()
try:
do_a_thing_that_might_cause_a_database_error()
except DatabaseError:
pass
do_another_thing()


If `do_a_thing_that_might_cause_a_database_error()` does in fact cause a
database error, then `do_another_thing()` will fail with
`TransactionManagementError` (assuming it tries to use the database)
because it's trying to continue with a transaction that's in an error state.

Assuming you can't prevent the possibility of the database error, here
is how you would solve the problem with the above code:

@transaction.atomic
def do_something():
do_the_first_thing()
try:
with transaction.atomic():
do_a_thing_that_might_cause_a_database_error()
except DatabaseError:
pass
do_another_thing()

By wrapping `do_a_thing_that_might_cause_a_database_error()` in an inner
atomic block (that is, a savepoint), and catching the DatabaseError
outside that atomic block, you allow the DatabaseError to first cause a
rollback of that inner atomic block, putting the transaction back into a
usable state.

This is covered in the docs. See the warning in this section:
https://docs.djangoproject.com/en/1.8/topics/db/transactions/#controlling-transactions-explicitly


Carl

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/558C3307.7080607%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.