Friday, October 2, 2015

Re: Corrupted Migration?

@Mike: I'm using Python 2.7.10 and Django 1.7. I'm not sure the ticket applies, but I appreciate the link!

@James: Sorry, I misspoke! [field] actually corresponds to a old field in state X. Before adding the new fields in state Y, the migration is trying to remove some old fields in state X but isn't finding them. I think this means that I'm unable to re-run that migration since it's looking for state X fields to remove, but can't find them (the models is already in state Y). My confusion is in the fact that the model state wouldn't roll back to state X after the first failed transaction (like the DB did).

This would be analogous to my writing you a check that will bounce. Once the check bounces, both your personal wealth and your bank statement should be back to what they were originally. But in this particular case, it looks like your personal wealth is stuck in a state that assumes the check has been cashed and your account has been credited, but your bank account disagrees :)

By the way, I talk about model state under the assumption that the Django ORM somehow internally keeps track of the models - I'm not sure exactly how it does this or if it does this at all.

On Thursday, October 1, 2015 at 1:17:43 AM UTC-7, James Schneider wrote:


> I have tried that (by the way, the offending migration didn't show up in the django_migrations table because the migration failed).
>
> I think we might be talking past each other. Sorry if I'm being unclear. Here's an example of what's going on:
> Before migration, my database is at state X, and my migrations files reflect that.
> I change my model file, updating the state (let's call it state Y). Then, I run make migrations, generating a new migration file that corresponds to state Y.
> I run the migrate command to get my database to state Y. It fails, so now my database is back at state X. There is nothing in the database to delete (not even from the django_migrations table) because the transaction was rolled back.
> I delete the migration file that corresponds to state Y. My database is back in state X, and my migration files again reflect that.
> I run the migrate command again, and it fails (same problem). The error is django.db.models.fields.FieldDoesNotExist: [Model] has no field named [field].

After you delete Y migration, have you searched your migration files to verify that there are no references to that field?

> [field] corresponds to a new field in state Y, not in an existing field in state X. I'm confused - why is it looking for a field that doesn't exist in the pre-migrated state (state X)?

Do you have any RunPython or other custom modifications in your migrations that would cause an object based on your modified model to be created (and saved), possibly relying on a default value for the new 'field' specified in your Y-version model, maybe as part of a data migration?

> So, I assume that the model state somehow is de-synced from the database/migration state. I hop into the Django shell and check the model metadata. The fields exactly match state Y. My database is still in state X.
> Does that make sense?

Makes perfect sense when you think about it. Being out of sync in this manner happens constantly, and is exactly why migrations exist.

Django makes the assumption that the current values in your models.py files exactly match the layout of the DB, since you should be syncing everything using migrations and validating via tests.

Any time you modify a model, the DB and you model definitions become out of sync, with your models being 'ahead' of your DB since you haven't informed the DB of your model changes via migrations yet.

Think of it like someone writing you a check. Your relative personal wealth increases as soon as you receive the check, but your bank statement won't agree with you until you submit the check to them so that they can credit your account accordingly.

Feel free to write me a check (for a large amount, bigger the better) so I can demonstrate the effect it has on my bank account. ;-)

-James


On Thursday, October 1, 2015 at 1:17:43 AM UTC-7, James Schneider wrote:


> I have tried that (by the way, the offending migration didn't show up in the django_migrations table because the migration failed).
>
> I think we might be talking past each other. Sorry if I'm being unclear. Here's an example of what's going on:
> Before migration, my database is at state X, and my migrations files reflect that.
> I change my model file, updating the state (let's call it state Y). Then, I run make migrations, generating a new migration file that corresponds to state Y.
> I run the migrate command to get my database to state Y. It fails, so now my database is back at state X. There is nothing in the database to delete (not even from the django_migrations table) because the transaction was rolled back.
> I delete the migration file that corresponds to state Y. My database is back in state X, and my migration files again reflect that.
> I run the migrate command again, and it fails (same problem). The error is django.db.models.fields.FieldDoesNotExist: [Model] has no field named [field].

After you delete Y migration, have you searched your migration files to verify that there are no references to that field?

> [field] corresponds to a new field in state Y, not in an existing field in state X. I'm confused - why is it looking for a field that doesn't exist in the pre-migrated state (state X)?

Do you have any RunPython or other custom modifications in your migrations that would cause an object based on your modified model to be created (and saved), possibly relying on a default value for the new 'field' specified in your Y-version model, maybe as part of a data migration?

> So, I assume that the model state somehow is de-synced from the database/migration state. I hop into the Django shell and check the model metadata. The fields exactly match state Y. My database is still in state X.
> Does that make sense?

Makes perfect sense when you think about it. Being out of sync in this manner happens constantly, and is exactly why migrations exist.

Django makes the assumption that the current values in your models.py files exactly match the layout of the DB, since you should be syncing everything using migrations and validating via tests.

Any time you modify a model, the DB and you model definitions become out of sync, with your models being 'ahead' of your DB since you haven't informed the DB of your model changes via migrations yet.

Think of it like someone writing you a check. Your relative personal wealth increases as soon as you receive the check, but your bank statement won't agree with you until you submit the check to them so that they can credit your account accordingly.

Feel free to write me a check (for a large amount, bigger the better) so I can demonstrate the effect it has on my bank account. ;-)

-James


On Thursday, October 1, 2015 at 1:17:43 AM UTC-7, James Schneider wrote:


> I have tried that (by the way, the offending migration didn't show up in the django_migrations table because the migration failed).
>
> I think we might be talking past each other. Sorry if I'm being unclear. Here's an example of what's going on:
> Before migration, my database is at state X, and my migrations files reflect that.
> I change my model file, updating the state (let's call it state Y). Then, I run make migrations, generating a new migration file that corresponds to state Y.
> I run the migrate command to get my database to state Y. It fails, so now my database is back at state X. There is nothing in the database to delete (not even from the django_migrations table) because the transaction was rolled back.
> I delete the migration file that corresponds to state Y. My database is back in state X, and my migration files again reflect that.
> I run the migrate command again, and it fails (same problem). The error is django.db.models.fields.FieldDoesNotExist: [Model] has no field named [field].

After you delete Y migration, have you searched your migration files to verify that there are no references to that field?

> [field] corresponds to a new field in state Y, not in an existing field in state X. I'm confused - why is it looking for a field that doesn't exist in the pre-migrated state (state X)?

Do you have any RunPython or other custom modifications in your migrations that would cause an object based on your modified model to be created (and saved), possibly relying on a default value for the new 'field' specified in your Y-version model, maybe as part of a data migration?

> So, I assume that the model state somehow is de-synced from the database/migration state. I hop into the Django shell and check the model metadata. The fields exactly match state Y. My database is still in state X.
> Does that make sense?

Makes perfect sense when you think about it. Being out of sync in this manner happens constantly, and is exactly why migrations exist.

Django makes the assumption that the current values in your models.py files exactly match the layout of the DB, since you should be syncing everything using migrations and validating via tests.

Any time you modify a model, the DB and you model definitions become out of sync, with your models being 'ahead' of your DB since you haven't informed the DB of your model changes via migrations yet.

Think of it like someone writing you a check. Your relative personal wealth increases as soon as you receive the check, but your bank statement won't agree with you until you submit the check to them so that they can credit your account accordingly.

Feel free to write me a check (for a large amount, bigger the better) so I can demonstrate the effect it has on my bank account. ;-)

-James


On Thursday, October 1, 2015 at 1:17:43 AM UTC-7, James Schneider wrote:


> I have tried that (by the way, the offending migration didn't show up in the django_migrations table because the migration failed).
>
> I think we might be talking past each other. Sorry if I'm being unclear. Here's an example of what's going on:
> Before migration, my database is at state X, and my migrations files reflect that.
> I change my model file, updating the state (let's call it state Y). Then, I run make migrations, generating a new migration file that corresponds to state Y.
> I run the migrate command to get my database to state Y. It fails, so now my database is back at state X. There is nothing in the database to delete (not even from the django_migrations table) because the transaction was rolled back.
> I delete the migration file that corresponds to state Y. My database is back in state X, and my migration files again reflect that.
> I run the migrate command again, and it fails (same problem). The error is django.db.models.fields.FieldDoesNotExist: [Model] has no field named [field].

After you delete Y migration, have you searched your migration files to verify that there are no references to that field?

> [field] corresponds to a new field in state Y, not in an existing field in state X. I'm confused - why is it looking for a field that doesn't exist in the pre-migrated state (state X)?

Do you have any RunPython or other custom modifications in your migrations that would cause an object based on your modified model to be created (and saved), possibly relying on a default value for the new 'field' specified in your Y-version model, maybe as part of a data migration?

> So, I assume that the model state somehow is de-synced from the database/migration state. I hop into the Django shell and check the model metadata. The fields exactly match state Y. My database is still in state X.
> Does that make sense?

Makes perfect sense when you think about it. Being out of sync in this manner happens constantly, and is exactly why migrations exist.

Django makes the assumption that the current values in your models.py files exactly match the layout of the DB, since you should be syncing everything using migrations and validating via tests.

Any time you modify a model, the DB and you model definitions become out of sync, with your models being 'ahead' of your DB since you haven't informed the DB of your model changes via migrations yet.

Think of it like someone writing you a check. Your relative personal wealth increases as soon as you receive the check, but your bank statement won't agree with you until you submit the check to them so that they can credit your account accordingly.

Feel free to write me a check (for a large amount, bigger the better) so I can demonstrate the effect it has on my bank account. ;-)

-James


On Thursday, October 1, 2015 at 1:17:43 AM UTC-7, James Schneider wrote:


> I have tried that (by the way, the offending migration didn't show up in the django_migrations table because the migration failed).
>
> I think we might be talking past each other. Sorry if I'm being unclear. Here's an example of what's going on:
> Before migration, my database is at state X, and my migrations files reflect that.
> I change my model file, updating the state (let's call it state Y). Then, I run make migrations, generating a new migration file that corresponds to state Y.
> I run the migrate command to get my database to state Y. It fails, so now my database is back at state X. There is nothing in the database to delete (not even from the django_migrations table) because the transaction was rolled back.
> I delete the migration file that corresponds to state Y. My database is back in state X, and my migration files again reflect that.
> I run the migrate command again, and it fails (same problem). The error is django.db.models.fields.FieldDoesNotExist: [Model] has no field named [field].

After you delete Y migration, have you searched your migration files to verify that there are no references to that field?

> [field] corresponds to a new field in state Y, not in an existing field in state X. I'm confused - why is it looking for a field that doesn't exist in the pre-migrated state (state X)?

Do you have any RunPython or other custom modifications in your migrations that would cause an object based on your modified model to be created (and saved), possibly relying on a default value for the new 'field' specified in your Y-version model, maybe as part of a data migration?

> So, I assume that the model state somehow is de-synced from the database/migration state. I hop into the Django shell and check the model metadata. The fields exactly match state Y. My database is still in state X.
> Does that make sense?

Makes perfect sense when you think about it. Being out of sync in this manner happens constantly, and is exactly why migrations exist.

Django makes the assumption that the current values in your models.py files exactly match the layout of the DB, since you should be syncing everything using migrations and validating via tests.

Any time you modify a model, the DB and you model definitions become out of sync, with your models being 'ahead' of your DB since you haven't informed the DB of your model changes via migrations yet.

Think of it like someone writing you a check. Your relative personal wealth increases as soon as you receive the check, but your bank statement won't agree with you until you submit the check to them so that they can credit your account accordingly.

Feel free to write me a check (for a large amount, bigger the better) so I can demonstrate the effect it has on my bank account. ;-)

-James


On Thursday, October 1, 2015 at 1:17:43 AM UTC-7, James Schneider wrote:


> I have tried that (by the way, the offending migration didn't show up in the django_migrations table because the migration failed).
>
> I think we might be talking past each other. Sorry if I'm being unclear. Here's an example of what's going on:
> Before migration, my database is at state X, and my migrations files reflect that.
> I change my model file, updating the state (let's call it state Y). Then, I run make migrations, generating a new migration file that corresponds to state Y.
> I run the migrate command to get my database to state Y. It fails, so now my database is back at state X. There is nothing in the database to delete (not even from the django_migrations table) because the transaction was rolled back.
> I delete the migration file that corresponds to state Y. My database is back in state X, and my migration files again reflect that.
> I run the migrate command again, and it fails (same problem). The error is django.db.models.fields.FieldDoesNotExist: [Model] has no field named [field].

After you delete Y migration, have you searched your migration files to verify that there are no references to that field?

> [field] corresponds to a new field in state Y, not in an existing field in state X. I'm confused - why is it looking for a field that doesn't exist in the pre-migrated state (state X)?

Do you have any RunPython or other custom modifications in your migrations that would cause an object based on your modified model to be created (and saved), possibly relying on a default value for the new 'field' specified in your Y-version model, maybe as part of a data migration?

> So, I assume that the model state somehow is de-synced from the database/migration state. I hop into the Django shell and check the model metadata. The fields exactly match state Y. My database is still in state X.
> Does that make sense?

Makes perfect sense when you think about it. Being out of sync in this manner happens constantly, and is exactly why migrations exist.

Django makes the assumption that the current values in your models.py files exactly match the layout of the DB, since you should be syncing everything using migrations and validating via tests.

Any time you modify a model, the DB and you model definitions become out of sync, with your models being 'ahead' of your DB since you haven't informed the DB of your model changes via migrations yet.

Think of it like someone writing you a check. Your relative personal wealth increases as soon as you receive the check, but your bank statement won't agree with you until you submit the check to them so that they can credit your account accordingly.

Feel free to write me a check (for a large amount, bigger the better) so I can demonstrate the effect it has on my bank account. ;-)

-James


On Thursday, October 1, 2015 at 1:17:43 AM UTC-7, James Schneider wrote:


> I have tried that (by the way, the offending migration didn't show up in the django_migrations table because the migration failed).
>
> I think we might be talking past each other. Sorry if I'm being unclear. Here's an example of what's going on:
> Before migration, my database is at state X, and my migrations files reflect that.
> I change my model file, updating the state (let's call it state Y). Then, I run make migrations, generating a new migration file that corresponds to state Y.
> I run the migrate command to get my database to state Y. It fails, so now my database is back at state X. There is nothing in the database to delete (not even from the django_migrations table) because the transaction was rolled back.
> I delete the migration file that corresponds to state Y. My database is back in state X, and my migration files again reflect that.
> I run the migrate command again, and it fails (same problem). The error is django.db.models.fields.FieldDoesNotExist: [Model] has no field named [field].

After you delete Y migration, have you searched your migration files to verify that there are no references to that field?

> [field] corresponds to a new field in state Y, not in an existing field in state X. I'm confused - why is it looking for a field that doesn't exist in the pre-migrated state (state X)?

Do you have any RunPython or other custom modifications in your migrations that would cause an object based on your modified model to be created (and saved), possibly relying on a default value for the new 'field' specified in your Y-version model, maybe as part of a data migration?

> So, I assume that the model state somehow is de-synced from the database/migration state. I hop into the Django shell and check the model metadata. The fields exactly match state Y. My database is still in state X.
> Does that make sense?

Makes perfect sense when you think about it. Being out of sync in this manner happens constantly, and is exactly why migrations exist.

Django makes the assumption that the current values in your models.py files exactly match the layout of the DB, since you should be syncing everything using migrations and validating via tests.

Any time you modify a model, the DB and you model definitions become out of sync, with your models being 'ahead' of your DB since you haven't informed the DB of your model changes via migrations yet.

Think of it like someone writing you a check. Your relative personal wealth increases as soon as you receive the check, but your bank statement won't agree with you until you submit the check to them so that they can credit your account accordingly.

Feel free to write me a check (for a large amount, bigger the better) so I can demonstrate the effect it has on my bank account. ;-)

-James

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/7f757c38-58db-4214-8626-9e553fb8be06%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

No comments:

Post a Comment