Friday, February 12, 2016

Ideas for easier migration from South to Django 1.7+

Hi everyone

I'm looking for a way to do a semi-automatic migration of databases from Django 1.6 with South to Django 1.7+ migrations.

We have thousands of sites, each using many re-usable apps with many migrations. I'm looking for a way to make this transition as painless as possible. I've not found an existing solution yet. If you know of one, please let me know.

The problem: There are many re-usable apps. Many of them have both South migrations and Django migrations and have been supporting both systems for a while now. But when the time comes to switch to Django 1.7+ there is a lot of manual work involved to pick the right migration to --fake to.
Without manual work you don't know exactly what versions of the individual apps were installed and hence what their exact migration state is. Also you might have updated the code and dependencies already, but want to restore an old database dump from which you have no idea what the exact versions of various apps were when it was made.

I don't think an automated matching is feasible (have Django introspect the database tables, columns and fields and automatically detect the matching migration it can --fake to). Hence my alternative proposal below.

Proposal:

Make a helper app that comes with a management command that proxies the built-in "migrate" command, but adds some checks first. This app would only work in Django 1.7+.

It would use a mapping between South migrations and new Style migrations to decide what migration it can safely --fake to.

The mapping can come with the individual apps (as a string attribute on the new Style migration, pointing to the corresponding South migration), or in a set of contributed mappings for common apps that comes with the migration helper app.

When running "smarter_migrate":

* If there are no entries for an app in the "django_migrations" table, look if there are any in "south_migrations". 
* Pick the latest already applied entry in "south_migrations" and look for a new Style migration that matches it. 
* If it finds a match, fake to that new style migrations and then do a normal migrate.

There would also be a "dry run" command that reports all the clearly matched migrations and any migrations that still need to be manually matched. And ideally, a way to make it really easy to create a pull request to the app's GitHub repo to add any mappings developers come up with while migrating projects.

We at Divio and Aldryn would be happy to kickstart and maintain such an app for the benefit of the wider community. I'm especially interested if anyone has a better idea on how to tackle this problem.

Cheers
--Stefan

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/66e4fccc-0dc3-4f60-aa63-b4776ed5390c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

No comments:

Post a Comment