Re: Why doesn't Django support managing Raw SQL changes using migration system like models

Hi Cmp,

Well, Django has to be database agnostic for things to work with any database. If raw sql is allowed, how do you can be sure it will work with any database?

So far, you mentioned a bunch of PostreSQL things. But what if, tomorrow, some higher ups decided that support is required and buy Oracle (because higher ups usually aren't that smart). How things will work? How do you make sure your raw SQL commands will work? (IIRC, there are differences in SQL/DML between Oracle and MySQL).

The biggest problem I see is that you're using triggers. Django knows nothing about them and since you're giving the database control to Django (via its migrations and querysets), you should relegate all the things to Django. And yes, there are drawbacks on that too, but still, everything should be one single place -- and if you need other systems to add data to your database, you should provide them with an API instead of letting them add directly to the database.

The biggest problem I see on your solution, is that your giving powers over your data to two different things -- Django and the database via triggers -- and that's the first step into mess.

PS: 76 migrations is still nothing. I worked with databases with 150+ migrations, and nobody cared about what the migrations did, because the models -- in their current, up-to-date form -- is the canonical representation of the data. All those 150+ migrations were just the way to reach the current state.

On Tue, Feb 27, 2018 at 8:53 PM, cmp <me@cmp.is> wrote:

I have an problem that I wanted to solve recently using a Postgres Trigger + Procedure that I do not want to do via Signals or within Python itself.

I want to make a log which when an entry is created the result is applied to a row. We need the log to be the source of truth and I don't want to write a whole async system using something like Kafka or whatever everyone recommends.

The issue here is that I need to iterate on the procedure which updates the rows as features come along and make it easy for other developers to find/read/etc.

Django only allows storing things like this as RawSql migration commands inside the migrations folder but this App already has 76 migrations and I think it's probably unreasonable for me to assume a normal django developer will think to read all those migrations without running in to some problem with my triggers. Not to mention the main block of sql will need to be copy and pasted between changes to new migrations... etc.

This seems like something the migration system should support somehow.

I found a package which seems to do this, but it appears unmaintained (little to no activity): https://github.com/klichukb/django-migrate-sql

I think it'd be a nice feature for Django to support maintaining a set of Raw SQL via the migration system so more advanced features of databases can be used.

For instance in Postgres setting up other things like full text search and special search indexes were difficult in 1.8. I realize it's easier since 1.10 to add a full text search index, but adding that feature is much more difficult than the RawSQL migrations we did to implement adding the index before.

Making it easy to manage triggers and procedures via the migration system is very difficult, but I think supporting RawSQL items which are managed via migrations would be relatively easy to implement and add a lot of value.

I'm posting this for two reasons: 1) I think it's a good idea and 2) to let others show me where I'm missing something or why this wouldn't be a good fit for Django.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/8043d044-3f7c-4bd2-bdc4-c504faf5120b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Julio Biason, Sofware Engineer

AZION  |  Deliver. Accelerate. Protect.

Office: +55 51 3083 8101  |  Mobile: +55 51 99907 0554

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAEM7gE3k2ugP0X7RwJwOrxrKB-LQZ48uQ%3DMjbsF86xe04vg2wg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.