Re: Help with a huge database query.

If I use the  values()  or  values_list()  the request in database has a long waiting time and some time I also got a  MemoryError (because 4GB RAM is not enough).

My examples:

Apple.objects.select_related('Orange').filter(state='fresh').values_list(

                               # Fields I don't want:

                               'field1',

                               'field2',

                               'field3'

                               # ..... ,

                               'field37'

                               )

( yes I want 37 different values ).

I also try with  .only()  with very nice response time, but in the end I will get all the fields. For example:

temp = Apple.objects.select_related('Orange').filter(state='fresh').only(

                               'field1',

                               'field2',

                               'field3' )

temp_json = serializers.serialize('json', temp)

return HttpResponse(temp_json, content_type='application/json')

it returns me all the fields! Not only the 'field2', 'field2' and 'field3'.

Could you explain me why ?

Finally, I also tried to run a query with  raw()  but the response time is disappointing.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Friday, May 28, 2021 2:13 AM, David Nugent <davidn@uniquode.io> wrote:

I think a better approach to using QuerySet.defer() is to use .values() instead, explicitly specifying which fields you need to export.

Just FYI rest_framework handles this out of the box and provides an API to specify not only which fields to export but filters as well.

Regards

/d

On Fri, May 28, 2021 at 8:10 AM 'odrzen' via Django users <django-users@googlegroups.com> wrote:

Hello Django community o/

I want to create a API call in order to get all data from a specific model ( table ) - but I want to exclude-remove only specific fields from this.

So, I try to find a right way to execute this difficult and "heavy" query/request with a efficient way.

This is what I have achieved so far:

```

#from django.http import HttpResponse, HttpResponseRedirect, JsonResponse

#from django.core import serializers

from apples.models import Apple

from oranges.models import Orange

published_fruits =  Apple.objects.select_related('Orange').filter(state='published').defer(

                               # Fields I don't want:

                               'field1',

                               'field2',

                               'field3'

                               )

So, I have the following results:

[

  {

    "model": "apples.apple",

    "pk": "5326t236-8415-48f4-89e5-1789vc9of442",

    "fields": {

      "id": "apple-type1",

      "name": "Brazil",

      ".....": "....",

      "exports ": []

    }

  },

  { ... }

  {

    "model": "apples.apple",

    "pk": "6435673472-fret2-523t-523t-d41159t23432213",

    "fields": {

      "id": "apple-type2",

      "name": "India",

            ".....": "....",

      "exports ": []

    }

  }

]

```

My proble is :

1. The fields who I don't want (exclude), finally I got it.

2. I want to directly pass all this response to a JSON http output for the users and I have it with the following way:

```

json_response = serializers.serialize('json', published_fruits)

return HttpResponse(json_response, content_type='application/json')

```

I get all the respone in JSON, but ( as you can see above ) with the following stracture:

```

  {

    "model": "apples.apple",

    "pk": "6435673472-fret2-523t-523t-d41159t23432213",

    "fields": {

      "id": "apple-type2",

      "name": "India",

            ".....": "....",

      "exports ": []

    }

  }

```

Please, could you help me to avoid the following part from each record  :

```

    "model": "apples.apple",

    "pk": "6435673472-fret2-523t-523t-d41159t23432213",

    "fields": {

```

from the JSON ?

Thank you very much in advance!

--

You received this message because you are subscribed to the Google Groups "Django users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/MvvGE0gJ6x5v1Geo7zgnHdB8TLr79HKJwubqhQzrOKUSB3Q4oHPeWCctdFmOAJ74PbqQ9gHkg44hKVrPqp9BFiZ7Bn7E-LzM30aNrbhcf7s%3D%40protonmail.com.

--

You received this message because you are subscribed to the Google Groups "Django users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAE5VhgX04DNGPHS%2BgpC7UdT2Kwj40%2BZ6UqKckSQ0G8RL5mHb0A%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/mlI7MqIhUIK5iaGcpHMCV2te7ixZ7sczFPnrDUv_j8MmyeKVWZe1Z2qdWCJNhxBqIVAr3B2KtWWoEEQKKR9WA6EB27-gEvMg0eCnO2ZcPd0%3D%40protonmail.com.