Monday, August 21, 2017

Re: Correct pattern to call external api during form saving



On Mon, Aug 21, 2017 at 5:16 AM, Chris Wedgwood <wedgemail@gmail.com> wrote:
Hi

My requirement is

1)User adds postcode to form
2)I call external api with postcode to get details of the
postcode,local authority,MP etc
3) save this data into another model 

My question is what is the correct pattern for doing this? Where
should I call the api?
Should this be done before the user form is saved(is_valid)?


It depends. Does the validity of the form depend on the return value of the API? If so, I would have the API calls inside of the clean_foo() methods of the form. If the API returns an error or times out, you would raise the respective validation error. 

If the data values can/will be saved regardless of the API calls, then I would place it either in the save() method of the Form (be careful if the form is reused elsewhere), or in form_valid() in the view. It probably makes more sense to keep everything in the form, though, and let the view handle the interaction with the end user. You may also consider offloading the API calls to a batch job processor such as Celery to fill in the blanks later.
 

What if the api goes down? How to handle that gracefully?


It depends on your workflow and the level of dependency you have on the API. You'll probably end up raising validation errors in the form to the effect of 'your address cannot be validated at the moment, try again later'. Or, you can accept the form values and have a django-admin command run through to resolve them later when the API returns. 

One thing to watch out for is how long your API calls take. The request/response cycle back to the client will be held up for the entirety of all the API calls, meaning that the end user will be staring at the loading bar on their browser saying "waiting for reply...". This could be a negligible/unnoticeable delay for low-traffic sites and API's, but if there is any sort of delay, or the API goes down entirely, all of your worker processes for your web server may be stuck waiting for the API, and your entire site goes down because the web server process (Apache/Nginx, etc.) is starved for resources waiting for the calls to timeout. This is one of the advantages of handing the requests off to Celery to be resolved using a separate pool of resources, with the consequence of the API calls being handled separately from the user form (ie if the user submits a bad value and Celery is checking possibly minutes later, what does it do with the existing information?). 

TL;DR; API calls by the server process should be minimized and tuned with short timeouts, otherwise even a low-traffic site can quickly become overwhelmed just waiting for API responses. 

Another option would be to push the API calls out to the browser via JavaScript, if it is a public API. This provides several benefits, such as instant validation with no request back to your server, and keeps your web server processes free and clear. This may not be possible in all cases. You're also dealing with CORS, and JavaScript, both of which bring their own set of issues.

-James

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CA%2Be%2BciXKO6EPDmfuOS-EErY5njarVTM%2B1G_m9NmGUDEtQw-eNg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

No comments:

Post a Comment