Sunday, January 25, 2015

Re: Smoke tests

Hi Tom,

Personally I'm not convinced by the concept of smoke tests in
production, if you have a proper development workflow, and your build
works in dev, then you should be confident that your build will work
in prod. Testing URL endpoints in prod should be part of your devops
testing, and kept completely separate to your application. Testing
database settings, cache, email etc should be part of your
bootstrapping, and the application should report a failure via a
reliable tracking system (NR, Sentry etc) of such problems. This is
not smoke testing, it's just good practise. Deployment testing is not
smoke testing, it's deployment testing. The concept of testing your
application in production is an anti-pattern, but the concept of
testing your deployment in production is a necessity, and the two
should be kept completely separate.

Some people use BDD, which I'm personally not a fan of, and others
will use tools such as Selenium and Mocha to ensure pages are working
correctly. If you have set up your application correctly, then you
will be catching these errors as they happen (e.g. Raven JS).

I don't know why "smoke tests" are suddenly becoming the new buzz phrase....

Anyway, hope this helps a bit

Cal

On Sat, Jan 24, 2015 at 3:20 PM, Tomáš Ehrlich <tomas.ehrlich@gmail.com> wrote:
> Hello,
> last few weeks I've been thinking about implementing smoke tests into my
> deployment process. Last week I wrote simple test runner
> (https://github.com/djentlemen/django-smoked), but still I'm missing
> methodology *what* should I test and *how*. Since smoke test has very wide
> definition for different types of software —
> https://en.wikipedia.org/wiki/Smoke_testing_(software), my idea is: After
> every deployment run small subset of tests with *production* settings and
> just check, that app was deployed successfully. If not, rollback to previous
> version immediately.
>
>
> Few such tests might be:
> — check responses of few URL endpoints (like homepage)
> — check database settings are valid (since most tests runs on
> testing/development database)
> — check cache, email settings, etc (for the same reasons as above)
>
>
> I wonder how do you test your apps? Do you use some kind of „smoke tests"
> described above?
>
>
> Cheers,
> Tom
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscribe@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/467626FA-A50D-4AF9-991C-5BD05637693F%40gmail.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAHKQagE-O8Suxcf25WKtNTU6skLtXbQCMLm5S1d-wwZk7dmtfg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

No comments:

Post a Comment