Re: django testing. random test order with some 'dependency constraints'

Hi Russel,

Thanks for your answer.

On 11/28/2011 04:12 AM, Russell Keith-Magee wrote:
> On Mon, Nov 28, 2011 at 9:39 AM, Gelonida N <gelonida@gmail.com> wrote:
> . . .
> Unit tests are supposed to be isolated. They shouldn't depend on
> execution order, or have any pre-requisite other than those specified
> as part of the setUp()/teardown() methods. As a result, you won't find
> a whole lot of documentation on how to do what you're describing.
>
> The right way to approach this is to identify the required
> preconditions for each test, and either construct those preconditions
> in the setUp() method,
Yes. However setUp() is being called prior to every test,

So either I had to add caching into the setUp() section
or I had to use setUpClass() / tearDownClass()

I think that with a mix of SetUpClass() / setUp() and some 'caching'
logic I should be able to benefit from results of previous tests if I
wanted to.
Benefiting from previous tests is mainly for run time reductions, which
will allow to run more tests in a reasonable amount of time.

or use Django's test fixtures to install test
> data.
>

> This is all covered in the documentation for Django's testing system;
> most notably in the section on fixtures.
>
> [1] https://docs.djangoproject.com/en/1.3/topics/testing/#fixture-loading
>
I try to avoid fixtures, as they don't allow to create smart data
(except I run a fixture generation tool prior to running the tests)

But perhaps I could create the fixture in the setUpClass() section if
not already existing. and just load them if they already existed.

I just had to find some logic, which would expire outdated fixtures (due
to migrations / etc)

> More broadly, you might want to dig into the documentation for
> Python's unittest module, which is what Django's unit test framework
> is built on.

Now I just have to look at how to best do the shuffling of tests in
order to have a random execution order.
Probably only shuffling only within TestCase classes and shuffling the
order of the TestCases would be what gives least penalty for run time,
but some unpredictable order.

So far I found only sortTestMethodsUsing(), which is not exactly
shuffling but might be one way to change the order.

I'll look also at
load_tests(), discover() / etc.

Though I'm still not sure about the *best* place to introduce shuffling
(without dependency checks)

The documentation mentions, that care has to be taken about test
fixtures *if* one does randomize the order. However no suggestion is
made *how* to best randomize the order if one desires to.

I will do some more searching / expereimenting.
The reasons why I asked this question is to avoid, that I implement a
working, but rather inelegent / inefficient solution to a problem, which
has already been tackled by others in a more elegant way.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.