On Mon, Dec 2, 2013 at 8:24 AM, Paul Whipp <paul.whipp@gmail.com> wrote:
I have test cases that pass when tested individually, pass when the full app is tested but fail when the tests for the entire project are run:
(lsapi)~ $ django test Creating test database for alias 'default'... .................................................................................................................................................s.....................................................................................................................................x..........................................................................................................................................................................................................................................................Exception RuntimeError: 'maximum recursion depth exceeded' in <function remove at 0x13447d0> ignored Exception RuntimeError: 'maximum recursion depth exceeded' in <function remove at 0x13447d0> ignored Exception RuntimeError: 'maximum recursion depth exceeded' in <function remove at 0x13447d0> ignored ...ss........E.....................................................................................................................E.E.Setting content_object to liquid app one .EEE.EEEE............................................................................................................... ====================================================================== ERROR: test_get_direct_notes (lsapi.tests.DirectGetTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "ls-api/lsapi/tests.py", line 869, in _method .... File "ls-core/lscore/model/note.py", line 37, in company return self.content_object.company AttributeError: 'NoneType' object has no attribute 'company' ... (lsapi)~ $ django test lsapi.NotesTestCase.test_wrong_user_cannot_put Creating test database for alias 'default'... . ---------------------------------------------------------------------- Ran 1 test in 0.241s OK Destroying test database for alias 'default'... (lsapi)~ $ django test lsapi Creating test database for alias 'default'... ...................................................ss..................................................................................................................................Setting content_object to liquid app one ...................Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <_ctypes.DictRemover object at 0x46dac70> ignored ..................................................................................................... ---------------------------------------------------------------------- Ran 303 tests in 71.469s OK (skipped=2) Destroying test database for alias 'default'...The 'django' command is an alias for django-admin.py. So the tests pass if run as a single case or as a test suite when testing the entire app but fail with errors if run when running the tests for all of the apps in the project.
Investigating the error in the full suite test: It is the result of an attribute being None when it 'should' have a value. The attribute is a content_object set up in a fixture.
This is an example of a really annoying edge cases that exists sometimes in Django's own test suite. Despite all our best efforts, sometimes unit tests don't clean up after themselves, so the order of execution matters.
The test passes when it runs by itself because it gets a completely new environment when it runs. However, the test fails when run in the full suite because one of the tests that proceeds it leaves behind some detritus in the test database that effects the execution of the failing test.
Can anyone shed any light on this or let me know how I might address the problem?
There's two tasks here:
1) Find the individual test that, when paired with the failing test, causes test failure.
2) Fix the test that causes the problem.
The first task is laborious, but not especially difficult. Test execution order in Django is predictable, so you just need to do a binary search until you find a pair of tests that causes the problem. Lets say your suite contains apps A, B, C and D, and C.TestModel.test_problem that is causing the problem. Run:
./manage.py test A C.TestModel.test_problem
./manage.py test B C.TestModel.test_problem
./manage.py test D C.TestModel.test_problem
Normally, one of these three test executions will cause the failure, and the other two will pass. Let's B is the causes the failure. Now run:
./manage.py test B.TestModel1 C.TestModel.test_problem
./manage.py test B.TestModel2 C.TestModel.test_problem
./manage.py test B.TestModel3 C.TestModel.test_problem
Where TestModel[n] are the various test cases in the B app. Again, one of these will fail. Repeat again for individual tests; you should then have the problem narrowed down to a single test pair that causes the failure. Sometimes it will actually be every test in a given test case -- this will happen if the problem is actually being caused by the setUp method in a TestCase.
You've now identified the cause of the problem; now you just need to fix it.
This is a harder to answer without knowing the exact problem. However, you've suggested that the problem might be related to content types and fixtures. This is a common cause of the problem in Django's own test suite as well. It's caused because Django has an internal cache of content types to speed up the lookup process, and this cache *isn't* flushed at the start of each test by default. In most situations, it doesn't need to be, because the content types won't change between tests. However, if you're creating new temporary content types, or you've got fixtures that modify content types, the cache values will end up being incorrect, and will cause problems.
*If* this is the problem, you just need to manually flush the content type cache at the end of the test that is causing the problem.
def tearDown(self):
ContentType.objects.clear_cache()
The content type cache will be automatically repopulated on next use, so you don't have to worry about setting anything up again.
If content types aren't the problem, then you'll need to go spelunking to find the cause; essentially, you're looking for anything that *isn't* cleaned up at the end of the problem test that will cause the failing test to have a problem. Caches, stale files, and manually executed SQL are all examples of possible causes.
Hope that helps!
Yours,
Russ Magee %-)
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAJxq848%3DnNnpDF7tRbdUXFBkv2dM_1EgSifFvKpjfOw8JKrbng%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
No comments:
Post a Comment