nose-progressive 0.6

Progress Bar

nose-progressive indicates progress in a stationary progress bar at the bottom of the screen:

thing.tests.test_templates:TaggingTests.test_add_new         [===========-  ]

It is glorious. It supports a wide variety of terminal types and reacts to terminal resizing with all the grace it can muster.

Tracebacks: Realtime, Compact, and Closed-Loop

nose typically waits until the bitter end to show error and failure tracebacks, which wastes a lot of time in large tests suites that take many minutes to complete. We show tracebacks as soon as they occur so you can start chasing them immediately.

A few other niceties further improve the debugging experience:

• Strip the Traceback (most recent call last) line off tracebacks, and use bold formatting rather than two lines of dividers to delimit them. This fits much more in limited screen space:

  FAIL: kitsune.apps.notifications.tests.test_events:MailTests.test_anonymous
        $EDITOR +361 apps/notifications/tests/test_events.py
    File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/unittest.py", line 279, in run
      testMethod()
    File "/Users/erose/Checkouts/kitsune/../kitsune/apps/notifications/tests/test_events.py", line 361, in test_anonymous
      eq_(1, len(mail.outbox))
    File "/Users/erose/Checkouts/kitsune/vendor/packages/nose/nose/tools.py", line 31, in eq_
      assert a == b, msg or "%r != %r" % (a, b)
  AssertionError: 1 != 0
  
  ERROR: kitsune.apps.questions.tests.test_templates:TemplateTestCase.test_woo
         $EDITOR +494 apps/questions/tests/test_templates.py
    File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/unittest.py", line 279, in run
      testMethod()
    File "/Users/erose/Checkouts/kitsune/vendor/packages/mock/mock.py", line 196, in patched
      return func(*args, **keywargs)
    File "/Users/erose/Checkouts/kitsune/../kitsune/apps/questions/tests/test_templates.py", line 494, in test_woo
      attrs_eq(mail.outbox[0], to=['some@bo.dy'],
  IndexError: list index out of range

  The preceding doesn’t quite reflect reality; in an actual terminal, the two lines after FAIL or ERROR are bold to aid visual chunking.

• Identify failed tests in a format that can be fed back to nose, so it’s easy to re-run them:

  FAIL: notifications.tests:MailTests.test_anonymous

  To re-run the above, do this:

  nosetests --with-progressive notifications.tests:MailTests.test_anonymous

Editor Shortcuts

For each failure or error, nose-progressive provides an editor shortcut. This is a combination of a filesystem path and line number in a format understood by vi, emacs, the BBEdit command-line tool, and a number of other editors:

FAIL: kitsune.apps.notifications.tests.test_events:MailTests.test_anonymous
      $EDITOR +361 apps/notifications/tests.py

Just triple-click (or what have you) to select the second line above, and copy and paste it onto the command line. You’ll land right at the offending line in your editor! Just make sure the $EDITOR environment variable is set properly first.

In addition, we apply some heuristics to choosing which file and line to show for the above: we try to find the stack frame of your actual test, rather than plopping you down unhelpfully in the middle of a nose helper function like eq_(). Even if you create your own assertion helper functions, like xml_eq(), we still track down your test.

Custom Error Classes

nose-progressive fully supports custom error classes like Skip and Deprecated. We note the tests that raise them in realtime, just like normal errors and failures:

TODO: kitsune.apps.sumo.tests.test_readonly:ReadOnlyModeTest.test_login_error

However, when an error class is not considered a failure, we don’t show it unless the --progressive-advisories option is used, and, even in that case, we don’t show a traceback (since usually the important bit of information is that the test was skipped, not the line it was skipped on). This stems from our philosophy of prioritizing useful information.

Custom error classes are summarized in the counts after the run, along with failures and errors:

4 tests, 1 failure, 1 error, 1 skip in 0.0s
         ^^^^^^ Bold ^^^^^^

The non-zero counts of error classes that represent failures are bold to draw the eye and to correspond with the bold details up in the scrollback. Just follow the bold, and you’ll find your bugs.

Django Support

nose-progressive can run your Django tests via django-nose. Just install django-nose, then run your tests like so:

./manage.py test --with-progressive --logging-clear-handlers