.addError() not called on SkipTest, etc. #45
Comments
|
Google Code Info: |
ghost
assigned
|
I also get the same issue: I am using nosetests v0.10.4 Did anyone understand what is going on? Thanks, Google Code Info: |
|
Still doesn't work. Using nose 0.11.3 and Python 2.7 on Fedora 14. Below is a patch that calls addSkip() in the plugin. I realize the manual says it's deprecated, but it works for me. --- result.py.orig 2010-12-20 11:54:06.904684002 +0100
Google Code Info: |
|
I still have this problem on nose 1.1.2. Neither addSkip nor addError is called to handle a Skip. Same goes for DeprecatedTest. Google Code Info: |
|
Still a problem in nose 1.2.0, however I quickfixed this by giving my plugin a score above that of the Skip/Deprecated/Custom error class plugins (default score of 1000). That way my plugin gets the addError call before the error class which suppresses it. |
ErrorClass returns True on addError to prevent SkipTest and DeprecatedTest from getting to the pdb plugin. However this prevents all plugins from getting them, and the documentation specifies that these exceptions should get to the plugins. The pdb plugin should handle these exceptions explicitly as the documentation suggests. This also deprecates Google Code Issue nose-devs#101 so there is nothing to test there anymore. This fixes nose-devs#45.
|
this was broken since 773dc1c - looks like it was intentional to prevent --pdb from triggering on Skip errors. The problematic code is now in errorclass.py - since its score is high and it returns True in addError it prevents any other plugins from getting addError in case of Skip\Deprecated. |
|
Why this patch wasn't marged w/ master? About one year later it could be still problem. Eg. in my case I need upload test data to db so I'm using nosedbreport plugin. The problem I'm encountered was not working plugin in case of skip test. After applying this patch works much better. |
ErrorClass returns True on addError to prevent SkipTest and DeprecatedTest from getting to the pdb plugin. However this prevents all plugins from getting them, and the documentation specifies that these exceptions should get to the plugins. The pdb plugin should handle these exceptions explicitly as the documentation suggests. This also deprecates Google Code Issue nose-devs#101 so there is nothing to test there anymore. This fixes nose-devs#45.
|
I'm with @SebaM, can this be merged with master? |
|
@jpellerin I just hit this issue in my plugin as well. I agree with @SebaM and @davydany -- can we please merge the patch from @wiggin15 or some other relevant fix? (Pretty please!?) My plugin cannot process skips in it's current version. If there's a reason you won't merge the patch, please let us know. I will use the workaround @Bentis mentioned if this is not going to get fixed any time soon. I personally think it is more consistent to either have [addSkip, addFailure, addError] OR have addError called for skips, failures, and errors. Whatever the case, at a minimum it should work as the documentation currently asserts that it works. My plugin is called green. Anyone interested is welcome to take a look at it |
|
@CleanCut, @SebaM, and @davydany: FWIW, @jpellerin turned over maintainership to me--he's simple too busy. Can we talk about the opposite problem: how many plugins does this break? How many will see errors where they were seeing none before? Do we know what form do those errors take? And let's talk about the patch itself: why is While nose has a sizable test suite, it doesn't fully cover all edge cases--so it's not enough to simply pass the tests. For example, I believe the removal of There's a sizable plugin ecosystem that I'm worried about breaking since this changes the semantics of the plugin API--fixing an inconsistency only to break most of the plugin ecosystem is not a fix to me. I'm unwilling to merge is for a patch release since it changes plugin behavior--at least not without good arguments showing minimal breakage. So I've been saving it for 1.4.0 release, but I'm not sure when that will happen as I'm currently focused on trying to get a 1.3.2 release out the door. With 1.4.0, I hope we can drop support for 2.5 and below, and possibly 2.6 and below. But we should all keep in mind that Nose is in maintenance mode too. Anyone building new plugins should really be looking at nose2. I hope that helps, and I hope that all of you can shed some light on the expected fallout and help provide a sound review. Like many in the open source world, I have a day job and a family, so I only get a limited amount of time to spend looking at issues in-depth. Any help you can provide here is useful. |
|
@jszakmeister: Aaarrrg. I was unaware of the existence of nose2. I will pivot and rebase my plugin off of nose2. I'm not concerned with python older than 2.7 anyway. I presume nose2 will have less issues like this. I highly advise adding a big advisory on the nose home page pointing people to nose2. I've only been using/developing for nose for two weeks, but in my extensive reading and googling I never once came across anything mentioning the existence of a nose2 project. I would have simply started there if I had been aware. |
|
Just to defend the patch, in answer to your questions: I agree that there is an issue with existing plugins. In my opinion the real fix will be to bring back addSkip and addDeprecated - this will not break compatibility and be much easier for plugins to handle... As for nose2: I don't see this project as a reason to drop maintanence of nose, with so many users still using nose and its plugins and so few users and plugins switching to nose2 - I just don't think it ever took off (nose is ranked 13th in pypi, nose2 is ranked 3735th). I don't think we're going to agree here, though. |
It appears to me that any plugin implemented on that base is relying on the filtering that was removed to support the
I'm sorry you saw it as being impolite--that was not my intention. I did read the commit message and I understood that it was supposed to be part of the fix, but the commit message did not say anything to address the other concerns, and unfortunately your response hasn't either. :-( To put it plainly: does
That depends on your view of compatibility. It's certainly a change in behavior, and different than what is happening currently (and necessarily so), so it could be seen as a break in compatibility which is why I believe it needs to be handled delicately.
I agree it's a problem and that it should be fixed, but I need to understand the implications. And yes, the documentation should get updated if we're going to keep it in the current form.
Nose has some serious issues that cannot be fixed without serious effort (like plugins interacting with multiprocessing). Nose2 is built on a much better core, and doesn't have all the compatibility concerns. I'd love to say that Nose will get there, but it's hard to see the value in it since unittest has gotten so much better and Nose2 addresses the remaining issues. |
|
@wiggin15 So I spent some time with this on the plane. The good news is that your patch doesn't break However, there are some other considerations. We talk about the It also turns out that we have some interesting some interesting behavior, at least for the Skip plugin when using |
|
Just read this entire conversation as I'm running into this issue as well. So, what is a plugin developer who wants to handle skipped test to do, if they want to distribute their plugin and not ask users to merge the Nose patch manually? I'm currently writing a plugin and need to handle skipped tests. For now, I just set the |
|
@gardaud From my perspective, setting Since my project is mostly to scratch my own itch, I'm just moving to nose2. I'm liking nose2 so far -- plugin development is far cleaner. I almost have green refactored to work with it. (I haven't pushed up my refactoring work yet, though. It will be version 0.9.0) |
|
I discovered that it was actually much more straightforward (and less buggy -- no issues like this one to deal with) to just take the example TextTestRunner and TextTestResult from unittest and write new versions than to try to adapt to nose's or nose2's plugin API and behavior. So Green is now a standalone test runner. Green is runnable now. It doesn't aspire to have all the features that nose or nose2 have, but it's working pretty well for me. If anyone wants to check it out I would welcome feedback. |
|
I'm running into this issue as well. My plugin uses custom TestRunner and TestResult classes, so setting the plugin score >1000 doesn't do anything. Is there any way to work around this (aside from using nose2) so the test result gets the .addError and .AddSkip calls? |
ErrorClass returns True on addError to prevent SkipTest and DeprecatedTest from getting to the pdb plugin. However this prevents all plugins from getting them, and the documentation specifies that these exceptions should get to the plugins. The pdb plugin should handle these exceptions explicitly as the documentation suggests. This also deprecates Google Code Issue nose-devs#101 so there is nothing to test there anymore. This fixes nose-devs#45.
ErrorClass returns True on addError to prevent SkipTest and DeprecatedTest from getting to the pdb plugin. However this prevents all plugins from getting them, and the documentation specifies that these exceptions should get to the plugins. The pdb plugin should handle these exceptions explicitly as the documentation suggests. This also deprecates Google Code Issue nose-devs#101 so there is nothing to test there anymore. This fixes nose-devs#45.
ErrorClass returns True on addError to prevent SkipTest and DeprecatedTest from getting to the pdb plugin. However this prevents all plugins from getting them, and the documentation specifies that these exceptions should get to the plugins. The pdb plugin should handle these exceptions explicitly as the documentation suggests. This also deprecates Google Code Issue nose-devs#101 so there is nothing to test there anymore. This fixes nose-devs#45.
ErrorClass returns True on addError to prevent SkipTest and DeprecatedTest from getting to the pdb plugin. However this prevents all plugins from getting them, and the documentation specifies that these exceptions should get to the plugins. The pdb plugin should handle these exceptions explicitly as the documentation suggests. This also deprecates Google Code Issue nose-devs#101 so there is nothing to test there anymore. This fixes nose-devs#45.
ErrorClass returns True on addError to prevent SkipTest and DeprecatedTest from getting to the pdb plugin. However this prevents all plugins from getting them, and the documentation specifies that these exceptions should get to the plugins. The pdb plugin should handle these exceptions explicitly as the documentation suggests. This also deprecates Google Code Issue nose-devs#101 so there is nothing to test there anymore. This fixes nose-devs#45.
What steps will reproduce the problem?
What is the expected output? What do you see instead?
Expected: .addError() is called on P
Got: .addError() is not called on P (assuming
nose.plugins.skip.Skip.addError is called before P.addError by the plugin
manager)
See attached patch for a proper test.
Please use labels and text to provide additional information.
As well as having .addError called, we also still need an interface for
plugins to find out whether an error is ignored. For example, plugin debug
needs this, so that it doesn't try to drop into the debugger on errors that
won't cause an unsuccessful test run, like skips. ATM, this is done by
returning False, which is what stops other plugins' .addError() methods
from getting called.
Discussion:
http://groups.google.com/group/nose-dev/browse_thread/thread/b1d726998dc79bd9
Google Code Info:
Issue #: 169
Author: j...@pobox.com
Created On: 2008-03-10T23:47:10.000Z
Closed On:
The text was updated successfully, but these errors were encountered: