I’m new to Xojo (used to do some RB development 5-6 years ago) and I want to use Xojo Unit to do TDD while developing an application. To do this the way I want I need to run tests in a certain order. I don’t know if I’m missing something but I couldn’t find anything about test ordering in the doc (which is a bit weak).
I looked at the source code for Xojo Unit and it looks like there is no ordering of test methods being called. The TestGroups seem to be run in the order they are defined. And there is no support for “buildup” & “teardown”.
Add the Setup and Teardown event handlers to your TestGroup subclass.
There is no support for specifying the run order of the tests. The names are just pulled from Introspection and run, so the order is not guaranteed. There are a variety of ways run order could be added. Offhand, adding an “Order” attribute and then sorting the method list by that might work. I’d have to think about it more.
What additional info do you need in the docs? Also, Marc did an article about XojoUnit in a recent issue of xDev Magazine.
I’m a bit “afraid” of modifying the Xojo Unit classes, my assumption is that these will be the official unit testing classes that will be updated by Xojo and I don’t want to spend time modifying the updated classes (I had just written some similar classes myself right before I heard that RB would be Xojo and would include Xojo Unit )
I missed some kind of overview of how the classes are expected to work (not the mention that it took a while before I discovered that they were in the examples, I was looking for them in the standard library). Yes, the article was what got me going.
I am trying to use the Unit Test example application but it seems the setup and teardown methods are executed only once per TestGroup instance but I think those methods should be executed prior and after each test so that you may have an isolated environment per test.
we do this alot. ok not as much as we should but a lot.
we wrote a sort method that we call before invoking the tests so the tests in a given testgroup are run in a given order (alphabetically). And this has been done with miminal changes to XojoUnit so we can update to a newer version without large backports.
we also wrote more assert tests to compliment the XojoUnit suite.
besides the article in xDev, and the sample code in the IDE, I spoke last year at XDC on this very topic. If you have any questions please feel free to ask me. I would send you the slide deck but there was not many slides as most of the presentation was demos/live code/interaction with the crowd.
I tripped up on the methods not being run in a predictable order problem yesterday, and wrote up a feature request with a solution (that you can implement yourself now).
[quote=74508:@Ian Jones]I tripped up on the methods not being run in a predictable order problem yesterday, and wrote up a feature request with a solution (that you can implement yourself now).
32867 - Ensure XojoUnit.TestGroup methods can be run in a predictable order. <https://xojo.com/issue/32867>[/quote]
Most test unit frameworks do not do this, the argument being that there should not be interdependency between tests. In other words it should be possible to test any piece of code without caring what went before or after it.
Yes that is what most test suites do. Even open source ones. So if a given test fails for whatever reason it doesnt skew the results of all the other tests that come after it. Each test should be independent from other tests BUT should rely on the setup/teardown (or whatever they are called in a particular test suite) of the test group.
now my OCD/ADD/ADHD/Squirrel says they should be ran in order not because it is needed or suggested by the test suite but it looks nicer on the humans eyes when reading the tests. My tests for a give method, as their number increases (_01 to _02 to _03 …etc) the complexity of the individual test increases.
I’ve been doing a lot of Dart recently, their built in unit test framework has setup and teardown methods at the test group level that are called before and after every single test within the group. As the XojoUnit setup and teardown events are only fired at the beginning and end of the group’s usage I guess I was assuming (I know, cardinal sin in software development to make assumptions) that the individual methods are just super granular chained tests, and therefore needing a reliable order.
I do think it a shame the XojoUnit setup and teardown events are not fired for every test method in a group, setup and teardown are most useful for ensuring that every test in a group starts from a common base of potentially lengthy setup.
I also feel the same way as Scott, if I name/see my methods in a particular order in the navigator I expect them to run in that order too.
So, how are people getting over the need to do relatively complex setup/teardown for each and every test method in a group? Are you just creating your own methods and calling them at the top and bottom of every method?
Yes you are right, but also most commercial test suites provide Setup/TearDown prior/after each Test execution, I did a fast search and see what I found:
I think “Test order” is considered a practice to avoid see Test-driven development - Wikipedia in case that you depend from system states manipulated by previous test because your test is not a “Unit” test isolated from the others; and you also will find difficult to find a bug because a single one may break all test, then which test do I start debugging? . Also I am wondering if you are really doing TDD, I mean red, green and refactor cycle or you are just writing Unit Tests which is quite different?
the core of TDD is write a test for a method (or class or obj), then write the method, run test to see if you method meets the test’s requirements. if not go back and tweak the method until it passes the test(s). now do the whole thing over again for another method.
that is just the core of TDD. Depending on whose version of TDD, there is more layered ontop of that.
My version of XojoUnit now has sorted method usage with setup/teardown before each method in the group. Dead easy to do and working well.
Sure wish XojoUnit was fully open source (with permissive license like MIT or BSD) and available on GitHub or similar so that I could submit a patch and champion the change.
XojoUnit, like all the examples included with Xojo, is covered under the MIT license. The license is in the “ExamplesLicense.txt” file in the Example Projects folder.
I can certainly also add XojoUnit to our GitHub repo.
[quote=75683:@Paul Lefebvre]XojoUnit, like all the examples included with Xojo, is covered under the MIT license. The license is in the “ExamplesLicense.txt” file in the Example Projects folder.
I can certainly also add XojoUnit to our GitHub repo.[/quote]
Ah, didn’t think to go look in the Examples folder, generally expect a license to be within a project, sorry about that. But that’s awesome, thanks for having it MIT licensed.
It would be fantastic if you could add XojoUnit as a repo on your GitHub account.
)