I was amazed to see the XojoUnit example, as I started with Xojo.
So I gave it a try in my first Xojo-project. It works well for me and used it to develope some (not all) functionalities.
How do you test your apps?
What do you never test?
Do you also use other tools for testing?
Thanks
I do like Apple and Microsoft do… I deploy it to the public
(JUST KIDDING)
How do you test your apps? Develop Use Cases, apply them, fix the code for those that fail, rinse, repeat, Then I attempt to have one or more others do the same thing for me in order to have use cases that I didn’t see due to being so close to the code
What do you never test? Nothing… I attempt to test or have tested all areas of the program
Do you also use other tools for testing? Depends on if the app generates output to be used by another app or not, but for the most part no.
Dave beat me to the joke, but I’ll say my version of it anyway…
“Send it out to my user base, see who complains…”
Seriously, I think there’s some standard answers to this, and I’ll avoid them, For a great treatment on how to test any software, see Code Complete by Steve O’Donnell That gives you a comprehensive view that you can’t do without.
Use assertions. It’s too bad Xojo doesn’t have a assert macro, but create your own (evaluate only under #if DebugBuild) and pepper them liberally in your code.
Don’t be afraid to write extra code or functions, just for testing purposes, that your clients will never see, for testing purposes.
Create bulk loops on functions you allow access to, whether exposed or not, and run them to stress-test your app.
Pay special attention for unexpected incoming data; see above about writing special tests to diagnose how solid they are.
Another thing I did that I don’t regret is whether a customer encounters an exception that wasn’t supposed to be triggered, it “calls home” and sends me the what the exception was, where it was, and the current running log of the program via email. So when I say I use my user base for testing, I mean it - it’s not primary, but it’s extremely helpful. Clients have a tendency not to report, this helps matters.
I test as much as I can with Unit Testing suite that comes with Xojo (we have actually extended the suite that we use). We dont test the UI automagically. We run the application and do what most users do, click, click, click, click, “what the hell just went wrong?”
Does anyone do automated GUI testing?
I use MacDeveloper (http://macdeveloper.net) to weed out the embarrassing bugs. Very affordable and I had good results.
Regarding automated tests for the Mac: there is this company named for a vegetable, but their software is insanely expensive like 5000$ or so. Then there is the open source software Sikuli.
How much does it cost? Do I have the possibility to control what gets tested?
It costs 12 $ per month as far as I remember. No, you don’t have control, you simply upload the applications and interested testers download it. Usually, they get a free license if they find a bug. Which as beta stage is a given if they work with the app longer than a few minutes.
Normally software is tested as I’m developing it. I also use my wife, who is great at finding bugs.
My wives devises her own testing plans independently from me (as I develop the software), she not only finds bugs, but also usability issues.
Then I have a select few users who love to play with software before it’s released.
Lastly if you submit to the Mac App Store, they have their own testing procedure, which isn’t foolproof and if they find an issue, they’re very unhelpful when it comes to resolving it.
Here you go 
The ultimate test, after having been through the cerberus at the MAS, is the battle fire. If there is the slightest chance that a bug exists, end users WILL find it. And seek support. That is why the support link for an app is paramount. You don’t want the user to simmer in frustration when any problem occurs, but be able to react right away.
Pumpkin ? Cauliflower ? Potato ?
Microsoft provides a benchmark program that tests a lot of things in Windows Store applications prior to uploading. It churns for something like 20 minutes and tests all sorts of things about display, system calls, GNW and the kitchen sink. Does not cost an arm and a leg, it is free for developers.
That said, and even if they sometimes seem overzealous, I prefer Apple reviewers. Nothing beats a human being for testing software.
Sorry, I always forget the english translation. Eggplant = aubergine.
http://www.testplant.com/eggplant/
They even don’t have pricing on their website anymore.
[quote=110445:@Beatrix Willius]Sorry, I always forget the english translation. Eggplant = aubergine.
http://www.testplant.com/eggplant/
They even don’t have pricing on their website anymore.[/quote]
So it is a prospect heart test. When told the price, if the client grabs his chest and faints, he has a heart condition. After buying, he has a checkbook condition 
And I thought eggplants being part of the cretan diet, they where good for the cardiovascular system
cretan or cretin?
Cretan diet -> mediterranean diet. That’s from Ancel Keys (http://en.wikipedia.org/wiki/Ancel_Keys) the same guy we got the lipid theory for heart diseases from.
I discovered that I am not as good to test my project as I am with others software
I think that is because I am tired when it comes to test my project(s); I do not think it is a matter of I code perfect at first run thing.
It is incredible how I am good at finding bugs to software I do not have to think twice to get ideas to how can I test this ?: ideas comes by themselves (unlike when I test my project(s): in that case, I have to force my brain to work ad nauseam 'till I found where to try).
BTW: I also am very good at strange design / coding theories (things to trash very fast); hopely, very few of these comes to life).
At last, when something is wrong in a code where booleans exists, it is usually because I reversed the Boolean testings, so I just have to revert them and all is good.
Worse than testing : writing the user manual
Yes, but you wrote the manual after the debug time, and you have to redo the testings !
Have a nice week end !
Thanks for your experience of testing own software. 
Write the user manual FIRST - so it serves as the spec for what the user will experience.
Then write the software to match that spec.
I’ve done that once for a full product and it worked fabulously - there were 0 rejections based on functionality because it was what the user manual said. There were some bugs in the implementations but overall no spots where the users acceptance testing got to a point of saying “This makes no sense” - because it was all hammered out in the user manual on paper long before code was written.
But its hard to work in that order
[quote=111052:@Norman Palardy]Write the user manual FIRST - so it serves as the spec for what the user will experience.
Then write the software to match that spec.[/quote]
I remember you talked about that experience before. Great idea.
I was half joking about writing the user manual being worse than testing. From what I see around the MAS, a lot of brilliant developers seem to put very little efforts in documenting correctly their programs, let alone their online presentation