Arjan Einbu spills the beans on his experience with unit testing, busting one myth after the other. Like the “I don’t have time” excuse, if you’ve used any of these excuses, better find a new one. Scratch that, write a test.
And about the one where management doesn’t buy into it? Go guerrilla and do it behind their backs. You might not get the pat on the back, but they’ll be busy blaming all the other developers for breaking the app, while you’ll be gloating about your perfect code.
It’s a nice post, that covers the essentials, in addition to what I posted before.
A nice plus would be the ability to run integration tests as well. The build server should be able to at least run nightly, and in such cases, why stick to just unit tests?
The ideal is to build with every check-in, including running the unit tests, for getting quick feedback.. Then on a timely basis, run the integration tests as well. This way you’re covered.
A while ago, Eric Shupps wrote about TDD and SharePoint development. His follow up post is really a good read, because rather concentrating on the technique (TDD or Test After), he’s concentrating on why we do unit tests in the first place: Making better code, for less bugs and shortening development cycles.
And I like this quote:
"In fact, I would argue that, due to the complexity of the SharePoint object model, the various undocumented and often erratic behaviors, and the level of skill required to effectively troubleshoot and optimize such code, Unit testing has more value in a SharePoint context than in any other.”
Yes, you can replace SharePoint with your favorite technology and it’s still works…
At the same day I posted about different usages of Isolator in the real world, that are not just regular faking, Travis Illig contacted me about an experiment he’s doing, which obviously succeeded.
Travis, a Typemock MVP, used Isolator to change the behavior of a single method inside 3rd party code, running in a Windows service that was giving him pain. The pain was removed by allowing Isolator to run inside the production code, intercept the offending call, and redirect it to another implementation. Now, how cool is that?
I’ve discussed with Travis other ways to intercept and modify calls. The first one Travis suggested is using a Swap.AllInstances of the offending type and then using a WhenCalled-DoInstead combo, to intercept the call and execute another piece of code instead.
The second option is redirecting the calls to another object, as Travis solution does. In this case, you redirect calls from one object to another. Only methods that have the same signature get redirected. All others are executed on the original object.
The final option is using CThru. With CThru you specify all the method calls you want to intercept, and then implement an event handler that is invoked when the method is intercepted.
For all these to work, you need the Isolator engine to run, and that’s why the environment variables are set. Basically, you can run TMockRunner and do the same thing, although it does get a little heavy when handling Windows services.
Well done Travis! Another reminder that innovation comes from everywhere – when people use a tool differently than what it was intended for, great things can happen.
This week I played with Racer and Gallio, the current incarnation of mbUnit. Did you know that Gallio comes with Typemock Isolator extensions?
Anyway, Gallio has some thread manipulation classes, like TaskContainer and ThreadTask, as well as others. I tried to find deadlocks around those classes. No dice. I’m not discouraged though. I’m just going to need a more thread-complicated project where a deadlock is waiting to be found.
But until then, here are things I find during my exercises, useful ones, I think.
First, when using MSTest, which I do, you need to turn off deployment in the solution’s test configuration file. There’s a small bug that causes a weird exception when you run with the default deployment on.
Next, running with Isolator. Yes, Typemock tools do work together. When you have both installed, and you want to run Racer, you should disable Isolator and enable Racer. This is the default mode, since Racer runs the regular code paths.
However, there come times where you need dependency isolation. In my tests, I found that the code I want to Racer to test calls system files dependencies. In another case, the function got an abstract class as an argument. In both, those didn’t have any impact on what I was looking for, namely thread manipulation methods. So I faked them with Isolator.
In order to make that work, you need to disable from the Typemock menu and enable Isolator. Then you need to link Isolator with Racer, using Isolator’s Configuration program. And that’s it, you can now use Isolator’s APIs to your heart’s content. This can make the process of running much shorter, because as we know dependencies can be heavy on the processing side, while not important for the purpose of the test.
So far I haven’t found a worthy opponent. And by worthy I mean a complex enough multi-threaded project that might actually cause deadlocks. What I’ve encountered until now are simple locks. What do you suggest I try next?
This post came from reading a question on our forum, as well as reading one on TDD experience. It’s about fragile tests, and the cost of maintenance.
We’re talking about refactoring. And not just for making the code prettier and maintainable. We refactor and then our tests break. Well, sometimes they do, depending on how we wrote them. But if we change functionality, yes they will have to change.
It becomes worse when our tests know too much about the inner logic of our components. If component A uses B in some way, and you change that, it maybe that integration tests for A are still passing, but the ones where you fake B break.
What should we do? Are we doomed? Is this our fate? Is there no way out?
Alas, there is no simple answer. First, remember that the cost of maintaining your tests is largely outweighed by having these tests in place at all. Having no tests at all costs a lot more than having to maintain a few tests. So if someone tries this on you, show them the door.
The second thing is to come to terms with the fact that test code is just that - code, and therefore is subject to the lifetime changes such as production code is. If you are using Isolator or any mocking framework out there, you’re bound to hit this sometime.
But you can minimize the risk of creating fragile tests. Here are a few suggestions:
What’s you learned-the-harsh-way lessons for avoiding fragile tests?
I like this one. I usually hear it in a much larger organizational context, but hey one developer is a micro-cosmos of the entire org, right?
Alk talks about the cost saving of writing a test in terms of just running and debugging an app for a bug. We always talk about how writing tests impacts development now, but saves so much during the entire lifecycle of the application. But, writing a unit test to fix a bug is the one case where the developer sees in his own eyes how much time gets wasted when he’s debugging. It’s time saved for him, not for “everyone”.
When developers see they don’t need the entire app just to tests a single method, it’s one of those “Aha” moments we all love. If you want to convince a developer that unit testing can help him, not only the entire project, show him how a writing a simple test can save him a lot of quality time with the debugger. And then say: “Oh this little tool? It’s called Typemock Isolator. Saves me hours, and it can help you do that too”.
