I suspect that many come to QCT as I did, looking for tools to support a more agile, test-driven development process. But as I’ve worked to integrate QCT into our continuous integration (CI) build process, it’s become clear there’s a fundamental disconnect. This creates obstacles that are not necessarily show-stoppers but make life difficult. There’s a big opportunity for a tool that can close this gap.
The problem: CI is file-based, but QCT isn’t. Here’s what I mean. In a CI process, it all starts with source files extracted from SCM. From there, an automated process transforms those source files into a testable artifact, runs the unit tests, and produces a deliverable artifact. Along the way, it must load things into a database and tranform database contents. But at the end, what’s left in the database is of no consequence – it is purely a transient residue that will be wiped out and totally replaced the next time through. What counts is what is in the source files – nothing else matters. That’s what I mean by file-based.
But QCT isn’t like that. It treats test definitions strictly as database objects. Their connection to source files is not really a QCT concern. Contrast this with other IDEs (e.g. Eclipse) which explicitly consume and produce files. Eclipse always knows exactly what file your test comes from. You can’t change a test without changing a file. You can’t delete a test without deleting a file. But QCT doesn’t do that.
I’m not saying that QCT is just like Eclipse. Nor am I denying that QCT must be concerned with the database in ways that don’t apply to Eclipse. What I am saying is that this disconnect between QCT and the source files causes trouble. Because it puts the burden on developers to manage the connection manually. And this is hard to do without screwing up. And the screw-ups cost us time, money, and good will.
For example, when I finish a test in QCT, I’m not really finished. I have to do two more things. First I have to export it to exactly the right spot. Then I have to commit my changed file to SCM. And BTW, since QCT export always adds fresh timestamps, it always looks like the test is changed even when it hasn’t. That means I can’t simply export all. I have to remember exactly which tests have really changed and only export those.
Deleting a test is tricky, too. I can delete it from QCT, but that really doesn’t do anything at all. I have to remember to delete the file from SCM. And if I forget, I’m in trouble, because QCT is completely unable to tell me about the diff between the tests in QCT repository and the tests in the SCM repository.
I want a QCT that can integrate with Subversion. I want to say “QCT, let’s look at the tests I’ve got in Subversion. Then let’s load and run that test. Then let’s change that test. Then let’s commit the new version to Subversion.”