Toad World® Forums

Testing system output


#1

I have been trying to set up test outcomes dealing with system output content (produced by dbms_output.put_line() calls. Unfortunately I could not find any good help dealing with this issue. Tutorials and videos deal with testing collections, but none deals with system output directly.

Here are the issues.

  1. In Preferences ->Results there is a check box “Show output to screen…”. It looks like a display option, no more. For some reason the test outcomes dealing with system output were all failing (stating the the program produced no output) until I turned this option on. Is this the expected behavior?

  2. I have to add an empty line to the expected value of the collection for the number of elements between the f_System_Output1 and e_System_Output1 to match. That is, I always have to add a line below (after the 9 lines of output I expect to get) for the test to run.

e_System_Output11(10) := ‘’;

  1. Some tests run when executed individually, but fail when run in a suite. In one such test I expect 10 lines of output (including the fake empty line). It runs fine when executed individually. When I run all the tests for the same package, this tests fails. The message states that the program produced 17 lines of output instead of 10. Where do those extra line come from? Does the tool separate the output produced by different tests when a series of tests is executed? This is a serious problem, since the tests fail for an unexplainable reason.

  2. When the tool states that a line of output does not match the expected value, it is very hard to find out what the actual value is. If I include a System Output-related test outcome for a test, no system output for that test is displayed in the test result viewer. The only way to see the actual output I could find was to delete the system output outcome and rerun the test. This is very cumbersome. Also, I would expect the error message to be more informative by displaying the actual and expected values, not just saying that they differ.

Michael


#2

Michael,

I am sorry about your frustrations in this area.

We have fixed several of these issues in 1.6, which is now available as a very solid beta (it is undergoing final testing now) at:

http://unittest.inside.quest.com/beta.jspa

For example, we now automatically deal with that silly extra line at the end of the output buffer.

We also straightened out the incorrect dependency between the preference and output testing.

Regarding (3): this is still a problem with 1.6, unfortunately. Generally, we were not sophisticated enough in our backend management of the DBMS_OUTPUT.BUFFER.

Also in 1.6, if you have calls to DBMS_OUTPUT in your customization code, that will cause false failures.

In 1.7, we empty the DBMS_OUTPUT buffer and repopulate it behind the scenes, ensuring that the outcome tests ONLY system output placed there during the program call.

So…1.6 is in beta, about to go production (within a month). At that same time, we will release our first beta of 1.7, which will fix the issues listed above. My apologies for not being able to offer a clean solution RIGHT NOW…

As for (4), you are absolutely right, we should give better information about this. I will look into what is possible. The challenge is that the output testing is based on the same template as the generic collection testing. It is easy to see how to grab and show you differences in the system output buffer. Far less clear to see how to do that when we have a collection of records, in which each field is an object. See what I mean?

But that’s my problem not yours!

SF


#3

Steven,

Thank you for the quick reply. Can I try 1.6 beta? When will it be available for purchase? Will I have to uninstall this version to install 1.6? If so, how to keep the existing tests from being deleted?

Is there any solution to or workaround for problem #3? Any customized code to make the system output outcomes to test correctly?

As for #4, in Java (which is where I spend most of my time) there is a notion of toString() method for an Object. Many collection and arrays override toString() method to provide a human-readable representation of themselves. For an arbitrary object with no toString() implemented, the default implementation will tell you the class of the object, but not its content. Is it possible to approach the task of displaying the value of arbitrary PL/SQL collection in the same manner? If types are known (number, date, varchar, etc.), convert the values to their string representation, if not, display the types of objects.


#4

Sure, you can try the beta. The link appears in the thread above. You can install it on top of your existing install of Code Tester and all your test definitions will remain intact. I will have to get back to you regarding #3 and workarounds.

Do you still have the problem if you run the test definitions independently, outside of a suite?

#4 - nice ideas. Not sure how feasible they are in PL/SQL.

More later…
SF


#5

Steven,

My company is considering purchasing the tool. I need to know whether workarounds for checking standard output in a suite of tests are feasible.

Thank you.

Michael


#6

These issues will be fixed in 1.6.1, due out in the Fall.

Note: 1.6.1 is what I referred to above as 1.7. So yes, the issues will be resolved, and you can check out betas starting in August to verify the fix or report issues that we still need to fix.

SF

Message was edited by: StevenFeuerstein