I recently began to experienment with this software and find it quite amazing compared to coming up with our own benchmarks. One thing I am unclear of is let’s say I set up a TPC-B benchmark and run it against Oracle10g1, once it’s finished run it against Oracle11g1, the finally against Oracle11g2. All three databases are on the same host. However I do see different results. I would expect at least some consistency between the 11g databases but so far I have not.
Is it fair to assume that Benchmark Factory should be running the same benchmark each time the job is submitted? Or is there some other way to accomplish the task I am trying to do which is to compare two identical databases and expect similiar results. I’ll continue reading the documentation to see if such a scenario is mentioned.