Toad World® Forums

Error in comparrison graphs?


#1

Hi, Im using tpc-c ,

I have noticed that when comparing two runs, the comparrison graph shows results for user loads thayt dont add up to the individual values from the single test

for example

comparing two tests (a and b)

if i take the results for userload = 1 for test a.

the individual transaction time averages are: 0.023, 0.005, 0.056, 0.00, 0.017

However when i look at a comparison (Transaction time vs User Load) for tests a and b… test a at user load 1 has a transaction time of 16.00 ms

16.00 ms doesnt equate to the average of (0.023, 0.005, 0.056, 0.00, 0.017)

so whats going on?


#2

BMF calculates the overall average transaction time but summing to total transaction time (the total time spent executing all transactions) divided by the total executions. Your calculation is an average of averages and not a true average.

So as an example in one test run I have a Overall transaction time of 15ms with transaction averages of (11, 9, 4, 77, 57). All times are in ms. Taking the average of these averages yields 31.6. But doing a reverse calculation of the total transaction times by taking the individual transaction averages multiplied by the number of executions for that transaction (26, 17, 2, 1, 4) gives times of

26 * 11 = 286
17 * 9 = 153
2 * 4 = 8
1 * 77 = 77
4 * 57 = 228

Now the sum, or total transaction time, is 752, which when divided by the total executions, 50, yield an overall average transaction time of 15.02 ms.