Toad World® Forums

TPC-E Transaction Mix error

I am experimenting with BF and using version 6.1.1 build 228. When I ran TPC-E Transaction Mix against Oracle 11g (11.1.0.7.0 – 64 bit), I got this error (my job was set to Stop job after first error):

Agent(myagentname) Error: TradeResult insert holding_summary failure.

My question is what the condition(s) to throw this error? Is it time out, or some other threshold(s) reached? Seems there was no dead lock as total_deadlocks = 0 in dbo.bf_test_results table. Vmstat showed that runqueue was 1 - 3 and CPU (user + system) usage was 81 - 94% in the last 5 minutes before the job stopped.

I am hoping to get the answer to better understand the test results.

Thank you,
Vivien

There will be more error information in the Agent error log located in the /My Benchmark Factory/Error Logs directory of the machine where the Agent was running. The Agent only sends the first error back to the console so as to minimize network traffic.

Thank you. This is what I find in the error_.log for that test:

Insert Into E_HOLDING_SUMMARY(HS_CA_ID,HS_S_SYMB,HS_QTY) Values(2821,'HVA ',200)
TradeResult insert holding_summary failure.
ORA-00001: unique constraint (VIVIEN.SYS_C0018333) violated

How could this constraint be violated? Is it related to benchmark testing? or we are expecting this error when some conditions meet.

Since the TPC-E is a read/write benchmark you will need to make sure to refresh the data between test runs. Are you reloading the data between test runs?

Yes, my job includes drop and create objects for each run. The error happened when the user loaded was at 30. The user load setting was 1 - 30 with interval = 2.

I reran the same job with no more error. But I did have early runs with the same error.

  • Error for scale = 2; user = 1 - 50 with interval = 10:
    Insert Into E_HOLDING_SUMMARY(HS_CA_ID,HS_S_SYMB,HS_QTY) Values(1522,'VBA ',200)
    TradeResult insert holding_summary failure.
    ORA-00001: unique constraint (VIVIEN.SYS_C0017607) violated

  • Error for scale = 1, user 1 - 50 with interval = 10:
    Insert Into E_HOLDING_SUMMARY(HS_CA_ID,HS_S_SYMB,HS_QTY) Values(144,'KZA ',100)
    TradeResult insert holding_summary failure.
    ORA-00001: unique constraint (VIVIEN.SYS_C0017942) violated

Is this error an indicator for BF threshold or a bug?

Any advice for those errors? I am getting the similar errors when I ran BMF against SQL Server. Are these errors indicating any problems either from the application (BMF) and/or the server/disk capacity? Are the test results reliable with those errors?

Error examples from Microsoft SQL Server 2008 R2 (SP1) - 10.50.2500.0 (X64):

ODBC Error (23000,A43) - [Microsoft][SQL Server Native Client 10.0][SQL Server]Violation of PRIMARY KEY constraint
‘PK__E_TRADE__83BB1FB2636EBA21’. Cannot insert duplicate key in object ‘dbo.E_TRADE’. The duplicate key value is (87892026).

ODBC Error (23000,A43) - [Microsoft][SQL Server Native Client 10.0][SQL Server]Violation of PRIMARY KEY constraint ‘PK__E_TRADE___4C12EC822C1E8537’. Cannot insert duplicate key in object ‘dbo.E_TRADE_HISTORY’. The duplicate key value is (87892026, CMPT).

ODBC Error (24000,0) - [Microsoft][ODBC Driver Manager] Invalid cursor state
Execute TradeStatus statement failure.
ODBC Error (24000,0) - [Microsoft][ODBC Driver Manager] Invalid cursor state
Execute TradeResult Frame2 insert holding_history failure.

Thank you.

This is a known issue with 6.1.1 which has been fixed and available for download. Please download the latest version of BMF (v6.6).

Thanks

Thank you.
Can you tell whether these errors affect benchmarking performance? Are the report using version 6.5.1 still accurate if any these error in it? Wondering whether it is necessary to redo all the tests (time consuming).

Errors will affect benchmark performance especially since BMF will not count a transaction that had an error in its TPS calculation, or any other statistic. Certain statistics for the successfully executed transactions are still valid and can be used, such as response time, which is a better indication of server performance.

Thank you. This really helps to clear my doubt.

ODBC Error (23000,A43) - [Microsoft][SQL Server Native Client 10.0][SQL Server]Violation of PRIMARY KEY constraint ‘PK__E_TRADE__83BB1FB2155B1B70’. Cannot insert duplicate key in object ‘dbo.E_TRADE’. The duplicate key value is (18924787).

I am still getting Violation of PRIMARY KEY constraint errors from version 6.6.

example:

Applying the newest version BMF 6.6 does not help.

this is happening in v. 6.8.1 too…I refresh the data between every run.

At what scale factor are you running the TPC-E test?

Dear Friends,

I get the following error:

“Execute TradeUpdate Frame2 statement2 failure” in the error log.

I only execute two bmf users, one executing trade-order and the other trade-update. I get this error roughly in half-an-hour execution of the workload. The bmf crashes.

TPC-E is at the benchmark scale 1.

Is it the similar problem the folks on this post are having, or is it something different?

thanking in advance

Rizwan

There will be additional error information in the Agent error log located at;

/My Benchmark/Error Logs

directory. Could you post the additional error information you are receiving? Thanks

Hi,

I’m getting similar errors running Benchmark Factory TPC-E.

Database: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

BMF version: 6.9.2 Build 621

100 users, scale factor 1, using “Create Objects for TPC-E (with Delete)” to start fresh each run.

Insert Into E_HOLDING_SUMMARY(HS_CA_ID,HS_S_SYMB,HS_QTY) Values(7272,'JIA ',400)

TradeResult insert holding_summary failure.

Transaction “TradeResult Transaction” run failed by virtual user 23 in agent 1 of hg-dmcache33

ORA-00001: unique constraint (TPCC.PK_E_HOLDING_SUMMARY) violated

Transaction “TradeResult Transaction” run failed by virtual user 23 in agent 1 of hg-dmcache33

Insert Into E_HOLDING_SUMMARY(HS_CA_ID,HS_S_SYMB,HS_QTY) Values(1292, 'ZBA ', -100)

Execute TradeResult Frame2 insert into E_HOLDING_SUMMARY failure.

Transaction “TradeResult Transaction” run failed by virtual user 35 in agent 1 of hg-dmcache33

ORA-00001: unique constraint (TPCC.PK_E_HOLDING_SUMMARY) violated

Transaction “TradeResult Transaction” run failed by virtual user 35 in agent 1 of hg-dmcache33

Insert Into E_HOLDING_SUMMARY(HS_CA_ID,HS_S_SYMB,HS_QTY) Values(6091, 'WBA ', -400)

Execute TradeResult Frame2 insert into E_HOLDING_SUMMARY failure.

Transaction “TradeResult Transaction” run failed by virtual user 37 in agent 1 of hg-dmcache33

ORA-00001: unique constraint (TPCC.PK_E_HOLDING_SUMMARY) violated

Transaction “TradeResult Transaction” run failed by virtual user 37 in agent 1 of hg-dmcache33

Unrelated Question: Why are you running scale factor 1? The TPC-E does not have a req of N scale factor per number of users like the TPC-C, but there are mins suggested by the spec for meaningful results. That’s why I tell every BMF user read and know the TPC spec you are running or else results are worthless. Here’s text from TPC-E spec that raises question:

The Scale Factor is the number of required customer rows per single Transactions-Per-Second-E (tpsE). The Scale Factor for Nominal Throughput is 500.

I’m using Benchmark Factory as an Oracle load generator and am interested in performance deltas – specifically the difference in observed TPS when a flash-based read cache is employed for the .dbf files. TPC-E with scale factor 1 (the BMF default) and 100 users produces a reasonable volume of I/O to the backend storage on my test system, so it is meeting my requirement.

That said – I’d rather the workload be realistic and the results be meaningful. However, I have difficulty understanding how the BMF Scale Factor parameter for TPC-E can map to the Scale Factor referenced in the TPC-E spec. To me, it seems more reasonable that the BMF Scale Factor parameter actually controls the number of Load Units. The BMF documentation doesn’t offer much insight on this, however, or at least I couldn’t find it.

Any clarity on this topic, and also how the number of BMF virtual users applies in the context of the TPC-E spec, would certainly be appreciated.

Here are some more quotes from the TPC-E spec which may help to explain my confusion:

2.6.1.2 The initial database population is based on the number of customers. The benchmark Sponsor selects the CUSTOMER table cardinality, based on the desired transaction throughput. Clause 6.6.7.2 defines the Nominal Throughput that may be reported for a given number of rows in the CUSTOMER table. The minimum number of rows for the CUSTOMER table is 5000. The size of the CUSTOMER table can be increased in increments of 1000 customers. A set of 1000 customers is known as a Load Unit.

2.6.1.4 The Scale Factor is the number of customer rows per single Transaction-Per-Second-E (tpsE). The Scale Factor for Nominal Throughput is 500.

6.6.7.2 The Nominal Throughput of the TPC-E benchmark is defined to be 2.00 Transactions-Per-Second-E (tpsE) for every 1000 customer rows in the Configured Customers.

6.6.7.3 Another way of expressing the Nominal Throughput is by using a Scale Factor. The Scale Factor is the number of customer rows per single Transaction-Per-Second-E (tpsE). The Scale Factor for Nominal Throughput is 500.

**