Toad World® Forums

Toad 2 - Data Compare Tool


#1

I really like Data Compare tool in Toad and rely heavily on it to perform regression testing for critical stored procedures.

It works very well with smaller data sets under 50 000 rows and 20-30 columns. Toad becomes very unstable and crashes often with out of memory exceptions if the data set is bigger than that.

I run the tool on a PC with 16 GB of RAM and Quad Core CPU it is it surprising that we get such errors.

Can anyone from the development team comment on possible solutions and whether any work is being done to address this in Version 3?

Thanks for a great product!


#2

Hi ZeenKork,

couple of questions:

  1. Can you do Help | About | Application Data Directory and open up the DataCompareAndSync.log file in …\plugins\Data Compare and copypaste the lines around the crash including the error here in this thread?

  2. Which version of Toad are you using and which connection is that on?

Thank you!


#3

Hi Dan,

Thanks for a follow up.

Currently used Toad Version: 2.0.1.209

When I tried using the 2.1 version the data compare seemed to be less stable with more errors (mostly unknown data type errors) so I switched back to 2.0.

I am using Toad with Sybase 15.5 (2602 (EBF 20630 SMP ESD#5.2))

Below is the log output

////////////////////////////////////////////////START LOG OUTPUT SNAPSHOT ////////////////////

8/8/2014 11:19:31 AM; Info; Comparision completed.

8/8/2014 1:59:22 PM; Info; Querying record count.

8/8/2014 1:59:22 PM; Info; Comparision started.

8/8/2014 1:59:23 PM; Info; Comparision completed.

8/12/2014 3:13:42 PM; Info; Querying record count.

8/12/2014 3:13:42 PM; Info; Comparision started.

8/12/2014 3:13:54 PM; Error; Table Reading error.

8/12/2014 3:13:54 PM; Error; ----------------------------------------------- Exception -----------------------------------------------

8/12/2014 3:13:54 PM; Error; System.OutOfMemoryException

8/12/2014 3:13:54 PM; Error; Exception of type ‘System.OutOfMemoryException’ was thrown.

8/12/2014 3:13:55 PM; Error; StackTrace : at Sybase.Data.AseClient.AseDecimal.ToBcdDecimal(Int32 outputPrecision, Int32 outputScale, MathFunctions mathFunction)

at Sybase.Data.AseClient.AseDecimal.ToBcdDecimal()

at Sybase.Data.AseClient.AseDecimal.ToString()

at Quest.FastData.SystemDatatypeConversion.ConvertItem(Object val)

at Quest.FastData.SystemDatatypeConversion.ConvertItems(Object[] row)

at Quest.FastData.SystemDatatypeConversion.DatatypeLoadDataRow(DataTable table, FastTable fastTable, Object[] originalRow, Object[] newRow)

at Quest.FastData.SystemDatatypeConversion.LoadDataRow(DataTable table, FastTable fastTable, Object[] originalRow, Object[] newRow)

at Quest.FastData.FastTable.GetDataTable(IFastConversion conversion, IFastTask& task, FastDataTaskStartedHandled started, Func`3 filterPredicate)

at Quest.FastData.FastTable.GetDataTable(IFastConversion conversion)

at Quest.DataCompare.PLFastData.FastDataCommand.ExecuteDbDataReader(CommandBehavior behavior)

at System.Data.Common.DbCommand.ExecuteReader()

at Quest.DataCompare.PL.ComparisonEngine.OpenSecondQuery()

8/12/2014 3:13:55 PM; Error; ---------------------------------------------------------------------------------------------------------

8/12/2014 3:13:55 PM; Error; ----------------------------------------------- Exception -----------------------------------------------

8/12/2014 3:13:55 PM; Error; System.OutOfMemoryException

8/12/2014 3:13:55 PM; Error; Exception of type ‘System.OutOfMemoryException’ was thrown.

8/12/2014 3:13:55 PM; Error; StackTrace : at Quest.DataCompare.PL.ComparisonEngine.ExecuteQuery(ObjectWithColumns iTableSource, ObjectWithColumns iTableTarget, String sQuerySource, String sQueryTarget)

at Quest.DataCompare.PL.ComparisonEngine.CompareData(SaveFlags saveFlags, ComparisionOptions cmpOptions, String dataStoragePath, Guid sessionId, ICommandProgress overallProgress, ICommandProgress detailProgress, ConfirmRetryHandler retryHandler, Func`3 setValueLength, Boolean extractDataOnly, Int32 rowLimit)

at Quest.DataCompare.PL.ObjectWithColumnsPair.DoCompare(CompareTypesOptions CompareOptions, ComparisionOptions ComparisionSettings, String dataStoragePath, Guid sessionId, ICommandProgress overallProgress, ICommandProgress detailedProgress, ConfirmRetryHandler retryHandler, Func`3 setValueLength, Boolean extractDataOnly, Int32 rowLimit)

at Quest.DataCompare.PL.PLFasade.DoCompare(CompareTypesOptions cs, ComparisionOptions co, String tempFolder, Guid sessionId, ArrayList ErrorList, ICommandProgress overallProgress, ICommandProgress detailProgress, ConfirmRetryHandler confirmHandler, Func`3 setValueLength, Boolean extractDataOnly, Int32 rowLimit)

8/12/2014 3:13:55 PM; Error;

////////////////////////////////////////////////END LOG OUTPUT SNAPSHOT ////////////////////


#4

Hi ZeenKork,

thanks for a quick reply.

As a matter of fact not long ago we found the same issue as you are having and are dealing with it right now. The number of the issue is TSY-1102.

Thank you,


#5

Dan, thanks a lot for the update! Is it possible to track progress on the ticket or I should keep an eye on the release notes to know when it’s completed.


#6

No, unfortunately it is not possible to track the progress other than ask here.

Currently the issue is in testing phase, so the teams are verifying whether it has been fixed. Please, watch the resolved issues in release notes.

Also I wanted to ask you if you can add here some details about the other bugs that you mention in your post from 20 Feb 2015 at 3:37pm.

Thank you!


#7

I don’t have a newer version installed, so I can’t reproduce the issues I was seeing.

There is one thing that comes to my mind regarding the Data compare tool as well.

  1. Have at least two datasets available in Result Sets tab.

  2. Right click within one them and choose Compare To-> Set 2 (or other set available)

  3. Set Columns and Define Comparison Key dialog appears.

  4. Without changing anything click on Cancel button in right lower corner

  5. Issue: An empty Data Diff Viewer window appears even though Cancel button was clicked.

I expect it to return me back to the Result Sets tab

Thanks again!


#8

Hi ZeenKork,

Thank you for comment. I agree with you and I created task TSY-1109 for fix.

Filip


#9

I installed Beta 3.0.0.604. The data compare tool seems to be more robust and appears to work faster. I was able to compare a dataset with 50K rows and 74 columns. The tool also handled comparison for a single key column for 2 datasets with 100K rows. I got out of memory error when I selected all 74 rows to be part of the key for a set with 50K rows. It looks like you guys are moving in the right direction.

Thank you and great work !

2/26/2015 10:22:01 AM; Info; Comparision completed.
2/26/2015 10:31:52 AM; Info; Querying record count.
2/26/2015 10:31:52 AM; Info; Comparision started.
2/26/2015 10:32:03 AM; Error; Table Reading error.
2/26/2015 10:32:03 AM; Error; ----------------------------------------------- Exception -----------------------------------------------
2/26/2015 10:32:03 AM; Error; System.OutOfMemoryException
2/26/2015 10:32:03 AM; Error; Exception of type ‘System.OutOfMemoryException’ was thrown.
2/26/2015 10:32:03 AM; Error; StackTrace : at System.IO.BinaryReader.ReadBytes(Int32 count)
at Quest.FastData.FastStringSerialize.Deserialize(BinaryReader inStream)
at Quest.FastData.FastTable.DeserializeRow(BinaryReader reader)
at Quest.FastData.FastTable.RowCollection.TemporaryDeserialize(Chunk chunk)
at Quest.FastData.FastTable.RowCollection.DeserializeChunk(Chunk chunk)
at Quest.FastData.FastTable.RowCollection.RowCollectionEnumerator.get_Current()
at Quest.FastData.FastTable.GetDataTable(IFastConversion conversion, IFastTask& task, FastDataTaskStartedHandled started, Func3 filterPredicate) at Quest.FastData.FastTable.GetDataTable(IFastConversion conversion) at Quest.DataCompare.PLFastData.FastDataCommand.ExecuteDbDataReader(CommandBehavior behavior) at System.Data.Common.DbCommand.ExecuteReader() at Quest.DataCompare.PL.ComparisonEngine.OpenFirstQuery() 2/26/2015 10:32:03 AM; Error; --------------------------------------------------------------------------------------------------------- 2/26/2015 10:32:03 AM; Error; ----------------------------------------------- Exception ----------------------------------------------- 2/26/2015 10:32:03 AM; Error; System.OutOfMemoryException 2/26/2015 10:32:03 AM; Error; Exception of type 'System.OutOfMemoryException' was thrown. 2/26/2015 10:32:03 AM; Error; StackTrace : at Quest.DataCompare.PL.ComparisonEngine.ExecuteQuery(ObjectWithColumns iTableSource, ObjectWithColumns iTableTarget, String sQuerySource, String sQueryTarget) at Quest.DataCompare.PL.ComparisonEngine.CompareData(SaveFlags saveFlags, ComparisionOptions cmpOptions, String dataStoragePath, Guid sessionId, ICommandProgress overallProgress, ICommandProgress detailProgress, ConfirmRetryHandler retryHandler, Func3 setValueLength, Boolean extractDataOnly, Int32 rowLimit)
at Quest.DataCompare.PL.ObjectWithColumnsPair.DoCompare(CompareTypesOptions CompareOptions, ComparisionOptions ComparisionSettings, String dataStoragePath, Guid sessionId, ICommandProgress overallProgress, ICommandProgress detailedProgress, ConfirmRetryHandler retryHandler, Func3 setValueLength, Boolean extractDataOnly, Int32 rowLimit) at Quest.DataCompare.PL.PLFasade.DoCompare(CompareTypesOptions cs, ComparisionOptions co, String tempFolder, Guid sessionId, ArrayList ErrorList, ICommandProgress overallProgress, ICommandProgress detailProgress, ConfirmRetryHandler confirmHandler, Func3 setValueLength, Boolean extractDataOnly, Int32 rowLimit)
2/26/2015 10:32:03 AM; Error; ---------------------------------------------------------------------------------------------------------
2/26/2015 10:32:03 AM; Info; Comparision completed.