Toad World® Forums

Comparator – most bugged part of TDM?


#1

It seems for me that comparator has more bugs than any other part of TDM :frowning:
I have two db models. Both are MSSQL2005, each TXP file is about 60 MB. I wanted to compare those two models at entity-attribute name level. I realize that in my schemas there are acceptable differences at identity seeds, etc (in fact some of those low-level differences come from business requirements). I loaded first TXP, then loaded second one (it took loooooot of time), clicked COMPARE, switched to settings tab and unchecked everything. Then I picked up model2, and after some time wait I have chosen “Generate report”. The only thing I checked was “Entity”. Nothing more. And I have got some information about columns… Since I know that id fields differ only in identity seeds, why are they displayed in this report?
I think filters what to compare simply do not work :frowning: Second attachment shows part of comparator window (again “Entity” was the only things checked, subitems of “Entity” were again unchecked) … why is object dbo.Accepted listed there? It is perfectly ok to list dbo._SysTables and dbo._tmpFKFullInfo, since they do not exist in model CARECMT, but why list dbo.Accepted ?
Version 3.3.3.3….
Regards,

Michal
moz-screenshot-13.jpeg


#2

Hello Michal,

Thanks for your message.

  1. Filter in comparator + alter report. Yes, we managed to simulate the problem and need to further verify it. Thanks for your notification! CR # 54 560.

  2. If Comparator shows that item is different, it is different - any its property, something that you don’t have to know or realize, e.g. attribute order etc.
    In any case, we will further test it. CR # 54 561.

Regarding the dbo.Accepted object, please try to select other settings in comparator (not only Entity) and see what exactly is different in this object. It should be displayed then.
Thanks.

Thanks very much for your great co-operation!

Regards,

Vladka


#3

Hi vladka,

Thanks for fast response.
I know that dbo.Accepted is different. Id Fields differ in identity seed value, while instanceID differ in default value. But – maybe I do not understand the comparison rules – I wanted to compare only on entity level. Those databases are huge, they have got more than 600 tables, so the first step is to figure out if there are some missing entities (tables) in schemas. Since merging requires some business discussion, we want (and have to because there is no alter script generation for MSSQL) to do it manually. In the future, when I will be sure that both databases match at entity level, time will come to compare if entities have the same attributes and if those attributes have the same datatype. Again, this is a living system so changing datatype must be conducted with great care.
So our modus operandi is to compare databases in few steps, comparing them more and more carefully, and this is why I was happy to see possibility to filter and limit the comparison “depth”… (this is how I thinks comparison settings should work)…
And also performance stuff:
I don’t know how much memory usage is optimized, but I am using dual core Lenovo R61i 2GB notebook, and after booting up Windows takes about 700 MB of RAM. Loading TDM and one model adds next 200 – 300 MB, loading second model is next 100 MB, so TDM with two big models takes about 300- 400 MB. Turing on comparator often pumps it to 1.2 – 1.5 GB! The memory taken by comparator was not released until whole TDM was closed, so it suggests some memory leak, please check it.

Regards,
Michal


#4

Hello Michal,

Thanks for your reply.
Yes, we will definitely deal with the problem. Thanks for your patience.

Regarding alter script generation for ms sql 2005 and 2008 - it is supported in latest Beta 3.3.3.3. Please feel free to check out this new feature. In case of any problems, please write us.
The alter script generation for ms sql will be supported in upcoming full version.

Comparator and memory issues - We will check it out. CR # 54 605.

Thanks very much once again!

Regards,

Vladka + TDM Team


#5

Hi all,

I have conducted a test on Sunday. I loaded my two big models, CAREMTX and CARECMT. Then I turned on comparator, turned on alter script and went for a walk  When I came back I saw “Out of memory” exception. You can see it on first attachment (tdm1GB.jpg) (log file also attached). Task manager below is sorted by memory usage – as you can see next in the line is scanning process from ZoneAlarm and some Windows stuff. TDM takes 1 GB.
The second picture (taskmenWhenTDMDied.JPG) shows task manager graphs, eg. Memory usage graph.
The third (closingtdm.jpg) shows closing TDM after all Out of memory error – notice “The grand canyon” on the right where TDM process was closed.
It would help if you put more Application.ProcessMessages in the code of comparator  Or maybe some progress bar? I’ll check how much RAM I can add to this machine…

Regards,
Michal
tdm1GB.jpeg


#6

Hi all,

I have conducted a test on Sunday. I loaded my two big models, CAREMTX and CARECMT. Then I turned on comparator, turned on alter script and went for a walk  When I came back I saw “Out of memory” exception. You can see it on first attachment (tdm1GB.jpg) (log file also attached). Task manager below is sorted by memory usage – as you can see next in the line is scanning process from ZoneAlarm and some Windows stuff. TDM takes 1 GB.
The second picture (taskmenWhenTDMDied.JPG) shows task manager graphs, eg. Memory usage graph.
The third (closingtdm.jpg) shows closing TDM after all Out of memory error – notice “The grand canyon” on the right where TDM process was closed.
It would help if you put more Application.ProcessMessages in the code of comparator  Or maybe some progress bar? I’ll check how much RAM I can add to this machine…

Regards,
Michal
closingTDM.jpeg


#7

More files…
outofmem.txt (57.8 KB)


#8

More files…
taskmenWhenTDMDied.jpeg


#9

>It would help if you put more Application.ProcessMessages in the code of comparator  Or maybe some progress bar?

Hi Michal,

Thanks for the information.
Yes, it is a problem, we know. We will deal with it. Thanks very much for your patience!

Thanks. CR # 54 769.

Regards,

Vladka + TDM Team


#10

I have encountered new error using new beta. The error message is
“Thread creation error: W magazynie brak miejsca dla wykonania tego polecenia.”.

Here can be found MS-translation of this error message:
http://support.microsoft.com/kb/928202/pl

(there is the only MS web page within Microsoft.com with this phrase). Full description in attachement.

Regards,
Michal
20081022.txt (81.2 KB)


#11

Hello Michal,

Unfortunately, I have bad news. There is not enough memory to accomplish the task. It is definitely a problem of Toad Data Modeler. We know about it and will work on it.
Let us apologize.

Michal, we’re afraid, we can suggest only the following at the moment:

  1. Re-install TDM and install only databases that you work with. (If you do not install all database systems that TDM supports, fewer system packages will be loaded. This could help significantly.)
  2. When you know you need to compare two big models, please do the comparison as the first thing in TDM (in other words, do not perform many other tasks in TDM before the comparison).
  3. Reduce number of opened applications to minimum (if possible).

Well, you may try these tips. Let’s hope they will work for you.

Regarding to memory consumption and speed optimization, let me repeat - we know it is a problem of TDM and we will work on this.

Michal, if it is possible, please send us the two big models you work with - they will be used for the testing purposes. Thanks very much in advance.
(Please send them to: modeling@quest.com).

Let me return to the problems related to model compare you reported. - CRs 54 560 and 54 561 (above in the thread).
Our developers made many changes ad fixed bugs in Convertor/Comparator. When we got the Beta 3.3.4.9 for testing, we failed to reproduce the two problems again. So, we can’t say that the two CRs were fixed. (It seems they were fixed while another problem was fixed…) Now we’d like to ask you to try out if you encounter the problems in current Beta again or not. We keep failing.
Thank you very much for your great co-operation!

Michal, if you have any questions, please do not hesitate to ask.
Thanks very much for your patience and understanding.

Regards,

Vladka + TDM Team


#12

Hello vladka!

I have conducted few tests. Our test machine is DELL Optiplex 745 workstation with 4 GB RAM and Win XP SP2.

Test one: alter script. Result – the same :frowning: Exception after 22 minutes of work. TDM allocated about 2 GB of RAM and crashed. Log shows that there were still about 1 GB of free RAM. So closing other apps won’t help. Alter script generation tries to allocate more RAM then TDM can handle :(. File: memleft.txt

Test two: minimal installation of TDM and alter script. Result: the same. Only this time it took slightly more time to crash. File: simple.txt.

Test three: report generation. This was quite ok, but there are some things I do not understand. I turned on comparator and checked only Entity (01.jpg). Wait some time to do comparation, clicked report, and again clicked only Entity. So I wanted ONLY to have differences reported on entity level – I did not want a report of Indexes, Contstraints, and so on (02.jpg). So, the report was generated: you can see huge differences between systems (03.jpg). We have totally 157 missing tables on both databases. This is why I wanted only to compare on entity level – I am concerned about those 157 tables first, I will check those 491 unequal (but existing in both databases) tables later.
And on picture 04 a piece of report can be seen: dtproperties is missing in one schema, and it is reported perfectly. It is ok – I wanted to know such things. But table CaseEvents … why it was reported? I did not checked Indexes or Constraints on “What to report page”, and I still got indexes and constraints reported. Maybe I do not understand the idea, but there should be a difference between:
a) what to compare
b) what to report

Regarding dbmodels, I am now working to get appropriate permissions to send you those TXP files.

Regards,
Michal
simple.txt (62 KB)


#13

Hello vladka!

I have conducted few tests. Our test machine is DELL Optiplex 745 workstation with 4 GB RAM and Win XP SP2.

Test one: alter script. Result – the same :frowning: Exception after 22 minutes of work. TDM allocated about 2 GB of RAM and crashed. Log shows that there were still about 1 GB of free RAM. So closing other apps won’t help. Alter script generation tries to allocate more RAM then TDM can handle :(. File: memleft.txt

Test two: minimal installation of TDM and alter script. Result: the same. Only this time it took slightly more time to crash. File: simple.txt.

Test three: report generation. This was quite ok, but there are some things I do not understand. I turned on comparator and checked only Entity (01.jpg). Wait some time to do comparation, clicked report, and again clicked only Entity. So I wanted ONLY to have differences reported on entity level – I did not want a report of Indexes, Contstraints, and so on (02.jpg). So, the report was generated: you can see huge differences between systems (03.jpg). We have totally 157 missing tables on both databases. This is why I wanted only to compare on entity level – I am concerned about those 157 tables first, I will check those 491 unequal (but existing in both databases) tables later.
And on picture 04 a piece of report can be seen: dtproperties is missing in one schema, and it is reported perfectly. It is ok – I wanted to know such things. But table CaseEvents … why it was reported? I did not checked Indexes or Constraints on “What to report page”, and I still got indexes and constraints reported. Maybe I do not understand the idea, but there should be a difference between:
a) what to compare
b) what to report

Regarding dbmodels, I am now working to get appropriate permissions to send you those TXP files.

Regards,
Michal
memleft.txt (65.3 KB)


#14

Att.

MB
01.jpeg


#15

Att.

MB
02.jpeg


#16

att.

MB
03.jpeg


#17

att.

MB
04.jpeg


#18

BTW: we have managed to extract desired information from difference report simply by opening HTML source, sorting it and extracting lines with „Object blahblah is missing in model….”

MB


#19

Hello Michal,

Thanks very much for taking the time to check out the problems in current Beta.
I’m sorry to hear that none of the tips worked. I’m terribly sorry I don’t have another suggestion/solution for you.
Our developers will deal with the problem of memory consumption, however need more time. Thanks very much for your patience.

Comparison report: We have discussed the issue of comparing only entities. During the comparison, TDM considers entity as an object together with its properties. This is the reason why the comparator returns e.g. that entityX is not equal with entityY - e.g. because of different indexes or PKs etc.

If you have any questions, please write me back.
Thank you once again for your great co-operation!

Regards,

Vladka


#20

I have encountered strange problem.
I have created a database in TDM. I have made a script, created DB in MSSQL 2005. This database has a table named OSOBY. Today, I have created new table - ZATWIERDZAJACY_ZLECENIE. I have created a relationship from table OSOBY to my new table. Since OSOBY PK is named OSO_ID TDM created field OSO_ID in my new table. Since all fields in my new table have prefix ZAT_ I have changed OSO_ID to ZAT_OSO_ID and also clicked the button to update caption accordingly.
In order to create alter script I did RE of current database and created alter script between my TDM model and RE model of current database. And… I have one extra field, see the attached script and screenshot of a model. Relationship56 is also wrong…
This happened in latest comercial, but since this thread is here, I post this message here.

Regards, Michal
workspace.jpeg