Update model for selected schemas/objects and RE errors

Tested on TDM 4.1.5.8, postgresql 8.4

I have original model created by RE using TDM4.0.
When using function Model Update I have used opportunity to select tables/views/functions to compare. I have used schema filter, and then check all tables/views/functions for selected one schema.

After RE I can see objects which have not been selected. Those objects are originated even from another schemas. Looks like even if I selected objects to compare, comparing is performed against all objects in model.
So, let's say, if I have 100 schemas in model, and want to update single one, still objects originated from 99 schemas will be reported as missing and moreover SELECTED TO BE REMOVED from model!!! Of course '(not exists)' comment is not true. Objects exists but have not been selected to RE
Is it really intended to screw my model up, force me to manually uncheck all objects from 99 schemas or I'm missing something?

I attached RE settings as pictures and result screenshot

Additional thing, in the RE stage I got a lot of errors like:
'Error during assigning data type to attribute "trigger_schema". User datatype "information_chema.sql_identifier" is not in model'.
There are more similar notices, referring to trigger_name, event_object_catalog, event_object_schema event_object_table, action_condition and so on.
Log from RE process attached also.

with regards

ReLog.txt (13.2 KB)

Hi Michal,

believe me that there is no intention to screw anyone’s model.

To put it clear. You are doing Model Update. What type of objects are being reversed although they were not selected or they are from a not selected schema? Which objects were these? They for sure shouldn’t be reversed but it may happen. We will investigate on that further.

Model Update is de facto RE + Model Merge and thus, as a workaround, you can use wildcard filter and uncheck them- it’s the easiest way and we already have CR registered for this.

Concerning “Unknown datatype information_schema.sql_identifier in attribute trigger_catalog”:
Information_schema is a system schema that is not loaded deliberately (we do not expect any user objects to be saved there). Is the attribute “trigger_Catalog” in any of your tables and does it have reference to the user data type “information_schema.sql_identifier”? If you know which attributes, we would appreciate to know to be able to investigate on it further.

Thank you very much for your patience and cooperation, Michal.

Regards,
Lukas & R&D Team

As I wrote, I selected a few objects (out of more), and only those has been reversed. Which is OK.
But my model contains more objects which has not been intended to be updated. That’s objects has been described by Update function as “not present” on “RE side” and marked to be deleted.

If you want I can prepare some reference material to reproduce this issue. But I hope it is straight clear.

About second part:
How can I check “trigger_Catalog” attribute presence?
I hear about this for the first time. But as model is created based on existing system, everything is possible.

thanx
with regards

Ahh, btw: TDM does RE of system schemas: pg_temp and pg_toast
I know that switch for excluding system objects doesn’t reffer to schemas, but excluding system schemas would be usefull as well.

Hi Michal,

sorry for not having responded instantly, but I have been really busy.

So to the issue. In the first part, in RE, the selection serves to omit only those parts of the database that are not in the destination model at all - to make the RE faster. It isn’t supposed to serve as selection of objects you want to update. This selection for update is made in the phase of model update itself in the differences grid, where you should only uncheck the Action checkboxes for those differences/objects that you do not want to apply.

And concerning the system schemas reversed- you mean just empty schemas being reversed, don’t you? Is there any functional reason or is it just not to have them in the model? We will consider it, but as it is not anyhow disturbing functionally, it’s of low priority.

Regards,
Lukas

Hmm.
So to the issue. In the first part, in RE, the selection serves to omit only those parts of the database that are not in the destination model at all - to make the RE faster. It isn’t supposed to serve as selection of objects you want to update.
Of course, why to make things easy if they may be done as complicated as possible? :wink:
This selection for update is made in the phase of model update itself inthe differences grid, where you should only uncheck the Action checkboxes for those differences/objects that you do not want to apply.

If I have 100 schemas in a model, and want to update only single one… do you think I am ready to pay a time to uncheck thousands of objects in various branches (entities, relations, functions, etc). Are you realy thing it is a correct way?

It is realy hard compare ONLY selected objects?

Is there any functional reason or is it just not to have them in the model? We will consider it, but as it is not anyhow disturbing functionally, it’s of low priority.

Try to answer a question: why should I want to RE 50 temporary/system schemas like pg_tmp or pg_toast? Do you think it may be usefull for something? I need a fast way to document db systems. It doesn’t help at all.

Hmm.

So, of course, the easier the better, but the development has been always quite fast and we tried to implement the functions in an easy way and therefore Model Update function works that way. Of course it would be better to develop individual components optimized for the particular function (RE, Sync&Convert, Model Update etc.). Believe me that R&D Team are doing their best to match your, customers' requirements as long as there are sources for it.

For the problem with updating just objects in one schema (or more as you can repeat it) see the screenshot I took, uncheck the Action checkbox at the top which unchecks all action checkboxes of changes and use the filter as indicated, which will select objects/changes of only the one schema. Ain't it easy?

And concerning RE of those temporary system schemas- RE of those empty system schemas doesn't take almost any time, so it's not about how fast RE works but of course, it's no use there and we will consider removing them. They were left there only because it didn't do any harm and there have been more important things and issues to solve.

Thanks for your priceless feedback, Michal!

Regards,
Lukas

Thank you for your tips but it is not enough. I hope thanx to this discusion the product will be better in near future. Unfortunately current state is not enough for productivity. It is just a bunch of functions - just some study rather then tool helping daily work. Sorry to say.

Now, I found a few issues.

  1. I have some model incl. entities, functions etc and have stored alias for RE.
  2. I’m selecting this alias and press load
  3. TDM connects to db giving me oportunity to select tables and functions.
  4. I’m selecting ONLY function and hit Finish
  5. What is my surprise seeing that all db objects are reversed (indexes, fk’s etc)… Finaly I feagured out why… but I’ll back to this issue in a moment.
  6. After a few minutes (yes… my db has hundreds of objects, I wanted to compare only 12), I have to choice comparison rules and then “select objects type and properties” My choose is “Compare All without graphics”. After comparing… guess what: Note Lines has been compared… What a stupid… I have no lines in physical database, and because of that Updater suggest me to remove lines from model.

After that, it is not surprising to me, that functions are reported to be different, just because my model contains Notes for function objects, which are of course not present in database.

Come back to 5th point for a while. After selecting and loading an alias, the page is switched to selecting tables/function. Again I have no idea what is an intention to NOT show other settings - type of objects selection to RE. Yes… it is available… you have to hit “Previous” button twice. Who the hell will come back to aliases page, and hit “previous” button again (which will not be present on that page while selecting alias) to set up object types? Do you think it is logical and stright forward? Any of my coleagues denied such advance as correct one.
Additionaly there are issues while using previews/Next buttons a few times. sometimes It resulting with doubled tables/function on selection list. But this is minor issue comparing to overall problems.

Sorry guys. But I have to say… current stage of TDM doesn’t deserve v4 number. Not even v2 (excluding v1 and v2 reserved for CaseStudio). It is still alpha stage of some main features. Even if you can call it “secondary features” those should not be present in release untill fixed.
But I have a feeling you are fine with such solutions, doesn’t matter it’s not intuitive, work in wrong way etc, and I’m going tired reporting things again and again.

TDM should help me on my work, not assigning me as beta tester or forcing me to constantly feaguring out how to dodge things to get expected result. If I want to compare one single object - it should compare only one given single object - especialy if there is an option to select this object to compare. If I’m comparing database to model, I’m expecting that any objects which cannot be present in database (graphical objects,notes etc) will not be compared at all. Currently they does for some even if I select an aprioriate option.

Please, be honest and say what will you do in near future to get TDM working as expected.

with regards

The story continue…

I did everything I should

  • after selecting and loading alias, a press ‘previous’ 3 times.
  • i have selected only functions to RE
  • go forward and selected only 12 functions from chosen schema
  • and perform comparing.

In results:

  • NameMode in DB is different than NameMode in model (1 vs 2). I don’t know what it is. But model is created by using RE. So those values should be the same even if I don;t know what it is.

  • There is entities in model reported to be deleted. We talked about this issue before. It is unacceptable!. There is no matching mask to disable all entities even in a few passes. Yes… I can disable all and then manualy enable only functions I want to compare. Do you see any sens of performing previous selections, if I have to do it again on comparison list?

Additionaly I don’t understand while unchecking child table (with FK), also related relation ship and parent table is unchecked automaticaly. OK - you may argue about sens of comparing FK relation. but why parent table?
It is not my case though, but noticing it as possibly p.i.a for somebody.

  • Difference in functions which are the same. But model contain Notes for functinos - I wrote about it before.

  • Note lines - there is no such obj on DB. and I don’t want to remove those lines from model. Wrote about it before.

  • InstanceUserGropuReferences - I didn’;t wanted to RE or compare those ones

now I feagured out that after RE I can select object types to compare selecting Detailed settings. But still I think selecting the same thing twice, jumping between pages forrward and backward and in result still got issues is a bit too much.

Hi Michal,

I think the problem is that you use one single model, one big database and that you try to update just a part of your model using Model Update.

That brings several problems and probably the key problem is that for the software there is no difference between objects that were deleted from database and objects that were not selected for comparison/synchnonization.

Example: you have model A with table Customer. During Model Update another model (model B) is created (in fact we need to do reverse engineering to get another model to be able to compare the two models) and in the second model table Customer doesn’t exist. Currently the software is not able to find out why table Customer is missing in the second model. One solution might be in doing reverse engineering of all objects during model update and marking “unselected” items somehow. But I don’t think this would be a solution that will improve your productivity.

Also, there are ways to get the same result already.

I will not recommend you to split your model to smaller parts, for some cases this can be pretty good solution, but I know sometimes it is necessary to manage a model with hundreds of schemas. Let’s take it as a must. What I can recommend you is the following:

A) Keep managing your large model, but instead Model Update do a full RE of your database and then compare just the items you wish to synchronize, like functions for example. In this case, Toad Data Modeler will be able to recognize objects that exist in your original model as well as in the reversed model. Problem of “missing/not-selected” and “missing/deleted” objects would disappear. Also, this way you can keep just one alias for RE with all items selected and without the necessity to use back/forward buttons. And if you do not want to compare logical information, like Notes on functions, this can be easily defined in Sync & Compare Wizard (button Detailed Settings, step Select Object Types).

B) If you wish to compare only functions, create new model and copy/paste there only functions from your original model. Then do RE of your functions and compare the two small models. This might be the fastest and “acceptable” solution (if we take productivity factor into account).

RE version number: I disagree and I bet a lot of Toad Data Modeler users would disagree too. I can imagine that we meet somewhere and discuss it in person. \

Other bugs and improvements: I do not know any software that covers all users expectations and is totally bug-free. I personally tested various high-priced competitive products. If you had the same knowledge/experience you would be surprised :slight_smile:

I’d like to ensure you that we do maximum to bring as many powerful features as possible and fix bugs. BTW: Did you notice that we added to the product some improvements based on your feedback?

Let me apologize for not following up all items in your post - one by one. I think some items will be fixed in future, but now I am not able to say what will appear in next version, what will be removed (in this case I suppose that ideal solution would be in removing model update feature and force you to use improved model merge instead) etc. Thank you for understanding it.

Re alias: some data are stored with alias. This is good if you plan to use the alias with exactly the same settings, otherwise you should create multiple aliases for the same connection. Imagine there are users who has a single model, single database, do changes into a model and forward the changes to database. Only critical changes are made to database first. Model update is in this case a feature that just brings minor changes to exsiting model and is used in scenario whole model-whole database…

Finally let me thank you for your feedback, thank you for the time you spent on reporting TDM features and thank you for all your input.

Have a great day,

Vaclav