How to clean up disk after data compares?

C:\Users\Toad\AppData\Local\Quest Software\Toad Data Point 5.6Quest_DC_Temp

I went out to my TIC server after it was reporting that it was out of disk space. There was 14 GB of junk in this directory. Does TDP just do a poor job cleanup up it's temp files, or is this related to other issues I'm having, in which data comparisons just hang up and never finish?

Very likely what you said... sounds like the Quest_DC_Temp dir is related to TDP's Data Compare, probably via automation jobs that contain Compare tasks. If your Compare jobs are hanging, then this likely means that the volume of data to compare is very large, and there's not enough disk on TIC to process the Compare. If TDP gets trounced, it may not get the chance to reclaim the temp space. I'm guessing it's ok (if no other TDP/TIC jobs or TDP clients are running) to empty this folder and re-start, to see if this fixes the issue (unless your compare volume is larger than available disk, that is.)

PM and dev teams take note.

Meanwhile, any chance to increase the temp disk footprint on the TIC server?

This happened on Sunday - the disk filled up and alerted our support team, who allocated an additional 10 GB of storage.

How much disk space is reasonably required? I'm able to run these on my own workstation, and I don't have as much storage as available as the TIC server.

Does running multiple Data Compare Automations in a larger Automation flow prevent it from reclaiming the disk space as each individual task completes?

Sounds like you've opened up a Support ticket, and i just confirmed that the engineers are conferring with the dev team on this issue, so stay tuned.

I'm curious, however, on a couple of things...

  1. How many data compares (tasks) are you running in your automation job?
  2. Are they being serially executed, or are they running in parallel?
  3. Temp space is being reclaimed properly when the job is running on your workstation?

My team is tasked with comparing the rows in approx. 1300 tables that are involved in a CDC process between DB2 and SQL Server. We've split them up by table prefix into 41 groups - not equal sized. Some groupings have 5 or fewer tables, while others have several dozen. There are three of us working on this and we are each creating a "Master" automation. The "Master" automations execute their tasks in a serial manner. We've come to the conclusion that any grouping that contains a single table (or multiple tables) which require several millions of rows being compared should not be in the Master flows and are scheduled in intervals of 30 or 60 minutes apart.

My Master automation consists of 8 smaller automations.
A second Master automation currently consists of 6 smaller ones.
A third Master automation currently consists of 3 smaller ones (he's been on vacation).

These three Master automations are all scheduled for 1:00 AM on Sundays.

On our local workstations, when the data compares themselves complete successfully, we see the temp directories being deleted. When the data compares either crash our TDP or cause it to freeze up and be exited from Task Manager, the temp directories require manual cleanup.

I haven't submitted the large Master automation on my workstation to see what happens in my temp directory.

With our drive mapping standards, we really don't want to expand the C:\ drive much larger. There's a Y:\ drive mapping with 1.5 TB free, but I cant' get the TIC server off the C:. I've logged in as the Run As user and changed it on the TDP instance, but when I retested, it didn't use the drive mapping I specified.

How can I change this?