We have Toad Data Point v5.6, and I have an issue with the "Find and Replace" File Activity deleting the file that will eventually be used for the "Send Email" activity that uses that file for the Body of the email.
I am using the "Find and Replace" activity that has 2 "Find/Replace" steps within it that is to update 2 variables, "YesterdaysDate" and "ReceivedDate" within a HTML file that will be used for the "Body" of the email. What I am finding is happening is, and this doesn't happen all the time, but does happen quite often, when it does the 1st update and sets the variable "<<'YesterdaysDate'>>" (sorry I couldn't get the variable within the less-than/greater-than characters to show correctly without having to duplicate the characters and also to wrap the variable within single quotes) and saves the file as "EmailBody_Today.html" that works perfectly fine. However, when the 2nd Find/Replace step runs that uses the "EmailBody_Today.html" as the "Source File", the file is mysteriously deleted and then the job fails because the "Source File" cannot be found.
So what I have done to work around this, as this happens a lot with other jobs as well, is I have backup version of the HTML file in the folder where this HTML file resides, named "EmailBody_Today_Backup.html" and when this error happens, I simply make a copy of the "Backup" file and rename it to "EmailBody_Today.html" and then rerun the job, and it works. Everytime I do this, it works without error, which is very strange to me as this doesn't really make a whole lot of sense. But it works and that's what I am having to do until we can figure out why this is happening like this.
Again, this doesn't happen all the time, but it does happen a lot more than expected.
I'm gonna sound like a Support guy, but have you been able to test your automation job using the latest TDP 6.1 release? I would strongly suggest always using the latest version, for obvious reasons.
I have a very similar automation job that I run from time to time. My job also has two steps in the FIND-REPLACE task...
-
The first replace step
a. Reads in a FormLetterIN file (this file never changes, is READ ONLY)
b. Replaces the first variable placeholder in the file with a value from a database table.
c. Saves the output to a file called FormLetterOUTfinal.
-
Second replace step
a. Reads in the FormLetterOUTfinal file as input
b. Replaces the second variable placeholder in the file with a value from the database
c. Saves the output to the same FormLetterOUTfinal file.
Incidentally, I've used both TXT and HTML file types for my FormLetterOUTfinal, and I've never had any issues like the one you're seeing. AND, I've been running this automation job here and there since TDP 5.6.
All that said, where are your email body files kept? Network shared drive? Is it possible that other apps or users or processes have access to the files?
How many occurrences of replacement are done in the first replace step? If there are MANY, then I'm wondering if TDP tries to run your second replace step while the file system still has a handle on the file and not finished with all the replaces from the first step? If that's the case, one work-around might be to use two FIND-REPLACE tasks (instead of one task with multiple replace steps.)
Unfortunately this is my work system and we don't have access to v6.1, nor do I even have access to download a newer version.
The files used for the email are on a network drive and no one else has access to these. Only the Toad job itself, so nothing else would be using the file when the job runs.
With regards to the occurrences of replacements, just like yours, mine has 2 replacement steps within a single Find/Replace task.
Just as a FYI - I have also tried to break this out into 2 Find/Replace steps instead of doing the 2 replacement steps all within 1 Find/Replace, and placed a Pause step in between, thinking just like you said, maybe the file hasn't had time to complete it's "Save" due to network traffic, and that did work for a few times, but then that too started to fail when it hit the 2nd Find/Replace task.
Just curious, does the automation log content give us any insight?
Otherwise, raising awareness to Quest R&D team. Also, would help if you could open a support ticket, for tracking purposes.
Hi Gary, the log just errors with the message below. If I look in that folder where the HTML file is to be found, it is gone. I actually watched it this morning when the Toad automation was running and the file was there when it got to that step, and after the 1st variable that was to be "replaced" the HTML file did get updated with today's timestamp, and then "Poof" the file was gone and the job failed with the error below:
Exception Messages:
Find_Replace_EmailBody_1 - Access to the path '\my network path to the HTML file\EmailBody_Today.html' is denied.
FYI - when I am replying to, or typing a new Post, how come when I put in the greater than and less than characters, anything between them is removed from the comment?
Not sure if this issue has been reported to Quest or not, but this week I am getting this error on Automated jobs that have never had this behavior happen. The job that threw this error has been working for over 3 years without any issues until this morning. The odd part is, other jobs that are running prior to this one have completed without any issues and they too have this same type of "Find/Replace" behavior step within it.