We're exploring Snowflake and it looks like it doesn't provide for workflow/automation capabilities. Not knowing too much about Snowflake, I don't see any special integration between TDP and Snowflake other than at the data source level: Snowflake creates the data sources and brings in the data while TDP creates the automation workflows and gets the data it needs from the Snowflake data sources (not really an integration if you were to ask me!).
Just for the sake of integration, one could probably try to create a workflow in TDP which will call out (maybe using the "Run Program" activity) to Snowflake tasks just to say that TDP talks to Snowflake directly. Even if possible, and I doubt it, I wouldn't be a fan of this given the tight coupling between TDP and Snowflake tasks and the nightmare of readjusting if one of the tools gets replaced by another tool.
Anyway, are there any architecture or design patterns for TDP-Snowflake integration? I'd appreciate any input on this. Thanks!
Thanks, Dan. Out of curiosity, once the TDP-Snowflake integration is supported, what would be the approach or the main use-case for a TDP workflow to connect and use data from a Snowflake data source?
For example, let's say that I have an Oracle database which I move from my on-premise data center to Snowflake/AWS. How would I reuse my TDP workflows to now work with the Snowflake data sources?
For starters, your workflow steps would need to be revisited/modified if they use employ some connection to a data source. For example, Export/Import tasks, etc. will all need to be modified to specify the correct source and target connections.
If the tables, column names/types, etc. in Snowflake are the same as in Oracle, you might get by with minimal changes to your automation flow, depending on how sophisticated your task logic is.
Thanks, Gary. To keep things simple, if we assume that everything in Snowflake (data source name, tables, columns, etc) is identical with the on-premise environment, would my TDP workflow need to change? My workflow has the following activities: connection to data source, SQL query, export results to Excel, zip Excel file, email Zip file.
In other words, does Snowflake add any layer/barrier in TDP accessing data sources within the Snowflake environment versus TDP accessing data sources within an on-premise data center?
Hi Chris, What do you mean the ODBC driver is bad? do you mean the embedded ODBC drivers of TDP have any issues?
And you can install your own ODBC driver in windows and connect to remote datasource by the generic ODBC connector in TDP.
Hi @dan, would you have any news since your January post regarding the timing / availability of a native Snowflake Toad Datapoint (TDP) connection release? Thanks in advance.
Hi folks - we are expected to have a new native connector for Snowflake in Toad Data Point by the end of this year. This would allow Toad Data Point Pro users to connect to Snowflake as they do with any other source in Toad Data Point. We are also expecting to support cross-query with Snowflake as well as import/export with Snowflake as either a target or source. If you are currently using Toad Data Point and are looking to use Toad Data Point with Snowflake, we would love to reach out for a conversation to understand your Snowflake env and your use cases! you can email me directly at julie.hyman@quest.com