Toad World® Forums

TDP integration with Snowflake

Toad Data Point
#1

We're exploring Snowflake and it looks like it doesn't provide for workflow/automation capabilities. Not knowing too much about Snowflake, I don't see any special integration between TDP and Snowflake other than at the data source level: Snowflake creates the data sources and brings in the data while TDP creates the automation workflows and gets the data it needs from the Snowflake data sources (not really an integration if you were to ask me!).

Just for the sake of integration, one could probably try to create a workflow in TDP which will call out (maybe using the "Run Program" activity) to Snowflake tasks just to say that TDP talks to Snowflake directly. Even if possible, and I doubt it, I wouldn't be a fan of this given the tight coupling between TDP and Snowflake tasks and the nightmare of readjusting if one of the tools gets replaced by another tool.

Anyway, are there any architecture or design patterns for TDP-Snowflake integration? I'd appreciate any input on this. Thanks!

#2

Hi, Snowflake is not supported yet but we have been working on its support with TDP.

#3

Thanks, Dan. Out of curiosity, once the TDP-Snowflake integration is supported, what would be the approach or the main use-case for a TDP workflow to connect and use data from a Snowflake data source?

For example, let's say that I have an Oracle database which I move from my on-premise data center to Snowflake/AWS. How would I reuse my TDP workflows to now work with the Snowflake data sources?

#4

For starters, your workflow steps would need to be revisited/modified if they use employ some connection to a data source. For example, Export/Import tasks, etc. will all need to be modified to specify the correct source and target connections.

If the tables, column names/types, etc. in Snowflake are the same as in Oracle, you might get by with minimal changes to your automation flow, depending on how sophisticated your task logic is.

#5

Thanks, Gary. To keep things simple, if we assume that everything in Snowflake (data source name, tables, columns, etc) is identical with the on-premise environment, would my TDP workflow need to change? My workflow has the following activities: connection to data source, SQL query, export results to Excel, zip Excel file, email Zip file.

In other words, does Snowflake add any layer/barrier in TDP accessing data sources within the Snowflake environment versus TDP accessing data sources within an on-premise data center?

#6

Hi...
Please reach out here-Snowflake Community.

Regards,
Srija,
Snowflake Developer

#7

I would love to get back to using TDP, It is my favorite tool, but the ODBC driver is bad so I had to switch products

#8

Hi Chris, What do you mean the ODBC driver is bad? do you mean the embedded ODBC drivers of TDP have any issues?
And you can install your own ODBC driver in windows and connect to remote datasource by the generic ODBC connector in TDP.