Sync Out for Snowflake

3.7
(3)

The Tableau CRM team spent a bunch of time in 2020 working with our partners at Snowflake, and as a result of the collaboration, we came up with a unique feature that will allow Salesforce customers to send raw data to Snowflake for data lake hydration purposes.

Tableau CRM already had the key pieces needed to make this work:

  • A mechanism to pull data from Salesforce via the Bulk API
  • A write connector to send data to Snowflake.

So our engineers pieced the two features together to give an ability to leverage the data sync schedule, incremental load support, and execution along with a crafty auto DDL (table create and recreate) execution to create Sync Out for Snowflake. Enabled with a few clicks, this is a great way to keep the data in Snowflake refreshed. 

Note: In case you prefer a video rather than reading or in conjunction with? Then please check out the Two Minute Connector Series.

Not all customers would want all the raw data in Snowflake, so they can use the same Output Connector to Snowflake to write curated data to targeted tables via Data Prep Recipes.

  • The Data Sync for your Local Salesforce Data allows you to pull data from Salesforce into Tableau CRM.  
  • The Snowflake Output Connector can be used to write data to a Snowflake table of your choice via a Data Prep Recipe; this connector is also how you create the connection for Sync Out to use. 
  • The new topic here: Our Spring ’21 feature Sync Out for Snowflake lets you write the data to Snowflake based on the execution of Data Sync. 

Pro Tip: Data Sync can be scheduled at your desired interval (no lower than 15 min) and you can setup a Recipe to execute based on Data Sync success; so you can use Sync Out and the Output Connector in a unified strategy (some raw data and some curated)

Setting up Sync Out

The setup for Sync Out is quite simple assuming you are familiar with the concepts and features of Data Sync.

  1. Have your Admin enable the Connector in Analytics Settings by selecting Enable Snowflake output connection.
  1. Create a Connection in Data Manager using Connect
  1. Enable Sync Out on the Salesforce Local Data Sync Connection(s) you wish.
  1. Lastly, when your Data Sync runs on-demand or via schedule, the data is written to Snowflake (A manual run could be scheduled as well). Once you have run the data you can head to Snowflake to see the new tables that have been created.

That’s how you can keep your Salesforce data up to date in Snowflake without the need for a third-party ETL tool.

How useful was this post?

Click on a star to rate useful the post is!

Written by


12 thoughts on “Sync Out for Snowflake”

  • 1
    Nicola Mitcham on April 12, 2021 Reply

    Hi Chris, thanks for the post, would this require a Tableau CRM subscription just to access the sync out to snowflake connector?

    • 2
      Chris Ames on April 12, 2021 Reply

      Nicola, Sync Out is a feature of Tableau CRM therefore you need to be licensed for Tableau CRM to use it.

  • 3
    Mike Peppou on April 13, 2021 Reply

    Hi Chris
    Thanks for this. We are lapping it up and thinking of all sorts of use cases for writing data back to salesforce. Data Prep is such a powerful tool for ingesting and cleaning data!
    Curious if you have considered using it as a data integration tool? E.g. to pull in an external data source, fix up the data, then write to Salesforce. This would appear to work well when you are writing to single objects – trickier if you are creating related records I expect. In our case we have a staging object that we use to stage our integrations so it should work.
    Our scenario is web donations that sit in an S3 table.
    Thoughts?

  • 5
    Joe Thomas on April 23, 2021 Reply

    Chris, do you think this could be used by the customers of a TCRM OEM partner like us at FinancialForce, would love to chat about this with you and some of my dev team.

    • 6
      Christopher Ames on June 2, 2021 Reply

      Joe, no reason why not it is TCRM functionality. Reach to out to discuss as needed.

  • 7
    Phillip Schrijnemaekers on June 1, 2021 Reply

    Greetings, Chris.

    Outstanding post. Quick question: what type of exception is used in the outbound connector to snowflake?

    • 8
      Phillip Schrijnemaekers on June 1, 2021 Reply

      Encryption* damn autocorrect. What type of encryption is used in the outbound connector to snowflake

      • 9
        Christopher Ames on June 2, 2021 Reply

        Philip, we don’t publicly disclose security/trust details out of practice. So I can’t tell you the “type”. We do have customer self-service on our accreditations which allow you to see the details of our core requirements including TLS 1.2 or greater for data transfer encryption. Keep in mind we enforce at a product level not a feature by feature so we can’t take something to market without adhering to the set standards and levels.

        https://compliance.salesforce.com/en/services/salesforce-services

  • 10
    Nic on June 18, 2021 Reply

    Hi Chris

    Follow up question, is it a limitation that formula fields created on SF Objects don’t sync out to snowflake?

  • 11
    Dilip Dhar on November 15, 2021 Reply

    Very unfortunate that it required TCRM subscription to get your data across to Snowflake. What if I have a Sales/Service cloud subscription and want to move my data to Snowflake?

  • 12
    Mridul Pandey on August 9, 2022 Reply

    Hi chris,
    Thanks for the article – super helpful
    Wanted to confirm a small detail – is this sync-out feature able to handle schema changes?
    Thanks in advance!

Leave a Reply to Christopher Ames Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.