Snowflake Streams for Staging Increments

urn:js:virtue:aspire:pattern:.

TL;DR

A pattern for helping a transform view identify data in a staging table to read when referenced. The pattern is one stream per transform view where a staging table can have 1 or more streams.

Instructions

For each transform view:

1. Create an Append-Only stream over the staging table the transform view will read from.
2. Write the transform view SQL ensuring it reads data from the stream created from it.
3. Create the insert/merge statement to take the data from the transform view and load it to the target.

Diagram

Appendix

Optional - Misc details related to what is being proposed.