Link Search Menu Expand Document
Virtues

Snowflake Release Cycle

urn:js:virtue:aspire:proposal:56.1

TL;DR

Changes to the Snowflake platform will be announced with 3 months notice, before being implemented. Documentation will be provided describing the actions required. Notifications will be sent to all Snowflake Users via their registered user email.

Rational

We want our cloud data platform - Snowflake, to be agile and keep up with modern business needs. We want to modernise, simplify, and transform our data estate and therefore we want to improve Snowflake platfrom on regular basis. As of the time of writing, we have around 800 users of Snowflake at Sainsbury’s. We want to provide greater visibility, transparency and advance notice to all of the users ahead of any regular changes, upgrades and modernisation taking place. Changes, upgrades and platform modernisation should be done in agile, modern, transparent way that aligns with our Sainsbury’s tech and ASPIRE strategy, and in the same way that Snowflake releases changes to us:

  1. We are informed about the change, with a notice period
  2. We are provided with detailed instructions to guide us through any actions that we may need to take as a result of the change
  3. The change is then released on the stated date

We propose to implement this same visible, transparent and agile process, aligned with our strategy for our internal management of the Snowflake platform. The main difference is that we will deprecate the use of these systems after the time period and move them out of support.

This means that if users raise a support request for Snowflake and are found to be using any deprecated feature, then they will not receive any support until they have moved away from the feature in question. To make this process clear and transparent, we will be providing:

  • 3 months notice for items moving out of support
  • A confluence page detailing the upcoming changes, the date of the deprecation, and the actions required
  • We will make it as easy as possible for users to understand if they are using currently any deprecated or unsupported features with a new views in Snowflake. These will show which roles / warehouses / users etc… are currently utilising the feature, and therefore would be out of support once the date passes

Once all users have migrated away from the feature, we will then drop / disable it.

Background

Users have been signalling to us that prioritasation of work can be sometimes challenging, especially when balancing innovation, compliance, business as usual and tech debt management. Having greater visbility, transparency and advance notice on what work is happening in Snowflake platfrom and when changes are happening, and when technology is coming to the end of life will help squads to prioritise effectively and ensure high value deliveries are not impacted.

One of the motivations around projects like Barbossa was to be able to migrate away from legacy, manual processes in Snowflake. These are particularly important to reduce the burden on the very limited administration resource & funding that we have around Snowflake. We are moving to a self-serve, automated way of managing Snowflake, which will allow us to save time across the department, and improve our agility.

Benefits

  • Aligning with Sainsbury’s Tech and ASPIRE strategy of ‘Modernising, transforming and simplifying our data estate’, ‘Delivering value early and often’, and creating ‘World-class experience, effectiveness & efficiency’
  • In-advance visibility - upcoming changes, modernisations and their rollout date will be clearly published on Confluence and communicated in advance so that users know what changes are upcoming, who it will effect, and how it will affect them. This will help with squad prioritasation, backlog and capacity management
  • A clear communications line for Snowflake changes
  • Increased agility & speed as the Administration team can more easily remove tech-debt that slows down the evolution of the platform, enabling teams to progress more rapidly
  • Time saving & efficiency benefits: Snowflake & administration teams will be able to focus on high priority, high complexity use cases
  • Better user experience

Impact / Downsides

  • Potential Blast Radius: All Sainsbury’s Snowflake users, however, the impact of each of the individual changes may be only on a small subset of users (e.g. Users using feature x only)
  • Impact: Less agility in the way we modernise our cloud data platfrom compared to now, where there are no SLAs / agreements on these kinds of changes. This is in favour of giving users more time to follow best-practice and prepare for any upcoming changes.

Implications

  • Easier for squads to prioritise, manage their backlog and ensure delivery of high value stories
  • Our data cloud platfrom (snowflake) will be more resilient, stable and modernised on regular basis - meaning we will be de-risking and futureproofing it in an agile way
  • Users using features that have reached end-of-life, or not following the latest version guidelines, will not be receiving priority, hands-on support from Snowflake team (support in the form of user guides, training, community of practice, drop-in clinics etc will still be available)
  • Some teams will not receive support from the Snowflake team if they are found to be using Out of Support features.