Do you remember back in the old days of OSS/BSS releases into production? They were a little bit stressful…. especially for the Release Managers.
I remember one Release Manager who used to set up the packages, type out the commands, then his hands would be poised above the keyboard, literally shaking. He would then position his finger above the Enter key and look away from the screen whilst pressing it!
He then couldn’t look back at the screen for at least a couple of minutes… maybe taking a sneaky peak from time to time.
It was pretty hilarious to watch!
It was like watching a nervous soccer fan at a big game being decided by penalties. Look away! Can’t watch!
Roll-backs upon fail were equally amusing to observe (for me at least).
But luckily, we can avoid these big-bang cutovers in many cases with load balanced application architectures. This article by Danilo Sato describes Canary / Blue-Green Deployments of new / update software.
The steps are summarised in Danilo’s diagrams below:
The canary release model provides a technique of migrating a few selected users (eg internal / project users) to the new solution. If something happens to the canary release and rollback is required, it’s simply a case of routing all users back to the old version.
Danilo also states, “Canary releases can be used as a way to implement A/B testing due to similarities in the technical implementation. However, it is preferable to avoid conflating these two concerns: while canary releases are a good way to detect problems and regressions, A/B testing is a way to test a hypothesis using variant implementations. If you monitor business metrics to detect regressions with a canary , also using it for A/B testing could interfere with the results. “