The Warehouse Group (TWG) is a New Zealand success story, having evolved from a single store to become the country’s largest general merchandise retail group operating across market-leading retail brands including The Warehouse, Noel Leeming, Warehouse Stationery, Torpedo7, and TheMarket.com.
TWG’s vision is to build New Zealand’s most sustainable, convenient and customer-first company, guided by their core mission: to help Kiwis live better every day. To deliver on this vision TWG embarked on a journey to drive efficiencies in their software development squads.
The “Golden Path” project was an initiative to put in place systems and processes to support the development squads to achieve these company goals. It provided training to help teams on board with new ways of working and enabled squads to test and experiment with technology.
ClearPoint and TWG worked together in a partnership to roll out Backstage. Backstage is a software engineering support system developed by Spotify that allows genres of projects to be templated including standing new projects up in a source repository right through to configuration of feature-flagging systems, build pipelines and more. Backstage supports fast scaffolding of new projects allowing developers to focus on business value rather than setup.
Our Golden Path framework has quality at its core. With the upfront alignment we now know what we need to build and how to test.
Matthew Law, Chapter Lead – DevOps, TWG
"These test results are automated at build and the results flow through the pipeline into ServiceNow. We also know where our logs and metrics are thanks to the linkage in Backstage, and Code Coverage via SonarCloud is front and centre in the code merge," says Matthew.
The Approach
TWG provided a clear directive to engineering squads to hone and harmonise the use of core technology and dev-ops tooling in order to eliminate barriers to getting projects off the ground. The goal was to establish a smooth path to production deployment and set that as a norm moving forward.
The approach was centred around a number of key points;
- Hone and harmonise the use of technology across the business
- Put in place good quality support systems for developers to use
- Automate developer processes as much as possible
- Decouple and shift left on development and deployment processes
- Adoption of DORA framework to inform a common approach
The goal was to reduce lead time to establish a new service from weeks to minutes thereby saving time and creating efficiencies that would have a direct impact on business outcomes. With historical challenges including lengthy onboarding and approval processes for new systems, engineers had to wait for weeks in a holding pattern waiting for projects to get off the ground. These manual processes clogged delivery pipelines and hampered progress.
Implementing Backstage and Templates
Central to the Golden Path effort was the roll-out of the open-source Backstage system. By implementing the Backstage service, the team knew that standard practices could be put in place in a cookie-cutter fashion, saving an enormous quantity of manual work in each project.
Using standard technology such as Java, SpringBoot and other libraries, The Golden Path team has worked to fashion great starting points for TWG squads working on new projects within the business. Repetitive setup for aspects such as logging, change-log, and code style are all embodied in the Backstage templates meaning that an engineer starting a project can instead focus on creative business engineering and not on tedious basic plumbing.
The Golden Path’s “north star” was the DORA metrics. The DORA metrics are a set of simple measurements that can be made against a software engineering team that helps to understand dev-ops performance. By improving the DORA metrics of the squads, the Golden Path team would be able to improve the overall dev-ops performance.
Golden Path has also gone on to configure and customise the Backstage deployment to provide a one-stop-shop for developers wanting to find out about a project; links to other resources, documentation, test results, APIs and more are exposed via this mechanism. By having all this information available in one place, the developer no longer has to bumble through a number of systems to find information about their project – again presenting considerable time savings.
On the 25th of March 2022, our Integration Modernisation squad moved into the Elite range as measured by the DORA metrics. At 2:11 pm they merged a final code cut of a middleware component to be deployed into Kubernetes. At 3:01 pm they were in production and still met governance commitments for ITSM, Testing and Code Coverage.”
Matthew Law, TWG
The Backstage implementation was also integrated with a number of other OTS products to create a cohesive development ecosystem. For example, LaunchDarkly was introduced to support feature-flagging. Amongst other things, feature flagging supports the separation of deployment and release which feeds into improvements in the DORA metrics. Other OTS products that Backstage was integrated with include ServiceNow for delivery governance, Harness for deployment management and Sonarqube for static code analysis.
Backstage was integrated as a key player in TWG’s schema curation workflows. The Backstage templates have also been written to dovetail into this mechanism supporting TWG’s overall systems architecture.
Shifting development processes “left” from build and deploy time back onto developers’ own computers allows for faster and more satisfying workflows for developers whilst reflecting positively in the DORA metrics. One way this was achieved was by enabling component testing on developers’ own computers so that developers could run component tests decoupled from other projects locally. This was sometimes a challenge to establish in the Backstage templates but was very successful in the end and formed an important part of the project.
Check out Backstage in action, with Matthew Law, Chapter Lead, DevOps, TWG
Training
As some of the new practices were foreign to the squads, the Golden Path ran a set of training sessions on the approaches as well as posting guides from the Golden Path within squads to assist in gaining familiarity with Backstage, the templates and the new techniques/tools. The guides were able to provide assistance and ran targeted training sessions to help team members get up to speed with the new ways of working as well as creating a sense of buy-in.
A key aspect of the Backstage templates has been automated testing. Specific approaches have been used to allow for a number of different genres of testing. Helping squads get acquainted with these techniques and where to best use them was a specific focus for the Golden Path team. As an example, an exercise called the “test pyramid” has been used successfully within the programme to help guide engineers’ testing effort so that the right testing is being done using the right techniques.
Key outcomes and learnings
The key outcome from Backstage implementation is time savings, service onboarding previously took up to 6 weeks and now takes a couple of minutes. The efficiencies included saving an estimated 25,000 hours a year and that is a significant outcome for the team and business.
With a clear directive and endorsement from the business up front, the squads and wider business functions were empowered to embark on a fundamental shift in mindset that enabled them to change and experiment. This empowerment provided the squads with the tools to take ownership, and have the autonomy to make informed decisions. The results speak for themselves: the Integration Modernisation Squad within TWG shifted from a 120-hour deployment process prior to just one hour after implementing the changes.
Finally, a measurement and learning framework was crucial to this project – with both short and long-term impact. Adopting the DORA framework and metrics made it easier to gain buy-in, explain the benefits and ultimately inform the Guide Strategy. A transparent feedback loop and clarity upfront is critical.
ClearPoint are leaders in engineering, design and technology transformative solutions. Gain a competitive edge in a digital-first world. With a combination of technology and business knowledge expertise, we make sure your organisation can seamlessly transition into the digital age, empowering you to drive lasting business value. Find out more.
