By Morné Niemand, Software Developer, ClearPoint

As businesses go through their digital transformation, they also want to increase profitability, productivity, market share, customer satisfaction and be able to respond fast to market shifts and disruption.

Taking ClearPoint’s own practical experience and DORA’s (DevOps Research & Assessment) Accelerate State of DevOps 2018 report, we’ve highlighted the key things you need to know for 2019. Regardless of your industry, it’s important to understand high performance in the context of software delivery and the factors that predict it.

Key areas companies should consider to continuously deliver great digital solutions include:

1. Leverage the Cloud

According to DORA’s report, the cloud improves software delivery performance. Teams that leverage all of cloud computing’s essential characteristics are much more likely to be high performers. The survey found that most respondents said the primary application or service they were working on was hosted on a cloud platform. They also asked companies which cloud provider they use and this was the response:

DORA Report_Usage of Cloud Providers

DORA Report_Cloud Provider Usage

Source: Accelerate: State of DevOps 2018: Strategies for a New Economy Report

The report found that companies who adopt the following cloud design practices benefit the most and fall into the elite performers group.

  • Platform as a Service (PaaS)
  • Cloud-native
  • Infrastructure as Code
  • Companies that leverage containers

When implementing a cloud strategy, it’s important not to lock things down so much that people can’t actually solve and diagnose issues in production.  Some cloud implementations still require that users raise tickets in order to access critical resources to accomplish their day to day tasks. Not only does this decrease throughput times, it also wastes time which leads to lost revenue.

It was found that for companies that adopt the PaaS model, most of their teams used libraries and infrastructure defined by the PaaS as the basis for their applications.  They also noted teams can deploy their applications into the cloud on demand using a single step, and can perform self-service changes on-demand for databases and other services required by their application.

An essential ingredient in DevOps is Infrastructure as Code. We like cloud agnostic tools like Terraform to control your infrastructure in an automated fashion. Infrastructure as code makes provisioning testing and deploying to production environments fast and reliable. The report found that almost half of cloud adopters agreed or strongly agreed that environment configuration and deployments use only scripts and information stored in version control, with no manual steps required (other than gated approvals).

2. Explore using open source technologies

The report found that open source components, libraries, and platforms are more likely to be extensively used by the elite/highest performers, who are also much more likely to expand open source usage in the future.

Adopting an “Open Source First” philosophy is a good way to start building and running your software on Open Source foundations, whilst actively and continuously contributing to the Open Source community.

At ClearPoint we have various Open Source projects which are actively being worked on to help our clients through their Digital Transformation journeys.

3. Outsource when needed

Nobody likes to admit that perhaps it’s time to outsource a piece of work to either contractors or 3rd party vendors, but given the right requirement this just makes sense.  Paying for expertise normally pays off in the long run, even though getting the business on board initially might be quite a daunting process.

The report highlights that a popular outsourcing model is to assign work to individual organisational functions – for example, application development, testing/QA, and IT operations.  However, this model has drawn criticism from Agile and DevOps communities because they are perceived as a barrier to high performance as this model creates silos.

Low-performing teams seem to be more likely to use functional outsourcing than elite performance teams which suggests that outsourcing by function is rarely adopted by elite performers.  A popular outsourcing model is the embedded contractor model where the outsourced staff operate and behave as part of the primary organisation’s cross-functional product or technology teams.

The most important thing in your customers digital experience is owning and controlling the outcomes.  Whether you build it internally, or partner with suppliers, having control of your customer’s experiences is key.  That is, controlling their experiences today and controlling the roadmap for their experiences tomorrow.

4. Key technical practices drive high performance

DORA highlights key practices that are essential to successful technology transformations. These include:

  • The use of version control
  • Deployment
  • Automation
  • Continuous integration
  • Trunk-based development
  • Loosely coupled architecture

This year, they also found that using efficient monitoring and observability solutions, continuous testing, and integrating database changes into the software delivery process all positively contribute to continuous delivery.  Integrating good security practices earlier in the software development process is also key and often neglected.

We’ve seen that bringing SecOps (Security Operations) into the planning, design and delivery journey earlier pays off and helps identify issues before go-live dates. The sooner the security teams are involved in the overall software delivery cycle, the more efficient the overall delivery process will become over time. Thinking of proper information security (infosec) controls up front is also vital as most companies are either required to be PCI compliant or meet regulatory compliance requirements.

A key item which stood out is the fact that systems are growing in complexity which drives more discussion about monitoring and observability. Having comprehensive monitoring in place and also knowing what information to log and what not to log allows better continuous delivery.  Elite-performing groups are identified as normally having proper logging & monitoring practices in place.

When it comes to continuous delivery and database changes, best practice is to use scripts in version control and manage these changes in the same way as production application changes. Furthermore, when changes to the application require database changes, these teams discuss them with the people responsible for the production database and ensure the engineering team has visibility into the progress of pending database changes. Having the various teams communicate effectively is key here.

5. Use Test Automation practices

By leveraging automation for repetitive and manual tasks, teams and organisations can improve work quality, repeatability, and consistency. It can also free team members from spending time on low-value tasks. With more work automated, elite/high performers free their technical staff to do innovative work that adds real value to their organisations.  Try to encourage developers to practice test-driven development (TDD) by writing unit tests before writing production code for all changes.

Something we’ve learned along the way is don’t try to put all your eggs into one basket. This means don’t try put all your testing scenario’s into GUI automation tests, as this will most definitely slow down your deployment pipelines. Try to find the right balance between what should be a unit test, component test and GUI automation test.

Companies can use code scanning tools to create Quality Gates which help with continuous code inspection as your engineers commit their code. This gives a company a reasonable level of comfort that the code they release to production will at least meet the minimum agreed quality and coverage levels. The report identifies a J-Curve diagram which illustrates the transformation teams go through when it comes to doing manual work vs automating.

DORA Report_JCurve of Transformation

Source: Accelerate: State of DevOps 2018: Strategies for a New Economy Report

When teams are doing work, some of the key questions to ask are:

  • are your teams able to focus their time and devote effort and energy on developing new features and supporting infrastructure?
  • Or do your teams spend most of their time correcting problems, fixing issues, and responding to defects and customer-support work, ie. fixing issues that arise because quality was not built in up front.

6. Trust your teams to be more autonomous and self-organising

Whether you’re talking to software or DevOps engineers, the key is to have more faith in your employees. The people who write the code should be able to debug and diagnose production issues and take ownership for the work they’ve delivered.  After all, who is best suited to fix an issue than the team responsible for writing the initial code?

In larger corporations where security can become a hot topic, it’s important that the security, DevOps and development teams are all involved in the design and implementation stages from the beginning.  A classic example is that once code hits production, developers need to be able to easily debug and monitor any resources they’re responsible for. ie. dont have only a select few individuals with NASA level clearance to access the production logs.

According to the report, teams get classified into high, medium, and low performers and find that they exist in all organisation types and industry verticals. A new group has emerged called the “elite performers”. The first is that we see the high-performing group growing and expanding, suggesting the overall industry is improving its software development and delivery practices. This trend signals that high performance is attainable for many teams in the industry and is not something reserved for an exclusive group of teams with unique characteristics. The second is that the elite group demonstrates that the bar for excellence is evolving across the industry, with the highest performers still optimising for throughput and stability.

The proportion of high performers has grown year over year, showing that the industry is continuing to improve. We also see low performers are struggling to keep up, widening the gap.

The below summarises these different groups by various categories. It’s quite obvious the Elite teams are very efficient in their delivery processes.

DORA Report_Aspect of Software Delivery Performance

Source: Accelerate: State of DevOps 2018: Strategies for a New Economy Report

DORA Report_Elite Group against low performers

Source: Accelerate: State of DevOps 2018: Strategies for a New Economy Report

The report highlights a very important point in that we often hear from organisations that prefer to take a cautious approach to software development and delivery. They believe releasing code infrequently can be an effective strategy as they use the extra time between deployments for testing and quality checks to minimise the likelihood of failure.

However, companies that take this approach normally take the longest time to restore a service and recover from possible data corruption.  When they happen multiple times a year, they can quickly take over the work of the team so that unplanned work becomes the norm, leading to burnout and lost revenue, an important consideration for teams and leaders.

The highest performers continue to see low change fail rates, while the low performers are increasingly likely to have changes impact their systems. This suggests that building resilient systems, or systems that we expect to fail, is increasingly important.

7. Remember, Organisational culture is key

Going through a digital transformation can be a disruptive and daunting journey, but organisations need to take into consideration culture, and be open to and prepared for certain mind-set shifts.

Technical and management practices shape culture and that culture in turn helps to improve performance outcomes. The report highlights that team dynamics are much more important to team effectiveness than a particular set of skills among team members. This means that when teams have a good dynamic, their work benefits at the technology and organisational level.

Also, when leaders give their teams autonomy in their work it leads to feelings of trust and voice. Trust and voice, in turn, positively affect organisational culture. There are some ways in which leaders can help their teams gain autonomy. These are:

  • Establishing and communicating goals, but letting the team decide how the work will be done
  • Removing roadblocks by keeping rules simple
  • Allowing the team to change rules if the rules are obstacles to achieving the goals
  • Letting the team prioritise good outcomes for customers, even if it means bending the rules

It’s shown that once teams trust their leaders it directly contributes to a stronger organisational culture.  Transparent communication is also very important as it helps improve organisational culture.  A good example of this is having open and “blameless” retrospectives or so called “post-mortems”. It is important to note that when teams can learn from their mistakes and failures and then turn these into opportunities to improve how they work, their tooling, processes and procedures and most importantly, team culture will automatically improve as there’s always a sense of trust in the teams.

The report found that elite performers are more likely to consistently hold retrospectives and use them to improve their work. Companies should also have a good climate for learning and factor these into working hours and not expect their employees to always study after hours to improve their overall capabilities and skills. Hackathons are a great way to encourage this, or alternatively allow small but frequent sessions to allow software/DevOps engineers to purely focus on doing “cool and new” stuff.

We believe constantly having a “Growth Mindset” is also a key ingredient in ensuring great software delivery. Teams need to be encouraged to be the best at what they can do, and be given the space to do so. In doing so, a better culture will automatically come as a result.

Fixed vs Growth Mindset

Source: Medium.com > medium.com

To read more about how to build a great engineering culture, see our blog on ’10 Team Culture Ideas to Carve Up the DevOps Wave.

Wrapping up

Delivery is where the rubber meets the road as we move from ideas and prototypes into real working software. The key focus is strong engineering and software delivery capabilities and the ability to move fast, whilst maintaining a growth mindset and a high level of quality through automation and DevOps practices.

Implementing good software delivery practices during technology transformations pays off in terms of organisational performance as well as generating quality outcomes.  Companies should strive to create more autonomous, cross-functional and elite performing teams and in doing so, do significantly less manual work, encourage automation as much as possible and spend more time doing new work that delivers actual customer value. Also, don’t be scared to embed contractors in a cross-functional team as this is most likely to give you quick wins with minimal effort.

Teams should strive to use more Open Source technologies and adopt essential cloud characteristics. Leveraging PaaS, Infrastructure as Code and using containers seems to push teams to fall into the elite performance group category.

Information Security teams should have input into the design and planning of applications right from the start in order to avoid last-minute rework and unplanned surprises.  Fast feedback on the quality and deployability of the system should be available to everyone on the team and acting on this feedback should be team member’s highest priority.  Having proper monitoring and logging tools in place is critical to ensure fast throughput in diagnosing problems.

Companies that develop and deliver quickly are better able to experiment with new ways to increase customer adoption and satisfaction, pivot when necessary, and keep up with ever-changing compliance and regulatory demands. And remember, maintaining and encouraging a great company culture is very important even though this can be very challenging. As soon as a great culture is in place, you’ll automatically reap the rewards and see how this affects your software delivery cycles.

Something we believe is vital in delivering great software is having a great collaboration tool for communicating amongst teams.  Slack continues to be one of the favourites used by many companies, including us here at ClearPoint.

As you move forward into 2019, we hope we’ve given you a clearer understanding of what you can do to improve your software delivery processes to add business and customer value.

If you or your business needs help in your DevOps and continuous delivery of software, we can help. Get in touch with us today.