What is DevOps and why is getting it right so important for the cloud?

8 Feb 2021

One of the most over-used and misunderstood concepts in IT today is DevOps. It’s right up there with Multicloud and Agile as ideas that get more lip service than understanding or actual comprehension. Most companies today have started to implement DevOps. DevOps engineer jobs are constantly posted. Yet most companies are unhappy with the return relative to the investment.

In a 2020 IDC survey of U.S. enterprise organizations, more than 77% of respondents indicated that they have already adopted DevOps in some capacity. But most of these enterprise organizations are using DevOps for less than 20% of their application estate. Those same companies plan to double DevOps usage in 2021.

So why is DevOps so hard, and why is it so important to get it right? And why is it important for your company’s journey to cloud?

Probably the biggest impediment for organizations implementing DevOps (or DevSecOps as its increasingly known) is a lack of understanding or agreement on just what it is.

What exactly is DevOps?

The term DevOps was first coined in 2008 at an Agile conference in Toronto. The definition proposed was:

DevOps is a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production, while ensuring high quality. (source Wikipedia)

Note that in that 2008 definition there is no mention of tooling. The concept is about new processes and practices to move faster. Implementing DevOps will help accelerate software project development cycle by minimizing costs and driving efficiency. Most organizations seem to think that implementing a ‘DevOps toolchain’ is all that’s required. And of course tooling is essential to automate, simplify, and accelerate the process.

But without changing the relationship between the IT Development team and the IT Operations team, and changing the underlying philosophy of both those teams, DevOps is just another way to waste IT funds by purchasing under-utilized tools. Like the people who think that having a ‘daily standup meeting’ means they’ve implemented agile, buying the latest toolset (or downloading the opensource versions) does not equal ‘implementing DevOps’.

As its name implies, DevOps all about bringing together the two main historical camps of IT: The development team and operations team. Although they have many skills in common, these two groups have always approached IT with very different perspectives:

- Developers want unlimited test environments to rapidly develop and test different code streams in parallel. They want to be able to develop test, iterate and not lose any changes. They want the latest cool feature that they’ve created implemented immediately and are happy to ‘throw It over the wall’ into production.
- The Operations team wants stability. Change is risk. The most likely time for an outage is right after something new has gone in. Cost and time taken to set up and manage all these overlapping environments distracts from the main goal – keeping the production environments safe.

There are other stakeholders with additional views:

- Business users want the change they want immediately and not wait for quarterly cycles. But like the electrical grid, they expect the systems to be always available and only notice when it is not.
- Senior IT executives may be primarily concerned is about improving overall operational metrics around cost, risk, quality, productivity and speed in the development cycle. More bang for the buck.

DevSecOps means taking the DevOps approach DEV_SEC_OPS and embedding a security perspective into it right from the start. Security is not a standalone practice but instead a shared responsibility. Designing and building security in rather than having it imposed by a separate ‘compliance’ function.

To implement DevOps think of the acronym CAMS

Think of the acronym CAMS (Culture, Automation, Measurement and Sharing) as the factors necessary to truly adopt embrace DevOps.

Gears Successful implementations of DevOps change the Dev and Ops cultures to make sure we stop thinking either/or in order to get both rapid access to change and maintain production stability. They highly automate the steps of software creation and use from requirements definition through development, integration, testing, deploying and operating. They use combined processes and tools to shorten development cycles, and allows for more frequent deployment to production.

Well implemented DevOps processes give developers and testers the ability to create, customize and delete their own environments; it is about keeping track of all the components necessary for a successful implementation of a new function into an environment, and it’s about ensuring that change is promoted to production smoothly, efficiently and often while retaining the necessary approvals and controls.

To track progress in implementing DevOps and to continuously improve, it is important to identify and track a set of measurements. In true DevOps fashion, these metrics should be simple, highly automated and accessible to all involved. While each implementation should focus on the pain points of your own organization, some of the key items to consider include:

- Frequency of code changes, number of stories/function points, features are being incorporated
- Defect rate at each step in the process. Numbers caught at each stage including in production
- Automated test case coverage
- Frequency of deployments, and deployment success
- Manual versus automated steps in the process. “The objective is to automate everything: Developers should be able to send new and changed software to deployment and operations without humans getting in the way of the processes.”

Track how these measures interact. As DevOps capabilities mature, you should be seeing significant gains in quality and functionality per unit time. As you deploy to production more frequently, are defects in production increasing? As you automate creation of test environments, is code integration becoming more challenging? How have these numbers changed since before you started your DevOps journey? Calculating the number of releases per period over time will give you an indication of improved velocity. Tracking the number of defects per release in production over time will indicate how quality is improving.

(You can find lots more on DevOps measurements here and here. )

Like any project designed to change behaviour, regular, frequent communications among the stakeholders is critical. Sharing pain points and successes helps pave the way to improve and expand your DevOps implementation. “Done right, DevOps is a feedback development lifecycle where assessment and feedback are the foundation and are embedded in every step.”

In 2020 we’ve seen the importance of automation and tooling to allow this work to be done remotely. And yet, “One of the biggest challenges to pursuing DevOps is a lack of urgency.”

Why is it so important today? And how does it impact our Cloud work?

Multiple trends have increased the importance of properly implementing DevOps.

- End-user expectations conditioned by the ‘it just works’ deployment of the latest features on new apps on their Android or iPhones means both increased expectation of the rate of change, higher availability expectations AND more success outcomes expectations.
- Agile development and continuous integration continuous deployment (CI/CD) create the need for automated deployment and the expectation of easily measuring outcomes.
- The ability to rapidly create and delete environments in the cloud (particularly development and test environments) means it is even more important to automate code deployment so that you know exactly what is where, and how to recreate it.
- Increasing use of microservices architecture (or APIs) for software development means there are more “things” to manage in development and production.
- As use of microservices increase, that in turn is one of the factors driving the increasing use of containerization as the deployment vehicle for cloud workloads. Containers are foundational for the API economy by providing efficient scalability simply and automatically. In order to deploy and manage those APIs in containers requires enterprise grade DevOps.
- Other trends like infrastructure as code, network as code, SRE (Site Reliability Engineering) rely on DevOps or DevOps concepts to be successful (see here.)

Migrating to the cloud – like any new technology – brings its own new operational procedures. It is all too easy as we build in new environments to recreate the mistakes of the past – or to use something which was successful in the past but doesn’t take advantage of the new approach. Fundamentally, the cloud is about creating a virtualized environment. As companies implement more solutions on more clouds, the proliferation of workloads to manage will mean the need for consistent tooling to manage while increase. DevOps process are the approach to managing that virtualization. The right culture, process and tools are critical to IT success in the cloud world. Containers, service mesh, data mesh … these can package up tools, data and technologies to make development faster, quality higher, operations smoother, and our business users happier. And that’s what DevOps is all about.

What’s Next in DevOps?

Already we see people trying to take the next step in DevOps by bringing in Artificial Intelligence concepts into the DevOps process (also known as AIOps). Moving beyond automation to self-healing, or cognitively-assisted touchpoints to improve service resiliency and make the user experiences better. Automatic suppression of false alerts, using predictive insights to predict and prevent failures, and log-file analytics to provide real-time problem determination are all possible.

Want to learn more? Some other insights on DevOps found here. (And thanks for all the research by my colleague Abdel Koumare who sifted through these to find the best ones.)

DevOps and hybrid cloud

Why DevOps is important for the business