A survey of 418 DevOps professionals finds that while DevOps teams closely monitor and observe the performance of applications, not nearly as many are able to correlate the value of those efforts to the business.
Conducted by Catchpoint, a unit of LogicMonitor, the survey finds more than two-thirds (67%) believe application degradations are either always or often as important to the organizations as outright downtime. However, only 26% directly evaluate business metrics after any improvement in application performance is achieved, compared to 32% that sometimes make that evaluation.
In fact, only 22% have any type of formal financial model for tracking actual downtime costs and even fewer (21%) have formal key performance indicators for tracking business metrics, the survey finds.
Leo Vasiliou, director of product marketing for Catchpoint, said the survey makes it clear that there remains a significant disconnect between the metrics that DevOps teams typically track and their relevance to the business. Most organizations are all too well aware this issue exists, but not enough political will exists to bridge that gap, he added.
As a result, many organizations are not correlating efforts to improve application performance to an actual return on the investment that was required, noted Vasiliou.
Overall, the survey finds 67% of respondents make use of dashboards and alerts to monitor applications, with 54% actively making use of synthetic probes and testing. Less than half (45%) are using passive monitoring tools such as real-user monitoring (RUM), while 50% use load testing/capacity planning tools (40%.
Less than a quarter (24%), however, are tracking metrics such as service level objectives (SLOs) or latency/error budgets and even fewer (22%) have any ability to detect anomalies using artificial intelligence (AI).
On average, 34% of the work respondents do could be classified as toil, the survey finds.
Additionally, 42% also noted their organization has a tolerance for any planned disruption, such as conducting chaos engineering tests.
Among the small percentage of respondents that work for organizations that have adopted AI (18%), just under half (49%) report seeing a reduction in toil, compared to 16% seeing an increase.
Just over a third (34%) said purchased third-party AI capabilities (34%) versus 33% that built their own. A third (33%) also report using free or open source AI tools, the survey finds.
A full 60% of those respondents are optimistic about AI, but only 42% said they are moderately confident in machine learning/AI versus 13% that are extremely confident.
At the same time, 65% of respondents spend less than two hours a month on learning new skills, which suggests that many DevOps teams are spending most of their time trying to optimize application environments using tactics and techniques that with each passing day are becoming more antiquated.
Each organization will need to determine the degree to which improving application performance provides meaningful value to the business, but software engineers are likely to continue paying close attention to any set of metrics they can directly influence. The challenge, and opportunity, is to get the rest of the organization to understand the importance of those metrics as it pertains to an actual impact on the bottom line that business leaders appreciate.
