2022 Market Research: Benchmarking Production Operations

Survey of over 300 on-call practitioners reveals half of incident response time remains toil despite millions being spent. Findings include:
Watch the video

2022 Market Research: Benchmarking Production Operations

Survey of over 300 on-call practitioners reveals half of incident response time remains toil despite millions being spent. Findings include:
  • The average company is spending $2.5M annually for on-call operations.
  • 48% of incidents are straightforward and repetitive.
  • 55% of incidents get escalated and then take 3X longer to resolve.

Get the Market Research Report

The Surprising Cost of On-Call

In this report, we surveyed more than 300 on-call practitioners, managers and executives running businesses that ranged from managing fewer than 20 nodes to more than 10,000. A lot can be learned about the industry simply by looking at a theoretical “average company”. This data uncovers a number of fascinating challenges and opportunities for the industry.

Key Findings:

  • On average, 8.7 major incidents occur each year, with 62% escalated to the C-suite
  • 55% of incidents are escalated to second-line responders
  • The average time to resolve escalated incidents is 10.7 hours
  • 48% of incidents are straightforward and repetitive
  • Human error caused major incidents 5x more frequently than bugs in automation
  • The average cost of on-call operations is $2,500,000 per year

About Dimensional Research

Dimensional Research® provides practical market research to help technology companies make their customers more successful. Our researchers are experts in the people, processes, and technology of corporate IT. We partner with our clients to deliver actionable information that reduces risks, increases customer satisfaction, and grows the business. For more information, visit dimensionalresearch.com.


About Shoreline

Shoreline is helping software engineers and site reliability engineers take the toil out of on-call. We make it easy to debug new issues in the moment and then automate them away forever. Shoreline runs across clouds, VMs, and containers and lets you manage your fleet as if it were a single box. Shoreline is working with large and hyper-growth companies like Dataiku, Domino Data Lab, TigerGraph, and Incorta.

Shoreline gives users a path to automation with a CLI for debugging and immediate issue resolution. Shoreline allows users to capture a debug and repair session in a Jupyter-style notebook built for on-call. Then, share notebooks with your entire team to safely fix issues using a list of pre-approved repair actions. Finally, when you are confident that you have precisely matched the repair action to the alarm, you can automate away common issues permanently.

Shoreline provides many pre-built automations, called Op Packs, that any customer can use. Shoreline is offering the JVM Op Pack for free so that users can eliminate JVM memory issues permanently.

Trusted by industry leaders

Modern businesses are powered by fleets of servers that are increasingly hard to operate, but customers expect 100% availability.
Dr. Jay Yu
VP Product and Innovation and
GM San Diego Innovation Lab
"If you need to manage a multi-cloud fleet, Shoreline is your best answer."
Niall Murphy
SRE and engineering leader at Amazon,
Google, and Azure. Author Google SRE Books
“Shoreline is not just a production debugging tool, it helps you make your whole team better.”
Paul Lewis
Senior Site Reliability
Manager at Domino
Shoreline actually allows our SRE team to fix a problem before it causes a 3AM wake-up call.”

Shoreline integrates with your stack

Broad platform and integration support smooths cloud operations.