Shining a Light on IT Operations Metrics

Modern IT departments operate in a pressure cooker. Their highly distributed infrastructures are spread across legacy data centers and hybrid and multiple public clouds. Now, the Internet of Things and the edge are part of the mix.

Massive amounts of data are flowing through these multi-faceted environments, and it falls on IT to make sense of it all, ensuring that issues don't become problems and that it’s viable to produce measurable outcomes for the business. Dashboards are instrumental for visualizing and quickly making sense of that data. 

Metrics – numeric representations of data measured over time – give IT operators and site reliability engineers (SREs) a window into how a system has behaved historically. That data provides insights as to how the system(s) should perform in the future and aids investigations when something goes awry. IT monitoring tools act as giant data lakes holding time series data, while dashboards give everyone from IT administrators to C-level executives ways to easily digest the data.

The visualization market is evolving. We’re seeing different kinds of databases, such as Prometheus, which bring with them new tools to leverage the data. Prometheus (PromQL) is growing in the DevOps space and becoming the standard for monitoring containers and microservices because it makes it easier to manage time series data. PromQL also comes with client libraries that include four core metric types:

• Counter: Represents a single monotonically increasing counter where the value can only increase or be reset to zero on restart.
• Gauge: Represents a single numerical value that can arbitrarily go up and down, such as CPU utilization.
• Histogram: Samples observations and counts them in configurable buckets. It also can provide a sum of all observed values.
• Summary: Samples observations and provides a total count of observations and a sum of all observed values. Configurable quantiles are calculated over a sliding time window.

Selecting visualization charts

Selecting the appropriate visualization dashboard for each user is important to maximize the value of the data. There are traditional line and bar charts, single value tiles – which display a single data point at a specific point in time – and list and gauges, which offer a single value at a specific point in time in a series.

Newer visualization types are now becoming common to address the various use cases in ITOps and DevOps. Let’s walk through a few:

1) Honeycombs offer big-picture views of IT infrastructure yet also display detail. The hexagonal display mirrors those in a honeycomb and help DevOps pros slice and dice the data based on attributes or tags. Each hexagon represents a specific host or resource and can be organized in logical groups of elements. The colors change when thresholds are reached, and the hexagons can be sized differently based on the value of the metric.

Consider a case of many AWS EC2 instances scattered across the globe. Each hexagon can represent an individual instance, and they can be grouped based on the "region" attribute. This honeycomb chart would let administrators see how each infrastructure site is performing.

2) Gauge metrics use a language like PromQL to set ceiling and floor. If the user creates alert conditions based on particular thresholds, the dashboard will indicate if a certain resource has reached its threshold by changing the color. A gauge that turns red may indicate a critical problem.  Managed service providers have SLAs with clients to maintain uptime. If a problem arises, a red indicator on a gauge tile lets the IT administrator know immediately that the SLA agreement has been breached and an issue needs to be addressed.

3) Heatmaps provide a graphical representation of the volume of data points within bucketed ranges. The color of each cell of a heatmap directly reflects the density of certain metric value ranges. Heatmaps are used to display group trends and spotting outliers, which make them useful when viewing metrics with a wide distribution of values.

Role-based requirements

Picking the right chart depends in part upon the people viewing the data. CXOs, IT managers, sysadmins, SREs, network engineers, and network operations center (NOC) operators will have different yet overlapping requirements.

For instance:

• SREs want to ensure the reliability and resiliency of the infrastructure, so through their dashboards, they look at everything from availability and latency to performance, efficiency, and capacity planning. They can quickly spot outliers like server downtimes or memory issues and immediately begin to troubleshoot.
• CXOs, conversely, don’t need to get into the weeds of infrastructure operations. They use dashboards to gain a macro view of metrics for performance over time across websites and networks. They need to understand the correlation between IT metrics and business services and applications. Dashboards should be able to connect those dots.

Other considerations

Finding the right visualization option is important. Once that is done, there are some other steps to take to ensure you get the insights you need from your dashboard:

• Positioning and ordering: Dashboard tiles should be placed in a logical order, with important high-level statistics near the top and the deeper and more granular metrics progressively moving to the bottom to give users a quick understanding of system performance.
• Design: A well-designed dashboard with consideration to color schemes and themes also will make it easier for stakeholders to digest and understand the data.

 IT infrastructure performance is crucial to the health of a company. Massive amounts of data streaming out of the systems can help IT administrators and others keep up to date about that performance and the overall end-user experience. Dashboards offer flexible ways to quickly view and understand the metrics; IT organizations now have a growing range of charts to choose from that can present the data they need in a way that works best for them.

Jay Menon is Assistant Product Manager at  OpsRamp.