Data Disconnect, Data Overload

I don’t know when I first noticed that restaurants know how to talk to children better than to adults. I suspect the prerequisite for this knowledge was having children. Since then, the differences in communication have become obvious.

For example, I’ve noticed the difference between the regular menu and the children’s menu. The regular menu has many more options, sometimes so many that it’s difficult to make an easy decision. The children’s menu, however, usually has just five to seven items, making it easy for a child to select what he or she wants. The regular menus at some restaurants can be so difficult that experts are employed in order to help patrons. At a fancy enough place, there may be a sommelier to help people choose the right bottle of wine to go with their dinner.

A recent experience made me realize how similar the complex dinner menu is to what happens inside businesses of all sizes. The experience involved too much data not presented in a clear way, groups of people that could not communicate with each other, and experts being brought in to help everyone make sense of it all.

We’ve all experienced the silos that exist inside of corporate IT. Teams that need to be able to work together struggle to communicate with one another. Database and systems administrators are a prime example. The rise of DevOps is simply raising awareness to an issue that has existed for decades: We are drowning in data, much of it with little or no context. And making matters worse, each silo has its own data about what’s going on within the infrastructure --  its own version of the truth, if you will.

As data grows in importance and value for corporations and individuals alike, data is also becoming easier to obtain and distribute. So much data is flowing into, and out of, our hands each day. But without context and a way to ensure that everyone is aligned, like some kind of Rosetta Stone, once the data leaves the silo it originates in, it has largely lost its meaning.

Figure 1:

This is why the different groups that make up the traditional IT department struggle to communicate. In a way, they aren't speaking the same language. So it's no surprise that silos are formed over time. As data professionals, we can do better. We must do better.

Here's what I see as necessary in order to facilitate better communication across all IT teams in any organization.

• Stop and learn something about yourself and your team. What are your top three priorities as a team? Find the metrics that support those three priorities. Some of the metrics you may be able to get yourself, and some may need to come from other teams. What's important here is that you focus on just the top three metrics and goals, keeping in mind that the top three will (and often do) change over time.

• Share what you have learned about yourself with other groups. Whether individuals or managers do the sharing, it’s important that teams get together to talk about their top three metrics and goals. By sharing this data in a meaningful way, it will help everyone to understand what the needs and motivations of all the teams are.

• Collaborate on the overlaps. This will allow you to push meaningful data to other teams as opposed to the dreadful weekly summary report emails that scroll forever and offer little to no value for anyone. The emails will be clear, concise and offer the metrics that will help teams communicate better. Think how much better the world would be if comments such as “the server is slow” were replaced by “this specific piece of code in this specific application is causing performance issues.” This comes back to operating off the single version of truth I mentioned earlier. Having tools that provide end-to-end infrastructure visibility can go a long way to getting here.

As data professionals, we need to find ways to reduce the need for experts to help teams communicate. We need to have aligned data that can be shared in a meaningful way. And we need to help people understand what actions, if any, are necessary based on this data.

Otherwise, you end up with the equivalent of unhappy children being served food they don’t like. Trust me, nobody wants that.