Enough headaches already!

Welcome to the first in our re-launched Blog series of 5 minute reads, designed to help you get under the skin of DataShaka and understand what we are all about.

So let’s start with why we even exist. Back in the midst of time (the early 10’s) the owners of a digital agency were getting frustrated. Actually, they were being driven insane by how difficult it seemed to be to get value from the data they, and their clients, were swimming in. So, like any good entrepreneurs, they decided to do something about it. And they wanted to solve this problem for regular business people, like them. DataShaka had to work for business leaders.

The core challenge to solve was being able to bring multiple data streams together to then generate insight. They had started off thinking that it was generating the insight that was going to be hardest to achieve. But, pretty soon, they had realised that it was the data preparation, that led up to this, that was actually taking up all the time. The 80:20 rule was the wrong way round, it was 80% Prep 20% (at best) insight. They vowed to flip this ratio on its head.

At that time, and arguably still, there was no easy way to have different data sets and formats talk nicely to each other, so they initially focussed their attention there. Out of this came Katsu, a data fabric that breaks all time-series data down into the constituent parts of Time, Context, Signal and Value. All time-series data had these elements and they knew this was the key to unlocking the puzzle.

Of course no-one else actually cared about Katsu, just that it worked and enabled different data sets to be brought together (exactly as it should be). Once the data was able to play nicely together, that’s when things started to get really interesting. With Katsu in place, next came along Tractor, and Chopstick - giving modern new ways to ‘orchestrate’ and manipulate data. If you’re that way inclined, you can find more on these here. There are posts and ShakaSnippets to come that will dig into these elements more deeply.

But, for now, back to business. Not only was DataShaka deployed successfully within its parent agency, it also went out on the road. Talking to organisations who were also suffering from the same data headache; too much time on preparation and management, for too little value. And it didn’t take long to find companies and agencies wanting to get stuck in. You can find Case Study examples elsewhere on our site, or click these links to download now: BBC, Hall & Partners, Bacardi.

A common trait shared by all our clients, past and present, is the value they place in human contact. In a world festooned with ‘plug & play’ API led solutions, the reality for many companies and situations is that an expert hand is often required to achieve the desired result. And they want to be supported around the clock and around the globe. We have always believed in this ‘people first’ approach and remain committed to being personally on hand for our clients. Sometimes an FAQ just doesn’t cut it.

So what have we doing for the last 10 years? How does the DataShaka of today differ from the DataShaka of a decade ago? Well, if you were to look under the hood, you would see a set of tooling very different from that of launch. As both thinking and tech has evolved, we have evolved with it. Originally highly tied to the Microsoft ‘stack’ we have evolved to a more modern, flexible architecture able to take advantage of the very best tooling, regardless of provider. There’s still a fair chunk of Redmond in there (most notably Azure, giving us and our clients the global footprint and security confidence they, rightly, require) but these days it’s also nuzzling up with the likes of MongoDB and Linux.

The other big evolution has been the visibility and control we are able to share with our clients. The heart of the client experience is the DCP - Data Control Panel. This gives clients the ability to see the status of their data as it is being processed. They also have the ability to manage that data through the DCP, including Adding New Data and Deleting Data. The DCP is a never-ending journey as we respond to the evolving demands of our clients. There will be more on the DCP in future posts and ShakaSnippets.

The other major innovation has been our Performance Dashboard. Built using the brilliant tooling of DataDog, the dashboard gives clients live performance stats on our query performance, as well as a live status position on all our critical tooling, how did we ever exist without it...

So how do we feel about life a decade on? The good news, for us, is that the Data headache has not gone away. The need to solve this challenge for organisations feels more important today than ever.

Our immediate focus is to continue working as the defacto Data Operations (DataOps) team for clients. Our clients detail the brief, we make sure the data is available, fit for purpose and easy to work with. Our clients generally don’t care what goes on behind the scenes, just that we do our job, and rightly so.

As we move forward, alongside our core service-led proposition, we will be developing our self-serve tooling, so that data professionals can have the power of DataShaka in their hands too, watch this space.

None of us like headaches, we like to think of DataShaka as the pill that takes the pain away. The Data Advil if you like.