Part 1 – Starting out and getting over the first hurdle! 

So, about ALL of that data you have… 

Pushing data into the cloud was once seen as a skill confined to tech companies, who had either started with at least one foot in the door or had the skills readily available to make it happen. However,  it is now much more commonplace for organisations to want to lift and shift their data and operations into the cloud, with the promise of releasing both savings and efficiency gains. 

This move to cloud–centric data usage is something that Oakland fully endorses; it is what comes next that is often the real issue. Moving operations to the cloud is a time–consuming and expensive job, affording it the luxury of its own owner and associated strategy. However, whilst a good strategy may be in place for getting your data away from on–premise systems, a question that will likely stump many companies is what do you plan to do with afterward? 

Shifting an organisation onto a cloud provider isn‘t cheap, which explains why most of Amazon‘s operating revenue is driven by AWS. As such, it‘s often tempting to offset this cost with large projects, similar in many ways to fabled ERP implementations. However, with the promise of significant rewards comes significant risk, and for many companies, this is an unknown area that can often lead to a whole new set of problems. We’ve certainly seen evidence of these issues working with our clients. With sprawling new data architectures designed to link core processes, development teams locked into long projects with little flexibility and struggling with ways of working that inhibit more than they improve. 

And that is why we have a slightly different approach to recommend, something that takes the focus off large–scale returns and instead looks to build an organisation‘s capability to deliver rapid solutions that prove their value. Throughout this and the following article, I‘ll be delving into the world of cloud-based proof of concepts (POCs). These projects combine discrete process elements with the myriad of cloud technologies in order to build out internal capabilities, ways of working, and business investment. They typically run over a much shorter development timeframe with high amounts of business input and a focus on achieving the most value from minimum change, i.e., not chasing the edge cases!  

Through these articles, we‘ll be able to highlight the successes that we‘ve seen first-hand from this methodology, drawing on real-world examples from the past twelve months. 

Where to start on your POC journey? 

So, we’ll assume that you‘re on board with the idea; where would you begin to look for something to turn into a proof of concept, and how can you help make it as successful as possible? Luckily Oakland has lots of experience here, so this is our advice. 

1. It doesn‘t have to be something brand new – Developing a POC doesn‘t always mean you have to build from the ground up; changing how something is done, or its core technology is just as valid and fruitful. 
2. Pick an area that will have strong buy–in, either through interest or the value it may drive. This also doesn‘t have to be a purely “tech“ area either; updating a process related to, say, data governance or improvement cycles using new technology can be a great way to make incremental gains.  
3. Ensure that your POC has an owner, someone to drive it forwards and make decisions. This is key when moving it into production and beyond, both in terms of features and budget! 
4. Consider access to data and environments (does a sandbox exist, or does one need creating?). These can be long–term blockers, especially if an external company is doing the work. This may impact which project you choose. 
5. Factor in the long-term costs of the project, not just the cost to get a minimum viable product; otherwise, you can easily fall at the first hurdle when you try to productionise your POC. 

These may seem rather obvious tips at first glance. Still, we‘ve certainly learned the hard way that it‘s not always the case, with the most common issues being the lack of a strong POC owner and an understanding from the organisation of what it will take to get the finished POC into production.  

Here are a couple of examples that give you a ballpark idea of the costs you could be looking at. 

POC 1 – Converting a process from a proprietary piece of licensed software to work in a more understood and maintainable manner, using open–source tooling and the client‘s cloud environment. The process itself encompassed around £300m in costs per annum and, while being fairly critical, wouldn‘t be too perturbed by delays in development. The initial work was carried out with a £50k budget to test what was possible, with an expected £200k cost from start to finish. 

POC 2 – Taking an in–demand, data–heavy, excel based analysis and developing it into a scalable cloud application. The results of this analysis were an in-demand product, but users wanted a better way to access the information and add additional features. The initial work was completed for roughly £80k, with a total cost of £130k to bring it into production. 

With these examples in mind, it should be possible to put together a shortlist of your viable options, each with assigned owners, stakeholders, and budgets, from which you can pick as many as required to go into initial development. 

Time to get moving?! 

With at least one project to kick off into development, the work can finally start, right? It‘s time for our next piece of advice; don‘t skimp on the planning! This doesn‘t just relate to planning how the work will be split (though always helpful), but other elements such as: 

• When will stakeholders be updated? 
• What are the must–haves and stretch goals of the POC? 
• What decision points are needed and when? 
• What are the criteria needed to be worth productionising? 
• Who is needed at core meetings to keep information flowing? 
• What governance points are required, and what do we need to cover these? 
• What team mix do we need to prevent blockers? 

Once you have this information adequately documented and available to share, you‘re in a perfect place to start. This might eat into your initial development time, but that‘s better than losing a week because you can‘t get a key person to explain how something works, or your decision–maker is on leave! Armed with the plan, the next steps are to start regularly communicating with your internal team and start laying the foundations. This will enable the team to develop something with real potential. Try and follow the 80/20 rule (80% value for 20% effort) and don‘t get hung up on the edge cases that can be filled in later; concentrate on getting the core concept working. 

Wrap up 

So, we‘ve covered the initial process to get started on the POC journey, including a few of our examples and experiences to try and mitigate any of the pitfalls that are associated with development projects! As such, this seems an excellent place to wrap up for now, with my next article going over getting past the cut–off point and where things can fall over moving into production and beyond. 

Richard Louden is a senior data engineer at Oakland