In recent years, Network Rail has been impacted by significant organisational, digital and cultural change.
Growing demands for greater efficiency and effectiveness have resulted in a need for more intelligent reporting and decision analytics, especially within Network Rail’s Regional Capital Delivery teams.
Adopting a devolution strategy required Network Rail to ensure that each region had the business intelligence and intelligent forecasting capabilities necessary to deliver major capital projects that gave outstanding value to the taxpayer whilst meeting stringent government guidelines.
Working in partnership with the Devolved Regions, the Network Rail Centre of Excellence embarked on an ambitious initiative to equip the capital delivery teams with the next generation of data analytics capability.
To deliver on this mission, Network Rail sought the experience of the Oakland Group to help craft a new approach to reporting and decision analytics, leveraging the very latest in data analytics technology and infrastructure.
This case study charts the digital strategy and successful outcome that followed.
Sweeping change creates an opportunity for improvement
Rail projects need to be delivered faster, at less cost and with greater value to the taxpayer than ever before.
The government has challenged the rail industry to discover new ways of working to cut the time and cost of delivering critical infrastructure projects.
This demand for greater efficiency and value within Network Rail is typified by Rail Project SPEED (Swift, Pragmatic and Efficient Enhancement Delivery.
“Rail Project SPEED, forged alongside Network Rail, is a call to arms to cut unnecessary red tape, be bold and purposeful, and empower the railway to be radical in its thinking to halve the time and reduce the cost of delivering infrastructure projects.”
Chris Heaton-Harris, Rail Minister
SPEED is just one of many drivers for change across Network Rail.
A series of reports including: The Hendy Report, Nichols Review, and the unprecedented Covid-19 pandemic have significantly impacted Network Rail’s capital project delivery.
In a bid to ‘put passengers first’ and create a more customer-focused, service-driven organisation, Network Rail devolved the delivery of capital projects to the regional level.
However, this presented several challenges for the existing capital projects reporting platform:
Legacy obsolescence
The existing business intelligence and forecasting technology used for Regional Capital Delivery reporting became dated and out of touch with the modern demands of complex engineering and infrastructure programs. Source systems were becoming impenetrable, and there was a danger that local workarounds would cause excessive complexity and confusion.
Need for a unified data strategy
Whilst each region had its own reporting needs, Network Rail still required coordinated governance of the data definitions, data structures and dictionaries across the group. Regions needed local autonomy but within the constraints of a common data strategy. Therefore, it was essential to empower the business with a single set of data, so everyone could do their own thing using a common data set – a significant challenge for the earlier reporting infrastructure.
Delivery process upheavals
Network Rail had transformed its process and standards for major engineering projects, transitioning from the earlier GRIP (Governance for Railway Investment Projects) process to the new PACE (Project Acceleration in a Controlled Environment) approach. This shift would require the legacy reporting system to support even more change it was ill-equipped to deliver.
Transitioning to the next generation of Project Reporting Analytics
Network Rail faced a dilemma.
Do they adapt the older technology to support the new devolved structure or take the opportunity to transition to a new solution that would meet current needs and deliver a platform for future growth and innovation?
In particular, Network Rail was looking for an agile solution that would move faster to allow for the ambiguity around technology changes. They needed a flexible platform to have their changing needs addressed as the business became more advanced.
In the face of growing demand for change, Network Rail opted for the path of innovation.
Which brought the Oakland Group to their attention.
Why Oakland?
Given our deep expertise across capital delivery and technology programs, the Oakland Group were selected to work with Network Rail to lay the foundation for reporting transformation within the capital delivery organisation.
“Central to our replacement (of a larger consultancy) was our delivery mindset.
The Oakland team was small and lean yet highly skilled.
We had the experience of talking business (capital programs) and truly understanding what the delivery community wanted.
But our team was also able to deliver change down to the deepest technical layers so that Network Rail could address all aspects of software, data, infrastructure and security.
Where most firms prefer to pitch for larger teams, with many handoffs and interfaces, we prefer leaner, more agile delivery teams that reduce the cost and complexity, ultimately delivering better, faster, higher-quality projects.
Our highly specialised, all-encompassing skill set delivers the right approach for the type of technical and business transformation that is common within large enterprises such as Network Rail.”
Richard Corderoy, Managing Director
Predicting the future of self-service reporting
Oakland began by helping Network Rail reimagine what could be achieved with intelligent forecasting and reporting.
In addition to a shift towards predictive analytics, it was clear the regional teams needed a faster and more flexible way of harnessing reports. The capital delivery teams didn’t want to wait for technical teams to design new reports. The regions needed autonomy to craft reports that met local needs within a coordinated framework and centrally governed data strategy typical of a progressive data-driven enterprise.
Network Rail users had become frustrated that due to the restrictions of the previous Oracle BI reporting toolset, they could not customise and update their reporting needs quickly. Therefore, delivering self-service analytics was a crucial shift in Network Rail’s culture and technical landscape, but one that Oakland was all too familiar with from previous implementations.
The challenge with building an entirely new reporting and analytics capability was that Network Rail needed to understand the art of the possible before committing to any final designs and solutions.
As one team member described it: “It’s hard to know what advanced analytics can deliver if you’ve never seen it before!”
“Network Rail had a legacy capital projects reporting capability. It was giving them insight into what had happened but without any foresight.
Capital delivery teams not only needed the ‘so what’ but the ‘what next’.
They needed a predictive capability.”
Martin Pocock, Delivery Lead
To compound the challenge, Network Rail was structurally, operationally and technically, very much a ‘moving target’ in terms of varying demands on:
Structural
The organisation had recently devolved control and now effectively operated as several distinct businesses.
Operational
With a shift from GRIP to PACE (the delivery standards within Network Rail), major changes were expected in how the organisation delivered major capital programs from an operational standpoint.
Technology
Against this backdrop of organisational change, Network Rail had also planned a roadmap of enterprise technology standards that were still evolving but would also have to be followed.
Commercial
Network Rail relies on a varied pool of transformation and delivery partners; part of the appeal of working with Oakland was our ability to map out the technical, commercial and process-related interfaces with each partner to ensure smooth delivery of the new data platform.
Fortunately, Oakland has over three decades of experience working in these complex climates of layered change and technical innovation.
Building quick wins through the ‘Lighthouse Projects’ concept
It was evident from the program’s early days that a long, drawn-out ‘waterfall’ style program would be inefficient and unlikely to deliver. Events were dynamic within Network Rail, so Oakland adopted an agile and pragmatic approach to solution delivery.
The solution was to prototype quickly, gain feedback, then refine and release to the business.
The Oakland Lighthouse Projects approach to digital transformation ideally suited the needs of Network Rail.
Because our team was small yet highly specialised, they could showcase concepts to the business, gather requirements, and then turn around an operational solution in a fraction of the time of traditional IT projects.
Rather than waiting months for a new feature or capability to ship, the Oakland team would engage the frontline capital projects teams to gain a deeper insight into the type of decisions and forecasting required, both at the regional and central levels.
Due to the variety of stakeholder demands, we used a select number of user stories after requirements analysis to keep the scope of the project to a reasonable size while meeting most of the demands.
This insights discovery process helped the team to build precisely what was needed.
Oakland understood the data and business operations intimately and routinely suggested new ideas and changes based on what they knew would help the business perform more effectively.
As each release was ‘shipped’ and refined, engagement within the capital teams grew, which didn’t go unnoticed with senior sponsors.
“We didn’t walk in with a pre-built solution and impose it on the business. Instead, we made a conscious effort to integrate and work with them.
A key difference is that we’re not just tech-heads or management consultants. Our teams span across business, data and technology.
Combining this type of team with the Lighthouse Projects approach meant we could deliver faster, in discrete stages, so the client got tangible value from each drop.
This consistent effort to deliver value soon built a belief and confidence in what we were trying to achieve within Network Rail.”
Martin Pocock, Delivery Lead
Switching to a more open and efficient architecture and delivery model
Network Rail was keen to mature towards open technology standards to move away from ‘lock-ins’ with a particular vendor or supplier. As a result, we opted for standard ‘off-the-shelf’ tools readily available in the marketplace, such as Microsoft Azure and DataBricks.
This open approach contrasted with the existing technology, which had become increasingly complex, costly and impenetrable to access or extend. Adopting a more open approach created a more sustainable and longer-term technology platform at a lower cost of ownership.
In addition, the previous solution consisted of two data pipelines loosely joined together, which we merged into one, creating less maintenance and faster update times from 24 hours of processing to 4 hours or less.
Microsoft Azure (the underlying platform selected) has a vibrant marketplace of tools and plenty of skilled resources. All of the software chosen was affordable and extensible – desirable standards within Network Rail.
Data Lake UI and Azure Data Factory have allowed pipelines for the manual upload of files to be built faster. Previously it required a website to upload the files as well as a pipeline. These both needed to be edited whenever a new type of file got added, significantly adding cost and delays.
Network Rail had not yet standardised the business on the Azure platform at the beginning of the project, so it was rewarding for the Oakland team to share their guidance and experience from many previous implementations.
“Building a data platform during a shift in enterprise data architecture standards presented an exciting challenge.
The team had to make a calculated guess as to what the central organisation would build in future releases.
They would collaborate and ‘dock’ the reporting platform to the moving target of their enterprise infrastructure, often predicting how the central design teams would adapt their solution, three or four releases into the future.
It was ultimately successful, which in all honesty, many people found surprising given the tremendous amount of ‘shifting sand’ the team were building on”.
Richard Corderoy, Managing Director
Building a security apparatus that satisfied users and stringent approvals
Despite the open architecture, there remained a stringent demand for security due to the evident sensitive nature of the data.
Oakland has a formidable background in public sector infrastructure combined with deep technical expertise. This experience helped design security protocols and policies that met clear guidelines and enabled the regional teams to create and share reports with far greater flexibility. This self-service approach dramatically boosted the level of internal productivity and decision intelligence.
Oakland delivered this solution within a security framework compliant with the approvals process for critical national infrastructure.
Rapid delivery through a small, highly skilled multi-disciplinary team
It’s a popular misconception that assembling large teams to deliver a data platform will accelerate delivery. In our experience, the exact opposite is the case.
Our approach on this project was to assemble a team with an integrated skill set, covering not only hard-core data engineering knowledge but also proven experts with deep knowledge of Dev-Ops, business intelligence, machine learning, cloud architecture, data architecture, data strategy, networking and cybersecurity.
When delivering a complex data initiative that transforms a large volume of disparate data into one consistent data lake and warehouse, it’s essential to have great communication, research, planning, tenacity and trust in the character of the team.
Engaging the business and technical teams to accelerate change and take ownership of the solution
Central to the initiative’s success has been the skills increase we’ve delivered into the business.
Engaging the business community
With the previous system, there was growing resistance to the reporting platform because the users felt frustrated at its complexity and lack of accessibility.
With the new system, we not only trained the users on how to master the core functionality of tools like Power BI, but we also helped them to ‘think like analysts’ and become more creative with the types of reporting they can produce.
In reality, most of the reporting needs will remain straightforward, but the reports are now accurate, simple to access and easy to adapt.
Engaging the technical community
The client groups in Network Rail had limited experience of cloud technology and architecture beforehand, so we helped upskill them to understand the architecture options we had selected.
Equipped with this new level of expertise, they can now deliver first and second line support of the data platform without any dependence on Oakland.
“It’s been encouraging to see the senior leadership team take an active role in the new platform.
Through our roadshows and training events, we’ve not only been able to showcase the essentials, those day-to-day reporting staples, but also the more advanced predictive capabilities that leverage machine learning and artificial intelligence.
That ability to more accurately predict the future has captured the imagination and interest at a senior level.
So it is this top-down meets bottom-up approach that I feel was an essential ingredient for the overall adoption and success of the platform”.
Martin Pocock, Development Lead
The next generation of intelligent forecasting begins at Network Rail
Oakland has helped deliver a total evolution in decision support within Network Rail. Here are just some of the benefits this project has delivered:
Greater automation
The old data aggregation and reporting platform has been replaced with a new system that automatically pulls data from far more source systems, eliminating the need for manual addition.
Reduced lead times
The previous solution consisted of two data pipelines loosely joined together, which we merged into one, creating less maintenance and faster update times from 24 hours of processing to 4 hours or less.
Secured accessibility
Data is securely organised within a state-of-the-art ‘Data Lake’, but easily accessible via a separate business-friendly portal that is easy to use and understand.
Increased skills
The business now has the skill to utilise Power BI tools that deliver the reports, insights and analysis required across each region.
Smarter decisions
Machine learning and advanced analytics tools have been deployed, offering even greater forecasting and decision analytics than previously possible, potentially saving millions of pounds. These insights become just another information source that users can incorporate into their reports.
Greater trust
Operational data is verified, and data definitions are published and maintained, improving data quality.
Flexible access
Security is flexible enough to allow users to build reports for inter-region consumption or national-level analysis.
Future-proofed
The cloud architecture is open, scalable, aligned with corporate standards, and ready for the future.
Faster decisions
Reporting processes have been streamlined, resulting in faster, better quality decisions, without the manual cost and ‘Excel-hell’ of the previous system.
User enablement
Users benefit from the self-service nature of reporting and analytics delivery. Each region can now ‘do their own thing’ and leverage multiple levels of reporting, including static reports and custom reports (on static models). They even execute ‘build your own models’ and data pipelines with curated data, giving them full customisation of data processing – whereas they could only download Excel files from Oracle BI previously.
Capital savings
Intelligent project analytics has delivered far greater transparency into projects that have misallocated contingencies and can be ‘flushed out’ of the portfolio roadmap, leading to the potential for substantial savings each year.