How the Welsh Government migrated their technology to the cloud
Summary
The Welsh Government:
- organised the Future ICT project to move systems and services to the cloud between 2016 and 2019
- created a multidisciplinary team to organise the migration, which included business and change management, digital, and communications staff
- employed a small developer team for SharePoint and SQL database development
- used an Agile approach throughout the whole project
Project aims
The Welsh Government wanted to modernise their technology, and developed a strategy that was more aligned with the UK government’s policy.
The Welsh Government also saw the opportunity to move away from traditional working patterns and practices. One of the main benefits of the migration was to equip the business with devices, software, connectivity and collaboration tools so that jobs were no longer tied to a particular building or office. By changing HR rules on flexible working and moving away from working at fixed desks, the Welsh Government was able to introduce a smart working policy with greater focus on productivity and wellbeing.
The contract between the Welsh Government and ATOS, who provided a range of services such as application support, was ending and they could not extend it again. The Welsh Government decided they wanted:
- new ways of delivering services
- more control over their commercial lock-in options
- to equip the organisation with better infrastructure and technology, as laid out in their 2016 ICT strategy
About the Welsh Government
The Welsh Government has devolved responsibility for a portfolio of policy areas such as:
- health
- local government
- economic development
- natural resources
- agriculture
- education
The Welsh Government manages these policy areas through a single government body rather than separate, Westminster-style departments. It has an annual operating budget of about £250m and about 5,700 staff.
Preparing for migration
Initial preparation for moving to public cloud started in 2014. This was the first opportunity the Welsh Government had to end their outsourcing contract. But after considering a lot of options they decided that public cloud did not yet have the functionality and security they needed, and chose to extend their contract with their supplier. This extension gave them time to revise their ICT strategy and vision, which they completed in 2016. It also gave them time to plan how they would migrate to the cloud.
Outlining plans and priorities
During the planning stage they decided to start with the parts of their systems that were invisible to users, such as moving the infrastructure to Exchange 365. They chose Azure as the platform for their content management system (CMS) because they were already using that supplier and it was the only product at the time that met all their security requirements.
The project team also planned how to deal with their legacy technology and services. They kept all legacy well maintained and designed their new architecture to take over from the old architecture as it became obsolete. Towards the end of the project they added a Hyper Converged hosting solution (HCI) for services they could not transition to the new architecture. HCI allowed the team to create a flexible virtual infrastructure for these services. The services were still available, but they were able to run them at a lower cost.
The project team also looked at an initial selection of services that they assessed as being ready for the cloud. During the project planning phase, they prioritised these services based on the security assurance and cyber vulnerability analysis. As the team deployed services, they defined the security and supporting security accreditation processes which allowed them to deploy services faster.
Carrying out the project work
The Welsh Government continued with their outsourcing arrangements for the business-as-usual work while the in-house team of contractor and staff engineers did the Future ICT project work.
The team worked on:
- migrating to Exchange 365
- moving from GSI email and adopting the latest email standards
- migrating staff to new smartphones
- doing a major application migration to Azure
- migrating staff to new laptops
Migrating data
Migrating the data was the hardest part of the project. They had at least 33 million documents amounting to approximately 20 terabytes of data. Keeping old and new systems in sync while they migrated this much data was hard and time consuming. During the data migration phase for the Electronic Records and Document Management System (ERDMS) the team had to maintain constant service availability. They created a secure Virtual Private Network (VPN) transit from the storage source to the cloud destination and used the supplier’s utilities to bulk copy and sync file data changes.
The whole migration took 2 weeks, but to make that possible, they had spent the previous 2 years introducing and enforcing a rigid data and records management policy. This policy includes auto deleting any content stored in personal drives or One Drive after a certain amount of time. This was not always popular with the staff but it meant that they saved their files in the right format in the records management system, which made the data cleaner and easier to migrate.
Moving from the old to new domain
To make it possible to migrate to a new environment the team invested in some legacy technology to make it stable. The team also created a new Welsh Government domain to set up a new managed network infrastructure. This provided a clear line between the 2 infrastructure domains which meant they could set up the new domain to meet their needs and gradually migrate data and services from the old domain.
By choosing this migration method the team could continue to provide a seamless service whether it came from the old or new domain. Keeping the domains in sync was hard but the benefits were that the project team could deploy modern cloud-based user authentication, device management and up to date encryption on the new domain.
The team aimed to migrate all applications to the new domain, but knew that they could not migrate some legacy applications. They decided to keep a small legacy network until they could remove or replace the applications on it.
New technology for staff
One of the last things the project did was migrate all staff onto new laptops. The team did extensive consultation with the staff about their needs and device preferences, and ran a series of pilots with volunteer users from across the organisation.
The complexity in this process came in moving from a Windows 7 Citrix environment. The project team had to configure a hybrid type environment, allowing them to maintain multiple versions of Outlook throughout the rollout and software upgrade process. They had to deal with a fairly complex migration plan. This included strict dependencies to maintain interoperability between teams of users who were using these different versions of Outlook, and a mixture of on-premise mailboxes and cloud mailboxes.
They had to do all this while balancing the need to future-proof their technology as well. Some of the older versions of Outlook were not compatible with the new environment, making them harder to migrate.
Overcoming obstacles
The project had a few obstacles, some of which had technical solutions and some that the project team addressed through managed communications.
Getting stakeholder buy-in
The main obstacle was gaining the confidence of the organisation that such an ambitious project was possible.
The project team got permission from the organisation for each stage of the project and gained confidence by:
- keeping the Board and staff well informed about plans and progress
- completing one element of the project at a time to show the plan worked
- showing the Board the costs and risks should some of the older technology fail
- showing the Board the poor staff survey results about user satisfaction
Also 2 incidents increased the urgency to complete the project:
- the 2017/18 WannaCry malware – the Welsh Government was not affected but their systems were old enough to cause concern
- the Welsh Government experienced a couple of ICT service failures which underlined the organisation’s complete dependence on its systems
Storing some data locally
The Welsh Government still needed to maintain some data and services on premise for a variety of reasons, including security and system age. To mitigate this the team set up a hyper converged computer system. This has allowed them to consolidate the services they could not transition to the cloud, and also have a secure location for backup copies of key cloud hosted datasets.
The team expects they will always need to store some data locally and host a small number of services. The team does not expect to store ‘live’ production data locally, but would store things such as security copies and backups of their strategic cloud-hosted data.
Improving cost optimisation
The team had to do active management and cost optimisation of the cloud environment so that the organisation did not use more cloud services than they needed. They established a Cloud Consumption Manager role for support teams, to focus on cost optimisation. By controlling the consumption of services they were able to bring their costs down.
The project team recommends that any organisation migrating to the cloud should prioritise resources to cloud optimisation. The important thing to do is to identify and analyse early on any applications that you can configure to consume platform services. This could include databases and web hosting containerisation instead of using virtual machine deployment.
Deploying new laptops
Another obstacle included developing a credible rollout plan for laptop deployment and gaining more resources to speed up rollouts. The variety of operating systems used across the organisation made it a complex project.
The team’s solutions were to:
- have good engagement with rollout groups, who nominated champion users to assist with the laptop deployments
- maintain and sync the various operating systems until they could gradually move the data
- issue new laptops with the latest operating system
- mandate a short familiarisation session for all staff as part of receiving their new device
These steps allowed the team to transition to the new laptops while also addressing some technical shortfalls.
Completing the project
The team has confirmed that the new environment is working well. It is stable and the services are all functioning as intended. With the cost optimisation planning the Welsh Government is saving money and gaining from staff efficiency.
Staff running applications can now run a more effective DevOps cycle of improving the applications, because the new platform uses a ‘plug and play’ infrastructure.
The team did post-project surveys and asked for feedback from the staff. The results confirmed the staff are very happy with their new laptops and are also pleased that everyone gets the same kit throughout the organisation. The surveys also showed that the staff feel invested in, and the change in technology has changed the way the organisation works and communicates.
Future plans
To maintain the new environment the Welsh Government is including IT in their ongoing strategy. Wherever possible, the aim is for their platform service model to only use services where they are the service administrator. This is instead of managing cloud hosted infrastructure services.