How I Helped the Nation Spend Wisely for a year

Aka reflections on my year as a Data Science Intern at the UK National Audit Office.

Executive summary

I spent a year as an intern within the Analysis Hub at the UK National Audit Office, and I thoroughly enjoyed it. I learned that writing code is fun, that collaboration is hard (but essential), and that it’s important to focus on why I’m doing a piece of work.

Background

The National Audit Office scrutinises UK government spending, making sure that taxpayers are getting good value for money (among other duties). Within the office are three teams specialising in different applications of data analysis that join forces to form the “Analysis Hub”. As an intern I spent my year in the Analysis Hub, with roughly equal stints in the Modelling, Value for Money analytics, and Financial Audit analytics teams.

What I learned

Software development is fun

Before joining the NAO I hadn’t written code in a very long time, and I’d forgotten how satisfying and enjoyable it is.

My first project with Elia¹ was to build an R Shiny web app that would allow analysts to quickly pull together a timeline of events to include in their reports. It was a great first project - if done successfully it would solve an actual problem (creating timelines was super time consuming), but it was no big deal if it didn’t succeed. We jumped right in to development and were quickly confronted with an array of problems: unexpected crashes, unclear dependencies between different components, and huge functions that seemed to be trying to do many different things at once. We worked through these problems, getting used to Shiny’s reactive programming as we went and learning how to write cleaner, more robust code.

Throughout the internship I was able to work on a range of projects, but by far the most enjoyable for me were those where I was building tools that others would use. For example, when I was working on making our apps more accessible I created an R package for us to use internally, and from this I was able to learn about how packages were structured, how to write good documentation, and how to raise useful errors when functions are used incorrectly. I also got to take on a much more substantial development project at the end of my internship: redesigning and rebuilding our internal Data Service. I quickly realised that while it was easy to point out flaws with the previous design and its implementation, it was a lot more difficult to create a new design that was 1) intuitive to use, 2) provided most of the same functionality, and 3) didn’t require huge changes to the back end infrastructure. Nonetheless, by creating a wireframe first I was able to work with stakeholders to interrogate and tweak the design before getting bogged down with the specifics of implementation. Then I got to work with the other two interns to build the new service, using a modular design to make the code cleaner and make collaboration much easier. Which nicely brings me to the fact that…

Collaboration is hard (but essential)

Throughout university I had done very few group projects, and those that I had done quite naturally lent themselves to being split up into quite independent parts. Out in the Real World though, almost nothing is done as a completely individual piece of work. And although working with other people to write code is hard, when done right the end product turns out much better.

As mentioned above, my final project was to work with the two other interns to rebuild a data querying interface. In order to facilitate collaboration I designed the app so that it would be composed from a number of smaller modules, each only passing specific bits of data and parameters to other modules. We had to think about what the output of one section of the app would be before we built it, so that someone else could build another component that would connect to it. And whenever I was writing code I had to think “would this make sense to me if I was reading it for the first time? Are the variable names clear? Have I generalised enough so that this function can be used in other contexts? Will this handle unexpected inputs?”. Then when I put in my pull request I’d get a rigorous review from the other interns - they could spot bugs that I hadn’t noticed, suggest a faster or cleaner way to do something, or just ask clarifying questions when something wasn’t clear. Although it could be tempting to wave through pull requests without interrogating their contents, by taking the time to go over each other’s work we were able to catch bugs early and grow the app while still allowing each component to be easily understandable.

Inevitably we would have disagreements on design or implementation decisions, but I found it helped to…

Always focus on ‘why’

It’s fairly easy to specify ‘what’ a product/output should be (e.g. “a service that does X”) and ‘how’ it should be done (e.g. “using an R Shiny app”), but it can be more difficult to justify ‘why’ it is necessary. Yet ultimately if not enough focus is given to the why, you can end up with a technically impressive output that isn’t what the customer/user/commissioner wants.

That’s why throughout my internship I made sure I always understood why I was doing a certain bit of analysis or creating a certain product. I realised that just because I could do something (e.g. add a feature or do some extra analysis) didn’t mean I should, and that any work done that’s outside the scope of the project will just give me less time to focus on what is actually required. I learned how important it is to get a scope right if you’re creating one, and to constantly refer back to the scope when you’re carrying out a project.

Thank you

A huge thank you to the NAO and the Analysis Hub for giving me this unique opportunity. I want to also give a big shout-out to my performance coach Joe Weir, who has been incredibly supportive this year and has helped me navigate the many intricacies and acronyms of the NAO.

Next steps

After attending the TakeAIM applied maths conference in March 2024 (thank you Laura Cole at the NAO for bringing me along!) I realised that 1) I missed maths, and 2) applied maths is actually quite cool. Armed with these revelations I did some research and found what seems like the ideal course: an MSc in Computational Applied Mathematics at the University of Edinburgh. I applied, got in, and now in just over a week I’ll start classes. I’m excited for the hustle and bustle of uni life, and for learning important skills that will help me with either further research or a career in software engineering.