Background
Local authorities routinely collect vast amounts of data: council tax records, education and social care files, housing datasets and service logs. This data is generated simply by councils doing their daily work; it was not designed to be used for research or evaluation. My research starts from the premise that unlocking its wider value is not just a technical question. It is a question about how people, systems, and the regulation around them either enable or constrain the effective use of routine data beyond its original operational purpose.
That premise has empirical backing. A rapid systematic review of 28 UK studies by Moorthie and colleagues (2022) found that the most frequently cited barriers to accessing, linking and using local authority data were technical capability and data quality, followed closely by legal and ethical frameworks, funding and capacity, cultural factors, data fragmentation, and public trust. What is striking about that list is how few of the barriers are purely technical. Most sit at the intersection of technology, organisation, and culture, which is exactly what sociotechnical theory was built for.
Why sociotechnical theory
Sociotechnical theory traces back to 1949, when the British National Coal Board engaged the Tavistock Institute to investigate why some UK coal mines achieved high productivity and morale while others using similar machinery did not. Eric Trist examined the relationships between the social and technical components of mining work (Trist, 1981). The answer to the productivity puzzle was not the technology itself but how well it fitted the workforce: teams that were upskilled, given choice, and treated as complementary to their tools consistently outperformed those where technology was imposed.
At the heart of the theory is the idea that social factors such as leadership and culture, and technical factors such as systems and infrastructure should be jointly optimised, with a core principle that the two are equally important (Cherns, 1976; Whetton, 2005). Improving one subsystem while neglecting the other achieves little. A council can install the best software available, but if staff lack the capacity, training, or confidence to use it, the investment is wasted. Similarly, skilled and motivated staff working within siloed arrangements and risk-averse cultures will struggle to fully leverage the potential of routine data. Therefore, I will employ this theory to examine routine data use in Somerset Council for operations, evaluation and research.
Where the theory does its work
Clegg et al.’s hexagon model separates the social side into people, culture, and goals, and the technical side into technology, infrastructure, and processes, all sitting within a wider frame of stakeholders, regulation, and financial circumstance (Clegg et al., 1979).
Clegg and colleagues used the hexagon model to trace how the Mid-Staffordshire NHS failures emerged from interacting elements: a culture tolerating poor standards fed into an absence of quality goals, inadequate nursing capacity, no processes or technology for monitoring care quality (Clegg et al., 2017). Challenger and Clegg (2011) performed a similar analysis of the Hillsborough disaster, finding failures in every one of the six elements, from an official mindset fixated on hooliganism to radio failures and poorly laid out grounds. Neither case can be explained by a single broken component; both were failures of joint optimisation.
Applying this concept to local authorities’ routine data use reveals insights that a purely technical inquiry would miss. Legal ambiguity around data sharing is often treated as a compliance issue. But Moorthie and colleagues (2022) found that lack of familiarity with the legal frameworks governing inter-agency sharing contributes to a risk-averse approach; institutional caution, shaped by culture and incentive structures as much as by law, is what actually determines whether staff feel able to share data at all. Fragmented IT systems look like a technical problem on the surface: legacy systems, bespoke formats, and coding practices unique to individual teams mean data cannot easily move even between teams within a single council (Moorthie et al., 2022). The reasons the fragmentation persists are frequently organisational. The same review found no example in the literature of a designated senior officer for data within any local authority.
Councils are estimated to spend around 3% to 6% of budgets on IT, yet much social care data is unstructured and by one estimate up to 90 per cent of unstructured data is never analysed (Moorthie et al., 2022). Spending on technology is evidently not the main constraint.
Councils that succeed in unlocking routine data, for research and evaluation as well as operations, tend to be the ones where technical investment, governance clarity, and cultural buy-in move in step with one another. The cross-cutting factors Moorthie et al. identify are trust between stakeholders, leadership, and capacity, none of which can be bought as software.
Testing the theory in practice: Somerset Council
My PhD applies this framework as a single-case study. Using a mixed-methods design grounded in critical realism, I am combining semi-structured interviews across the council and the Integrated Care Board, document analysis of existing data strategies and governance frameworks, and a quantitative assessment of routine data for quality profiling. The findings will feed directly into practical recommendations for how Somerset Council and by extension other local authorities with similar context can better integrate, govern, and use the data they already hold.
A broader lesson
The potential of routine data for local authorities does not lie in better software nor in skilling up staff without securing the space to exercise the knowledge they acquired. It lies in recognising data use as a sociotechnical system where people, technology, and environment must evolve together. As local authorities attempt to do more with less, understanding how to achieve that joint optimisation may be one of the most valuable contributions this research could offer.
References
- Challenger, R. and Clegg, C.W., 2011. Crowd disasters: A socio-technical systems perspective. Contemporary social science, 6(3), pp.343-360.
- Cherns, A., 1976. The principles of sociotechnical design. Human relations, 29(8), pp.783-792.
- Clegg, C.W., 1979. The process of job redesign: signposts from a theoretical orphanage?. Human Relations, 32(12), pp.999-1022.
- Clegg, C.W., Robinson, M.A., Davis, M.C., Bolton, L.E., Pieniazek, R.L. and McKay, A., 2017. Applying organizational psychology as a design science: A method for predicting malfunctions in socio-technical systems (PreMiSTS). Design Science, 3, p.e6.
- Moorthie, S., Hayat, S., Zhang, Y., Parkin, K., Philips, V., Bale, A., Duschinsky, R., Ford, T. and Moore, A., 2022. Rapid systematic review to identify key barriers to access, linkage, and use of local authority administrative data for population health research, practice, and policy in the United Kingdom. BMC Public Health, 22(1), p.1263.
- Trist, E.L., 1981. The evolution of socio-technical systems (Vol. 2, p. 1981). Toronto: Ontario Quality of Working Life Centre.
- Whetton, S. 2005. Health Informatics: A social-technical perspective. South Melbourne: Oxford University Press.