New linked dataset available to provide insights into earnings and employment in Britain

Posted on

The ADR UK funded Wage and Employment Dynamics (WED) initiative that aims to provide new insights into the dynamics of earnings and employment in Great Britain has made a new dataset available. Accredited researchers can apply to use the de-identified Annual Survey of Hours and Earnings (ASHE) – 2011 Census linked dataset from the Office for National Statistics (ONS) Secure Research Service.

Using the ASHE – 2011 Census linked dataset, researchers can explore how factors such as gender, ethnicity, disability and migration affect individual’s wage levels and pay progression. Access to this data will provide fresh insights into things such as the experiences of young people entering the labour market, job mobility, and career progression to retirement. This will enable policymakers to make better-informed decisions to improve the experiences of people in employment.

About the data

The ASHE dataset is an annual survey based on a 1% sample of employee jobs, drawn from the Pay As You Earn (PAYE) register, and conducted by the ONS. This de-identified dataset contains information on employees’ earnings, paid hours, occupation as well as a limited number of personal characteristics: gender, age, and residential location.

To expand the number of de-identified, personal characteristics that can be observed for employees, the ASHE dataset has been linked to the 2011 Census. The 2011 Census dataset includes characteristics such as educational and vocational qualifications, health, disability, and household circumstances for individuals in England and Wales.

The 2011 Census has been matched to all employees observed in ASHE in either 2010, 2011 or 2012. The ASHE – 2011 Census linked dataset therefore covers the period 1997-2020, but only includes individuals who were in 2011 Census and were able to be matched to individuals in either ASHE 2010, 2011 or 2012. It also includes some enhancements to the basic ASHE data, such as minimum wage rates and survey dates.

The linked ASHE – 2011 Census dataset allows researchers to examine pay and career progression of a cohort of individuals who were in employment around the time of the 2011 Census. This data is linked and made available under the provisions of the 2017 Digital Economy Act, which provides a legal gateway for researchers to access government data in a secure way. For more information about the linked dataset, see the ASHE – 2011 Census user guide.

Huge potential for this dataset

This dataset will help to fill a crucial evidence gap in information about the experiences of individuals in the labour market. Importantly, for example, ASHE – 2011 Census will enable researchers to understand the underlying causes of intersectional pay gaps and provide a deeper understanding of how demographic factors have informed labour market transitions.

The dataset could help answer a range of research questions to inform policy, such as:

  • How do personal characteristics such as household structure affect wage progression?
  • How does wage progression differ depending on characteristics such as gender, disability, or ethnicity?  
  • What role do employers play in wage inequality?  
  • What are the returns on investment from education?
  • What is the relationship between migration and the labour market?

Tim Butcher, Chief Economist, Low Pay Commission said,

The WED project addresses weaknesses in our evidence base – improving the quality of longitudinal earnings data and extending coverage to a broader range of characteristics – that should enable researchers to give new and innovative insights into the wage and employment dynamics of the lowest paid.

How to access the dataset

The ASHE – 2011 Census dataset is available from the ONS Secure Research Service. Access to the dataset is managed securely in line with the Five Safes framework. Accredited researchers can apply to access the data by submitting a project application. For more guidance on submitting project applications, visit the ONS Secure Research Service.

The ASHE – 2011 Census linked dataset was produced as a collaboration between ADR UK, the ONS and the Wage and Employment Dynamics project team. The project team is led by Professor Felix Ritchie and Damian Whittard of the University of the West of England, and involves partners from UCL, Bayes Business School, the University of Reading and National Institute of Economic and Social Research.

Australia’s bold proposals for government data sharing

Posted on

By Felix Ritchie.

In August I spent a week in Australia working with the new Office of the National Data Commissioner (ONDC). The ONDC, set up at the beginning of July, is barely two months old but has been charged with the objective of getting a whole-of-government approach to data sharing ready for legislation early in 2019.

This is a mammoth undertaking, not least because the approach set out in the ONDC’s Issues Paper proposes a new way of regulating data management. Rather than the traditional approach of trying to specify in legislation exactly what may or may not be allowed, the ONDC is proposing a principles-based approach: this focuses on setting out the objectives of any data sharing and the appropriate mechanisms by which access is governed and regulated.

In this model, the function of legislation is to provide the ground rules for data sharing and management within which operational decisions can be made efficiently. This places the onus on data managers and those wanting to share data to ensure that their solutions are demonstrably ethical, fair, appropriate and sensible. On the other hand, it also frees up planners to respond to changing circumstances: new technologies, new demands, shifts in attitudes, the unexpected…

The broad idea of this is not completely novel. In recent years, the principles-based approach to data management in government has increasingly come to be seen as operational best practice, allowing as it does for flexibility and efficiency in response to local conditions. It has even been brought into some legislation, including the UK’s Digital Economy Act 2017 and the European General Data Protection Regulation. Finally, the monumental Australian Productivity Commission report of 2017  laid out much of the groundwork, by providing an authoritative evidence base and a detailed analysis of core concepts and options.

In pulling these strands together, the ONDC proposals move well beyond current legislation but into territory which is well supported by evidence. Because of the unfamiliarity with some of the concepts, the ONDC has been carrying out an extensive consultation, some of which I was able to observe and participate in.

A key proposal is to develop five ‘Data Sharing Principles’, based on the Five Safes framework (why, who, how, with what detail, with what outcomes) as the overarching structure. The Five Safes is the most widely used model for government data access but has only been used twice before to frame legislation, in the South Australia Public Sector (Data Sharing) Act 2016 and the  UK Digital Economy Act 2017.

The most difficult issues facing the ONDC arise from the ‘why’ domain: what is the public benefit in sharing data and the concomitant risk to an individual’s privacy? How will ‘need-to-know’ for data detail be assessed? What are the mechanisms to prevent unauthorised on-sharing of data? How will shared data be managed over its lifecycle, including disposal? To what uses can shared data be put? Can data be shared for compliance purposes? How can proposals be challenged?

These are all good questions, but they are not new: any ethics or approvals board worth its salt asks similar questions, and would expect good answers before it allows data collection, sharing or analysis to proceed. A good ethics board also knows that this is not a checklist: ethical approval should be a constructive conversation to ensure a rock-solid understanding of what you’re trying to achieve and the risks you’re accepting to do so.

This is the also the crux of the principles-based approach being taken by the ONDC: it is not for the law to specify how things should be done, nor to specify what data sources can be shared. But the law does provide the mechanisms to ensure that any proposals put forward can be assessed against a clear purpose test around when data may and may not be shared and that appropriate safeguards are in place…

Finally, the law will require transparency; this has to be done in sunlight. A public body, using public money and resources for the public benefit, should be able to answer the hard questions in the public arena; otherwise, where is the accountability? The ONDC will require data sharing agreements to be publicly available, so people can see for what purpose (and with what associated protections) their data are being used.

To some, this need to justify activities on a case-by-case basis, rather than having a black-and-white yes/no rule, might seem like an extra burden. The aim of the consultation is to ensure that this isn’t the case. In fact, a transparent, multi-dimensional assessment is any project’s best friend: it provides critical input at the design stage and helps to spot gaps in planning or potential problems, as well as giving opponents a clear opportunity to raise objections.

Of course, even if the legislation is put in place, there is still no guarantee that it will turn out as planned. As I have written many times (for example in 2016), attitudes are what matter. The best legislation or regulation in the world can be derailed by individuals unwilling to accept the process. This is why the consultation process is so important. This is also why the ONDC has been charged with the broader role of changing the Australian public sector culture around data sharing, which tends to be risk-averse. The ONDC also has a role to build and maintain trust with the public through better engagement to hear their concerns.

From my perspective, this is a fascinating time. The ONDC’s proposals are bold but built on a solid foundation of evidence. In theory, they propose a ground-breaking way to offer a holy trinity of flexibility, accountability, and responsibility. If the legislation ultimately reflects the initial proposals, then I suspect many other governments will be beating a path to Australia’s door.

All opinions expressed are those of the author.

Back to top