Expanding discovery across the UK health data landscape

The Cohort Discovery Service helps researchers securely explore and size potential study groups – without moving or exposing the underlying data – before applying for data access. It uses a federated approach, meaning that rather than centralising data, user searches are sent to each custodian who holds the data, and only aggregated totals are returned back to the researcher. This means researchers can understand feasibility and refine study ideas while data always remains under custodian control.

The service is designed with privacy at its core. Researchers never see patient-level data: Queries are run on pseudonymised data within secure custodian-managed environments and only aggregated totals of patient numbers available in searchable datasets are returned. Patient counts are also rounded, and small numbers are suppressed to reduce any risk of re-identification. Learn more about how Cohort Discovery keeps data secure.

The federated onboarding of data from the Clinical Practice Research Datalink (CPRD), the Cohort Discovery Service now provides approved researchers with far greater reach when assessing and defining study cohorts. This addition takes the total number of patient records available to search through the service to more than 100 million. These are health records rather than a population count of unique people, as individuals can have records in multiple places and records are included for both living and deceased individuals. Together, they provide a rich resource for population-level health research across the UK.

Infographic promoting the Cohort Discovery Service for custodians. The headline reads: Onboard datasets to the Cohort Discovery Service and enable federated searches across them. Illustrated researchers use a search interface to explore multiple datasets without sending individual enquiries. Search results help determine dataset availability before submitting data access requests, resulting in more targeted, better prepared requests. The graphic highlights benefits including fewer unnecessary enquiries, better quality requests, privacy safe queries, and increased dataset visibility and research impact.
Onboarding datasets to the Cohort Discovery Service enables researchers to perform privacy-safe federated searches across multiple datasets, helping them assess data availability before submitting access requests.

CPRD: unlocking new potential in primary care research

The CPRD database is the UK’s largest collection of routinely collected electronic health records from UK primary care practices. Spanning decades of demographic, diagnostic, prescribing and clinical data, CPRD provides a uniquely comprehensive view of real-world patient care.  

The federated onboarding of anonymised CPRD records for over 49 million patients into Cohort Discovery means that researchers can now identify larger, more representative study cohorts drawn from primary care data. 

The dataset has been transformed into the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM), part of the OHDSI (Observational Health Data Sciences and Informatics) community’s open, standardised framework.  

This standardisation ensures medical terms and variables are consistently structured and comparable across datasets, driving progress towards more reproducible, multi-source research that can accelerate insights and collaboration across the health data research ecosystem.  

Researchers can start exploring today via the Gateway’s free Cohort Discovery Service (approval and sign in required). Access to use anonymised UK population representative CPRD primary care and linked data for a research project is governed by a licence agreement and subject to protocol approval. 

Puja Myles, Director of CPRD, says:

“We’re delighted to see the CPRD OMOP Aurum data incorporated into the Gateway’s Cohort Discovery Service, ensuring these important data are findable by researchers looking for regulatory grade primary care data for research with public health benefit. By permitting researchers to explore multiple data sources, this service helps researchers undertake initial feasibility assessments and link with relevant data providers, streamlining the early stages of their research.”

Millions more records also discoverable through Cohort Discovery

Beyond CPRD, Cohort Discovery continues to grow with the federation of large-scale datasets that further enhances coverage across UK healthcare settings. Each dataset has been standardised to the OMOP Common Data Model, supporting interoperability and harmonisation across data sources.

The following examples illustrate how these datasets are expanding the breadth and depth of linked, research-ready cohorts and making them discoverable to researchers through Cohort Discovery.  

Infographic titled: Cohort Discovery – A walkthrough. It shows the six-step process for using the Cohort Discovery Service. A researcher first creates an account via the Health Data Research Gateway and applies for access. Once approved, they use a query builder to define a study population. The query is securely executed in real time across multiple pseudonymised datasets, returning aggregated, rounded counts of matching records from each dataset. Using these results, the researcher can then submit a better informed data access request to the relevant data custodians. Illustrations of researchers, dataset icons, arrows, and database symbols visually represent the workflow.
The Cohort Discovery Service helps researchers securely search datasets and submit better informed data access requests.

Barts Health NHS Trust: comprehensive secondary care data

The Barts Health Data Platform (BHDP) includes data on approximately 3.3 million individuals, covering inpatient, outpatient, pathology, radiology, maternity, critical care and pharmacy records from across the Trust. Capturing activity from 2012 onwards, it provides a rich, longitudinal view of secondary care in a large, diverse urban population. 

Integrated with the Trust’s SECTRA enterprise imaging solution, anonymised diagnostic images can be securely exported to the research environment, enabling researchers to combine structured health records with imaging for deeper clinical insights. 

The dataset extends across a wide clinical scope (hospital admission through diagnostics, treatment, and follow-up) in a UK (England) healthcare setting, making it well-suited for translational, outcomes and real-world evidence studies.

Optimum Patient Care Research Database

The Optimum Patient Care Research Database (OPCRD) is one of the largest and fastest growing data sources for routinely collected, primary care electronic heath records and patient reported outcomes across all major UK clinical systems. OPCRD [Gateway metadata] holds anonymised data for 28 million patients and is representative of the UK population when compared against nationally published datasets. 

OPCRD is developed and maintained by Optimum Patient Care (OPC), a not-for-profit social enterprise. OPC’s network of over 1,200 GP sites across the UK can uniquely support users to accelerate research delivery, including the identification and recruitment of eligible patients for clinical trials. Linkages to secondary care data sources, including Hospital Episode Statistics (HES) and Akrivia’s mental health trust data are also supported.  

Chris Price, Commercial and Legal Director for OPC UK, says:  

“OPCRD provides a rich source of real-world evidence for public health and outcomes research. We are delighted to collaborate with HDR UK to allow researchers to run feasibility searches against OPCRD through the Gateway’s Cohort Discovery Service, to explore the potential of OPCRD’s data before progressing to detailed research studies.

 

Access to OPCRD’s is supported by a streamlined governance procedure which enables datasets to be approved, and access provided within just three weeks of request. To date, OPCRD has been used in over 120 publications, and we remain committed to accelerating research delivery in the UK through our unique and fully supported delivery models.”

Public Health Scotland: deepening national coverage across Scotland’s health data

Public Health Scotland (PHS) was one of the first adopters of the Cohort Discovery Service, helping to establish national coverage early on and demonstrating the value of federated feasibility queries across large-scale health datasets. 

These include:

  • National Records of Scotland – Deaths Data Dataset, offering detailed mortality records to support studies of population health and outcomes [update in progress] 
  • Outpatient Appointments and Attendances – Scottish Morbidity Record, covering outpatient activity across Scotland’s NHS services [update in progress]
  • General Acute Inpatient and Day Case – Scottish Morbidity Record, providing rich data on hospital admissions, diagnoses, procedures and discharge information 
  • Mental Health Inpatient and Day Case – Scottish Morbidity Record, detailing admissions and episodes of care within specialist mental health services [update in progress]

Together, these datasets provide a comprehensive, longitudinal picture of health and care interactions across Scotland’s population, supporting studies in disease progression, service delivery, and health outcomes. Their federation within Cohort Discovery ensures that researchers can explore and define cohorts spanning all major healthcare settings – primary, secondary and mental health – within a consistent, standardised analytical framework.

Carole Morris, Head of Data and Modelling Services at PHS, says:  

“Much like designing a new kitchen, the Cohort Discovery Service gives researchers the space to explore what’s possible and better understand the data available to them, helping them shape ideas and assess feasibility at an early stage. From there, Trusted Research Environments can work alongside researchers to refine those plans, put the right permissions in place, and prepare the data, so researchers can focus on turning their ideas into meaningful research.”

Driving scale, consistency and collaboration in UK health data research

Through increasing participation from data custodians including CPRD, Barts Health NHS Trust, OPCRD and Public Health Scotland, Cohort Discovery now enables researchers to explore and size potential study populations across 100+ million patient records before applying for data access. Their collective availability and standardisation reflects increasing collaboration across the ecosystem and provides a secure, federated and consistent service for large-scale, reproducible health research. 

Get started

Researchers can begin exploring potential cohorts straight away through Cohort Discovery on the Gateway (access approval required). The service was recently updated to help users spend less time navigating technical complexity and more time exploring research possibilities. Find out what’s new and how the redesigned service can better support research opportunities in our latest blog. 

Data custodians can participate by onboarding their datasets and enabling cohort searches across them, helping to improve the quality and efficiency of research engagement. Visit the Support Centre to find out how to get started.

Book a demo

Explore what’s possible with Cohort Discovery and book a personalised demo to learn how the service can help you build queries, explore potential patient populations and assess research feasibility with greater ease.

Book a demo with our team

About the Cohort Discovery Service

Cohort Discovery helps researchers quickly assess whether relevant patient cohorts exist across multiple datasets, without accessing identifiable data. This enables a clearer and faster starting point for planning studies and supporting more efficient and informed data access requests. 

Available through the Health Data Research Gateway, the service is supported by a federated analytics ecosystem that enables queries to run securely within Secure Data Environments (Trusted Research Environments) using tools like Bunny (developed by the University of Nottingham as part of the HDR UK Federated Analytics programme) and data pre-processing and transformation support from the Health Informatics Centre (HIC) at the University of Dundee.

The Cohort Discovery Service is developed and maintained by HDR UK and builds on earlier work from the CO-CONNECT programme, which enabled researchers to rapidly discover and access COVID-19 data while ensuring patient information remained private and secure.