Small numbers, big challenges: Rare disease research in trusted research environments

By Robert Maddison-Jarvis a Public Health Researcher at Public Health Wales

Background

I did my PhD in cystic fibrosis genetics between 2023 and 2026. During it, I hit an unexpected problem.

Cystic fibrosis (CF) is an inherited genetic disorder which causes a wide range of problematic symptoms, including recurrent respiratory infection, respiratory decline, gastrointestinal issues and nutritional issues. CF occurs when a child is born with two defective copies of the CFTR gene. About 4% of people in the UK and Europe are carriers for the disease, meaning they have one copy of the defective gene and one functional copy. As such, when two carriers have children, there’s a 25% chance their child will inherit a defective copy from both of them and develop CF.

In Wales, all potential CF cases have received genetic testing since the 1990s from the All-Wales Medical Genomics Service (AWMGS). My project was to take all the CF genetic test reports generated over the years and explore how useful they are for research. The health system spends enormous amounts of money generating these data, so it is worth understanding whether there are useful insights we can draw from them and, crucially, how we do that.

But these weren’t any old data. They were patient records, containing details about a very significant and very sensitive event in each individual / families life. CF is a life-long condition which, despite recent advances, is also still life-limiting. Accessing, reading and then analysing these reports which mark the beginning of that journey was not something to be taken lightly. NHS employees have the ability to review patient records without special permissions (though they do need a good reason) as this is needed for them to fulfil their duty of care. Academics are a different story. In order for a researcher (let alone -gasp- a student!) access and work with these data, the data had to be anonymised and secured.

This was where the SAIL Databank came in. SAIL is a trusted research environment (TRE) which provides the technology to ensure data is protected whilst it is accessed and used for research. SAIL is locked up tight: I had to be provisioned a special USB key to access their platform, which made me feel like James Bond watching the long string of unique passcode characters stream into my computer whenever I used it. Security is one thing, but another key function of SAIL is that it can anonymise data whilst retaining the ability to link other data about those same individuals together. Each person is assigned a codename, a string of numbers and letters, which is used consistently for that person across all their records, enabling us to match them together. For example, my education records, my hospital records and my residential history together, all without revealing any personal details about me. This allows research using large public datasets to occur in a way which protects the individuals those data pertain to, whilst preserving as much useful information as possible. My project was therefore made readily possible through SAIL.

What was I going to do? One of the primary ways we use information about the genetic variants for a specific disease is to generate what we call allele frequencies; understanding how common these variants are in the wider population. The allele frequency is the number of times you observed a particular allele divided by the total number of alleles observed. For example, if you sample 100 alleles and see a particular allele 8 times, it’s frequency is 8%. These frequency values are then used by genetic counsellors for risk assessing patients, as well as comparing with other populations. The last time CF allele frequencies were calculated for the Welsh population was in a 1993 study that analysed samples from CF families in Wales. 30 years on, it’s a good idea to see if the frequencies might have changed.

The plan was to count the alleles in the new data, compare them with the old, and see if they were different. Simple.

Not so.

Problem

Primary research is messy. Research plans often gleam with well-thought through questions, clear intentions, timelines, deadlines and thoroughly colour coded Gantt charts, but no plan survives first contact with the enemy. The enemy, in this case, is an invisible, multi-faceted and multi-limbed monstrosity that doesn’t care about your project timelines or your suite of highlighter pens or desk organisers. It doesn’t care about much of anything, really. Only reality.

In February 2025, with just under a year before my PhD thesis was due for submission, I was preparing the allele frequency data for output review.

An output review is a complex little thing. SAIL is a secure digital environment which you access through a computer. You can access lots of data and do your analysis inside it freely, that’s what makes it so good, but you can’t take information out of that environment without going through an output review. The purpose of the review is to assess whether your output – be it a graph, a data table, even just a single number – is safe, in respect of protecting the identity of the individuals whose data were used to generate it. This is an extension of the strict protections the TRE applies to the data it holds: as the data used to produce the output are under strict controls to protect individuals, the outputs must be scrutinised to ensure they do not inadvertently undermine those protections.

And it was after requesting my CF allele frequency data be released from SAIL that I was informed I would not be allowed to because my outputs did just that.

This was a big deal. This work made up a quarter of my PhD thesis. I got in touch with the data team and found out more. As a rule of thumb, outputs pertaining to fewer than 10 individuals are considered unsafe. In my data, there were frequently <10 counts for many of the CF variants, as they occur rarely, and so the data were potentially disclosive of the patients who had those variants.

SAIL’s output review policy highlights that appeals can be made against decisions to restrict release of results. Whilst it was possible to present such an argument at the time, I didn’t take that path, primarily because I had to focus on delivering my thesis, and secondarily because it wasn’t necessarily evident what the argument was, what the counter-arguments might be, and who was in the right.

Anatomy of the problem

So, you can’t report small numbers – so what?

According to this paper, rare conditions occur in at most 1 in 2,000 people. Necessarily, then, there are few patients out there affected by these conditions, but when you focus on a small country like Wales, this will mean even fewer. Any study, then, that considers data about these conditions, about these patients, will inevitably run into small numbers. That is likely to be the case whether you’re working on CF or a more obscure disease, or even an ultra-rare condition which doesn’t even have a clear name. Even if you feel you have plenty of data going in to a study, and that you’re unlikely to cross the <10 threshold, you don’t know that for sure until you start segmenting and filtering your data.

Who does this reporting limitation affect?

It affects researchers, like me, who find they cannot publish results from their research study. That may seem frivolous but these studies take money, time, energy (both human and electrical – big computers eat a lot of power) and effort to set-up and fulfil. In a situation where an entire study could not complete due to this reporting limitation, it amounts to an enormous waste of precious resources. In this way, it affects the systems that support researchers, universities, research councils and charities, the system from which the research questions initially came, and the scientific industry at large, as what results can be reported must be round-about and far less detail oriented.

But it also affects patients. Many rare diseases don’t have a great deal of studies or epidemiological evidence around them. A good example may be a rare cancer. The advantage of a TRE like SAIL which facilitates linkage to other datasets is that a comprehensive disease history can be built around the patient by linking hospital records, GP records and pathology e.g., blood tests, together. This presents the opportunity to characterise the retrospective and prospective course of a disease very closely, offering key insights into how the condition functions, which paves the way to treating it. In this way, for rare disease patients, primary research is a fundamental prerequisite to their care, and such research is most empowered through linked platforms like SAIL. The potential inability to report important findings due to small numbers then becomes a cruel irony.

Of course, it’s important to be clear that this limitation is not the TREs fault. SAIL was established to enable population data science and exists because of its ability to facilitate highly secure data access. At population level, the <10 limit is hard to hit. The problem with rare diseases is that the relevant population is inherently small, and the <10 limit is a clear and present threat to the sciences ability to complete, but SAIL wasn’t set up to facilitate rare disease research. Another important point is that the <10 limit is a way of fulfilling the TREs legal requirements as set out in GDPR and the Data Protection Act. TREs have a responsibility to prevent unlawful processing of personal data, and that includes outputs: if an output has sensitive content, and this is used in some malicious way, then the TRE has legal accountability. You could imagine the headlines. Is it any wonder that SAIL goes to pains to ameliorate such possibilities?

Nevertheless, this limitation has an impact. In my study, that impact is relatively benign. The allele frequencies can be calculated using the same data within the NHS, but part of the reason they were the subject of an academic study in the first place was because of lack of resource within the service. However, in the case of rare diseases, it is likely to have recurring and significant impacts. What, then, are the possible avenues of resolution?

Solutions

The first solution is that of argument. SAIL’s output review policy allows for a case to be made to release the data. Let’s think about how that argument might be constructed. Why should the counts have been allowed to be released? The real question is what impact would releasing the data enable which is otherwise not possible?

In my project, this comes down to how useful having updated CF allele frequencies are. Most genetic tests are selected panels rather than agnostic; they look for whether a person does or does not have a list of specific recognised genetic variants, rather than looking for the presence of any possible variants. That means that when designing the panel you have to decide which variants to look for and that is often informed by which variants are most common. This makes sense because if you can use a test panel that accounts for 99% of the pathogenic variants in the population, then you only need to use more expensive and complicated methods 1% of the time, delivering a quicker service for the patient. In the scenario where you can’t assess which variants are most common over time, you lose the ability to monitor the true effectiveness of your panels. Suddenly, you might be running a panel that only accounts for 90% of variants, and you’re spending 10 times as much resource on more expensive methods, causing delay and waste.

In the case of rare diseases generally, the specific impact depends on the nature of the study and of the disease. My experience suggests that TREs may not currently be set up in a way to anticipate or deal with this problem and, whilst that is not necessarily their fault, it does mean there are questions about how worthwhile it is conducting this research within them. This is a problem because for many rare conditions, which lack therapies and clear epidemiological evidence, research is critical and so enabling research to be conducted becomes an equity issue. Where there is a clear benefit, the argument can readily be made to release the outputs – in theory.

Practically, wading into the arena of information governance is not for the faint of heart. It is a highly specialised area all of its own, and one weighted towards risk minimisation because there is a high price tag for getting it wrong. In rare diseases where primary research has such potential for impact, this is not so straightforward. What is the approach that minimises risk most when patient care can be so directly affected? How do you ensure consistency in the decision making processes when there are so many rare conditions?

So, even if there is a clear benefit to the patients, the arguments still need to be made and ultimately somebody needs to be prepared to answer those questions and say “It’s okay to release the data.” This is the heart of the issue: accountability. Who does the buck stop with if something goes wrong – if information is released and there is a negative consequence? Will the benefits of releasing the data outweigh those potential risks sufficiently that, if those risks bear fruit, that the fallout will still be worth it?

Ultimately, my thesis did not report the allele frequencies. I wrote up that chapter making a lot of the same observations I have made here, with the suggestion that in order to advance on this problem, we have to look beyond science. This is an ethical, legal and philosophical debate that will require input from patients, families, clinicians, legal experts in data protection, information governance, government, scientists, data controllers and other voices to find the common ground on which to build that sense of accountability. To enable someone to say ‘Yes, it’s okay to release the data,’ because otherwise this problem will become a barrier to rare disease research and risk leaving them behind.

Glossary

Gene: A defined length of DNA which fulfils a specific biological function, most often encoding a protein.

Allele / Variant: A version of a gene, most often used to refer to genes that bear a mutation which might cause disease. Variant is often used interchangeably with allele, though they are subtly different: variant refers to the change an allele carries, whereas allele describes the physical gene lives on. The term mutation has largely been replaced by the term variant.

Pathogenic: Disease causing, most often used to refer to genetic variants that cause diseases.

Note: Header image imagined from the text by artist Chris Glynn