5 Questions To Consider When Designing A Breath Research Study
Published on: 7 Dec 2020
Over the last several years we’ve supported a wide range of studies in the field of breath research. In the process we’ve learned a lot about what makes a good study – and what doesn’t. One of the most important things we’ve learned is that you need to devote enough time to planning your trial and asking yourself the right questions. You can always redo your analysis, but no statistical model or new analytical idea can retrospectively fix a badly designed study.
We support our customers throughout the design process to help them scale their ambitions to their resources and ensure the end results of their trial are relevant and valuable. With our tried and tested insights, specific to the breath biomarker research field, our experts can help you protect your study from many common design pitfalls.
We asked the team to share some advice on the key considerations that help them to design effective breath analysis research trials:
1. What is your specific aim?
This might sound an obvious place to start, but too often it’s skipped over. Consider your ideal outcome – what would success look like? A clear, focused objective that is specific and achievable is more likely to produce meaningful results.
Once you’ve established your goals make sure that they align closely with your experimental method, and ensure your ambitions are well-managed and realistic. There are plenty of published examples of small studies with overambitious goals. Often this leads to overinterpreted data which limits the impact of results, or even renders them unreproducible.
If you intend to conduct your own pilot study then be careful to manage your own expectations, be pragmatic about what your study can achieve. Pilot studies should demonstrate proof of concept, you can’t use them to detect cancer or stratify asthma patients in a single step. Have a clear idea of what the next step might be and how this study can help you to get there.
2. How will your study be useful?
There’s little value in a study that won’t lead to clinically or economically useful outcomes, or a better understanding of disease pathology. As an example, a breath test for a disease isn’t useful if existing methods are already extremely accurate, easy and timely. Studies often compare patients to healthy controls, but most doctors need a test to differentiate between diseases with similar symptoms. As such, depending on your goal, it can be more useful to include controls with similar symptoms or related conditions to compare against your disease of interest.
Make clinicians part of your study design team – healthcare professionals that are able to advise accurately on current clinical needs. We work closely with clinicians from a range of specialisms to maximize the impact of Breath Biopsy®.
Once you are certain of a disease area that would benefit from further investigation, consider carefully, how will your method improve on current tests? Which is more important, high sensitivity or high specificity (see boxed text)? Knowing what’s most important in a test to be developed in the long term will help you plan your study accordingly.
Lastly, have a plan for how you will validate your results. If there’s no clear next step towards clinical application then you may need to rethink.
Is it better to be sensitive or specific?When assessing the quality of medical tests two parameters are typically key:
Naturally, the ideal test would have both high sensitivity and high specificity, but achieving both isn’t always possible. The choice of whether sensitivity or specificity is more important typically depends on your clinical application. An early-detection cancer screening program that will be used to refer patients for more tests should be sensitive, you don’t want to tell someone they’re healthy if they’re not. But the test might not need to be specific. If a few extra people need to go for further testing that might not be so bad – unless the further tests are highly invasive, expensive or pose a risk to patient health. By contrast, if you’re developing a test to determine which asthma patients need steroid treatment, specificity might be more important than sensitivity, to avoid prescribing patients steroids unnecessarily. |
3. How will you measure success?
At the very least, what criteria will you need to meet to achieve a statistically powerful set of results? Ask a statistician to help if you’re not sure. This can particularly impact your sample sizes and choice of target groups.
While the intended use population should always be used for late-stage trials, in the early stages, groups are often selected to maximize the potential to detect difference. For example, a small pilot study is more likely to give conclusive results if you include patients with advanced illness – as the observed effect sizes may be larger. However, dependent on disease pathology, what you find working with late stage patients initially, might not help you later, if your goal is ultimately an early diagnosis.
If possible, even with a pilot study, it’s beneficial to be guided by existing knowledge. The more you can simplify and optimize your analysis around detecting particular compounds, the clearer your results will be. Where relevant, interventions such as EVOC® Probes, can also boost differences between groups, potentially increasing the likelihood of meaningful results in your trial.
4. What are your limitations?
There will always be a lot of constraints on a study. One of the most obvious is budget, which may in turn inform your other constraints. Even a pilot study on a limited budget can produce useful results, that can help to secure larger funds, if the goal is well defined. Combining this with a consistent analytical platform like Breath Biopsy can enable staged trials that grow in scope and quality as new funds become available to support them.
Some limitations, such as patient recruiting, are common to all trials and can be affected by factors outside your control. Others are more unique to breath research – such as the risk of noise from inconsistent ambient air environments clouding your results. Controlling for drift in analytical devices is another particular challenge that can only be mitigated with the ability to reliably store stable breath samples for long periods of time. These challenges have stunted progress in the breath field in the past but can now be overcome with access to technologies optimized specifically for breath.
While considering your limitations also assess your unique strengths. What resources can you access that no other study can? Are there healthcare systems, networks, or tools that can ensure your study has unique value? How can your study design make the most of these opportunities?
5. How might your study go wrong?
It’s standard practice to conduct a postmortem after a study, evaluating what went well, what could have gone better, and what the next steps should be, but have you ever considered holding a premortem? In any study, there are always lots of things that could go wrong and there’s benefit to predicting and trying to mitigate them before they happen.
Where possible, doing preliminary work can help you understand the risks in your study, so try to collect samples and test your analysis pipeline thoroughly. Similarly, plan the blanks and controls you will collect to span all relevant sampling conditions. We operate rigorous quality control throughout the Breath Biopsy analysis process to ensure consistency in results even in large-scale studies. Comprehensive data collection about test subjects as well as the breath collection process can also help to spot when things aren’t as expected. For example, the ReCIVA® Breath Sampler collects CO2 and air pressure data during sampling, which we use to check that breath samples are complete and suitable for analysis.
Final thoughts
We’ve arranged these questions in an order that should feel intuitive, but you may find yourself addressing these in different ways. We strongly recommend approaching them iteratively, running through the cycle multiple times to refine your aims and methods in line with your risks and limitations. Only once all the answers are aligned should you look to finalize your study design. As one of our team put it:
“a study with a well-addressed, less ambitious objective is far more
valuable than a poorly executed study with unrealistic objectives.”
If you’re looking for help planning your own breath research studies, get in touch.
Visit the Breath Biopsy Community and share your top tips for study design. What lessons have you learned from working in the field and how could study design be developed to provide more comparable results and support wider biomarker validation?
At the Breath Biopsy Conference 2020 an entire session was dedicated to ‘Study Design and Data Analytics’, including talks from Roy Goodacre (University of Liverpool), Pierre-Hugues Stefanuto (University of Liege), Laura McGregor (Markes International) and Julia Greenwood (Owlstone Medical). All of these presentations are now available online to catch up on or re-watch, for free.
Watch The Breath Biopsy Conference 2023
Further information:
- Senior Owlstone Medical staff contributed a chapter on ‘Clinical study design’ to Beauchamp et al., Breathborne Biomarkers and the Human Volatilome, 2nd edition, Elsevier, 2020, ISBN: 9780128223970
