How to satisfy MHRA guidance on human factors and usability when designing medical apps.
21 Oct, 2020
Chief Product Officer, Paul Cooper, gives practical advice on how to achieve Medicines and Healthcare Products Regulatory Agency (MHRA) standards for the usability of software designed for medical devices.
The body content of your post go es here. To edit this text, click on it and delete this default text and start typing your own or paste your own from a different source.
my mhealth build digital therapeutic apps that help thousands of patients to remotely self-manage long-term health conditions such as COPD, asthma, diabetes, heart disease and COVID-19. As a class 1 regulated medical device manufacturer, we are required by law to adhere to a user-centred design approach defined by the MHRA.
Primarily, MHRA regulations are designed to ensure that medicines and medical devices work and are acceptably safe. That means designers and developers of apps like myCOPD and COVID-19 Virtual Ward must reduce the risk of any “user errors” which could potentially cause harm to a patient. We can do that by ensuring a user-centred design process is built into the way we make software by default:
- Product iteration and improvement throughout the life-cycle
- Post-market vigilance and surveillance of similar devices
- Human factors considerations that promote optimal clinical outcomes
Even before MHRA, my mhealth apps had been designed with the end-user in mind, whether they were apps for a nurse, doctor, clinical manager or patient, our goal has always been to build tools that people want to use. By following a user-centred design process, our software has been optimised for usability and meets the highest accessibility standards, so patients of every skill level or physical ability can benefit. And that work continues.
A recent example of the success of an iterative design process, and in particular, using formative user-testing as a method to identify risks, are improvements we made to the COVID-19 Virtual Ward app. During alpha-testing with a cohort of care home nurses, we observed an issue where some carers needed to enter patient data themselves because the patient didn't own a smartphone or computer. So, we designed a new function that enables clinicians to register patients on their behalf.
Another example of the iterative design process: during the Tees Valley NHS Vanguard Unit trials by Hartlepool & Stockton Heath GP Federation (H&SH) on the COVID-19 Virtual Ward, in-depth user-research was conducted, and usability issues were identified that resulted in a design improvement to the patient list in the clinical app. We knew that nurses were using the app to telephone patients and request their symptoms over the phone, thereby enabling the nurse to enter the data on behalf of the patients. But only when interviewing the users did we fully appreciate the real-life complications of their workload. The nurses needed a way to identify who to contact from a long list of patients. The problem caused an increase in the amount of time to make the phone calls and impacted on the nurse’s efficiency. The simple solution of indicating this type of patient and including a checkbox to mark that the results had been assessed solved this issue.
Carl Gowland Head of Operations & Service Delivery at H&SH told us:
"The COVID-19 Virtual Ward app is really straightforward to use. By working directly with my mhealth to make changes based on our clinician’s experience when testing the app, we’ve been able to work more efficiently and help more patients during the COVID-19 pandemic."
Another success story of this user-centred design process is a post-market improvement made to the medication diary.
Following usability and ethnographic research with different patient types, our research team understood that the amount and frequency of medication required for all the conditions supported by the apps varied significantly from patient to patient. Some patients took up to ten different meds every day, therefore individually re-entering the same medicine every day was a significant time investment.
The solution, to ask patients to enter their meds once, then enable them to tap a "copy yesterday’s meds" button, has helped thousands of patients save time, and reduce the risk of error significantly.
These are just a few examples of how my mhealth have successfully followed a user-centred design approach within a formative and summative assessment research model to make patients and clinicians lives easier and less prone to risk.
If you're interested in hearing more about how we've shaped our design process to conform to the "MHRA Human Factors and Usability Engineering Guidance for Medical Devices", or would like to license our software to help your patients please
get in touch here or call us on +44 (0)1202 299 583.
Permission to use received from Rebecca Fowler View poster .
Results of a service evaluation using myHeart in a large London-based acute NHS Trust. We are pleased to report the outcomes from this recent service evaluation of myHeart and the potential benefit of using the myHeart app to supplement existing cardiac rehabilitation (CR). Cardiac rehabilitation is an essential evidence-based intervention that supports patient recovery following a cardiac event. It offers patients a structured education and exercise programme to aid recovery and support behavioural changes to help reduce the risk of future cardiac health complications. The myHeart app provides a structure educational and exercise intervention that mirrors current CR service delivery as well as supportive self-management tools. Overall, 721 patients were invited to participate in 1 of 4 groups (class-based CR, class-based CR with myHeart, home-based CR and home-based CR with myHeart). A total of 584 patients opted to use class-based CR and of these 43 chose to include myHeart to support them with this. There were 137 patients in the home-based group, with 54 choosing to include myHeart alongside their CR. This 12-week evaluation involved functional, physical and psychological assessments both before and after CR to explore potential changes. Patients were also asked to complete a rating of perceived exertion Borg RPE scale (Borg 6-20). Those in the home-based groups were contacted mid-way through the study. Results identified three key outcomes: 1. Blood pressure, cholesterol, LDL, BMI, HbA1c and exercise were all very similar across the groups with marginal differences across each measure. 2. Drop-out rates (DOR) of patients being invited to attend CR and attending CR were significantly lower in those groups with access to myHeart. * Class only: DOR = 58.2% Class + App; DOR = 25.6% * Home only: DOR = 73.5% Home + App; DOR = 42.6% 3. Those patients with access to myHeart and in the home-based group saw the greatest improvement in anxiety and depression scores. This real-world evaluation provides an encouraging insight into the potential impact of myHeart to supplement CR services, and is suggestive that, as an adjunct to support both class and home-based programmes, myHeart helps to reduce drop-out rates in CR and can assist in reinforcing continuous engagement with CR programmes.
Results have led to continued QISMET Accreditation for myDiabetes (QIS2015) and have revealed the app could play an important role in supporting structured patient education delivery for type 2 diabetes following initial diagnosis, and as an ongoing resource. Together with a large Health and Care Partnership, we led a multi-centre service evaluation to explore the impact of myDiabetes on education course attendance rates. T2DM is a serious and growing problem worldwide and affects more than 3 million people in the UK. Structured education is a large part of managing T2DM to promote a healthy lifestyle and improve blood sugar control. However, the uptake for education courses has been less than encouraging across the UK. By offering a digital alternative or adjunct to a class-based course those who are unable or prefer not to attend a class-based programme, are able to receive structured education. myDiabetes is an app to support patients and clinicians manage diabetes together, remotely and at scale. Overall, 83 T2DM patients were recruited by the healthcare team, of which 28 chose to use myDiabetes alone, 35 chose only usual care, and 20 chose to use both. Patient education usage was monitored over a 12-week period. During this evaluation we monitored changes in diabetes related clinical health outcomes where possible, including HbA1c, blood pressure and body mass index. Participants in all three sites were asked to complete the Problem Areas In Diabetes (PAID) questionnaire at the beginning and end of the evaluation to explore markers of improvement in diabetes related distress. Results showed the app was acceptable in this care setting with 31 of 42 patients using it alone or as an adjunct to usual care. A total of 586 education videos were watched, on average each patient watched 22.5 (SD 19.6) videos. There was a reduction in PAID scores across all arms, with the app only arm showing the greatest improvement. Patients using myDiabetes showed the greatest improvement in HbA1c (-7.5 vs –4.4 mmol/mol), Systolic Blood Pressure (-12.2 vs +3.3 mmHg) and PAID score (-6.8 vs –5.2). In this real world evaluation myDiabetes performed better than published education course completion rates and resulted in significant improvements in HbA1c and PAID score compared to classed-based programs. This supports the use of myDiabetes to support the delivery of structured based education for patients with type 2 diabetes.
NHS University College London Hospitals NHS Foundation Trust, part of North Central London ICB, is taking a significant step towards enhancing patient empowerment and optimising disease management. Asthma is a chronic condition that affects millions of people worldwide, often leading to severe health complications if not managed properly. Recognising the critical need for effective self-management tools, NHS University College London Hospitals NHS Foundation Trust has chosen the myAsthma app to provide patients with the resources they need to take control of their health. Dr Kay Roy PhD FRCP, Consultant Respiratory Physician University College London Hospitals NHS Foundation Trust, comments “We are thrilled to introduce myAsthma as a self-management tool to our community. It represents a significant step forward in empowering our patients with asthma to take control of their health. By providing them with personalised support, we believe this tool will greatly improve their quality of life. Additionally, the use of myAsthma in outpatient settings will help triage patients more effectively, ensuring they are seen in a timely manner and appropriately referred for the right investigations and services. Our team is excited to see the positive impact this will have on the asthma population across North Central London ICB." The myAsthma app, part of the my mhealth suite of digital health solutions, is designed to empower patients with comprehensive tools and information to manage their asthma more effectively. Key features include: • Personalised Action Plans: Tailored asthma management plans based on individual patient needs. • Inhaler technique training: Contributing to better health outcomes and reduced risk of exacerbations • Medication Tracking: Reminders and logs to ensure patients take their medication as prescribed. • Symptom tracking: Easy-to-use tools for tracking symptoms and triggers. • Educational Resources: Access to a wealth of information on asthma, helping patients understand their condition and how to manage it. As more NHS partners embrace the my mhealth platform, we're thrilled to witness its growing impact and the positive changes it is bringing to long-term condition care. For more information on this article or other my mhealth projects, please get in touch https://mymhealth.com/contact-us
Read the myCOPD Data Booklet.
Permission to use received from Rebecca Fowler View poster
Henry M.G. Glyde1Alison M. Blythin2 Tom M.A. Wilkinson3Ian T. Nabney4 James W. Dodd5 EPSRC Centre for Doctoral Training in Digital Health and Care, University of Bristol, Bristol, UK my mHealth Limited, Bournemouth , UK my mHealth and Clinical and Experimental Science, University of Southampton, Southampton, UK School of Engineering Mathematics and Technology, University of Bristol, Bristol, UK Academic Respiratory Unit, Translational Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK Abstract Background Acute exacerbations of COPD (AECOPD) are episodes of breathlessness, cough and sputum which are associated with the risk of hospitalisation, progressive lung function decline and death. They are often missed or diagnosed late . Accurate timely intervention can improve these poor outcomes. Digital tools can be used to capture symptoms and other clinical data in COPD. This study aims to apply machine learning to the largest available real-world digital dataset to identify AECOPD Prediction tool which could be used to support early intervention improve clinical outcomes. Objective To create and validate a machine learning predictive model that forecasts exacerbations of COPD 1-8 days in advance. The model is based on routine patient-entered data from myCOPD self-management app. Method Adaptations of the AdaBoost algorithm were employed as machine learning approaches. The dataset included 506 patients users between 2017-2021. 55,066 app records were available for stable COPD event labels and 1,263 records of AECOPD event labels. The data used for training the model included COPD assessment test (CAT) scores, symptom scores, smoking history, and previous exacerbation frequency. All exacerbation records used in the model were confined to the 1-8 days preceding a self-reported exacerbation event. Results TheEasyEnsemble Classifier resulted in a Sensitivity of 67.0% and a Specificity of 65% with a positive predictive value (PPV) of 5.0% and a negative predictive value (NPV) of 98.9%. An AdaBoost model with a cost-sensitive decision tree resulted in a a Sensitivity of 35.0% and a Specificity of 89.0% with a PPV of 7.08% and NPV of 98.3%. Conclusion This preliminary analysis demonstrates that machine learning approaches to real-world data from a widely deployed digital therapeutic has the potential to predict AECOPD and can be used to confidently exclude the risk of exacerbations of COPD within the next 8 days. Permission to use received from Henry Glyde. Read more on Heliyon website.
Charlotte Smith 1 Francesca D’angelo 2 University Hospital of Derby and Burton, Cardiac Rehabilitation Department, Burton Upon Trent, UK. University Hospital of Derby and Burton, Health and Wellbeing Department, Burton, UK To examine the effectiveness of physical activity outcomes using a web-based Cardiac Rehabilitation application compared with a conventional programme or a combination of both. University Hospitals of Derby and Burton NHS Foundation Trust poster presented at the BACPR Annual Conference October 5-6th 2023 Permission to use received from Charlotte Smith
Francesca D’angelo 1 Charlotte Smith 2 University Hospital of Derby and Burton, Health and Wellbeing Department, Burton, UK University Hospital of Derby and Burton, Cardiac Rehabilitation Department, Burton Upon Trent, UK. To examine the effectiveness of psychological outcomes using a web-based Cardiac Rehabilitation application compared with a conventional programme or a combination of both. University Hospitals of Derby and Burton NHS Foundation Trust poster presented at the BACPR Annual Conference October 5-6th 2023 Poster presented at the BACPR Annual Conference October 5-6th 2023 Permission to use received from Charlotte Smith
Christopher Duckworth 1 Bethany Cliffe 2. Brian Pickering 1 Ben Ainsworth 2 Alison Blythin 3 Adam Kirk 3 Adam Kirk Thomas M. A. Wilkinson 3,4,5 Michael J. Boniface 1 1 IT Innovation Centre, Digital Health and Biomedical Engineering, University of Southampton, Southampton, UK. 2. School of Psychology, Faculty of Environmental and Life Sciences, University of Southampton, Southampton, UK my mHealth Limited, London, UK. National Institute for Health Research Applied Research Collaboration Wessex, University of Southampton , Southampton , GB Faculty of Medicine, University of Southampton, Southampton , GB Mobile Health (mHealth) has the potential to be transformative in the management of chronic conditions. Machine learning can leverage self-reported data collected with apps to predict periods of increased health risk, alert users, and signpost interventions. Despite this, mHealth must balance the treatment burden of frequent self-reporting and predictive performance and safety. Here we report how user engagement with a widely used and clinically validated mHealth app, myCOPD (designed for the self-management of Chronic Obstructive Pulmonary Disease), directly impacts the performance of a machine learning model predicting an acute worsening of condition (i.e., exacerbations). We classify how users typically engage with myCOPD, finding that 60.3% of users engage frequently, however, less frequent users can show transitional engagement (18.4%), becoming more engaged immediately ( < 21 days) before exacerbating. Machine learning performed better for users who engaged the most, however, this performance decrease can be mostly offset for less frequent users who engage more near exacerbation. We conduct interviews and focus groups with myCOPD users, highlighting digital diaries and disease acuity as key factors for engagement. Users of mHealth can feel overburdened when self-reporting data necessary for predictive modelling and confidence of recognising exacerbations is a significant barrier to accurate self-reported data. We demonstrate that users of mHealth should be encouraged to engage when they notice changes to their condition (rather than clinically defined symptoms) to achieve data that is still predictive for machine learning, while reducing the likelihood of disengagement through desensitisation. Read more
Contact Us
(+44) 1202 299583
support@mymhealth.com
© 2024 my mhealth Limited Milton Gate, 60 Chiswell Street, London, EC1Y 4AG