It’s all part of the process (evaluation)

Stephanie Allan is undertaking a PhD as part of the EMPOWER team. Here she introduces herself and describes the importance of process evaluation for successful research.

The most basic question about any intervention is “does it improve the thing the researchers expected that it would?”. If an intervention manages to do this, it is successful. If it fails, then it is not. However, sometimes interventions that might be successful can fail for reasons not to do with the intervention at all. For example, if people are too busy to use the intervention. Similarly, an intervention might appear work when you test it over a period of a few months (because everyone you gave it to happens to be very keen) and you get excited and decide to offer it to everyone. However, then you might discover the general population are not as enthusiastic about using it in the long term. Might it have helped if you had explored potential barriers to general usage for your intervention as you went? Perhaps you could have asked “how long do you think you would actually use this for?”. These are the sort of questions that get asked in doing process evaluations.

Process evaluations are useful because they can help researchers answer questions about how the context where you are conducting research can affect your final result. For example, exploring if the context in which you tested the intervention (over a short time period) could have an impact on longer term use when you decide to roll it out more widely. A lot of process evaluations involve talking to people and their results can be very useful for predicting if an intervention is likely to become more widely used beyond small scale testing.  Furthermore, they can be useful for pinpointing the “active parts” of an intervention. For example, rather than simply looking at “do participants use an intervention?” we might ask “what makes you want to use the intervention? What parts are helpful to you?” – this is useful as it gives us a clearer idea about people’s reasons for using an intervention and can even help us improve and create interventions that are more useful to the needs of end users.

I am completing my PhD within the EMPOWER project and completing a process evaluation. There will be lots of different data generated from the trial (including how people are doing and how often the app was used). I hope through my role that I will get to learn a lot about what people think about and do with EMPOWER. There is no prescriptive guide on how to best do a process evaluation. Therefore, at the moment I am figuring out what methods would be the best for approaching this task. I have spent a lot of time reading about what other process evaluators have used in their work. Any intervention for any problem can be process evaluated. I am used to reading psychosis research so it has been an enjoyable challenge to read in different fields including blood pressure monitoring, smoking cessation programmes and even the usage of robots within surgical teams.

A lot of process evaluations involve talking to people and their results can be very useful for predicting if an intervention is likely to become more widely used beyond small scale testing.

The EMPOWER team spent a lot of time speaking to mental health staff, people with experience of psychosis and their carers in advance of the clinical trial. This has been very useful because it has given us a good idea of what people are already doing and it has been helpful in deciding what questions will be most relevant for our participants. We really value the expertise of our stakeholders and we hope by using this approach the results of our process evaluation will be relevant to their needs.

One thought on “It’s all part of the process (evaluation)

Comments are closed.