RCTs
RCTs
Design experiments to test your solution and understand how your changes impact behaviours with Randomised Controlled Trials.
Useful for:
Testing
Project stage:
Deliver
An RCT is a reliable tool that helps to accurately measure the effect of a specific change or intervention, while mitigating the influence of confounding variables.
Their execution requires meticulous planning, ethical considerations, and rigorous analysis. When conducted correctly, they yield powerful, evidence-based insights that can drive effective behaviour change strategies
It’s useful for:
Testing behavioural interventions in a controlled setting.
Reducing biases in experimental design and evaluation.
Enhancing the reliability and validity of your results.
Understanding the mechanisms through which your intervention affects behaviour.
Applications:
RCTs are pivotal when you have a behavioural intervention and want to measure its impact. They aid in minimising bias and verifying that any observed impact is attributable to your intervention.
Important reminders:
Test only one variable at a time.
Random assignment is essential to avoid selection bias.
A large, representative sample strengthens the validity of your results.
Remember the human aspect in your trials; treat participants with care and respect.
STEPS
Hypothesis Formulation: Formulate a clear, testable hypothesis. Your hypothesis should not only predict an outcome but also propose a psychological or behavioural process that leads to this outcome.
.Test Conditions: Carefully design your test and control conditions. The difference between the two should be just the intervention being tested.
.Participant Selection: Ensure that participants truly represent your target audience. Randomly assign them to the test and control groups. Aim for a sizeable, diverse sample for more generalisable results.
.Ethical Considerations: Before beginning, ensure all participants have given informed consent and that your study meets ethical guidelines.
.Preflection: Prior to executing your experiment, reflect on what you expect to happen. This helps you stay objective and prepares you for possible outcomes.
.Experiment: Conduct your experiment. Pay attention to potential sources of bias, and aim for precision and consistency in administering the intervention.
.Data Analysis: After the experiment, rigorously analyse your data. Use appropriate statistical tests to determine the significance of your results.
.Reflection and Interpretation: Reflect on the results. Do they support your hypothesis? Consider not only whether your intervention worked, but how and why it did so.
.Replication: If resources permit, consider running your RCT again. Replication enhances the robustness and reliability of your results.
.Reporting: Finally, make sure to report your findings in a transparent and accessible manner. Include the data and methods so others can replicate your study.
CONTROL
Control is when nothing has changed. Choosing a control can be difficult. A small change can influence behaviour. Just interacting with a control group can introduce another variable. One that could influence the result of your experiment.
Ask yourself:
What is the do nothing condition?
How can we access the control without introducing another variable?
Look to see where people already are and what their next natural step in behaviour is.
TEST
Test condition is the control condition plus one key variable. More than one and you won’t know what caused the results.
Ask yourself:
Have I changed only one variable?
Tip: Think about the context of your test condition.
Any small change or additional task can influence behaviour. Think about how to match one additional action to another in the control.
For example,
Test: Receive push notification with written reminder.
Control: Don’t receive the push notification reminder.
This isn’t a pure comparison.
The act of getting a ping may influence people - irregardless of what it is about. Instead include the cues of a notification into the control condition but remove the text.
Test: Receive push notification with written reminder.
Control: Receive a blank push notification.
Derived from:
RCTs have their roots in medical research, but have been adapted to the field of behaviour design due to their strong capability in isolating and testing individual variables.
