From the Field: The Importance of Meta Monitoring
Guest Writer: Alex Bozzette, PMC Global Health Corps fellow in Burundi
“She can’t write, so we skipped the signature. That’s OK, right?”
Wrong. When you’re interviewing a patient and navigating the complex world of rural informed consent: very, very wrong. As a project coordinator with Population Media Center in Burundi, one of my most rewarding experiences so far this year was had over the course of a week up-country. On a Monday, my co-fellow Jean Sacha and I left Bujumbura—Burundi’s loud, vibrant, gritty, juxtaposition-loving capital that is a stone’s throw from the DRC on Lake Tanganyika—for five days of rural clinic monitoring. More precisely, we monitored monitoring.
At Population Media Center, our team produces a radio soap opera called Agashi (“Hey! Look Again!”). Entertainment-education is a popular strategy for sparking public health behavior change, and radio dramas are an ideal approach here in Burundi, where everyone from NGO Country Directors to moto drivers tunes in to radio religiously. Our program follows a strict, evidence-based methodology that has been applied in 40 countries to date—a system where positive characters perform preferred behaviors, negative characters neglect them, and transitional characters serve as indecisive, approachable role-models struggling to choose their path. These transitional characters, over months and years of broadcasting, are designed to motivate listeners by learning about and gradually adopting socially positive behaviors—everything from healthy child nutrition to condom use. Here in Burundi, because of demographic challenges, we focus on family planning, maternal and child health, and community life with returning refugees, among other themes.
Gauging the impact of our work is essential, and clinic monitoring is a cornerstone of that effort. We use structured interview questionnaires to determine what type of information motivates clinic clients to seek health services and, as a result, what proportion of patients are influenced by our radio drama. Our team at Population Media Center-Burundi is small, which means that we have to contract out our major monitoring and evaluation work. Over two weeks, our contractors for this particular study interviewed 1,224 patients at 102 health structures in 10 of Burundi’s 17 provinces. Jean Sacha and I joined them.
Leaving Bujumbura on our first truly independent assignment as Global Health Corps fellows, we traveled up into the Burundian highlands—along the way experiencing omnipresent shouts of “Mzungu! Mzungu!” and watching men on colorful bicycles grab the bumpers of semi-trucks for a free lift. We arrived in cold, misty Mwaro that evening and spent the next four days bouncing our way around the country. Our routine: a 6:00am start, supervise four hours of clinic interviews, navigate occasionally unnavigable roads for two to six hours, arrive in a new province at night, meet the new interview team, repeat.
The week was exhausting, illuminating, challenging, productive, life-affirming, and fun. Most importantly—particularly for you, the public health-interested reader—it was crucial for our work. Mistakes were made despite extensive efforts to train our interview staff thoroughly (multiple days of training and a pilot study). People are people, and people are human.
We observed a random sample of interviews using a structured evaluation system that we developed to assess interviewer relations with clinic staff, patient selection randomization, informed consent, and adherence to questionnaire protocols. Generally, performance was excellent. But on rare occasions, informed consent explanations weren’t as long or precise as they could have been, questions were asked superfluously when skip patterns were neglected, and interviewers were tempted to fudge randomization strategies (every third patient exiting the clinic) when interview quotas and slow days created long waits. We corrected the case of an interviewer failing to explain that an illiterate mother could indicate consent with a quick line or symbol. When necessary, we communicated recommended changes to the wider interview team.
These issues happen. Recognizing that is essential, and as public health practitioners we should prepare for that reality by consistently dedicating funds to monitoring our monitoring. Fortunately for our work here, Population Media Center does. A drop in the bucket can help protect the integrity of our data and, most importantly, the rights of the people we serve.