A Practical Guide to Developing an M&E Budget

 

Research Team with TASO

SAFE believes that projects should be rooted in data–it is ethical to evaluate our work so that we understand what works and what doesn’t, because that is how we can have the most impact for moms and babies and be as responsible as possible with the funds we have available. We know that this isn’t just a struggle for us, but for many smaller organizations whose budget just doesn’t allow for huge M&E expenses.

That is why for the past year SAFE has worked hard to develop an effective, evidence-based, robust monitoring and evaluation (M&E) strategy in order to better quantify the amazing impact SAFE has in the communities where we work. Our strategy was intentionally developed to gather high quality, mixed methods data at a fraction of the cost and in a much quicker time-frame than other programs we have seen. We developed the method primarily through our work with the ACT for Child Health Initiative we recently launched, supported by the Children’s Prize, in which we surveyed 4,499 women in a comprehensive quantitative health survey, and interviewed more than 60 people as part of our baseline assessment (these included maternal death social autopsy, exit interviews, and in-depth interviews).

Yet, we really wanted to develop a methodology that could be useful in the long-term, not just to SAFE but also to other nonprofits wanting to incorporate more data collection on smaller budgets than originally anticipate as being required. That is one of the reasons we are so thrilled with a new partnership!

In January, SAFE was invited by the Center for Health Market Innovations (CHMI) to participate in their Primary Care Adaptation Partnership. This program allowed us the opportunity to learn more about a wonderful organization called Healthy Entrepreneurs (HE) that addresses health commodity security and supply chains in a sustainable, community-based business model. As part of a new partnership created with HE (more on this later!), SAFE offered its expertise in M&E in order to support the design and implementation of a survey in an area in which HE already works in Uganda. With a very modest 4 figure budget and less than 2 months to design and implement the tool, the SAFE and HE teams were able to successfully survey 913 participants in the Kibaale District of Western Uganda and will complete about 2000 added surveys by the end of April in an area where SAFE currently works in preparation for launching a new joint SAFE-HE project in that area later this year.

Now having been able to work with another organization to adapt our M&E model to their own context, SAFE wants to encourage other small organizations to prioritize monitoring and evaluation of their programs in order to better inform their interventions and ensure they are having the largest impact. The remainder of this post will be dedicated to sharing some ideas about M&E budgets in hopes of helping others (and we welcome you to reach out to us for more specific information if helpful). (Additional posts in coming weeks will focus on other aspects of M&E development, and our growing partnership with HE, so stay tuned.)

The first step in any M&E project is to draft an accurate budget to see if the work you want to accomplish can be completed with the budget at hand. We know that monitoring and evaluation can be a bit overwhelming if you have never done them before, so we wanted to offer some thoughts as to how to structure a budget and plan.

Truthfully, when we first designed an M&E budget, we didn’t really know much about what other people paid, and honestly that was probably a good thing since most M&E projects had a lot more capital to work with than we did. We just looked at the overall categories of other budget templates, and adapted them to our context. They included:

TRANSLATION: Your survey cannot be in English if the local language isn’t usually English, and it will severely compromise data quality if you have interviewers translating on the spot. For the survey to really be valid, your survey needs to be translated into the local language BEFORE you conduct any training or data collection. If you have staff who are really good at both spoken *and* written translation, and understand the differences between them, you can have an existing staff member work on it, but of course, you will have to pay them for that service. If your staff isn’t accustomed to these types of translation, you would be better to pay for translation. Remember that translation isn’t supposed to be verbatim translation, but rather it’s looking to translate the true meaning. For example, the term for “antenatal care” in our area literally translates to “drinking medicine”. So having someone translate “did you receive antenatal care during this pregnancy?” shouldn’t be translated verbatim; it should actually say “did you receive [drinking medicine] during this pregnancy?” Making sure you budget for a good translator will save you a lot of headaches later in the survey process. Don’t chintz on translation, and make sure you have the survey translated into ALL major languages (if you are in an area that has a lot of refugees, you will likely need more than one translation; not being able to survey someone because the survey isn’t in the correct language will introduce bias because the results will not longer be representative of the population on the whole; different people really do have different health indicators, especially those who don’t speak the predominate language in a particular region).

STAFF: Enumerators (quantitative surveys) and interviewers (qualitative) should be local people who are fluent in the major languages; this is very important, as having foreigners conduct the surveys, even with a translator, would introduce a tremendous amount of bias and render the results invalid. We chose to use relatively young (20s – 30s) enumerators/interviewers with at least a high school diploma and fluency in multiple languages, who were really looking to “prove” themselves, and budgeted time (and a little money) to train them. This saved us significant money, as more experienced interviewers would have been far more expensive. We budgeted to pay them a fair daily wage plus a small stipend for food and water during field work (which was very important for their health and satisfaction). We priced out compensation by asking partner organizations what they paid people of a similar training background. Additionally, as we have used these interviewers multiple times for surveys,  so our invest in their original training has paid off.

Also, don’t forget about supervisors. If you’ll have yourself, your more experienced staff, or an outsider coming to monitor the data collection (which ideally you should have), make sure that you budget for their time, food, water, and lodging.

TRAINING: Do NOT forget to budget for training: training space with electricity (how are you going to train all of these people in a small office room?), refreshments during training (as much as you might not want to pay this, it’s really crucial if you want this to work), renting a projector, and materials like notebooks, pens, a roll of big paper and markers (or a whiteboard and dry erase markers with an eraser). You should not chintz on this, as the quality of your training (and the length of your training) is directly related to the quality of the work your people will do, particularly if you’re using less-experienced interviewers.

SURVEY MATERIALS: What do you need to actually do the survey? If you have a large sample size, we found that using a tablet-based survey (rather than paper surveys) was much less expensive. But that means that you need Android-based tablets (and extra, because some will break or get stolen or malfunction or you will find you need one so you can follow along or you’ll need one more interviewer than you thought, etc.). You also need CASES to put the tablets in and a backpack for the enumerators/interviewers to carry the tablets and supporting materials. **DO NOT omit these items because they seem less important. If there aren’t cases, your tablets *will* break or not last as long because of water or dust getting inside them (we say this from experience). If you don’t have backpacks, staff will lose things (also from experience). A little upfront costs for these items can go a long ways in saving money later. Ways to cut costs include buying online, getting things donated (have a “Donate your old Android tablet party”, or a “donate your gently used backpack party”) when appropriate, going through suppliers like Amazon when you don’t have to pay shipping, getting local when appropriate (like the cloth sewn backpacks instead of commercial ones which are cheaper and have the added benefit of supporting local business), etc. Some other questions:

OTHER EQUIPMENT: If you will be doing population-based surveys, do not forget to get adequate charging equipment. We recommend getting a multi-USB charging device that you can plug into a cigarette lighter (so you can charge in the car), and/or staying somewhere that has a back-up generator or solar electricity. If you’re working in a low-middle income country where power outages are frequent, you WILL most likely have many times where power is out precisely when you want to charge.

MUAC TAPES: Will you focus on nutrition? If so, you might need tape measures, MUAC tapes, or scales. *Think about this ahead of time and budget for it.* Also remember that partner organizations might have things available you could borrow or pay a nominal fee to rent. 

AUDIO RECORDERS: Will you be recording an interview (if you need to do that)? If so, you need to either have them record with a tablet or get an audio recorder and ample batteries.

AIRTIME: Your enumerators and administrating staff will need cell phone airtime to communicate about data collection issues. We didn’t budget for this, but should have. A good rule for us was to budget about $0.50/day/enumerator and $5/day/supervisor. Encourage them to communicate by text whenever appropriate, and have the enumerator “beep” (call and hang up quickly) the supervisor to get a return call. This will save you money.

MAPPING AND ENUMERATION CENSUS: Most types of quantitative survey sampling methods will require that you generally know the number of households in each village and generally know the boundaries of each village. We found that there were no estimates available in some of our areas, and that those that were available were sometimes old and inaccurate. We used what are called “shape files” of the parishes in our area (each parish have multiple villages), and either Google Maps or QGIS (both free) to generate general maps of the area. You can also create your own map using a handheld GPS (which we did with no prior technical knowledge, and you can ask us if you want more info—we’re developing a manual!). You can then go work with village chiefs or elders to identify boundaries and number of homes. You can either ASK for the number of homes (knowing that there will be some bias here) or physically count them yourself. It’s even better if you can actually visit every home in the entire region and get a head count by gender and age, as that will be useful during sampling frame develop and survey administration… sometimes that’s possible and sometimes it’s not (for us, each “enumerator census” cost about $1,300 on average). BUT remember that you need to budget for it, the personnel, the technology (or printing), getting out to the village chiefs, often compensating them for their time in taking you around their village, etc.

FUEL AND CAR RENTAL: How will you get around the survey area? You will likely need a large vehicle that can accommodate all people. If your organization owns one, cool. If not, you’ll need to hire one with a driver, and then pay for fuel. Have your staff negotiate the price, in consideration that you will rent it for multiple days. Don’t just accept the first price that you get, and it’s even better if you develop relationships with particular drivers over time, because then you will consistently get better prices.

TERRAIN AND OTHER TRANSPORT MECHANISMS: If your region is particularly mountainous or treacherous, you may need to budget for your enumerators to get out of the hired vehicle and take local transport means (motorcycles, etc.) to get to certain areas that are inaccessible by car. Make sure you think about this and budget for it if you need to.

A WORD OF ADVICE: If you need help developing strategies, don’t be afraid to find a really dedicated MPH (or other professional) student. We use highly qualified and well-trained students in a lot our work; it saves us a lot of money, gives them really good experience, and gets us much better data than we would have if we relied only on our own devices. (If you want to use a recent MPH graduate who will do a good job and might not cost as much as a senior professional, let us know! We have some great people to recommend!!)

Now go out there and get surveying! If you or someone you know is interesting in learning more about conducting M&E on a budget or would like SAFE to consult on a project, feel free to email us at info@safemotherssafebabies.org.

Gifts that Give Life!

SAFE Holiday Email FundraiserSafe Mothers, Safe Babies (SAFE) has been partnering with rural Ugandan communities to save the lives of pregnant women and babies for the past seven years. Together, we serve more than 100,000 people every year, made possible by the generous support of people like you.

This year for the Holidays, we’ve teamed up with both Ugandan and U.S. artisans to give you many gift choices–from jewelry and cards to songs (for digital download) and paintings, along with our regular projects that you can sponsor in someone’s honor. Since all of the materials and talents were donated, 100% of your purchase goes directly to SAFE life-saving work with pregnant women, newborn babies, and young children. Shop with SAFE for Gifts that Give Life! 

Check out the catalog below. Easy online ordering here:

The SAFE Store

Or you can mail your order form and check, cash, or money order to:
Safe Mothers, Safe Babies
1160 Mayfield Dr.
Decatur, GA 30033

SAFE Holiday Catalog 2015 p1 SAFE Holiday Catalog 2015 p2 SAFE Holiday Catalog 2015 p3 SAFE Holiday Catalog 2015 p4 SAFE Holiday Catalog 2015 p5 SAFE Holiday Catalog 2015 p6 SAFE Holiday Catalog 2015 p7 SAFE Holiday Catalog 2015 p8 SAFE Holiday Catalog 2015 p9 SAFE Holiday Catalog 2015 p10 SAFE Holiday Catalog 2015 p11

Join our SAFE Summer Team–Application Open!

IMG_7152 Are you passionate about maternal and child health? Committed to participatory methods? Want to gain experience in international global health–the real, on-the-ground, challenging but rewarding kind of experience? Consider joining SAFE’s internship and practicum program! Each year, SAFE brings undergraduate, graduate, and professional students (and sometimes those taking some ‘gap’ time) to Uganda to work together with rural communities towards improved maternal and child health. Students gain valuable, real-world experience learning about participatory methods of development and applying them in practice to develop, implement, and evaluate SAFE projects. Some students work on programming, others conduct monitoring and evaluation, and others work on more formal research projects. If this sounds like something that would interest you–and you feel that you have skills that would benefit SAFE and our community partners–apply to join our summer team. Some of our anticipated summer projects are listed below; and an information sheet can be accessed here: SAFE Internship Overview.

Application Procedure: If you feel it could be a good fit, apply by sending your resume/CV and a 1-page cover letter to: info@safemotherssafebabies.org. We look forward to hearing from you soon!

Some current internship and practicum openings, SAFE Summer 2015

Research:

Voices of Maternal Health: Intern will develop an in-depth interview and/or focus group discussion guide(s) to facilitate qualitative interviews about maternal health, then oversee the implementation of that guide in-country using organization-employed translators. Component of a larger research grant SAFE is implementing on reducing child mortality in the first 1,000 days. Possibility for thesis and publication.

Third Delay Maternal Referral Tracking: Intern will track emergency maternal referrals between their place of origin and their intended destination among 4 maternal health facilities, in order to better understand the situation surrounding emergency maternal referral and the Third Delay. Possibility for thesis and publication.

Maternal Health Exit Interviews: Intern will develop an in-depth interview guide to evaluate women’s experiences in partner health facilities during antenatal care, labor, delivery, and postpartum care; then oversee the implementation of that guide in-country using organization-employed translators. Component of a larger research grant SAFE is implementing on reducing child mortality in the first 1,000 days. Possibility for thesis and publication.

Verbal Autopsy: Intern will develop a methodology for conducting verbal autopsies when women die during labor or experience a stillbirth, or after a child under age 2 dies; then oversee the implementation of that guide in-country using organization-employed translators. Component of a larger research grant SAFE is implementing on reducing child mortality in the first 1,000 days. Possibility for thesis and publication.

Programmatic Support:

Maternal Health Education: Student will review and edit SAFE’s existing maternal and child health education curriculum, then conduct refresher trainings with SAFE community groups. Intern will then watch the community groups’ use of curriculum in the community and help to improve the curriculum based on performance. Possibility for thesis (or special project).

Tablet-Based Maternal and Child Health Education: Intern will film community groups’ dramas and/or talks and training provided by local health providers; digitize the films; and put them on organization provided tablets that can be used by women during antenatal care to learn about maternal and child health topics. Should also include monitoring software to evaluate use of videos. Requires a student or a team of students with experience in filming, digitizing, and programming. Possibility for thesis and publication.

Drama Support: SAFE community groups use drama and songs to educate community members and important maternal and child health topics. Student will work with community groups to improve their acting skills, story lines, and audience engagement techniques, towards the end-goal of improved performances and greater audience reach. A background in drama, acting, theater, or musical performance strongly preferred for this project. Possibility for publication.

Community Group Assessment: Student will work design and conduct needs assessment of SAFE community groups, then work with the group members to develop long-term plans to address deficiencies and promote self-sufficiency.

Holiday Gifts and SAFE Motherhood!

This Holiday Season, you can find a perfect gift for your loved ones and support safe motherhood and childhood health in Uganda at the same time! Our Holiday Giving Catalog features jewelry handmade by SAFE volunteers using beads made by SAFE community groups. It also features handcrafted holiday cards and several projects you can sponsor in a loved one’s honor. To order and pay by mail, download our SAFE Holiday Catalog 2014 or view it below and print the order form. To order and pay online, click here. Thank you in advance for your generosity!

SAFE Holiday Catalog 2014SAFE Holiday Catalog 2014 p2SAFE Holiday Catalog 2014 p3SAFE Holiday Catalog 2014 p4SAFE Holiday Catalog 2014 p5SAFE Holiday Catalog 2014 p6SAFE Holiday Catalog 2014 p7

Success: SAFE’s Approach Achieves Results!

IMG_0642

SAFE recently completed an internal evaluation of our programs, and the results are exciting! Check out an overview of what we found below:

Background: Too little progress has been made in reducing maternal and perinatal mortality in Uganda since 2000, partly resulting from poor utilization of maternal healthcare services and poor facility infrastructure. Safe Mothers, Safe Babies (SAFE) addresses these deficiencies through an integrated intervention package targeting the three delays (Thaddeus and Maine), including: (1) increasing demand through participatory educational outreach with community groups; (2) improving access through motorcycle ambulances and personal savings programs; and (3) using innovative low-cost technology to improve quality of care through strengthening facility infrastructure, commodities, and medical training.

Objective: To evaluate whether Safe Mothers, Safe Babies’ approach has improved utilization of delivery care in target health facilities.

Data and Methods: We conducted a cross-sectional quantitative and qualitative evaluation through: secondary health data from 4 intervention and 3 control facilities assessing change in the number of monthly health center deliveries and distribution of types of obstetric emergency and related maternal referrals; qualitative data from 49 key informant interviews assessing program quality; a photovoice project conducted by program beneficiaries assessing community views; and six months of field notes from observing obstetric care in 14 facilities.

Results: Intervention facilities experienced a 40.63% average increase in health center deliveries 24 months after the intervention package, which three control facilities did not experience (t(5)=-2.8, p=0.038). This was confirmed by results from key informant interviews, which reported increased utilization of healthcare services in addition to satisfaction with all types of three delay projects.

Discussion and Implications: The intervention package has effectively improved the utilization of delivery care in target health facilities by addressing the three delays. This demonstrates a successful way that the three delays model can be united with community-based and facility-based approaches to improve maternal and child health, both in Uganda and potentially elsewhere.

International Women’s Day 2014!

We started the day very early in the morning at 1 AM so that we could join with 3 radio reports. Unfortunately it rained at night so we struggled, but it was still a very successful day and we reached more than 700 people in total…

The first event was organized by Matove Beads Development Association (MABEDA). They celebrated women and cleaned up the local health facility, and gave the health workers a chance to share their challenges with the community which helped the community better understand their position so they could work together to improve it. The rain had made it hard for people to reach the facility, but still more than 50 people turned up for the hour-long celebration and everyone was so happy.

The Lubira community groups–Balindhabeene, Balikyebuza and Alikyenda women groups–organized an incredibly successful day with more than 500 people attending the event! It started with cleaning Lubira Primary School, and then cleaned Lubira Health Center–they swept, slashed the grass down, and burned all the trash. With a clean health facility, each group presented a song and drama about the “war that women go through during delivery.” They were beautiful songs and dramas, and everyone loved them. One of the Lubira politicians gave a closing speech and praised the community groups for having the love and capacity to organize something so good to celebrate International Women’s Day even when the district failed to organize something. He sent his appreciation to the SAFE team for their mentorship and support, which was echoed by all the community groups and community members.

The Kalalu community groups–Kalalu Women’s Voice (KWV), Kamede Men’s Development Association (KAMEDE), Bugya Bukye, and Balibonerawo–held a cleaning activity of the local borehole along with dramas and songs, and more than 80 people attended. It was well-received and everyone celebrated the women of the community. What a successful day!

Launch Day: Health Services

eRanger2This past summer, a group of SAFE’s international interns worked to launch a new eRanger motorcycle ambulance program. The Launch Day celebration brought together some of the most compelling aspects of SAFE’s work. Over the next two weeks, we’ll be doing a series of posts about those efforts. Enjoy!

SAFE, along with its community partners, offered various health services at the Launch Day celebration, including maternal and pediatric immunizations, prenatal care and general medical consults. This highlights the way SAFE uses its projects for multiple purposes. In this instance, knowing the launch would draw a crowd of almost a thousand people, it was important to ensure the health facility was prepared to provide preventive and proactive medical treatment. Literally hundreds of people were able to be treated as a result!

HeathServices3HealthServices1HealthServices2

Launch Day: Dramas

eRanger2This past summer, a group of SAFE’s international interns worked to launch a new eRanger motorcycle ambulance program. The Launch Day celebration brought together some of the most compelling aspects of SAFE’s work. Over the next two weeks, we’ll be doing a series of posts about those efforts. Enjoy!

As part of the Launch Day celebration, various community groups performed dramas for the almost 1,000 people in attendance. They used acting, dance and song to communicate the importance of a health facility delivery, the elements of a safe delivery, the warning signs of complications and the potential impact of the eRanger program in tactfully overcoming common cultural barriers to maternal health.

Some of the dramas started an important dialogue about gender roles by incorporating gender role reversal in their productions, in which men acted like pregnant women and women put on pants to act like their husbands. The performances got each sex thinking about how the experience of reproduction may feel to their spouse.

Because these messages were shared by community members for community members in memorable visual ways, their impact was powerful. People laughed and learned. They continue to benefit from the regular outreach events community groups host to impart this important educational service to their fellow village members.

Dramas1Dramas2Dramas4Dramas5Dramas3