Monitoring and Evaluation of HWTS
Susan Murcott - Sunday 21 May 2006
Orlando Hernandez raised an important question the other day about measures for monitoring and evaluation of HWTS. I think it is very important that those of us implementing HWTS come to consensus about a common metric (or a set of common metrics) for determining long-term sustainability of HWTS implementation.
In my background paper to this conference, I offered several examples. One is from the Kenya Water and Health Organization (KWAHO) and their SODIS program in Kibera, Nigeria. They refer to their measure as “acceptance level,” and an example is shown - but it is not so easy to cut and paste into this message, so you can all refer to Table 6 in this EConf's background paper.
Another set of options are ones I suggest: "rate of adoption" and “rate of sustained use”
Rate of Adoption (ROA) = # of households using HWTS system after 1 month / # households reached (or trained)
Rate of Sustained Use (ROSU) = # of households using HWTS system after 1 year / # of households reached (or trained)
What does everyone think of these options and for those implementing HWTS, what are you currently doing to measure sustainability of your intervention?
Susan
Susan
Re: Monitoring and Evaluation
Tommy Ngai - Sunday 21 May 2006
Hi,
I would like to add to Susan's comment on monitoring and evaluation, but let me first introduce myself.
My name is Tommy Ngai. I am currently a PHD student at University of Cambridge studying sustainable development and the implementation of HWTS. I am also a research associate at Centre for Affordable Water and Sanitation Technology (CAWST) of Canada, as well as a research affiliate at MIT, where I have completed my Master degree on arsenic mitigation under the supervision of Susan Murcott 4 years ago. For the past few years, I have been working in Nepal to research, monitor, evaluate, and disseminate various household-level arsenic and/or microbial mitigation options.
Susan mentioned about need to have a common metric to evaluate the the sustainability of all HWT. Although we have not clearly agreed and defined this long-term sustainability measure, for simplicity's sake, let me call it "long-term acceptance" for now.
I agree that this is a very important measure, and will have many important implications. This measure not only compares the relative success of different implementation efforts in the past, but also can be used to plan future scale-up programs. During my work in Nepal, I have constantly faced the following questions, but I still don't know how to answer.
1. Assuming three technologies has been implemented in the same country, using the best practice social marketing model. Technology A, B, C has long-term acceptance levels of 20%, 50%, and 80%. Does it means that future funds should be focused to disseminate Technology C, as it has the highest acceptance?
2. Assuming in a country, the same technology has been implemented by two agencies using two different implementation models, and the long-term acceptance level for model A and model B are 30% and 80%. Does it mean model B is better?
3. How do you balance acceptance rate with cost effectiveness, water quality, and other evaluation measures? If implementation program A costs $10 per household reached, has a 20% acceptance, and the water quality meets the revelent standard 90% of the times; and if implementation program B costs $5 per household reached, has a 80% acceptance, and the water quality meets the revelent stardard only 60% of the times, then which is better?
Have anybody done any investigation and research in this area? Thank you for your comments and suggestions.
Tommy Ngai
tipping point - spontaneous spill over
heather lukacs - Monday 22 May 2006
First, I would like to thank the organizers of this e-conference and all of you for sharing your experiences and approaches related to household water treatment and storage. Second, I would like to introduce myself. My name is Heather Lukacs. Many of you I have either met or know by name through the years that I worked with Susan Murcott at MIT. I am still inspired by her approach as illustrated by her 'Mother's Day gift' way of considering HHWT technology – which very much resonates with me.
Now, my question – I would appreciate feedback or thoughts from you:
Have you observed cases where household water treatment technologies (or hygiene education practices) have begun to spread spontaneously, without outside intervention among people who before did not treat their water?
In the Orangi Pilot Project in Pakistan, for example, masons were trained in building low-cost sewers in peri-urban areas. After the project funding dried up, the masons continued building sewers in other communities nearby. They were able to market their skills and broaden the impact of the original project.
There are so many places in the world where community-based organizations are aware of and want to help provide SAFE drinking water to people living in their communities. It seems to me that the right technology packaged in the right way for these independently motivated people would catch like wild fire. I would love to hear stories, your thoughts, or potentially come and research cases where this has (or is in the process) of happening.
I would like to see our the discussion move from what is merely acceptable to what is preferred or desirable - after all who would want to give their mother an 'acceptable' gift for Mother's Day?
Peace to you all,
heather
------------------
Heather Lukacs
PhD student
Interdisciplinary Program in Environment and Resources
Stanford University
luka@stanford.edu