Welcome to ISQua Login | Register | Contact Us

Building in effective learning strategies in improvement work 

Case Study by Lani Marquez, MHSc, Knowledge Management Director at University Research Co., LLC (URC) for the USAID Applying Science to (ASSIST) Project

Lani Marquez, MHSc, is Knowledge Management Director at University Research Co., LLC (URC) for the USAID Applying Science to (ASSIST) Project, a global health systems strengthening initiative that supports health care improvement in low- and middle-income countries.




This case study looks at how to build in effective learning into improvement work. Projects often achieve significant improvements in quality of health care in demonstration sites that receive intensive support from project staff.

Please read the case study below and consider the following questions:

Introduction
This case looks at how to build in effective learning into improvement work. Projects often achieve significant improvements in quality of health care in demonstration sites that receive intensive support from project staff.  A key challenge for improvement science interventions is how to efficiently apply the knowledge gained in demonstration sites on a broader scale, with less resource-intensive methods.  Knowledge management (KM) approaches that facilitate transfer of tacit knowledge among peers and generate explicit knowledge that to enable others to replicate improvements in care offer a potentially cost-effective strategy for spreading improvement knowledge.

Case presentation
In Uganda, since 2010, encouraging men to undergo voluntary medical male circumcision (VMMC) has been a key national strategy for the prevention of HIV transmission for the Ministry of Health (MOH). Yet external quality assessments in 2012 showed serious gaps in VMMC care quality and compliance with MOH standards.  In 2013, the United States Agency for International Development (USAID) supported an 18-month collaborative improvement intervention with 30 USAID-funded sites which demonstrated significant improvements in compliance with VMMC standards (Byabagambi et al. 2015). USAID asked the project to scale up VMMC improvement to 100 more sites spread across the country.  The project applied a series of KM strategies to support this scale-up.  The first strategy was to document and synthesize learning across sites about specific ways that the demonstration sites improved VMMC services.  This process, which involved bringing representatives for some of the demonstration sites and the coaches that supported the sites together in a “knowledge harvest meeting” to review specific areas of VMMC services and gather ideas on what were the changes tested by these teams that led to improvements in each VMMC service area, led to the development of a written guide on how sites could improve compliance with MOH standards and patient outcomes.  The written guide constituted the second KM strategy.  

In addition to the written guide, the project brought representatives from 10 demonstration sites and five experienced coaches together with representatives from 10 new sites and new coaches to talk through how to apply the written guide to the most common problem areas the demonstration sites encountered in trying to improve VMMC services.  The format for the two-day meeting, known as a “knowledge handover”, provided many formats for knowledge sharing: representatives from demonstration sites presented aspects of their experience in plenary and during “talk shows” while small group discussions provided an opportunity for more in-depth discussion. During small group discussions organized by site, new sites were able to ask more directed questions of experienced sites. Sessions were organized topically around the seven domains in the MOH VMMC standards.  

The fourth KM strategy used to transfer knowledge about how to rapidly improve care was to have an experienced project improvement coach visit new sites.  These coaching visits, which lasted one to three hours, provided the opportunity for the coach to problem-solve with the new site on how to improve VMMC services. 

Management and outcome
All four KM strategies were implemented by in-country project staff with involvement of national and district level MOH officials and proved to be feasible for health professionals in a low-income setting to apply.  To evaluate the KM strategies, the project implemented a study to measure the costs of each KM strategy and the change in VMMC service quality (measured as compliance with MOH VMMC standards) observed by the new sites that were exposed to the written guide, knowledge handover meeting, and coaching visits.  The strategies were introduced in an additive way in three study arms: 1) the first arm received only the written guide; the second arm received the guide and participated in the knowledge handover meeting; and the third arm received the guide, participated in the handover meeting, and received three on-site coaching visits within five months of the handover meeting. Observations of compliance with VMMC standards pre- and post-intervention were analyzed in light of costs, to produce incremental cost-effectiveness ratios for each KM strategy.  Four areas of VMMC service delivery were evaluated: history taking, consent, anesthesia, and post-procedure instructions. 

The results showed that the guide plus handover meeting was only more effective in producing gains in providing complete post-operative instructions than the guide alone and that the most intensive intervention (sites receiving all three KM strategies) produced the biggest gains in history taking (34.8 percentage point improvement), anesthesia administration (19.9 percentage point improvement), and post-operative instructions (36.6 percentage point improvement). Qualitative interviews explored the experience of each KM strategy and found that participants valued both the written guide was well as opportunities for face-to-face learning through the handover meeting and coaching visits. The more intensive intervention of manual + participation in a handover meeting + on-site coaching yielded the greatest improvement but at about 10 times the cost per additional patient compliant with quality standards compared to only distributing the guide.

Discussion
This case shows how relatively simple KM strategies can be incorporated into the scale-up of better care to fully leverage an initial collaborative improvement experience of demonstration sites. Our Uganda safe male circumcision study found that KM strategies contributed to better care in new sites that did not participate in a full collaborative improvement activity. The written guidance on how the demonstration teams improved care, organized in a user-friendly structure that facilitated finding ideas on how to improve specific areas of VMMC services, was well received by the new sites.  At the same time, we found that the handover meeting and coaching visits enabled the new teams to make better use of the written guide by allowing them to explore key information in the guide with experienced teams and discuss questions they had about how the improvements were introduced. Face-to-face conversations between experienced improvers and novices allowed the new teams to ask questions, and we believe, better enabled them more effectively apply the learning garnered from the demonstration sites. Reducing the costs of face-to-face knowledge transfer could improve the cost-effectiveness of KM strategies.

We recommend all three KM strategies.  Synthesis of learning from demonstration sites is the essential first step in knowledge transfer and requires a deliberate process to gather that knowledge and organize it in a way that can be readily used by new sites.  At the same time, the written guidance alone did not seem to lead to significant improvement, perhaps because new improvers did not fully utilize the information until prompted to do so through face-to-face interaction.  The knowledge handover meeting, held in the country’ capital, was not cost-effective but still added value by engaging new improvers in understanding the issues and solutions covered in the guide.  On-site engagement of the whole improvement team through coaching visits proved to be the most effective strategy for improving VMMC service quality, underscoring the importance of ongoing support to improvement teams.

Questions
1. How do you gather insights about how to make care better in your improvement project? What process is used to integrate the experiences of different improvement teams?
2. How are these insights codified into written guidance for others?
3. How are written materials deployed?
4. Are there opportunities for staff who are starting improvement work or being asked to implemented tested changes, to meet and talk with staff who are experienced with implementing improvement? How are these encounters structured? 
5. What KM strategies could you see integrating in your next improvement project?
 


References

Byabagambi J, Marks P, Megere H, Karamagi E, Byakika S, Opio A, Calnan J, Njeuhmeli E. 2015. Improving the quality of voluntary medical male circumcision through use of the continuous quality improvement approach:  A pilot in 30 PEPFAR-supported sites in Uganda. PLoS One. Published: July 24, 2015. http://dx.doi.org/10.1371/journal.pone.0133369.


More information about KM techniques and strategies is available at:
• https://www.usaidassist.org/content/building-capacity-improvement (sections on documenting, synthesizing, and sharing learning)

• http://liberatingstructures.com (excellent resource for simple group sharing exercises)

• Designing Participatory Meetings and Brown Bags: A TOPS Quick Guide to Linking Development Practitioners: http://www.fsnnetwork.org/resource-library/knowledge-management/designing-participatory-meetings-and-brownbags-tops-quick-guide