Analysing 3429 digital supervisory interactions between Community Health Workers in Uganda and Kenya: the development, testing and validation of an open access predictive machine learning web app

This study explored the use of artificial intelligence to support supervision of Community Health Workers (CHWs) through a machine learning web application called CHW supervisor, which was designed to automatically code instant messages exchanged between CHWs and their supervisors. Developed using 2,187 messages from Uganda and validated on 1,242 messages from Kenya, the app’s performance was compared to human coders. While human coders showed high agreement (88–95%, Cohen’s kappa 0.7–0.91), the app achieved moderate accuracy (73–78%, kappa 0.51–0.56). The findings highlight that while AI tools like CHW supervisor show promise, human expertise remains essential due to the nuanced nature of supervisory communication, posing challenges for fully scaling digital CHW supervision.

Read More >>>