Retention and risk of burnout among health workers is one of the major challenges in the healthcare industry today. A healthcare manager in facilities supervises over 100 employees resulting in a lack of meaningful engagement and unnoticed signs of burnout. Result? Burnout risk and high turnover rates among healthcare workers.
To tackle the problem, a healthcare engagement platform reached out to us to employ AI in a solution that drives intelligent human engagement. This product helps to frontline healthcare managers to engage with employees for effective management and deal with employee burnout. The platform provides a simple, real time, automated approach for personalized engagement between managers and their team members. Thus, enabling managers to keep track of employee burnout ultimately helping them to predict burnout months before and trigger appropriate actions to reduce burnout rate among healthcare workers.
Burnout is a critical problem in the healthcare industry. According to a report published in 2017 by Medscape, there was a 25% burnout increase in physicians in the past four years. Not only physicians, but it is a major concern among other healthcare workers like nursing staff, physician assistants, and residents, and medical students. This has only been exacerbated by the recent Covid19 pandemic, during which many of these healthcare workers are in the frontlines.
Wikipedia defines burnout as:
A syndrome resulting from chronic work-related stress, with symptoms characterized by "feelings of energy depletion or exhaustion; increased mental distance from one’s job, or feelings of negativism or cynicism related to one's job; and reduced professional efficacy.”
The effect of the personal well being of the health worker as a result of burnout has a major impact on quality and adverse impact on patient care. For example:
Caboom helped the healthcare engagement platform by developing a prediction model to score the risk of employee turnover due to burnout in the platform. The model was deployed in its AWS cloud-based data pipeline.
When the client approached us, the engagement platform already had an existing rule-based algorithm in the product that gave a risk score for employee turnover. We were tasked with researching whether an ML-based solution would help improve the accuracy of the score and integrating it into their product.
The existing scores were generated through a mixture of researched and defined KPIs based on staff's working shifts, attendance, breaks, and so on. These KPIs were highly useful and accounted for information that couldn't simply be learned from the raw data. The existing KPIs are further feature engineered to make the machine learning algorithms more robust.
The objective of the ML solution was to not only provide a single highly accurate risk score for employee turnover but also to help the managers understand the contributions of the handcrafted KPIs on these risk scores. To accomplish this we explored both simple machine learning models that were explainable, as well as more complex ones that provided high accuracy at the cost of explainability.
After some iterations for our algorithm ended up using an ensemble combination of two models:
- Neural network
The Neural network model was 85% accurate while XGboost was 65% accurate.
The neural network was used where high accuracy was required and XGBoost was used to give insights into how KPIs are contributing to making an inference to add interpretability. Furthermore, logistic regression objective function was used in xgboost to get the probability of inference and probability of the contributing KPIs in inference. This system can be seen in figure above.
The scores from this model were then fed into a final scoring engine based on some hand-crafted business logic to provide the final turnover risk score.
The major finding from this work was that even small changes in human behavior, engagement, and interaction can be modeled by a complex system that recognizes which behavior triggers employee burnout. Thus, alarming us at least a month before turnover with an explanation of what may have caused it. Achieving the same kind of insights manually with a large number of employees is rather difficult. It would be like asking a human to draw a map of the world just by walking around. But machine learning models can handle the volume and complexity of this data and learn patterns hidden within them quite well, saving hundreds of hours of work for the managers.