A preliminary evaluation of the delivery of the “Leadership in Mental Health, Eastern Mediterranean Region” course

Abstract

Background

It is well established that the Middle East and North Africa (MENA) region urgently needs to scale up mental health services and that a major barrier is a leadership gap including both mental health specialists who lack public health/planning skills and public health system leaders who are not trained in issues relevant to mental health. To address the need for capacity-building in mental health leadership in the MENA region, a 10-day course was developed and delivered to 43 participants from fourteen countries. Both quantitative surveys and qualitative follow-up interviews were conducted with the course participants.

Results

A positive impact was found on participants’ knowledge, motivation, and implementation. Participants reported the greatest changes in their ability to integrate mental health into primary care, along with changes in ability to plan, take leadership, advocate for change, and establish networks between and within countries and felt better able to use research to advocate for mental health services and training.

Conclusion

The course is an effective tool for developing leadership skills in the region and demonstrated that participants used the knowledge gained to implement change in their settings.

Background

In 2007, a landmark series of papers published by the Lancet established psychosocial disabilities as a global priority in addressing unmet health needs, and an essential consideration in addressing global poverty [1]. Since then, a wealth of evidence has emerged, including a further Lancet series [2], a PLoS Medicine series [3], and a range of resources through the World Health Organization’s (WHO) Mental Health Gap Action Programme (mhGAP) [4]. Much of this evidence points to the urgent need to close the mental health treatment gap. It is estimated that about 13% of burden of disease is caused by mental and neurological disorders, but about 85% of people with serious mental disorders in low- and middle-income countries do not receive the care they need [5]. This causes a huge economic and social burden on communities and countries, as well as individuals and their families. It is vital, therefore, to significantly scale up services for people with mental disorders.

In order to achieve this scale-up of services, barriers that made previous efforts falter must be addressed [6]. One such barrier is a lack of the leadership skills necessary to sustain national program development. This leadership gap includes both mental health specialists who lack public health/planning skills, and public health system leaders who are not trained in issues relevant to mental health. Despite the fact that mental health is now emerging as a global health field in its own right, this lack of skills in important planning positions has resulted in mental health remaining a low priority in many countries. Even when countries make strategic decisions to strengthen mental health, these decisions have often not been supported by evidence-based interventions that ensure effective and affordable services are provided [7,8,9]. In recognition of these weaknesses, the Eastern Mediterranean Regional Office of the WHO included in its Regional Strategy an emphasis on adding a public health approach to the training of mental health professionals. Several recent guidelines have also cited leadership development as a key priority for global mental health [10].

Since 2007, a number of leadership courses have been implemented to build the capacity of key players to scale up regional mental health services. One such program is the “Leadership in Mental Health (LMH), Eastern Mediterranean Region”.

Research statement

The aim of this paper is to examine the experiences of the professionals who attended this course on their leadership roles and skills, and if so, what type of impact it had; an evaluation was conducted.

Methods

The course was developed by the Eastern and Mediterranean Regional Office of the WHO and the Department of Psychology at the American University in Cairo (AUC). Existing curricula served as a basis for the course design, and then local, regional, and international experts worked together to refine it. The final result was a 10-day residential course delivered in English and designed to give mental health professionals an introduction to mental health leadership skills. Topics included putting mental health on the public health agenda, mental health policy, advocacy, project design and evaluation, emergency response and vulnerable groups, promotion and prevention, addressing stigma, and developing community-based services and alliances. The course objectives included increasing students’ understanding of (1) the leadership skills required in order to improve mental health services, (2) how to develop and disseminate national mental health policy and strategy, and (3) how to decentralize mental health services in order to improve access and meet mental health needs. The aim was that by the end of the course, participants would be motivated to improve the mental health of their populations, enhance the care of service users, prioritize mental health on the wider health and development agenda, and pursue further training and development of their professional networks.

The course structure included delivery of didactic teaching, but also utilized active learning strategies such as case studies, exercises, group discussion, and over-arching group projects designing mental health plans. Participants also gave country-specific presentations on their own work and the status of mental health services in their country.

Participants

The course was delivered in July 2016 and July 2017 at AUC. Participants were enrolled into an online group for the purpose of communication during and after the course. Forty-four mental health professionals from fourteen MENA region countries attended (see Table 1). The age range of the participants was from 23 years to 60 years old, including 17 males and 24 females.

Table 1 Breakdown of participant countries and occupations

Statistical analysis and evaluation

The evaluation of the course relied on three sources of data: (1) quantitative daily evaluations assessing reactions to the training, (2) a retrospective test of learning given at the end of the training, and (3) 6-month follow-up interviews on how material from the training was being used in practice.

Daily satisfaction surveys

Participants filled in daily evaluations that asked how much they agreed with six statements related to the course material and the quality of the trainers, using a scale of 1 to 5, with 1 indicating strongly disagree, 2 disagree, 3 neutral, 4 agree, and 5 strongly agree. These daily evaluations indicated that across all of the units, participants agreed that the curriculum achieved learning objectives (4.3), that the information presented was relevant to the topic (4.4), that there was sufficient time for discussion (4.3), and that the exercises were helpful (4.3). Participants also felt that the instructors were easy to understand (4.2) and answered their questions (4.2). They agreed that the overall quality of the sessions was good (4.3). See Table 2 for a summary of these daily evaluations.

Table 2 Module evaluation summary

Overall learning survey

At the end of the training, participants completed a retrospective pre/post survey as an indirect measure of what they had learned. This approach was used in order to avoid response shift biases that might occur because participants were not able to accurately assess their knowledge prior to the training. Participants were asked how much the training had increased their knowledge in terms of unit learning outcomes. Each outcome was listed and participants indicated whether they had achieved “no difference in knowledge” (1), “small difference in knowledge” (2), “moderate difference in knowledge” (3), or “large difference in knowledge” (4).

Results

Across all of the units, participants indicated that they had experienced a moderate difference in their knowledge as a result of the training (3.2). Participants indicated that the largest change in knowledge was in the area of prevention and promotion (3.3), and the area of least change in knowledge was mental health law (3.0). See Table 3 for a summary.

Table 3 Overall learning survey

In addition to the questions on changes in knowledge, participants were asked two open-ended questions. In response to the question: “When I think about my work in the field of mental health, I believe this course has had the most impact on my ability to,” participants stated that the course had most impacted their ability to integrate mental health into primary care (n = 12), to take leadership and plan (n = 10), to involve stakeholders or network (n = 8), and to use research to create real change (n = 2). They said it impacted their learning and sharing knowledge (n = 4) and helped them in developing policy (n = 2) and advocacy skills (n = 5). In response to the question asking for suggestions for the course, participants stated that there was a need to change the time; either to have more time, shorter days, or fewer days (n = 9). They also asked for more practice and interactive teaching (n = 6) and better coordination between the lectures to avoid repetition (n = 3).

Six-month follow-up

To assess the impact of the course on the participants’ practice, phone interviews were conducted with 18 of the participants. The interviews were conducted in English, transcribed, and analyzed for common themes.

When they were asked “How have you used the material from the course in your work?,” participants stated that they had used what they had learned for the purpose of advocacy (n = 8) and had worked to integrate mental health into primary care (n = 10). As one stated,

It actually showed in which direction I have to start not only for the service delivery but also for the planning for mental health and that means to engage all the players, all the authorities from the officials in the ministry of health and the also the government hospitals and district hospitals how to have a room or to have a small unit for mental health services not just to provide psychiatric services but also to provide mental health in general.

Another described how they had used the knowledge from the course to work with the Ministry of Health in their home country to open a National Mental Health Center and to begin to prepare for a Mental Health Act

Another example also I worked with the ministry of health on how to start this national mental health committee, and how to prepare for a mental health act. I am saying that we worked with the ministry of health and authority to have a political willingness in terms of how to have a clear policy for the mental health establishment and providing the mental health for all the needed persons especially in displaced camps. As well to have a clear policy in terms of budget for the mental health and planning for the mental health as well with all sectors. At the same time we work with the community on how to avoid stigma about the mental health in emergency situations and in ordinary situations so we, a lot of utilization of the material that we had from our training in Cairo.

Participants felt they had received helpful information (n = 6), and many had used the material to train their staff (n = 10). For example, one said that

We have about 150 health officers- general practitioner, and we have 100 midwives, and 300 practical nurses and nursing staff, we are planning to train all of them on mental health and mental health gap and choose the modules which can be implemented within our health center. So, we just, we are, we are in the beginning, so the course was a very strong baseline for me.

Some cited in particular that it had been helpful to learn about working with emergencies (n = 4), monitoring, and evaluation (n = 6). They also reported that it had increased their motivation (n = 9).

When asked if the course had impacted their leadership in mental health, participants reported that it had enabled them to better integrate mental health into their work (n = 8), helped their team leadership skills (n = 4), and had helped them to engage stakeholders (n = 6). Other participants said that they had taken on more leadership, joined a UN task force, worked to change policy, or tried to empower community leaders. One stated that

When I was brought in the leadership, in the leadership room, I was told that I am not enough, I’m not enough to do this, in the head room. But with the help of the course, I really used it too much and I used what I have learned too much… They asked me to organize the clinical all around the hospital, and then to give the decision makers what we will do next to make the clinical affairs better.

Participants were asked if the training helped them build networks. They felt it had helped them build connections to others in their own countries (n = 11), and outside of their countries (n = 11), and was motivating and beneficial to hear from other countries (n = 9). Participants talked about how they shared information with each other and shared ideas, for instance one participant from Lebanon said

Yes, I got to know Egyptians the most, such as Dr. ***** and Dr. D*****. I always stayed in touch with them. My colleagues also helped me during the course, as did the group from Sudan; they gave me new ideas I can use in my work. The group from Sudan also said that they’d invite us as an organization once a course becomes available at their end. Since I work with elder people and PSS activity, I was able to give them some of my ideas and I sent them a few PowerPoints, and they also gave me some ideas as well.

When asked if there had been challenges or successes in implementing what they had learned in the training, the most common challenge was limited resources, including time and money (n = 9). Challenges in getting others on board with mental health integration was also mentioned (n = 4), as well as political challenges such as red tape and safety [6]. Participants felt that they were successful in developing plans for mental health integration into primary care (n = 6). One participant stated the following

Yes, because we open a new unit in the general hospital in Tripoli, because we only have only one big hospital for psychiatry in Tripoli, we opened a unit in the general hospital which is at the center of the capital. This is a new approach and a new idea that you open a unit in a general hospital, which is a good idea.

Participants further mentioned implementing monitoring and evaluation, a focus on prevention, networking for resources, and focusing on the needs of beneficiaries. Two participants mentioned a success of changing legislation, which given the 6-month time frame is a rather impressive achievement.

Finally, participants were asked if they had any recommendations for further training. They suggested that the LMH training be more practical (n = 9), include more group work (n = 7), and offer field trips to mental health clinics (n = 4); they also wanted to have more time for the course (n = 5) and follow up the course in order to maintain learning (n = 6).

One participant stated, “I would say it definitely motivated me to take further courses and not only me because the first week that I returned back from Egypt, my friends the recommendation for the leadership was to send some more people to this course if it was happening again in the future.” Participants said that they would like the opportunity for more training in the areas of utilizing traditional healers and child protection.

Discussion

The aim of the paper was to examine the experiences of the professionals who attended this course on their leadership roles and skills, and if so, what type of impact it had. The survey findings demonstrate that the LMH course had a moderate positive impact on participants over the two cohorts. However, the qualitative data demonstrates a strong positive impact. Both at the end of the course and at 6 months later, participants reported the greatest changes in their ability to integrate mental health into primary care, ability to plan, take leadership, and advocacy. This was particularly apparent in the follow-up interviews where the majority of participants described ways in which they had implemented change; several spoke of how they had used the knowledge from the course to effect policy change, such as working with the Ministry of Health in their home country to open a National Mental Health Center and to begin to prepare for a Mental Health Act, opening mental health units within primary care settings, and training staff. Such developments within a time frame of 6 months are to be commended.

The use of monitoring and evaluation skills drawn from the course were frequently mentioned, as was being better able to use research to advocate for mental health services and training. Perhaps most importantly, the majority of participants reported increased motivation to scale up services and provide better mental health care to their communities. One notable area of positive feedback was the impact of learning from international colleagues, sharing best practice, and being able to gain support from other professionals. This was particularly pertinent as many participants described feelings of isolation, a lack of support, and not knowing where to get support from at the onset of the course.

Challenges

The difficult economic situation in the region and problems related to obtaining Egyptian travel visas created challenges for some participants to attend. Although the organizers tried to limit costs, bringing international experts and delivering a residential course was expensive. To address this, low-cost accommodation was offered on site in the faculty residences and where possible scholarships were provided. Online teaching was considered as a way to address both costs and visa difficulties, but internet access and service can be problematic, and the value of bringing people together for group work, discussion, and networking would be lost.

Lessons learnt and recommendations for future implementation

Based on the results of the evaluation, it is apparent that more practical applications specific to the region need to be incorporated into the sessions. Facilitators should collaborate prior to course delivery in order to minimize repetition of material, ensure that the material is a good fit for the time allotted and is relevant to the region. In the future, more local experts will be utilized in order to reduce expenses and improve regional applicability.

While participants noted that they had developed many networks since the training, there were also a number who had not managed to sustain such networks. An important area for development of the course is to find a way of maintaining networks with course alumni and connecting past cohorts together in a meaningful way.

Conclusion

The results suggest that such training is an effective tool for developing mental health leaders to address the mental health gap in the MENA region. It is important when designing such trainings to emphasize practical applications that are relevant to the region and to give participants opportunities for hands-on learning. Making sure that networking occurs during the training and is maintained afterwards is key to providing both local expertise and ongoing support to participants. Finally, addressing challenges around affordability and travel concerns is essential to ensuring that those who need the training most are able to access it. As the number of health professionals who complete the course increases, hopefully we will start to see an impact on closing the mental health gap in the MENA region.