A key objective of the decentralized pharmacovigilance program is to increase the knowledge of in-service healthcare professionals in pharmacovigilance to enable them develop a culture and practice of adverse drug reactions reporting. It is imperative to evaluate the impact of the training offered, as it is a key component of the national decentralized pharmacovigilance program. Thus the aim of this study was to evaluate the impact of a one-day decentralized pharmacovigilance program training of healthcare professionals on pharmacovigilance in South Africa. Self-administered structured, pre- and post-training questionnaires were retrospectively reviewed. The healthcare professionals’ knowledge regarding pharmacovigilance in South Africa significantly increased after the one-day training intervention (P < 0.002). There was an increase in the number of correct answers to every question, although to varying degrees. However, despite this increase, it is clear that various aspects of the overall training need to be re-emphasized to have an even greater impact, and there is a strong indication of a positive shift in pharmacovigilance knowledge gained, though to varying degrees.
Pharmacovigilance (PV) is the science and activities related to the detection, assessment, understanding, and prevention of adverse drug reactions (ADRs) or any other drug-related problem (WHO Policy Perspectives on Medicines) (Anon, 2004). Pharmacovigilance ensures the safe use of medicines. The National Department of Health’s Pharmacovigilance Centre for Public Health Program (NPC) in South Africa (SA), established in 2004, embarked on a program to decentralized PV since 2010. Currently, the focus of this programme is on the management of ADRs in public health program such as Human Immuno-deficiency Virus (HIV)/Acquired Immune Deficiency Syndrome (AIDS) and tuberculosis (TB). To date, 2919 healthcare professionals (HCPs) have been trained in seven of South Africa’s nine provinces (NDoH, 2015). These include 169 physicians, 618 pharmacists, 1317 nurses and 798 allied healthcare professionals. Spontaneous reporting by HCPs has been shown to play an important role in identifying drug safety issues. (Begum et al., 2013). However, under-reporting of ADRs has been a real and persistent problem for PV program. In order to improve the reporting rate, it is important to educate the HCPs about PV approaches, its tools and its impact on both the cost and quality of patient care. Ideally, the most appropriate time to train HCPs would be during the undergraduate training. However, in the South African context, PV has not been a very strong component of undergraduate training, or if it exists, it remains largely undocumented (Malangu, 2014). Given the background of insufficient PV training of undergraduates in South Africa, with very large numbers of patient on HIV treatment, the in-service training of HCPs as part of the decentralized PV program in the public health arena is an important route to remedy the existing situation. A key objective of the decentralized PV program is to increase the knowledge of in-service HCPs on PV and ADRs and to enable them to develop a culture and practice of reporting spontaneously. As this in-service training is a key component of the South African National Department of Health decentralized PV program, it is imperative that the impact of the training is evaluated. The aim of this investigation was therefore to assess the impact of the one-day PV training provided during the roll-out of the PV program and advice the decentralized PV program accordingly.
Design and study setting
This is a short prospective, descriptive, before and after intervention study. The setting for this study was in South African public health facilities, specifically in Pixley ka Seme and Namaqualand districts of Northern Cape Province, where PV training was conducted between February and March 2015. All health care professionals in the two districts were invited to attend the training through a memorandum from the provincial head of department. In particular, the training was aimed at HCPs without previous pharmacovigilance training. The training contributes to professional development and participants were expected to have better knowledge, awareness and practice afterwards.
Training intervention
One-day didactic training sessions, delivered by NPC staff were held to introduce the theory and practice of PV, together with the various systems and processes involved in PV practice in South Africa. Four hours of theory, interspersed with four hours of discussion and ADR reporting practice was given to groups of between 30 and 50 participants, a heterogeneous assortment of doctors, nurses and pharmacists but also included a few social workers and laboratory technicians.
Sample size determination and sampling technique
Although, all health care professionals in the two districts were invited, on account of staffing challenges only a limited number of HCPs were allowed to attend the training. The study population was small and consequently, all the 129 participants that came for training were surveyed.
Data collection instrument
To compare the knowledge levels before and after training, a peer-reviewed, structured, self-administered anonymous multiple-choice questionnaire was used as a pre-test/post-test tool. The questionnaire is a standard department tool that has been used over the last few years by National Department of Health (NDoH). The tool was validated when it was initially implemented, hence the researchers could not conduct the traditional validation. The finding from this study may suggest amendments to the data collection tool. The researchers were informed prior to the administration, the tool was pretested and piloted with three staff members at the NDoH in order to identify questions that don’t make sense to participants, or problems with the tool. Although, developed and used by NDoH, the tool has not been previously used to analyze the effectiveness of its training intervention. The tool had sixteen items and used both open and closed questions. This study used twelve relevant quantitative questions (questions 2 to 13) that related directly to the aims of this study. Questions 1, 14, 15 and 16 did not form part of this quantitative analysis as they are part of another qualitative sub-study. In order to facilitate interpretation and discussion, the remaining questions were grouped into pharmacovigilance concepts and theory (Questions 2, 5, 7, 9, 10, 12 and 13) and systems and processes in pharmacovigilance (Questions 3, 4, 6, 8 and 11).
Data analysis
All collected data were captured in Microsoft Excel (Microsoft Corp, Redmont, WA, USA) and exported to STATA 13 (StataCorp. 2013). Stata Statistical Software: Released 13 College Station, TX: StataCorp LP) for the statistical analysis. The number of correct answers for each question before and after the training was the variables of interest. Descriptive analyses were run to determine proportions of correct and wrong answers per question and overall. One sample test of proportions was employed to determine differences between the results for each question of the questionnaire as well as the overall results. A p value less than 0.05 was considered statistically significant.
Ethical considerations
Full ethical approval for the study was obtained from the Humanities and Social Sciences Research Ethics Committee of the University of KwaZulu-Natal (HSS/1328/016D), and permission to use the data was obtained from the South African National Department of Health.
Gender and professional characteristics
Majority of HCPs who attended the training and responded to the pre and post-test were nurses followed by “other” health care workers, pharmacists and physicians, respectively. This information is of considerable interest when seeking to request permission for the proportions of HCPs to attend training. The proportions trained were found to be representative of the proportions of HCPs in the districts.
Responses to the pre and post-training questionnaires
A heterogeneous mixture of questions on PV, the systems and processes involved in PV practice were asked in the pre and post-test. The result of these tests did not show any specific trends that favored either an increase in the knowledge of the concept of PV or an increase in the knowledge of systems and processes. However, there was an overall positive shift in improved knowledge for each of the questions asked, though to varying amounts. In some areas where there were only small increases in the knowledge gained (Questions 2,9,11 and 12) were flagged for consideration. This may be due to contamination of the testing instrument, where there may be some ambiguity or misunderstanding with the question/s, and/or areas of weakness in the training. The latter may result from a gap between materials delivered versus specific questions asked. This discussion is grouped into two areas:
1. Pharmacovigilance concepts and theory (Questions 2, 5, 7, 9, 10, 12, 13)
2. Systems and processes in pharmacovigilance and its decentralization (Questions 3, 4, 6, 8, 11)
Questions 1, 14, 15 and 16 did not form part of this quantitative analysis as they are part of another qualitative sub-study.
Pharmacovigilance concepts and theory
Which objective of pharmacovigilance do you think is most important? (Question 2 in Questionnaire)
The participants were tested for their knowledge and awareness of the primary aim of pharmacovigilance which is patient safety before (n = 80, 62%) and after the training (n = 98, 76%). The results showed a statistically significant improvement (p = <0.01). Healthcare providers need to understand that with every medicinal product comes its own benefit-harm scale, and that they should always ensure it is more beneficial.
All of the following are threats to national ADR reporting in South Africa except? (Question 5 in Questionnaire)
The participants were taught the importance of reporting ADRs to understand that medicines safety data has to be collated, aggregated and analyzed in order to pick up signals. That there was no significant improvement in the before (n = 31, 24%) and after responses (n = 41, 31.8%) (p = 0.05) is an indication that the training needs more emphasis on the adverse consequences of underreporting on PV. The question is also difficult to evaluate and revision is suggested.
Which of the following persons should not attend meetings of the PV committee? (Question 7 in questionnaire)
The participants were tested for their awareness about who should be present in the cluster and/or pharmacovigilance committee meetings. Based on the programmatic recommendations, the patient should not form part of the committee. Before the training, only 72 participants (55.8%) answered correctly to this question. After the training, the number of correct answers increased significantly (n = 109, 84.5%, p < 0.01).
Which of the following is a requirement for proper reporting of an ADR to a PV centre? (Question 9 in questionnaire)
The correct answer to this question provided
information on the awareness of trainees to the importance of a properly and fully completed ADR form. Sixty six (51.4%) of the trainees gave correct answers before the training which increased significantly to 104 (80.6) (p < 0.01) just after the PV training.
Which of the following is/are true about spontaneous reporting of ADRs? (Question 10 in questionnaire)
The spontaneous reporting system is the easiest and cheapest to establish and presently, it is the bedrock of the current decentralized PV program in South Africa (Dheda, 2013). Before the training, 63 participants (48.8%) gave the correct answer which slightly increased to 70 participants (54.3%) after the training, an increase that was not statistically significant (p = 0.25). These results suggest that more training and information on the methods and the differences in the methods in pharmacovigilance be given.
Which of the following does not determine the increased concern about drug safety? (Question 12 in questionnaire)
Pharmacovigilance and drug safety concerns have recently come into the spotlight on account of the rapid scale up of ART (pre-exposure prophylaxis [PrEP], revised guidelines to include universal test and treats among others) as well as multidrug resistant tuberculosis (MDR-TB), concerns around co-morbidities, prevalence of traditional medicines use (Clayden et al., 2013). HCPs are required to understand the reasons for the increased concerns. That there was no significant increase (p = 0.8) between before (n = 32, 24.8%) and after the training (n =33, 25.6%) is of great concern. This training going forward should be revised to include materials that will put more emphasis on these growing concerns in large treatment program.
Which of the following does not support ethical PV? (Question 13 in questionnaire)
The training gives information on confidentiality, patient education and handling of patient personal identifier information when reporting. Unfortunately, there was no significant difference (p = 0.8) in the correct responses given before (n = 37, 28.7%) and after (n =39, 30.2%) the training. This is an indication that more emphasis needs to be placed on ethical consideration in training going forward.
Systems and processes in pharmacovigilance and its decentralization
Which of the following is/are responsible for monitoring ADRs in South Africa? (Question 3 on questionnaire)
Since 1987, the Medicines Control Council (MCC) of South Africa has been the regulatory body responsible for monitoring the safety of all medicinal products used in South Africa. In 2004, the South Africa NDoH formed the NPC (Dheda, 2016). The participants were tested for their awareness of the existence of the latter and the former bodies. Before the training, only nine participants (7%) were aware of either of the two bodies responsible for monitoring ADRs in South Africa. After the training, although a statistically significant result (p = <0.01) was found, the number only increased to 11 (8.5%). The results obtained, with a mere increase of two, suggest that the component of understanding the PV systems and processes in South Africa is still poorly understood. It is proposed therefore that this component of the training be reinforced in the future and that more information, education and communication material be distributed in public health.
What do you think would be the main advantage of decentralized PV in South Africa? (Question 4 in questionnaire)
The decentralized pharmacovigilance program was established in June 2011 and is currently been rolled-out into Mpumalanga, Northwest, Eastern Cape, Northern Cape, Limpopo, parts of KwaZulu Natal and the Free State provinces. This question tested the awareness of the participants on the program with regard to the purpose of a decentralized pharmacovigilance program. The results, 76 correct answers before (58.9%) and 78 (60.5%) after did not increase significantly (p = 0.64). What was interesting however was the fact that a large number of participants (>59%) clearly understood the main advantage of a decentralized pharmacovigilance process. During the training, information relating to the advantages of decentralizing PV in public health, especially in the era of HIV/AIDS and MDR-TB, and the benefits of the resulting rapid decision-making in rational medication use in patient treatment and management were emphasized.
Within a decentralized PV program, where would assessments of ADR interventions be discussed? (Question 6 in questionnaire)
The decentralized PV program is constituted of pharmacovigilance clusters and or mini pharmacovigilance centers (Dheda et al., 2013). The latter and the former are formed between facilities that have existing referral lines and/or proximity. The clusters themselves are multidisciplinary platforms that include doctors, pharmacists, nurses and other para-medical staff such as laboratory personnel and counsellors. It is within these structures that pharmacovigilance activities are expected to take place. These clusters form the backbone of the decentralized program and trainees are expected to clearly understand their role in the overall programme. Comparatively, before (n = 12, 9.3%) and after the training (n = 96, 74.4%) the awareness by HCPs increased significantly (p < 0.01). This gives an indication that participants really understood their enhanced role in the decentralized PV program post training.
Which of the following should not be a goal of the decentralized system of PV in South Africa? (Question 8 in questionnaire)
The question about the goals of pharmacovigilance was answered correctly by 23 (17.8%) of the trainees before and 31 (24%) after the training. The score did not increase significantly after the training (p = 0.11).
Which represents a logical flow of information about ADRs in a decentralized PV system? (Question 11 in questionnaire)
The correct answer to this question provided information on the awareness of trainees around PV systems and flow of data/information. 65 (50.4%) of the trainees gave correct answers before the training which increased significantly to 80 (62) (p < 0.01) just after the PV training. This together with the increase in knowledge of their enhanced role in the decentralized PV program is positive signs for both the training and the program.
General comments
The HCPs overall knowledge about PV significantly increased after the training (p < 0.002). The increase was for every question, though to varying degrees. However, despite this increment, it is clear that various aspects of the overall training need to be re-emphasized to have an even greater impact. It is also suggested that some questions (4, 5, 8, 10, 12, and 13) be reworded to improve clarity. The lack of correct answers in the post evaluation may also be due to the training focused on the positive (increased concern) meanwhile, these questions focused on the negative (does not determine the increased concern). It has been previously shown that healthcare professionals knowledge about PV was inadequate in some countries and it was reported that ADR reporting's increased after the training (Naritoku and Faingold, 2009; Shankar et al., 2006; Rehan et al., 2002; Hema et al., 2012). Therefore, healthcare professionals training are very important to increase ADR reporting. The knowledge of HCPs in pharmacovigilance has been conducted before, however, this is the first study of the impact of the training currently provided to HCPs in the South Africa public health system. Overall, the knowledge of HCPs increased soon after the training. However, a further study needs to be conducted to establish whether this increase remains steady in the long term. Studies have shown that HCPs have some knowledge about the PV program and their spontaneous ADR reporting rate was low and one of the reasons for this was inadequate awareness of reporting ADRs (Oshikoya et al., 2009; Hardeep and Bajaj, 2013; Pedrós et al., 2009). This therefore indicates the importance of appropriate PV training.
