Road crashes (RCs) are the 8th leading cause of death in the   world  (World  Health  Organization,  2018).  Middle-income and developing countries are the most affected. In Benin, RCs are the 6th leading cause of death among people aged 15 and above (Ministry of Health, 2018). The lethality of Road Traffic Injuries (RTIs) in the country has almost doubled from 0.94‰ in 2016 to 1.84‰ in 2017 (Ministry of Health, 2017, 2018). The burden of RTIs is not only limited to death, but also extended to physical consequences, psychological consequences, everyday life consequences and financial consequences (Hasselberg et al., 2019).  The majority of RC deaths are preventable (Oliver et al., 2017). This requires the existence of adequate prevention policies based on the realities of the country. Like several African countries, Benin does not have a road safety policy, which includes, among other things, the prevention of RCs and RTI care mechanisms (World Health Organization 2018). For this reason, the World Health Organization (WHO) recommends building a surveillance system for RCs that can provide information on their magnitude and guide the choice of preventive measures in all member countries (Holder et al., 2001). Information on RCs is simultaneously generated by two governmental institutions: The National Road Safety Centre (NRSC) of the Ministry of Infrastructures and Transports, and the Information System Management Service (ISMS) of the Ministry of Health (MoH).

The ISMS is based on one of the components of the National Health Information and Management System (NHIMS), Routine Health Information System (RHIS). This system includes service statistics, administrative data, community data, and epidemiological and surveillance data (Ly et al.). It provides data on all elements frequently encountered in the country's health facilities and serves as an epidemiological surveillance system for RTIs in hospitals. These data are important and needed for planification and resources allocation. Given the need for decision-making support in the field of RC prevention in Benin and the leading role that this system plays in meeting this need, it is important to ensure its effectiveness in monitoring RTIs. The objective of this study was to evaluate the performance of the RHIS for RTI surveillance in Benin in 2019, using CDC’s performance criteria.

Framework of the study

This study was carried out in two cities in Benin: Cotonou and Ouidah. Cotonou comprises thirteen city districts and four health districts. In terms of health infrastructure, there are four university hospital facilities, two district hospitals and 17 health centers (Ministry of Health, 2018). Ouidah city belongs to Ouidah/Kpomassè/Tori-Bossito health district and is home to its district hospital. In addition to this hospital, there are two major health centers. Ouidah city embodies then four city districts.

Type of study, study populations and information sources

This was an evaluative study, with a mixed approach (qualitative and quantitative). Data collection was conducted from March 25 to April 19, 2019. The study targeted actors of the RHIS at all levels, and data from epidemiological reports and health statistics yearbooks were also used. The sampling method was non-probabilistic with purposive sampling involving the head of the ISMS, statisticians in the health districts offices and those responsible for completing monthly epidemiological reports (monthly summary collection forms) in public health centers, public hospitals and private hospitals involved in the collection of routine data. Additionally, one quarter of the total number of monthly epidemiological reports over the last six months were selected by systematic random sampling. So depending on given criteria, the statistic units of analysis were RHIS actors or data.

Collection techniques and tools

A semi-structured individual interview was conducted with the head of the ISMS using an interview guide. Other participants were interviewed individually using a structured questionnaire. A document review with a tabulation sheet provided access to data from health statistics yearbooks, monthly epidemiological reports and normative documents from the ISMS. These different data collection techniques were only implemented after research authorizations had been granted in each of the health districts and free, informed and written consent had been obtained from the different targets.

Evaluation criteria

The evaluation was based on the performance criteria of the Centre for Disease Control and Prevention (CDCP) for an epidemiological surveillance system (German et al., 2001). The following criteria were subject to a qualitative approach:

1. Usefulness: It reflects the system's ability to produce information on the frequency of RTIs and to contribute to the prevention of RCs in terms of users' behavior and political decision-making. To assess the usefulness of the system, its results were compared with its predefined objectives. Achieving at least one objective matches with usefulness.

2. Simplicity: Takes into account system’s structure, with the number of participants and their perception of the tasks to be performed. To judge this criterion, participants’ opinions were taken on the clarity of the case definitions, the adequacy of the amount of information to be reported, the ease of obtaining the data to be reported and the ease of transmitting the data from one level to another. The participants rated each component of this criterion on a scale from 1 to 5 (1=not at all satisfied, 2=not satisfied, 3=fairly satisfied, 4= satisfied, 5=very satisfied). This scoring model for the components has been adapted from the evaluation model for national health information systems: Health Metric Network (Ministry of Health 2008). For each component of simplicity, the sum of the points awarded was divided by the maximum number of points to obtain. The simplicity score was the mean of the four components scores.

3. Flexibility: System’s ability to adapt to changes in information needs, operating conditions and resources. This criterion was determined on the basis of recent changes in the system and was judged on two components: The theoretical and practical level of adaptation to change. Theoretical adaptation focused on the structures  involved  in  data  collection.  To  measure  the  system’s theoretical adaptation to a change, the number of structures operating in accordance with the change was divided by the total number of structures. For the level of practical adaptation, the opinion of operational players was taken on the system's reaction to each change that occurred. The latter rated the adaptation of the systems to each change on a scale from 1 to 5 (1=not at all suitable, 2=not suitable, 3=fairly suitable, 4=suitable, 5=very suitable). The sum of the points attributed was divided by the maximum number of points to see how flexible the system was considered by participants. The system flexibility score was the mean of the two levels of adaptation.

4. The quantitative approach concerned acceptability, data quality, exhaustiveness, representativeness, stability and reactivity.

5. Acceptability: Reflects the involvement of the system's participants and health facilities. It was measured by the proportion of participation of facilities/units expected to contribute to routine data collection.

6. Data quality was judged on the basis of the completeness of the collection tools: Monthly epidemiological reports (B5a for the health centers and B5b for the hospitals). These tools provided information on ailments frequently encountered in hospitals. In this study, only cells reserved for RTIs were considered. To calculate completeness, reports received over the last six months prior to the collection period were sampled. The number of unfilled cells for each tool was identified. The proportion of filled cells represented the completeness of the system collection tools.

7. Exhaustiveness of the system judges its ability to identify all cases of the event of interest, RTIs. It was obtained by dividing the number of RTIs registered by the system by the total number of RTIs in Ouidah in March 2019. As the exact total number of RTIs was not available, it was estimated by applying the capture-recapture method to both NRSC and ISMS data sources. The total number of RCs during this period was obtained using Chapman and Seber's unbiased estimator (Gallay et al., 2002):

N1 and N2 is the number of victims identified respectively by the RHIS and the NRSC, and X11 is the number of victims identified jointly by the two sources. In order to obtain the number of victims jointly recorded by the two sources, a match between databases was made using the following variables: Date of crash, type of crash, age, sex and type of injury endured by the victims.

8. Representativeness: A surveillance system is considered representative if it correctly describes the occurrence of health events monitored over time and its distribution in the population in terms of location and individual characteristics. In this study, the geographical coverage of the system was the indicator of representativeness. Thus, the number of city districts with at least one health facility involved in routine data collection was divided by the total number of city districts in the cities.

9. Stability reflects the reliability and ability of the system to operate without failure. The number of statistical yearbooks produced by the system between 2008 and 2017 served as an indicator of stability.

10. Reactivity reflects the speed with which information is transmitted from one level of the surveillance system to another. The average time (in months) taken by the system to produce the statistical yearbook between 2008 and 2017 was used to assess the system's reactivity.

Data analysis

The socio-demographic and professional data of the system participants as well as those relating to the criteria of  simplicity  and flexibility were entered and processed using the software Epi Info 7. As the usefulness of the system was subject to a qualitative approach, its data were processed manually and analyzed according to the content. As for the other criteria, an Excel spreadsheet was used to process and analyze the data.

Ethical statement

The study protocol got approval of the Institutional Review Board of Regional Institute of Public Health and was conducted with respect of confidentiality.

In health facilities and health districts (zones) offices, the targets were: Twenty-two nurses, six statisticians, a midwife and an epidemiological surveillance officer. The male to female sex ratio was 1, with an average age of 42 years and an average number of years of experience of 15 years. Of the 19 health center workers, ten had not received any training in completing the monthly epidemiological reports. All those in hospitals and zone offices had had at least one training session.

RHIS description

The RHIS is a passive surveillance system based on the pyramidal configuration of the country's health system. At the peripheral level, agents of public health centers and private clinics enter the following data in the curative care register (B1) for each patient received: Serial number, surname and first name, neighborhood, age group, sex, constants, symptoms and other signs, conduct or treatment and observation. At the end of the month, the person in charge of the center counts each ailment (cases and deaths by age group and sex). The purpose of this analysis is to complete the monthly epidemiological record. Cases of RTIs are reported under the heading "Traffic Accident Trauma". Once completed, the B5-a forms (paper format) are sent to the Health District Office (no later than the 5th of the following month) to which the health centers and clinics belong. At the Health District Office, after checking for internal consistency, the data from the sheets are integrated into the country's national online warehouse (by the 20th of the following month at the latest) using the District Health Information System 2 (DHIS2) platform. Data entry is the responsibility of the Health District Office statistician. In private and district hospitals, still thanks to DHIS2, the compilation of B5-b forms is done automatically after entering individual patient data in the "Tracker" section of DHIS2.

At the intermediate and central levels, data from Departmental Hospitals and the National University Hospital Centre (NUHC) must also be directly integrated into DHIS2 as in district hospitals.  Once  data  are  made available in the online warehouse, it is accessible to any user of the platform. Participants at different levels use it to produce indicators on ailments monitored, including RTIs and feedback. It should be noted that only data produced by the RHIS are taken into account for the calculation of these indicators. There is no interaction between the RHIS and the other source of information on RTIs in Benin, the NRSC. This configuration, based on DHIS2, is relatively recent. Indeed, its integration occurred in 2015.

System performance evaluation

The performance criteria for the RHIS in the surveillance of RTIs were as follows (Table 1):

1. Usefulness : The RHIS produces information on the road crashes victims and injuries countrywidethrough the country, by departments and by health districts. It was therefore considered useful. 

2. Simplicity: The scores for simplicity components were as follow: 77.5% for the clarity of the case definitions, 90.67% for the adequacy of the amount of information to be reported, 78% for the ease of obtaining the data to be reported and 84% for the ease of transmitting the data from one level to another. Therefore, the system was considered simple at 82.54% by the agents responsible for data collection. 

3. Flexibility: The RHIS has undergone a major change in recent years: the advent of DHIS2 in 2015. Given its partial implementation at the peripheral level, only statisticians in district offices and hospitals judged the level of practical adaptation in using this platform. The sum of the points awarded by statisticians for that component of the flexibility criterion was 35, out of a maximum of  55.  The  level  of  practical  adaptation  was 63.63%. Since the DHIS2 is used countrywide, the theoretical adaptation was 100%.  The system was therefore 81.81% flexible with regard to DHIS2. 

4. Acceptability: Out of the 232 health facilities in the two cities covered by this study, 128 were involved in routine data collection, representing a 51.4% participation rate. In the public sector, only the NUHCCNHU and the Army Training Hospital did not participate.

5. Data quality: The assessment of data quality covered 198 B5-a forms. On each B5-a forms, 39 cells concerned RTIs for health centers with an inpatient service and 21 for centers that did not offer this service. Out of the 198 reports sampled, six were from centers offering inpatient services. Out of the 4,266 cells for RTIs, 694 cells were actually filled in. The completeness was, therefore, 16.27%.

6. Exhaustiveness: From March 1 to March 31, 2019, health facilities in Ouidah city recorded 91 cases of RTIs. As for the NRSCCNSR, it recorded six (06) RCs in the same period, resulting in 12 victims. After matching the two sources, four (04) victims were jointly recorded by the two systems; the total number of RTI victims was therefore estimated at (Table 24). The exhaustiveness of the RHIS in RTI monitoring was, therefore, 38.07%.

7. Representativeness: All the 17 city districts in the two cities that were included in the study had at least one health facility (public or private) involved in routine data collection. The representativeness is therefore 100%. 

8. Stability: The system was 100% stable because all statistical yearbooks between 2008 and 2017 were actually produced.

9. Reactivity: Between 2008 and 2017, the average time taken by the SGSISMSI  to produce health statistics yearbooks was 5 months.

Although the RHIS was considered useful in RTI surveillance, it provided little information (number of cases and deaths only). Information about the type of injuries was not produced. They are not collected due to the composition of the data collection tools. These data would nevertheless be very useful in assessing the country's preventive measures, such as helmet use by motorcyclists, introduced in 2015. In addition, it could help design accurate prehospital care policies, which really matter when it comes to decreasing the number of deaths due to RTIs (Eftekhari et al., 2019). It is also important to note that, this type of surveillance system is very limited in terms of information on risk factors (Puvanachandra et al., 2012).

The good representativeness of this system is based on the country's national health infrastructure coverage, which was 94.3% in 2017 (Ministry of Health, 2018). However, it does not ensure a perfect assessment of RTIs given the level of acceptability of the system. The acceptability of health facilities was relatively lower than that found by Parker et al. (2014) for the Road Crash and Victim Information System in Cambodia (RCVIS). The non-participation of the NUHC is a real challenge for this system considering the high number of victims it attracts due to the quality of its technical platform. The same applies to Cotonou Army Training Hospital, which is affiliated with the Ministry of Defense. In addition, as in other African countries, the RHIS suffers from low participation by the private health facilities (Seitio-Kgokgwe et al., 2015). Less than half (44.77%) of the private facilities were involved in routine data collection. Although efforts are being made to further integrate the private sector, its participation in surveillance is still low due to the informal or even illegal nature of many private health facilities. The quality of feedback would also play a crucial role in acceptability (Muringazuva et al., 2017). As far as the NUHC and Cotonou Army Training Hospital are concerned, their non-participation is linked to organizational differences with the other hospitals, which does not favor the integration of the RHIS collection tools into their health information system. This raises the whole problem of the inter-operationality of collection systems that coexist in the same environment. The current morbidity and mortality indicators produced by the RHIS on RCs could be more realistic and could be increased significantly if all these structures were involved in routine data collection.

Concerning the flexibility of the system regarding the implementation of DHIS2, the relatively low level of practical adaptation (63.63%) mainly reflected its difficult implementation. This difficulty is essentially linked to a problem with the country's digital development: Internet coverage (Dehnavieh et al., 2019). The proper functioning of DHIS2 requires a high quality internet connection, which is not available in all hospitals.

The level of completeness of the system's data collection tools is relatively low compared to surveillance systems for other diseases such as dengue fever and influenza (Abdulla et al., 2014; Nuvey et al., 2019). This was mainly due to the fact that the cells that should be filled in "00" were left empty by agents. The RHIS data collection tool integrates several ailments, with a considerable number of cells to be filled; this is not the case for disease-specific surveillance system tools. The low proportion of participants trained in the completion of collection tools could also be the basis for this practice. The study only focused on B5-a forms that are specific to health centers. Given the rarity of deaths in these centers, the cells referring to them were empty. The same was true for cases in children under the age of one and from one to four years of age. All these elements are factors that explain this low completeness rate. Monthly hospital epidemiological reports (B5-b) were automatically produced by the DHIS2 platform, which considerably reduces or even excludes the risk of cells being left empty. The implementation of a training plan by the ISMS for the agents responsible for completing the B5-a forms could help improve the completeness of the system's collection media. In addition, an improvement of the data collection tools, integrating the particularities related to the low completeness could also be an area for improvement.

The capture-recapture method has frequently been used to assess the exhaustiveness of epidemiological surveillance systems (Abegaz et al., 2014; Sango et al., 2016). The exhaustiveness of the RHIS was relatively low according to the results of this study. Similar results have been found in other  African  countries.  In  Mali  in  2012, Sango et al. (2016)estimated, using the capture-recapture method, the exhaustiveness of health system to detect RTI victims at 42% (Sango et al., 2016). In 2014, Abegaz et al found in Addis Ababa, Ethiopia, a higher proportion of victims identified by health sources, ranging from 55.2 to 56% (Abegaz et al., 2014). These observed variations could be explained by differences between data collection periods. Indeed, Sango et al. (2016)collected data over a four-month period from January 2012 to April 2012, while the data collection for the Ethiopian study extended from June 2012 to May 2013 a year.

The lack of connection between the two sources of data on RTIs in Benin, the RHIS and the NRSC, is a real obstacle to effective decision-making. This configuration creates confusion about the real magnitude of RTIs due to disparities in the information provided by these sources.

Overall, this evaluation revealed that the RHIS was useful in the surveillance of RTIs in Benin. Although simple, representative and stable, this system alone was not sufficient to determine the magnitude of RTIs in Benin or to guide decision-making for preventive measures. A system integrating data from different sources (health sector, police and insurance companies) would be more effective (Puello et al., 2013). The implementation of such a system requires an awareness of the problem and a realistic analysis of the stakeholders and the feasibility in order to guarantee its full functionality.

On the basis of the elements of the methodology and the specifics of the subject, some shortcomings have been identified. First, the results obtained cannot be generalized to the whole country. Indeed, Cotonou and Ouidah are cities that have a special status and are not representative of the country. In addition, as some criteria (simplicity and flexibility) are very subjective, there could be significant variations from one region of the country to another. Regarding the application of the capture-recapture method, its validity is based on six key conditions, one of which assumes that the study population must be closed (Gallay et al., 2002). This was not the case in this study. The population of every city is subject to movement. Finally, as with other evaluations of epidemiological surveillance systems based on CDC criteria, the Positive Predictive Value (PPV) was not determined (Parker et al., 2014). Its assessment was not considered relevant given the nature of the phenomenon being monitored. An evaluation of the injury surveillance system in monitoring RC deaths in Pakistan estimated its PPV at 99% (Nittayasoot et al., 2019). RTI victims are admitted to health facilities after the crash has occurred. The diagnosis of trauma following an RC does not require any testing; it is based on the patient's declaration.

The authors have no conflicts of interest associated with the material presented in this paper.

This study was conducted as part of a Master in Public Health funded by the project PRD-2017 IRSP/Benin of Académie de la Recherche et de l’Enseignement Supérieur (ARES). The authors are sincerely grateful to all participants and authorities.