Efficiency, quality, and management practices in health facilities providing outpatient HIV services in Kenya, Nigeria, Rwanda, South Africa and Zambia

Few studies have assessed the efficiency and quality of HIV services in low-resource settings or considered the factors that determine both performance dimensions. To provide insights on the performance of outpatient HIV prevention units, we used benchmarking methods to identify best-practices in terms of technical efficiency and process quality and uncover management practices with the potential to improve efficiency and quality. We used data collected in 338 facilities in Kenya, Nigeria, Rwanda, South Africa, and Zambia. Data envelopment analysis (DEA) was used to estimate technical efficiency. Process quality was estimated using data from medical vignettes. We mapped the relationship between efficiency and quality scores and studied the managerial determinants of best performance in terms of both efficiency and quality. We also explored the relationship between management factors and efficiency and quality independently. We found levels of both technical efficiency and process quality to be low, though there was substantial variation across countries. One third of facilities were mapped in the best-performing group with above-median efficiency and above-median quality. Several management practices were associated with best performance in terms of both efficiency and quality. When considering efficiency and quality independently, the patterns of associations between management practices and the two performance dimensions were not necessarily the same. One management characteristic was associated with best performance in terms of efficiency and quality and also positively associated with efficiency and quality independently: number of supervision visits to HIV units.

& With a mean process quality score of 34%, overall process quality was low and quality scores were low across the five countries. All countries could make considerable quality improvements. & One third of health facilities were classified in the bestperforming group with above-median efficiency and quality. Best-performing facilities were more likely to be larger and privately owned than poor-performing facilities. & Best-performing facilities were more likely to have more HIV unit supervisions and more managerial meetings. They were also more likely to have reward schemes for good performance. Conversely, they were less likely to use the rotation of well-performing health staff as an incentive. & Implementers should know that the patterns of associations between management practices and technical efficiency and process quality are not necessarily the same for the two dimensions of performance when they are considered independently. Number of supervision visits

Introduction
Despite significant progress achieved in addressing HIV globally, the annual number of new HIV infections among individuals 15 years and older remains high and almost one third of people living with HIV eligible for antiretroviral therapy (ART) do not receive treatment [1]. At the same time, following more than a decade of increases in financing for HIV services, there has been a leveling off in recent years [2]. To increase access to HIV prevention and treatment services in the context of constrained financial resources, there is an international commitment to improve the allocation of HIV spending in low-resource settings by focusing on the appropriate evidence-based interventions, populations, and geographic areas [3][4][5][6][7][8][9][10][11]. Likewise, improving technical efficiency in HIV service delivery, or ensuring that facilities delivering HIV services use inputs to produce the largest quantity of outputs possible, is also a global priority [12,13]. While this emphasis on improving the technical efficiency of HIV services is necessary, the quality of these services also requires attention [14]. There are concerns that there could be a tradeoff between efficiency and quality in HIV service delivery, and that quality could suffer if programs focus exclusively or excessively on cost reduction without also paying attention to service quality [15]. The relationship between cost and quality of health services is complex and remains poorly understood. Research on the relationship between health service costs and quality suggests that costs and quality are jointly determined, but the direction of the association remains uncertain [16]. While some research has focused on exploring the potential endogeneity in this relationship [17], other studies, specifically on benchmarking, have treated efficiency and quality as two independent dimensions of performance that one can benchmark on simultaneously [18]. In the area of HIV service production, there is a growing literature on the efficiency of health facilities delivering HIV services in low-resource settings [19][20][21][22][23]. However, there is only limited information available on the relationship between efficiency and quality of HIV services in these settings [21,22], with only one study differentiating between facilities that performed well in efficiency and quality from those that performed poorly in both and those that exhibited a potential trade-off [22].
As implementers around the world work to improve HIV service delivery, it is critical that they understand the relationship between efficiency and quality and the factors that determine both dimensions. Facility-level management characteristics may explain some of the variation in efficiency across facilities [21]. There are an increasing number of studies on the relationship between facility-level management practices and productivity, which come mostly from higher resource settings [24,25]. This work also finds that better management techniques in health facilities are associated with improved quality of care [25][26][27][28][29], though there is increasing evidence that managerial capacity in the healthcare sector is scarce, especially in primary care environments [30]. In HIV service delivery, there is limited evidence on the management factors that complement the production of efficiency and quality or those that could generate a trade-off between them [23]. Implementers are more likely to achieve global HIV service delivery goals if they understand how to avoid situations where efficiency in the production of HIV prevention services is achieved at the expense of the production of quality or vice versa.
To provide insights on the performance of outpatient HIV prevention units, we used benchmarking methods to identify best-practices in terms of efficiency and quality and uncover relevant factors with the potential to improve efficiency and quality. We focused on technical efficiency, defined as the best combination of inputs to achieve an optimal level of production [31][32][33] and process quality which refers to the way health services are provided to deliver desired outcomes [34]. To evaluate efficiency, we used data envelopment analysis (DEA) which produces a ranking of comparable decision-making units (DMUs) in terms of their technical efficiency, in a context in which their production function must accommodate multiple inputs and outputs [35]. DEA has been used extensively to assess the efficiency of public services, including healthcare, but most DEA studies have considered efficiency only, without considering service quality [36]. Among those DEA studies that have incorporated quality, some have included quality as an exogenous variable in the second stage of the DEA analysis [37], some have added quality as an output measure in the DEA model [38][39][40][41][42][43], and others have treated efficiency and quality as independent dimensions that are benchmarked at the same time [18,22,36]. In this study, we used the latter method of treating quality as an independent dimension of performance. This approach allowed us to identify and characterize bestperforming facilities with high efficiency and high quality. We could also identify organizational correlates of performance, drawing from a rich dataset on facility characteristics including management practices. We hypothesized that there are management practices associated with best practice (high efficiency and high quality) and that the patterns of associations between management factors and efficiency and management factors and quality may not be the same. Figure 1 shows an overview of our methods. We used data from a sample of health facilities providing HIV prevention services collected as part of the "Optimizing the Response in Prevention: HIV Efficiency in Africa" (ORPHEA) and the "Optimizing the Response in Prevention and Treatment: HIV Efficiency in Nigeria" (ORPTHEN) studies. We conducted our analysis in four steps. First, we estimated a technical efficiency score for each health facility using DEA, coupled with a post-hoc analysis which factored in contextual determinants of efficiency. Second, we estimated a process quality score using data from medical vignettes using data-reduction techniques and reliability analysis. Third, we mapped the relationship between the efficiency and quality scores and we studied the managerial determinants of best performance in terms of both efficiency and quality using a positive deviance approach. Fourth, we used seemingly unrelated regression to study the relationships between management practices and efficiency and quality independently and to assess whether some management factors might generate trade-offs between efficiency and quality.

Data
We used health facility data collected in analogous retrospective cross-sectional micro-costing studies conducted in Kenya, Nigeria, Rwanda, South Africa, and Zambia [44][45][46][47]. Data covered 2011 for Kenya, South Africa, and Zambia, 2012 for Rwanda, and 2013 for Nigeria. In-depth descriptions of the sampling and data collection methods used in Kenya, Rwanda, South Africa, and Zambia have been published elsewhere [46,47]. The sampling strategy was standardized to collect information comparable across these four countries and corresponding methods were used in Nigeria [44,45]. In each country, multistage sampling was used to first select sub-national areas and then randomly select health facilities providing at least one of the following outpatient services: HIV testing and counseling (HTC), prevention of mother-to-child transmission (PMTCT), voluntary medical male circumcision (VMMC), and in Nigeria also ART. Health facilities in selected sub-national areas were enumerated along with information on the services of interest stratified by ownership/management and level of service provision. Facilities were randomly sampled within these strata using probability proportional to size, with preference Fig. 1 Overview of methods given to sites providing more than one outpatient service of interest. Facility sample size was calculated to identify the minimal number of facilities necessary to detect statistically significant associations between average unit cost per facility and the determinants of efficiency. The analytic sample included 338 facilities with the sample breakdown by country as follows: 46 facilities in Kenya, 141 facilities in Nigeria, 52 facilities in Rwanda, 42 facilities in South Africa, and 57 facilities in Zambia. Nonhospital health facilities (i.e., health posts, health centers, and medical clinics) comprised two-thirds of the sample and hospitals made up the remaining third. A standardized set of ORPHEA and ORPTHEN survey instruments were developed. All instruments were piloted in all countries and minor adaptions were made for each country setting. Data collection was staggered by country between October 2012 and December 2013. Data on HIV prevention service inputs and outputs were collected retrospectively by month for the entire previous calendar year, whereas data on the process quality of services and the management characteristics of facilities correspond to the time of data collection.

Estimation of efficiency
DEA assumptions We estimated the efficiency of health facilities using DEA. DEA is a nonparametric linear programming technique used to measure technical efficiency in a sample of homogeneous decision-making units (DMUs) such as health facilities [48]. As a data-driven method, DEA identifies a best practice frontier from the most efficient DMUs in a sample and computes the distances between each DMU and the frontier, providing efficiency scores in percentage terms (where 100% means full efficiency and 0% means full inefficiency), and therefore an efficiency ranking of DMUs. DEA is one of two efficiency analysis approaches along with Stochastic Frontier Analysis and it is the dominant approach used to measure efficiency in health care [49,50].
We developed an output-oriented variable returns to scale (VRS) DEA model. Output-oriented DEA models maximize outputs with a given amount of inputs [50]. This means that our efficiency scores represent the proportional increases in outputs that DMUs could achieve using the same level of inputs, if they were on the efficient production frontier. We considered that the policy goal is to increase coverage and accessibility of HIV prevention services and not to minimize input use. We assumed a VRS production process, meaning that any change in output results in a variable change in input [48]. The returns to scale distribution of facilities by country is shown in Supplementary Material Table S1.
The output-oriented model is specified as follows: u jn y jn −w n ; ∀n ∈ f1; 2; …; kg s:t: Where θ n is the technical efficiency score of the n-th DMU, u jn and v in are the relative weights of the i-th input and the j-th output of the n-th DMU (as defined below), and w n indicates the different returns to scale. The values of outputs and inputs are positive or equal to zero, so u and v are positive vectors. The efficiency score that identifies the best practice frontier is found by solving this optimization problem. For all DEA analyses we used DEA Solver Pro (V13).
Outputs and inputs Prior work on HIV efficiency informed the parametrization of our multi-output VRS DEA model [19]. We defined outputs as the total number of clients-peryear in the following categories: 1) number of HTC clients; 2) number of HTC clients diagnosed as HIV-positive; 3) number of PMTCT clients tested; 4) number of PMTCT clients diagnosed as HIV-positive; and 5) number of PMTCT clients receiving antiretroviral prophylaxis. For the inputs, we considered labor (annual total cost of physicians, clinical officers, nurses, counselors, social workers, laboratory staff, administrative staff), the annualized capital cost of equipment and building, and utilities. We converted all cost data from local currencies to United States dollars (US$) using mid-year exchange rates for 2011 (Kenya: 88.81 Kenyan shillings and Zambia: 4860.7 Zambian kwacha) and 2012 (Rwanda: 614.3 Rwandan francs and South Africa: 8.21 South African rand), and then inflated to 2014 prices. All costs were adjusted for purchasing power parity (PPP) for non-tradable services, primarily staff salaries. The descriptive statistics for the input and output variables used in the DEA analysis are presented in Supplementary Material Table S2.
Second stage of DEA model Since traditional DEA estimates suffer from finite sample bias, Simar and Wilson propose a parametric bootstrap method to account for the (truncated) data-generating process of efficiency, in which full efficiency (θ i = 1) is in principle possible but occurs with zero probability [51]. Technically, this method combines truncated regression analysis, simulation of the unknown error correlation, and calculation of bootstrapped standard errors. Using this approach, we effectively parametrized different technologies for each country and facility type post-hoc. The data generating process of efficiency is modeled as: where b θ i is the bias-corrected efficiency score for facility i, z i is a matrix of contextual variables, and ε i is an idiosyncratic error term.
Contextual variables As contextual variables we included five variables that exogenously determine differences in production technologies and other (unobserved) constraints that affect the efficiency of outpatient HIV services: facility size (total number of outpatient clients per year), supervisions by national-level Ministry of Health staff, type of facility (hospital versus non-hospital), type of provider (public versus private), and country. The inclusion of these variables was informed by previous work in which exogenously determined contextual variables influenced different efficiency levels and thus determined different production possibility frontiers [52]. Health facility size is a proxy for facility catchment area, supervision visits by national-level staff represents independent national oversight, type of facility indicates the difference between multi-product facilities and their counterparts, type of provider signals the source of financing, while country reflects a series of fixed characteristics that are part of the health system organization of each country. For the second-stage DEA analysis, we used the Stata command simarwilson.
Supplementary Material Table S3 shows the results of the second-stage DEA analysis using a regression model with bootstrapping in which the raw efficiency score is the dependent variable and the contextual variables are on the left-hand side of the equation.

Estimation of quality
Process quality This study used Donabedian's structureprocess-outcome framework for evaluating quality of care where structure signifies the characteristics of the facilities in which care takes place, outcome refers to the impact of care on the individuals receiving care, and process represents the activities undertaken by providers to deliver and by patients to receive care [53]. The focus of our study was process quality measured by the technical knowledge of health providers on the appropriate care strategies for patients who access HTC and PMTCT services. Process quality captures the extent to which services provided include the elements or processes involved in the production of services when adopting best practices [54]. We developed a process quality score based on responses to vignette instruments used to assess the extent to which providers reported following existing national HTC and PMTCT guidelines [55,56]. Providers who had contact with HTC and PMTCT clients were randomly selected to participate in vignette interviews. The medical vignettes presented respondents with hypothetical scenarios describing typical HTC and PMTCT clients. Respondents were then asked a series of questions on the processes they would follow with the clients in these hypothetical scenarios. This method allowed us to assess the discrepancies between recommended practice according to national guidelines and the procedures that providers said they would follow, measuring provider competence. The HTC and PMTCT vignettes have been described in more detail elsewhere [47].
Quality score From the medical vignettes interviews, we constructed 107 indicators of compliance with HTC and PMTCT guidelines. At the facility level, we computed the percentage of providers at health facility i who reported complying with each of the indicators of recommended practice. We used principal component analysis (PCA) to construct the process quality score for each country separately [57]. We rotated the covariance matrix in the varimax sense, retained the first principal component, and rescaled it between 0 and 100. Reliability analysis was conducted, and the estimated Cronbach's α for each country sample varied between 0.90 (Kenya) and 0.97 (Zambia). Supplementary Material Fig. 1 provides additional information of the PCA analysis. We computed the quality score using the Stata command pca.
We opted to use a data dimensionality reduction technique in the absence of a clear data generating process (DGP) for quality, and because our goal was not to explain the correlation among survey items (over a hundred). PCA analyzes all the variance of the set of variables (survey items). We therefore adopted a simple approach given the high number of "factors" in our data and our goal, which was to summarize the data into a single scalar that maximized the variation in the quality score between clinics.

Benchmarking the relationship between efficiency and quality
To describe the relationship between health facility efficiency and quality, we first plotted the second-stage DEA efficiency scores against the process quality scores, to reveal the underlying heterogeneity of the two dimensions. We derived four efficiency-quality quadrants using the medians of the efficiency and process quality scores to determine thresholds [22]. The four quadrants reflect the possible configurations of efficiency and process quality score combinations in our sample of health facilities: above-median efficiency/above-median quality; above-median efficiency/below-median quality; below-median efficiency/above-median quality; and belowmedian efficiency/below-median quality.
Using a positive deviance approach to study the characteristics of the positive deviants or best-performing facilities [58], we explored the organizational correlates-or management practices-of the best-performing group, defined as those clinics at the intersection of those facilities in the upper half of efficiency (θ) and those in the upper half of quality (Q): W e u s e d a l i n e a r p r o b a b i l i t y m o d e l w i t h heteroskedasticity-consistent standard errors to explore these associations. We sought to identify the management characteristics associated with an increase in the probability of a health facility being classified in the quadrants described above.
Where X i includes binary and count variables that characterize the management practices described below.

Managerial determinants of efficiency-quality
Simultaneous equations estimation To explore the relationships between management practices and efficiency and quality independently and assess whether some factors might generate trade-offs between efficiency and quality, we estimated regression equations for health facility efficiency and quality simultaneously using seemingly unrelated regression (SUR) [59]. The SUR system of equations is based on the following assumptions: the existence of correlation of error terms across regression equations, and/or a relationship between the parameters included in the different equations of the equation system [59]. We assumed that both observable and unobservable factors simultaneously influence efficiency and quality with observed management practices concurrently influencing both health facility efficiency and quality.
The estimating equations were specified as follows: where b θ i is the bias-corrected efficiency score for facility i, and b Q i is the quality score. Since the strength of the association between management practices and performance is unknown a priori, we adopted an agnostic regression approach and implemented stepwise techniques for model selection. Therefore, X 1i aggregates the selected management practices for the efficiency equation, and X 2i for the quality equation. We set a threshold significance level of 0.2 and used only those covariates that showed a variance inflation factor (VIF) smaller than 2 to avoid inference problems due to multicollinearity. The β coefficients represent the partial correlation between a given management practice and the associated outcome, and ε 1i , ε 2i are idiosyncratic error terms. We conducted the Breusch-Pagan test for the null hypothesis that the covariance matrix of residuals is diagonal, i.e. Cov(e 1i , e 2i ) = 0.

Management practice variables
We derived the management variables we examined from responses to questions on facility-level management practices. These questions were based on the World Management Survey (WMS) developed by Bloom and Van Reenen for manufacturing firms [60] and subsequently applied to health facilities [24][25][26][27][28][29]. We adapted the questions to capture relevant information for health facilities delivering HIV services in low-resource settings. Managers responsible for HTC and PMTCT services were interviewed in person on management practices in five domains: operations management, performance monitoring, target setting, people management, and community engagement. We coded the management variables so that higher values indicated more or better management. The practices included in this analysis were: total number of supervisions made to HIV units in a year; whether funding received by facilities was based on the number of clients served; whether funding received by facilities was based on the quantity of inputs used; whether facilities or employees received payments or rewards for good performance in providing HIV services; whether well performing staff were rotated; whether staff received training as a result of good performance; participation of local community on facility governing board; existence of community advisory councils providing facility oversight; and frequency of managerial meetings in a year during which management and administrative issues related to HIV service delivery were discussed. Supplementary Material Tables S4 and S5 provide additional details on the management practices used in the analysis. Figure 2 shows the histogram of bias-corrected technical efficiency sc ores obtained from our DEA analysis (Supplementary Material Table S6 presents both the raw efficiency scores and the bias-corrected efficiency scores by country). These results show that average efficiency in the sample was 46%. The results also indicate considerable variation in the efficiency scores across the five countries. Facilities in South Africa had the highest average efficiency score with a mean of 73%. Rwandan facilities had the second highest score with a mean of 67%. Kenya, Nigeria and Zambia followed with average scores of 47%, 37% and 28%, respectively. The distribution of process quality scores by country are shown in Fig. 3. The average process quality score in the sample was 34% with over 50% of health facilities having a process quality score of 26% or less. The distribution of average process quality scores in the sample was skewed, with only a few health facilities having high process quality scores. Average process quality ranged from 22% in Nigeria to 51% in Kenya. Kenya had the largest proportion of health facilities with process quality scores above 50%. Figure 4 maps the bias-corrected technical efficiency scores and the process quality score for each health facility in the sample. The median values of the efficiency and process quality scores determined the distribution of facilities across the four quadrants. South African facilities were mapped either in the above-median efficiency/above-median quality quadrant (66.7%) or in the above-median efficiency/belowmedian quality quadrant (33.3%). Most Rwandan facilities Fig. 2 Distribution of bias-corrected technical efficiency scores by country Fig. 3 Distribution of process quality scores by country were mapped either in the above-median efficiency/abovemedian quality quadrant (75%) or in the above-median efficiency/below-median quality quadrant (17.3%). Kenyan facilities were distributed across the four quadrants with most facilities mapped either in the above-median efficiency/abovemedian quality quadrant (58.7%) or the below-median efficiency/above-median quality quadrant (28.2%). Facilities in Nigeria were also distributed throughout the four quadrants with most facilities mapped in the above-median efficiency/ below-median quality quadrant (19.1%), the below-median efficiency/above-median quality quadrant (20%), or in the below-median efficiency/below-median quality quadrant (51.1%). Zambian facilities were mapped mostly in the below-median efficiency/above-median quality quadrant (56.1%) or in the below-median efficiency/below-median quality quadrant (31.5%). When considering the sample as a whole, facilities were distributed across the four quadrants with a third of facilities mapped in the above-median efficiency/above-median quality quadrant, a fourth of facilities mapped in the below-median efficiency/below-median quality quadrant and the remaining facilities mapped in the above-median efficiency/below-median quality and belowmedian efficiency/above-median quality quadrants. More details are shown in Supplementary Material Table S7.
When comparing the best-performing facilities (n = 114) to the rest of the sample (n = 224), we found that 71% of the best-performing facilities were private, compared to 45% of facilities in the rest of the sample. We also found that bestperforming facilities were significantly larger in terms of size of HTC unit (4836 vs 3028 clients per year) and total number of outpatient clients (6161 vs 3603 clients per year). Table 1 presents results from a linear probability regression model examining the management practices associated with the likelihood of being in the best-performing group of facilities (above-median efficiency/above-median quality). We report the marginal changes in the probability of belonging to the above-median efficiency/above-median quality quadrant compared to being in the other three quadrants (below-median efficiency/below-median quality, below-median efficiency/ above-median quality, and below-median quality/abovemedian efficiency quadrants) given a discrete change in each variable and holding the rest of the covariates at their mean values. Facilities with above-median levels of efficiency and quality were 1.6 percentage points (p.p.) more likely to have an additional HIV unit supervision. The best-performing facilities were 8 p.p. more likely to have an additional managerial meeting. The best-performing facilities were less likely to use rotation of well-performing health staff as an incentive (19.1 p.p.). Compared to poor-performing facilities, facilities with above-median levels of efficiency and above-median levels of quality were also more likely to receive rewards for good performance providing HIV services (14 p.p.) and to receive training for good performance (11 p.p.). Supplementary Material Table S8 shows results for sensitivity Fig. 4 Bias-corrected technical efficiency scores and process quality scores analysis conducted around the cut-off points used to define the best-performing group. Table 2 shows results of simultaneous equations to identify the management factors associated with efficiency and quality simultaneously. To test the existence of a potential relationship between the two equations justifying the use of seemingly unrelated regression, we tested the correlation between the residuals of the two equations. We found that although the correlation was weak (0.0719), the Breusch-Pagan test for diagonality of the variance-covariance matrix of the disturbances of the two equations was statistically insignificant (p value 0.2161). We therefore failed to reject the null hypothesis of zero correlation between the two equations. The first results column shows the results for efficiency and the second shows the results for process quality. The number of supervision visits to HIV units was positively associated with both efficiency and quality. An additional supervision visit was associated with a 1.5 p.p. increase in the bias-corrected technical efficiency score and with a 0.69 p.p. increase in the process quality score. Funding based on the number of clients served was associated with a 8.2 p.p. decrease in the quality score. Funding linked to quantity of inputs used in health facilities was positively associated with the quality score (7.1 p.p.).
Rewards for good performance in the provision of HIV services provided to the facility or staff was positively associated with the efficiency score (16.7 p.p.). Similarly, we found a positive association between the number of meetings and the efficiency score (3.5 p.p. per additional meeting). The existence of a community advisory council was negatively associated with the efficiency score (6.6 p.p.). In contrast, the participation of local community in the facility governing board increased quality by 6.4 p.p. compared to facilities with no local community representation on the governing board.

Discussion
In this analysis of technical efficiency and process quality in 338 health facilities providing outpatient HIV services in Kenya, Nigeria, Rwanda, South Africa, and Zambia, we found overall levels of both efficiency and quality to be rather low, which is troubling. The mean technical efficiency score in the sample was 46% and the mean process quality score was 34%. We found substantial variation in both technical efficiency and process quality across countries. Whereas we found facilities in South Africa and Rwanda to be more efficient, Table 1 Linear probability regression results examining the management practices associated with the likelihood of being in the best-performing group of facilities (above-median efficiency/above-median quality) y=Above-median efficiency/above-median quality Model estimated with 338 observations. dy/dx denotes the marginal effect of the covariate on the outcome. Asterisks denote significance levels: *** p < 0.01, ** p < 0.05, * p < 0.10. Robust 95% confidence intervals in brackets. Above-median efficiency/above-median quality defined as the joint probability that facility i ranks above the 50th percentile of Efficiency and Quality facilities in Nigeria and Zambia had relatively low levels of technical efficiency. Process quality scores were highest in Kenya and lowest in Nigeria. Our findings on efficiency were consistent with most other studies on the technical efficiency of HIV services in similar settings [21][22][23], though one study of facilities providing HIV treatment in Zambia found mean efficiency to be significantly higher [19]. While we are unaware of previous work using vignettes to assess HIV service process quality, the low levels of quality we found were comparable to those of studies that have used this technique to examine quality of other health services in low-resource settings [56,[61][62][63][64][65][66].
Mapping technical efficiency scores against process quality scores, we found that one third of facilities were mapped in the best-performing group (above-median efficiency/abovemedian quality). One fourth of facilities were mapped in the below-median efficiency/below-median quality quadrant which is concerning. Just over 40% of facilities were mapped in the above-median efficiency/below-median quality and below-median efficiency/above-median quality quadrants. A recent study on the efficiency of health facilities integrating HIV and sexual reproductive health services in Kenya and Swaziland applied a similar mapping approach but used the 75th percentile instead of the median as the threshold to divide the four quadrants [22]. They found that no facilities were mapped in the high efficiency and high quality quadrant, two-thirds of facilities mapped in the low efficiency and low quality quadrant, and one-third of the facilities were distributed in the other two quadrants.
Delving into the characteristics of positive deviant facilities with above-median technical efficiency scores and abovemedian process quality scores, we found that most bestperforming facilities were Rwandan, South African, and Kenyan. We found that best-performing facilities were more Model estimated with 296 observations (42 facilities did not consent to the medical vignettes survey, and therefore missing values on process quality). 95% confidence intervals in brackets. Asterisks denote statistical significance: *** p < 0.01, ** p < 0.05, * p < 0.10 likely to be privately owned than poor-performing facilities. Best-performing facilities were also significantly larger than poor-performing facilities in terms of size of HTC unit and total number of outpatient clients. This finding on patient volume and what appears to be associated provider specialization is consistent with study findings from other settings and health areas that observed that patient volume and provider specialization were associated with better quality in terms of process quality and health outcomes [67][68][69].
Our assessment of the management characteristics of positive deviant facilities suggests that facility-level management characteristics may explain some of the variation in the performance of HIV prevention service delivery. Positive deviants or best-performing facilities were more likely than poorperforming facilities to implement certain management practices and less likely to implement others. Assessing the relationship between facility management characteristics and best performance in terms of both technical efficiency and process quality together, we found that compared to poor-performing facilities, best-performing facilities were more likely to have more HIV unit supervisions and more managerial meetings. Best-performing facilities were also more likely to have rewards for good performance providing HIV services and to have training for good performance. Conversely, bestperforming facilities were less likely than poor-performing facilities to use the rotation of well-performing health staff as an incentive.
Our findings also suggest that to improve HIV service delivery, understanding the management practices of positive deviants is important but not sufficient. Considering the relationship between management practices and technical efficiency and process quality independently, we found that the patterns of associations were not necessarily the same for the two dimensions of performance. One of the management characteristics was positively associated with both the efficiency score and the quality score: number of supervision visits to HIV units. Some management characteristics had significant associations with the efficiency score, but not with the quality score or vice versa. Rewards for good performance and number of meetings were positively associated with efficiency whereas the existence of a community advisory council was negatively associated with the efficiency score. The associations between these management characteristics and quality were not statistically significant. Conversely, though the associations between these management factors and efficiency were not statistically significant, funding based on the number of clients served was negatively associated with process quality and funding linked to the quantity of inputs used was positively associated with quality. We did not find any evidence of a possible trade-off between efficiency and qualitywe did not find management practices that were positively associated with the technical efficiency score and negatively associated with the process quality score or vice versa.
One of the strengths of this study is that we benchmarked at the same time both the technical efficiency and process quality of facilities delivering HIV prevention services. Our study contributes to the literature on the joint analysis of efficiency and quality conceiving these two concepts as two separate dimensions [36]. Our study also expands on previous work mapping facilities delivering HIV services [22] since it not only describes the relationship between efficiency and quality but also describes the relationship between management factors and efficiency and quality. Another strength of the study is our operationalization of the management variables. Studies exploring the management practices of health facilities often construct additive management scores [23,[26][27][28][29] making it difficult to identify the specific actions implementers can and should take to impact productivity. We studied discrete management variables that providers can recognize and act on.
Several limitations should be kept in mind when considering our findings. ORPHEA and ORPTHEN study data covered different years-2011 for Kenya, South Africa, and Zambia, 2012 for Rwanda, and 2013 for Nigeria. Because we retrospectively collected information on inputs, prices, and outputs, data on process quality of services and management characteristics of facilities correspond to slightly different time periods, although there was never more than a 12month difference. We used routine monitoring data to capture information on outputs, and the level of detail, quality, and completeness of these data varied across facilities. Our quality measure derived from medical vignettes assessing the technical knowledge of health providers is focused on process quality, we did not have measures of structural quality or patient outcomes. Our measures of contextual variables were limited, and it is possible that we may have missed relevant factors related to the organization of the health systems in each country. Contextual characteristics such as national governance, financing mechanisms, budget allocations, donor relations and others may be important in explaining variations in efficiency and quality [70,71]. Although our questionnaires on management practices were based on previous studies, we may have omitted some practices relevant to each of the health systems in the five countries. In terms of our analyses, because the DEA approach orders health facilities according to the efficiency frontier generated from the sample analyzed, results might be different if comparisons were to be performed within countries given the change in the point of reference. This is likely to underpin the differences between our results on the efficiency of Zambian facilities and results in the previous study mentioned above, which found a mean efficiency score of 83% among Zambian facilities providing HIV treatment [19]. Our study findings on the levels of efficiency and quality should not be generalized to other settings. However, our findings provide useful insights on the relationships between management factors and efficiency and quality.

Conclusions
We found important differences in the technical efficiency of health facilities providing outpatient HIV services in Kenya, Nigeria, Rwanda, South Africa, and Zambia; all countries could make substantial efficiency gains. Of particular concern, process quality scores were low across the five countries; all countries could make considerable quality improvements. Only one third of facilities were mapped in the bestperforming group. We identified several management practices associated with best-performance in terms of both efficiency and quality. We also found that when considering efficiency and quality independently, the patterns of associations between management practices and the two dimensions of performance were not necessarily the same. We found one management characteristic associated with best-performance in terms of efficiency and quality which was also positively associated with efficiency and quality independently: number of supervision visits to HIV units. As implementers around the world work to improve HIV service delivery, it is critical that they understand the relationship between management characteristics and efficiency and quality so that they can identify the practices that they can influence to improve performance. Additional research exploring the quality and efficiency of HIV service delivery in low-resource settings is urgently needed, as is further investigation of the causal relationship between management practices and efficiency and quality.