The Technological and Agency Eects of IT: Randomized Evidence from Credit Committees

Daniel Paravisini LSE Antoinette Schoar MIT

April 12, 2012

Abstract Information technology may aect productivity by reducing agents information processing costs (technology), and by making agents actions easier to evaluate by the principal (agency). We distinguish these mechanisms empirically in the context of the randomized adoption of an IT innovation credit scoring in a bank that lends primarily to small businesses. We nd that the output of committees that evaluate credit applications increases for applications that contain a score. The same is true when committees information set is held constant: output increases in anticipation of the score being available in the future. This eect is uniquely consistent with the agency mechanism, and explains over two thirds of the total output increase. Additional evidence suggests that IT shifts problem solving from upper to lower hierarchical levels of the bank, and has no discernible eect on the quality of projects chosen.

We are grateful to Heski Bar-Isaac, Greg Fischer, Luis Garicano, Andrea Prat and Lucia Sanchez for helful comments and discussions. We wish to thank BancaMia for their support, and Isabela Echeverry and Santiago Reyes for excellent research assistance. We thank IPA for the funding that made this study possible. Please send correspondence to Daniel Paravisini (d.paravisini@lse.ac.uk) and Antoinette Schoar (aschoar@mit.edu).

Introduction

The diusion of information technologies (IT) since the advent of the computer has been positively associated with increases in productivity in organizations.1 Ascertaining empirically the channel through which IT aects performance, however, has proved elusive. The main diculty lies in the dual role played by most IT innovations. The adoption of IT services raises productivity directly by reducing information processing and communication costs as well as allowing for greater standardization of decision rules. At the same time IT raises productivity indirectly through improved monitoring which reduces information asymmetries. Distinguishing between the technological and agency channels is a key input for understanding the implications of these innovations on the internal organization of rms and their boundaries.2 Existing empirical work had to rely on ex ante classications of whether the technology adoption channel or the agency channel will be dominant. For example in the seminal study on the trucking industry, Baker and Hubbard (2004) use the introduction of an on-board computer system to test the impact of better monitoring on incentives and performance.3 But new technologies usually are a bundle of features that also interact with other dimensions of the organization such as job descriptions, compensation structures or even the allocation of authority (see Milgrom and Roberts (1990)). The present study explores empirically how the introduction of a new IT based credit scoring model aects worker productivity at a bank in Colombia that lends to small enterprises. We worked with the bank to randomize the roll out of the scoring model across the dierent bank branches. Prior to the adoption of the IT system, credit committees
For early surveys, see Brynjolfsson and Yang (1996) and Brynjolfsson and Hitt (2000). See, for example, Aghion and Tirole (1997) Antras, Garicano and Rossi-Hansberg (2006), and Alonso, Dessein and Matouschek (2008). 3 Hubbard (2000) identies two classes of on-board computers and argues that one helps the principal provide better incentives while the other only improves coordination. Bloom, Garicano, Sadun and Van Reenen (2011) also classify technologies into communication enhancing and information enhancing, although not for the purpose of separating the technology and agency channels.
2 1

at each branch perform the rst evaluation of a loan application and try to determine whether the application should be approved, and conditional on approving it what the terms of the loan should be. By varying the timing of the roll out along the committee decision process, we are able to cleanly dierentiate the individual impact of the scoring model through the technology and agency channels. For the purpose of our experiment we randomly select a fraction of credit committees (branches of the bank) to receive independent credit scores concerning the estimated default probability of a new applicant. Bank headquarters developed this scoring model based on historic data of the bank data, e.g. using borrower characteristics such as age, gender, leverage, assets etc. The score ranges from one to zero and provides an estimated default probability based on past behavior of similar applicants. The characteristics used to calculate the score are a subset of those contained in the application, and are thus fully observable by the committee even before the scores are provided. There are two ways in which the credit score might improve the decision making of the credit committee: On the one hand it can provide a dierent weighting of how the applicants characteristics factor into the default probability if loan ocers use a model that is less well calibrated than the central one (technology channel). On the other hand providing the score gives an ex ante indication to the credit committee and the manager about which cases are easier or more dicult to analyze and therefore should not be pushed up to the manager (agency channel). We nd that committees are more likely to make a decision regarding an application when a credit score is available. The change is economically signicant: scores reduce by 4 percentage points the fraction of cases where no decision is reached (from 11% in the control group). We nd no statically signicant eect of relying on the scores for the average loan size that is approved nor the performance of the loans in the rst six months after approval. These results suggest that scores improve the willingness of the committee

to make decisions and, as a result, scores improve the productivity of the rm. We also nd that the average time committees spend evaluating applications increases by 9.3%, which might be a sign that scores increase credit ocer eort. When a committee does not reach a decision, it passes the case up to the manager, or it defers the decision on the case until more information is collected. In the control group of applications, fewer than half of the non-decisions (43%) involve passing the case up to the manager. Yet, when we explore how scores aect the two dierent non-decision margins we nd that close to 70% of the overall reduction in non-decisions is explained by passing fewer cases up to management. The results imply that scores improve committee output mostly by increasing the frequency with which committees solve problems relative to upper management. As mentioned before this productivity improvement can be driven by a technological explanation: more information allows committees to solve more complicated problems, either by reducing complexity in evaluating all the dierent dimensions of the credit applications or by having a more accurate forecast model.4 Alternatively scores can reduce the agency problems between the manager and the committee. If the committee is compensated on the quality and size of its loan portfolio, committees may nd it more comfortable to send an application up to the manager whenever there was a doubt about a case. However, if the score provides a clear indication to the manager and the committee whether a le was dicult to evaluate, sending too many cases up for evaluation can hurt the reputation and career prospects of the committee. To dierentiate these two explanations we add a separate treatment arm where we introduce a credit score for the manager but hold constant the information set for the credit committee. In a randomly selected sample of treatment applications, committees

The importance of the technology eect will depend on the weight that loan ocers put on the local information obtained from their own prior lending experience relative to the weight assigned to the population information contained in the credit score.

are asked to make an interim evaluation of the application before observing the value of the score. Interim committee output increases by 3.5 percentage points relative to the control group, despite the fact that both have the same information at the time of making a decision. Although once the score value is disclosed committee output increases even further, 74% of the output increase in this treatment group occurs before observing the score. These estimates imply that the adoption of the scoring model has a rst order eect on output through the incentive mechanism. When we compare the total output of treated branches in experimental weeks relative to non-experimental weeks (or relative to propensity score-matched branches during experimental weeks) we do not nd any evidence that the experiment aected the number, amount, or performance of approved loans. Thus, the potential eciency gains from having a credit score in this setting derive exclusively from the transfer of decision making from management to workers in lower levels in the hierarchy. However, one could conjecture that over time (and once the organization has been able to observe the change in processing speed) the time savings at the management level could lead to growth in other parts of the rm. The results taken together imply that the introduction of scores improves committee decision-making through both the technological and agency mechanisms. The information mechanism is consistent with the theories of the optimal organization of knowledge in production, as in Garicano (2000). The agency mechanism is consistent with theories of optimal delegation, surveyed in Mookherjee (2006). In our set up the agency mechanism explains the bulk of the eect of scores on output, highlighting the importance of improved monitoring via IT solutions to encourage decision making lower down in the hierarchy. It can ultimately facilitate the decentralization of work and organizations. The specic application we focus on, credit scoring, is of particular importance given the large literature in nance and banking on relationship lending and the role of loan

ocers in the lending process. This literature has largely focused on the trade o between using soft less standardized and dicult to communicate versus hard information (see for example, Rajan (1992) and Petersen and Rajan (1995)). Stein (2002) specically conjectures that loan ocers face weaker incentives in soft information regimes, but this link has not received much attention in the empirical literature.5 Our paper provides the rst direct evidence to support this conjecture and characterizes an economic mechanism behind it: the adoption of a standardizing technology in the context of a soft information lending process can mitigate agency problems inside the bank. The rest of the paper proceeds as follows. We provide in Section 2 a description of the tasks and incentives of the credit committees, the characteristics of the credit scoring system, and the specics of the experimental design. Section 3 presents the results of introducing the score on committee output and productivity, Section 4 disentangles the technological and agency eects, and Section 5 examines the potential aggregate eects of the introduction of scores. Section 6 concludes.

Setting and Study Design

The study was implemented with BancaMia, a for prot bank in Colombia that focuses on micro and small enterprises. In October 2010, the month prior to the roll out of the study, the bank issued 20,119 loans totalling $US 25,9 million through its 143 branches. Historically the bank relied on a relationship lending model were loan ocers go into the eld and collect detailed information from the potential applicants. This information collection mechanism is necessary since small enterprises in Colombia do not have any audited nancial statements or other secondary data that a bank could use for credit assessment. The bank relies on a sophisticated information system that allows the data
A number of studies have analyzed the implications of soft information for bank function and organizational design. See, for example, Berger, Miller, Petersen, Rajan and Stein (2005), Liberti and Mian (2009), and Hertzberg, Liberti and Paravisini (2010)).
5

collected by loan ocers in the eld to be automatically uploaded via PDA devices to a data storage facility in the banks headquarters. All the information related to an application, including both new information collected by the loan ocer, past information about the borrower in BancaMia if the borrower has a credit history in the bank at the time of the application, and any external secondary source information (e.g. credit score of the borrower from a private credit rating agency) is put together by the system in a single application le.

2.1

Credit Assessment Process

An application le is reviewed at the branch level by a credit committee, composed of the loan ocer that collected the information in the eld, the head of the branch, and one or two additional credit specialists, who are typically other loan ocers associated with the branch. The credit assessment is based on the information that loan ocers collected from the borrower in the eld. General information about the industry and a macroeconomic outlook are taken into account as well. A committee can take four possible actions regarding a loan application. First, it can reject the application. Second, it can approve it, in which case the terms of the loan must be decided. The committee can adjust the terms of the loan at will in order to improve the acceptance rate. For example, the committee may decide to approve a $500 loan when the requested loan amount in the application is $1,000. When a committee takes any of these two actions we consider that the committee has reached a decision regarding an application. When a committee cannot reach a decision, it has two additional actions at its disposal. The rst is to send the application le to a regional manager, whom evaluates the application and reaches a decision.6 The second is to postpone the decision and send
loans above 8 million pesos go directly to the regional manager for approval. Randomization insures that this mechanical relationship between loan size and approval level is orthogonal to the scores. Also, adding requested loan amount as a control in the specications does not change the estimated eect of scores.
6

the ocer out to collect additional information about the borrower. During informal interviews, bank managers expressed that such non-decisions by committees represent a substantial cost to the bank in terms of the opportunity cost of time of managers and ocers. Nevertheless, committee member compensation is a function of the loans issued at the branch and their repayment performance, regardless of whether the decision was made by the committee or by the upper level manager. There are two potential reasons for this compensation scheme. First, penalizing committees for asking questions to upper level managers may lead to too many bad decisions at the committee level. And second, committee members must be compensated for monitoring the performance of the loans after origination, even when the decision to approve is made at an upper level of the hierarchy.

2.2

Credit Scores

In 2010, BancaMia developed a credit risk model to establish the statistical relationship between the banks historic quantitative and qualitative information in loan applications and the repayment performance of issued loans. For the quantitative part of the score, loan ocers are asked to collect information such as: gender, age, location, number of years in business, frequency of late payments in past three years (if the loan applicant already has a credit history with BancaMia), level of overall indebtedness, house expenditures as a percentage of total income, among other variables. For the qualitative part, loan ocers are asked to collect information based on more subjective variables such as: overall knowledge of business, general sense of the level of organization, quality of information provided, quality of business location, quality of crops being cultivated (agricultural loans only), stability and diversity of income, among other variables. The stated objective of introducing the credit scoring system was to improve identication of the best and worst clients, decentralize the loan approval process, and reduce 8

the labor costs involved in loan application evaluation. The idea was to include the score as an additional piece of information in the application le, to be used by the committees at the time of evaluation. The score is a proxy for the expected default probability of the loan. Figure 1 plots the non-parametric relationship between scores, approved loan amounts, and default probabilities in the population of loans issued during October 2010. A loan is considered to be in default if interest or principal payments are more than 60 days overdue, and we measure default at six months after the loan is issued. There is a strong positive association between credit scores and requested loan amounts, and a negative one between scores and default probabilities.

2.3

Experimental Design

Before the full roll-out of the scores, we implemented a pilot program with a randomized control trial design in eight of their branches to evaluate the eects. The branches were chosen to be representative of the average urban branch of the bank.7 The pilot consisted of randomizing, at the application level, the introduction of scores in the application le at the time of the committee meeting. At the initiation of the discussion of an application in a committee, our research assistants used the last digit of the time in the research assistants cellular phone to allocate a le to the control group or two treatment groups. The information of which group the le belonged to was available to committee members during the deliberations. In the control group, the committee evaluates the application without the score. In the rst treatment group (T 1), the score was added to the application before the beginning of the evaluation. In the second treatment group (T 2), the committee rst evaluated the application without the score and chose an interim action. The treatment status was
BancaMia also operates rural branches, with a larger fraction of loans associated with agricultural micro-enterprises.
7

randomized before the committees evaluation process and the committee could ascertain this status while deliberating the interim decision.8 Thus, the information set under which committees made interim decisions in treatment T 2 is the same as the information set of the control group, except for the fact that the committee had information about the future availability of the score. After recording the interim outcome, the research assistants disclosed the score and the committee revised its choice if necessary. We report in Appendix Table A.1 the number of control, treatment T 1 and treatment T 2 loans per branch in the study sample.9 We present statistics grouping all the treatment applications together, and delay until Section 4 the discussion regarding the distinction between treatments T 1 and T 2. Table 1, Panel A, shows descriptive statistics of pre-determined application characteristics for control and treatment applications. The average requested amount and the score of loan applications in the treatment and control groups are not statistically dierent. Figure 2 plots the cumulative distribution of scores and requested loan amounts for the treatment and control applications. The score and amount distributions are indistinguishable between the treatment and control groups in a two-sample Kolmogorov-Smirnov test for equality of distributions, with corrected p-values of 0.81 and 0.94 respectively. These ndings corroborate the internal validity of the experimental design. Table 1, Panel B, presents the statistics for committee choices and loan outcomes. Committees reach decisions (accept or reject) in 89% of the applications in the control group. Interestingly, in only 0.3% of decisions in the control group the committee rejects a loan. Committees were more likely to reach a decision, and conditionally on making a decision, more likely to reject a loan, in treatment applications than in control ones.
The research assistant present during the committee evaluations had the treatment status and gave it to committees upon request. 9 In a short training workshop before the roll-out of the scores, branch directors and loan ocers at the eight pilot bank branches were provided with a detailed description of the treatments, a general explanation of the credit risk model and the scores, and a discussion about the objectives of researching the accuracy of the credit risk model in predicting client performance.
8

10

Neither the average time to make a decision, or the loan characteristics conditional on approval are statistically dierent in the treatment and control applications. We delay until Section 3.2 the discussion about committee actions in the case of non-decisions.

3
3.1

Results
Committee Output and Performance

We use the following reduced form equation to estimate the eect of credit scores on the output of credit evaluation committees:

Yi = Scorei + Xi + i ,

(1)

where Yi is a nal outcome regarding loan application i (we discuss the eect of treatment T 2 on interim decisions in Section 4). For example, the outcome Decidei is a dummy equal to one if the committee reaches a decision on whether to approve or reject a loan, and zero otherwise. Some outcomes, such as loan amount and loan default, can only be measured if the committee approves the loan. This introduces an additional complication in interpreting the results that we discuss below. The variable Scorei is a dummy equal to one if the loan application is in the treatment groups, i.e., if the score was available to the committee at the time of making a decision, and Xi is a vector of application characteristics that includes the credit score of the applicant, the requested loan amount, a time trend (in days), and a full set of branch dummies. The results are presented in Table 2. We present the results of specications that include all controls. We only include controls that are plausibly predetermined, and for the purpose of addressing omitted variable issues. Appendix Table A.2 presents the results without controls and nd little qualitative or quantitative dierence in the estimated 11

eects. The estimated in equation (1) represents the eect of introducing a score on a committee output. The estimated eect on Decidei is 0.0395 (Table 2, Column 1) and signicantly dierent from zero. It implies that introducing a score in an application reduces by close to 4 percentage points the probability a committee does not reach a decision. This decline is economically signicant relative to the proportion of cases where no decision is made in the control group (11%). The estimated eect on average decision time per credit application, measured in minutes, is 0.41 (Table 2, Column 2), a 9.3% increase relative to the average time sent per application in the control group. The eect is statistically signicant at the 5% level. If one assumes that the entire increase in decision time is due to those applications in which the the treatment led the committee to reach a decision when it would not have done so otherwise, the estimate implies that the marginal cases require an additional 10.4 minutes to decide (0.41/0.0395), or over twice the average time required to decided the inframarginal ones (the average decision time in the control group is 4.7 minutes). Although this increase is potentially an overestimation of the additional decision time for marginal cases, the result suggests that the introduction of scores increases the amount of time devoted by committees to evaluate and decide marginal cases.10 The estimated eects on approved loan amounts and the probability of default in six months are positive but not statistically dierent from zero (Table 2, Columns 3 and 4). These estimates are obtained only from approved loans and thus represent biased estimates of the average treatment eect. The direction of the bias depends on the average size and quality of the marginal loans, those that are approved by committees due to the treatment. One can infer from the sign of the estimated coecients on requested amount
Treatment T 2 requires committees to make two decisions, one before receiving the score and one after. This design may increase mechanically the time spent per application. We do not nd, however, the eect of treatment on time spent per application to be signicantly dierent between the two treatments (see Appendix Table A.3).
10

12

and the score in Table 2 (Columns 1 and 2), that applications for larger loans and with larger credit risk scores are less likely to be decided on and take more time to decide. It is likely then that marginal loans are larger and riskier than inframarginal ones, which would imply that the estimates in columns 3 and 4 are biased upwards. This bias increases the likelihood of type II error. Thus, it is unlikely that the failure to reject the = 0 null is driven by estimation bias.

3.2

Information Collection and Problem Solving

Overall, the ndings above indicate that scores increase the decision making capacity of credit committees, increase the eort by committees in deciding marginal cases, and have no signicant eect loan size or default probability. Take together the results imply that the introduction of scores increases the productivity of committees. We cannot evaluate overall productivity gains, because the cost savings from more committee decision-making are dicult to quantify. We do know that the potential cost savings are due to: 1) a reduction in the time devoted to collect additional information from loan applicants, and 2) a reduction in the time spent by upper level managers in evaluating loan applications. In this subsection we explore to which degree each of the two margins is aected by the introduction of scores. We are interested in modeling how the introduction of a score aects the committee choice between making a decision, collecting additional information, or sending the application to a manager in a higher hierarchical level to make the decision. We model this choice with the following multinomial logistic specication: P (Di = m) = m Scorei + Xi m + mi , P (Di = 1)

ln

(2)

where Di represents the committee choice to decide, collect more information, or send the 13

decision to the manager. We use the committees choice to make a decision, Di = 1, as the reference category. All the right-hand side variables are as in equation (1). There is a predicted log odds equation for each choice relative to the reference one, e.g. there is a m for the choice to collect more information and one for the choice to send the application to the manager. A positive estimate for m implies that committees are more likely to take action m than to make a decision (accept or reject) in the treatment group relative to the control group. The results are presented in Table 3. The m estimate is negative for, both, the choice to collect more information and to send the decision to the manager, although it is only signicant for the latter.11 This implies that observing a score with a credit application reduces the probability that the committee will send the decision up to the manager relative to making a decision itself. To evaluate the economic signicance of the eect, we report on the bottom rows of Table 3 the implied marginal eect of treatment on the probability of each choice. Observing a score decreases the probability of sending the decision up to the manager by 2.3 percentage points, a substantial decline relative to the proportion of applications sent to the manager in control applications (4.8%). Comparing the marginal eects suggests that this eect accounts for close to 70% of the overall increase in decision making by committees. The economic signicance of this magnitude is highlighted by the fact that delegating to the manager accounts for 43% of non-decisions in the control sample. The results suggest that scores increase committee decision making ability by reducing the degree to which they rely on managers in upper levels of the hierarchy to solve problems. In contrast, the eect through reducing the need to gather additional borrower information is not statistically or economically signicant. There are two potential interpretations for this dierence. First, information provided through scores may be a
The coecients on the treatment regressors m are signicant at the 1% level in a joint test across the three choices)
11

14

substitute for the superior information or information processing capacity in upper levels of the hierarchy, but not for additional information that can be gathered at lower hierarchical levels. This is a natural statistical implication of scores, since scores are obtained estimating borrower performance models using population data, while lower hierarchical levels observe performance in small samples. Under this interpretation, scores increase decision making at lower levels of the hierarchy because they reduce the cost of problem solving at this level relative to upper management. A second potential interpretation comes from agency theory, since scores may reduce the cost of inducing eort from committees. From the compensation contracts perspective, there is no cost for committee members from sending decisions up to managers: committee members are remunerated for the loans originated in the branch irrespective of whether the nal decision to approve the loan is taken by the committee or the upper level manager. Despite the lack of contractual incentives, less than 5% of applications are sent to the manager in the control group. By revealed preference, there must be non-contractual costs associated to sending applications to the manager. These costs may arise if committees care about looking competent in dealing with applications. For example, the probability of being red may increase and the probability of a promotion may decrease if the committee send seems incapable of assessing the credit quality of easy-to-evaluate applications. Although the rst best outcome involves committees deciding on easy-to-evaluate applications and managers deciding on hard-to-evaluate ones, the agency problem arises because managers cannot assess without error the diculty of evaluating an application. As a result, committees save on their own eort costs by sending problems up to the manager too often. If a score makes problem diculty observable, then it may lead to less problems sent to the manager and more decision making at the committee level. In the next section we disentangle the information and the agency mechanisms by

15

analyzing separately the two treatments described in Section 2.

Information and Incentives

The results presented so far are obtained using the nal decision of the committee for all treatment applications, and thus can be interpreted as the eect of observing a score on committee output. In this section we turn our attention to evaluating the eect of treatment T 2 on interim decisions. In treatment T 2 the committee performs an evaluation of the application and reaches an interim conclusion before observing the score (e.g. with the same information set as the control applications). The committees had available the information about which treatment group the application belongs to during the deliberations of the interim choice, and thus, about whether the score would be ultimately available in the loan application or not. In principle, we can use this treatment to evaluate how ratings aect committee decision-making holding the information set of the committee constant. One caveat of this treatment is that committees may have weak incentives to perform a thorough interim evaluation, because committee members know that they can revise the decision after observing the score. Although study participants were explicitly asked to perform a thorough evaluation of the application regardless of the treatment status of the application, we interpret any observed eect on interim decisions bearing the potential weakened incentives in mind. We estimate the following variations of the OLS equation (1) and the multinomial logit model (2):

Yiint = T 1 ScoreT 1 + T 2 ScoreT 2 + Xi + i , i i

int P (Di = m) T T = m1 ScoreT 1 + m2 ScoreT 2 + Xi m + mi , i i int P (Di = 1)

(3)

ln

(4)

16

int where Yiint and Di represent, as before, the committee decisions, but with the interim

decision replacing the nal decision for treatment T 2. We estimate this specication again using the full sample of applications, including both treatments T 1 and T 2 so as to perform inference tests on the dierence between two eects. Therefore, we include indicators ScoreT 1 and ScoreT 2 which reect whether application i belongs to treatment i i T 1 and T 2. The coecient on the rst indicator has the same interpretation as before.
T The coecient m2 represents the pure incentive eect of the score, since by the time the

committee makes the interim decision it has not observed the score. We present in Table 4 the results. The eect of the score on the probability of making a decision estimated using the linear model is positive and signicantly dierent from zero at the 10% condence level in both treatments (Column 1). The magnitude of the eect of treatment T 2 in interim decisions is 0.035 and its dierence with the eect of treatment T 1 on nal decisions is not statistically signicant. Treatment T 2 does not have a statistically signicant eect on the interim approved loan amount (Column 2). The coecient estimates on the treatment dummies in the multinomial logit model are negative as before, and signicantly dierent from zero only for the decision to send the application to the manager, also as before (Column 3). The estimated marginal eects imply that the probability that committees send the application to the manager in interim decisions is 2.5 percentage points lower in treatment T 2 applications relative to control ones. The ndings imply that committees are less likely to send problems up to higher levels of the hierarchy when a score is available in the credit application, even holding the information the committee has about the borrower constant. This is consistent with the incentive interpretation that scores improves committee output because of the information they provide to managers. An alternative interpretation of the treatment eect on interim decisions is that, due to the weak incentives mentioned above, these interim decisions are fake, made half-

17

heartedly with the intention of substituting them with a real decision after the score is observed. We can explore this possibility by comparing how committees revise their interim decisions in treatment T 2 applications after observing the score. Table 5 presents in matrix form the transitions between interim and nal decisions for all the applications in treatment T 2. There is a large concentration of the observations on the diagonal indicating that observing the score does not have a rst order eect on the interim decisions made by committees. Committees change a decision in only 1.5% of the over ve hundred applications. This represents prima facie evidence that interim choices were informed. In every instance in which the committee changes its decision, the change is between accepting the loan and sending the application up to the manager, and vice versa. In seven out of the eight changes, the committee changed the decision from sending up to the manager to accepting the loan. Several conclusions can be drawn from the observed patterns in Table 5. First, there are very few reversals (one) of decisions to whether accept or reject the application. This suggests that interim decisions were not fake or made mostly with the purpose of appearing diligent for the purpose of the experiment and amending them later. Second, observing the score has an additional positive eect on committee output in treatment T 2. The implied eect of observing a score on the probability of sending the score to the manager is -1.14 percentage points ((12 6)/523), although it is marginally signicant (a two sample test of proportions rejects that the dierence is positive with a p-value of 0.07). This eect can be interpreted as a pure information eect, after the incentive eect of scores has been netted out. Third, in not a single instance did observing the score aect a committee decision between acceptance and rejection of an application in treatment T 2. This would suggest, that the information content of scores regarding this margin of committee decisions is, at most, secondary. Fourth and nal, the magnitude of the pure information eect on sending applications up to the manager appears to be smaller than

18

the incentive eect. We obtain a back of the envelope estimate of the relative magnitudes of the agency and information eects on committee decision making by comparing the eect on treatment T 2 on interim decisions from Table 5, with the eect on nal decisions for the same treatment (shown in Appendix Table A.3). The dierence implies that the incentive eect explains 74% (0.0353/0.0472) of the overall change in committee output.

External Validity

The study is designed to randomize the treatment at the committee level, once the information in an application is already collected. It is appropriate, thus, to treat application characteristics, such as the rating of the borrower or the requested loan amount, as predetermined and orthogonal to the treatment. This does not imply, however, that the experiment had no eect on the application characteristics. In anticipation of the expected treatment loan ocers may have, for example, changed their information gathering eort, manipulated the entry of data into the system to aect the score of an applicant, inuenced the borrower to change the requested loan amount in the application, or postponed certain types of applications to the committee until the pilot implementation in the branch ended. In any of these cases, the introduction of scores would have aected the application pool characteristics in a branch, and had overall eects on lending outcomes that are not captured by our experimental design. In principle, it is possible to test whether the experiment aected the applicant pool characteristics, for example, by comparing outcomes of the experimental branches during the weeks of experimentation relative to other weeks, or by comparing them with propensity score-matched non-experimental branches of the bank during the same weeks. However, the bank only keeps record of approved loans. The data on applications was collected for the purposes of the study and for study branches only. Given this limitation, we test whether the experiment changed the aggregate branch behavior on approved loan 19

characteristics using the following OLS regression:

Yi = ExperimentW eeki + Zi + i ,

(5)

where Yi is either the score of the borrower, the approved loan amount, or a dummy equal to one if the loan is in default six months after issued. ExperimentW eeki is a dummy equal to one if the loan was approved during an experimental week in the branch. Zi is a vector of controls that includes a full set of branch and week dummies, and branch-specic trends. We present the results in Table 6 estimated on two subsamples. Panel A shows the estimates using experimental branches only, using all the loans approved starting four weeks before the experiment began on the rst branch (week 41 of 2010), and four weeks after the experiment ended (week 26 of 2011). Panel B shows the estimates using experimental branches and the same number of propensity-score matched branches during the same period. Branches were matched based on size (number and total amount of loans approved), average approved loan size and borrower score during the month prior to the beginning of the experiment. We nd no statistically signicant change in the score, loan amount, or default probability of approved loans during experimental weeks across all specications in Table 6. This is consistent with the randomized results, in which we found that the introduction of a score in the application had no eect on approved loan amounts or default. These results imply that either the experiment did not change the applicant pool, or it changed the applicant pool in a way that exactly oset the eect of introducing scores on loan outcomes. All the results put together imply that the introduction of scores had a rst order eect on credit committee eort and increased the frequency of decisions taken at lower hierarchical level of the bank with no detrimental eect on the quality of decisions.

20

Conclusions

Information technology that make agents problems and decisions observable by the principal may have ambiguous eects on the productivity of dicult-to-evaluate workers. In moral hazard contexts where the principal and the agent are symmetrically informed about which actions are appropriate, observing the agents decisions reduces the cost of inducing eort by the agent (Holmstrom (1979)). In contrast, when agents have career concerns (Holmstrom (1999), Dewatripont, Jewitt and Tirole (1999)) and have private information about the productivity of their actions (Prat (2005)), IT innovations may under certain circumstances reduce performance. The present paper uses a randomized controlled trial to identify the incentive eect of an information technology innovation in the context of a micronance institution. We measure the eect of including in a credit application le a unidimensional metric of a borrowers credit repayment probability based on her observable characteristics a credit score on the output and eciency of credit evaluation committee. To distinguish the incentive eect, we use a treatment in which committees make decisions in anticipation that the score will be available in the application, but before observing the actual score. We nd that credit committees are more likely to make credit decisions accept or reject an application as opposed to passing the decision up to a superior in the hierarchy, when a credit score of the borrower is available, even holding the information set of the committee constant. The results imply that credit scores aect the committee productivity through the incentive mechanism, by making its decision easier to observe by the principal.

21

References
Aghion, Philippe, and Jean Tirole (1997) Formal and Real Authority in Organizations. Journal of Political Economy 105(1), 129 Alonso, Ricardo, Wouter Dessein, and Niko Matouschek (2008) When Does Coordination Require Centralization? The American Economic Review 98(1), 145179 Antras, Pol, Luis Garicano, and Esteban Rossi-Hansberg (2006) Organizing Oshoring: Middle Managers and Communication Costs. NBER Working Paper. 12196 Baker, George, and Thomas Hubbard (2004) Contractibility and Asset Ownership: OnBoard Computers and Governance in U.S. Trucking. Quarterly Journal of Economics 119(4), 14431479 Berger, Allen, Nathan Miller, Mitchell Petersen, Raghuram Rajan, and Jeremy Stein (2005) Does function follow organizational form? Evidence from the lending practices of large and small banks. Journal of Financial Economics 76(2), 237269 Bloom, Nicholas, Luis Garicano, Raaella Sadun, and John Van Reenen (2011) The Distinct Eects of Information Technology and Communication Technology on Firm Organization. Available at http://cep.lse.ac.uk/pubs/download/dp0927.pdf Brynjolfsson, Erik, and Lorin Hitt (2000) Beyond Computation: Information Technology, Organizational Transformation and Business Performance. The Journal of Economic Perspectives 14(4), 2348 Brynjolfsson, Erik, and Shinkyu Yang (1996) Information Technology and Productivity: A Review of the Literature. In Advances in Computers, ed. Marvin Zelkowitz, vol. 43 (Elsevier) pp. 179214 Dewatripont, Mathias, Ian Jewitt, and Jean Tirole (1999) The Economics of Career Concerns, Part I: Comparing Information Structures. The Review of Economic Studies 66(1), 183198 Garicano, Luis (2000) Hierarchies and the organization of knowledge in production. Journal of Political Economy 108(5), 874904 Hertzberg, Andrew, Jose Maria Liberti, and Daniel Paravisini (2010) Information and Incentives Inside the Firm: Evidence from Loan Ocer Rotation. The Journal of Finance 65(3), 795828 Holmstrom, Bengt (1979) Moral Hazard and Observability. The Bell Journal of Economics 10(1), 7491 (1999) Managerial Incentive Problems: A Dynamic Perspective. The Review of Economic Studies 66(1), 169182 22

Hubbard, Thomas (2000) The Demand for Monitoring Technologies: The Case of Trucking. Quarterly Journal of Economics 115(2), 533560 Liberti, Jose M., and Atif R. Mian (2009) Estimating the eect of hierarchies on information use. Review of Financial Studies 22(10), 40574090 Milgrom, Paul, and John Roberts (1990) The Economics of Modern Manufacturing: Technology, Strategy, and Organization. American Economic Review 80(3), 511 528 Mookherjee, Dilip (2006) Decentralization, Hierarchies, and Incentives: A Mechanism Design Perspective. Journal of Economic Literature 44(2), 367390 Petersen, Mitchell, and Raghuram Rajan (1995) The Eect of Credit Market Competition on Lending Relationships. Quarterly Journal of Economics 110(2), 407443 Prat, Andrea (2005) The Wrong Kind of Transparency. The American Economic Review 95(3), 862877 Rajan, Raghuram (1992) Insiders and Outsiders: The Choice between Informed and Arms-Length Debt. The Journal of Finance 47(4), 13671400 Stein, Jeremy (2002) Information production and capital allocation: Decentralized versus hierarchical rms. The Journal of Finance 57(5), 18911921

23

Figure 1: Population Credit Risk Scores and Loan Characteristics, Before Experiment
(a) Requested Loan Amount, by Score
Requested Loan Amount (USD) 1000 1100 1200 1300 1400

Local polynomial smooth

.1 Score 95% CI

.2 lpoly smooth

.3

kernel = epanechnikov, degree = 0, bandwidth = .05, pwidth = .07

(b) Default Probability, by Score

Local polynomial smooth

.04 0 0 In Default after 6 Months .01 .02 .03

.1 Score 95% CI

.2 lpoly smooth

.3

kernel = epanechnikov, degree = 0, bandwidth = .05, pwidth = .08

Non-parametric relationship between the score assigned by the credit risk model and requested loan amount (a) and default probability (b) estimated on the sample of all loans approved by BancaMia during October 2010, one month before the roll out of the randomized pilot program.

24

Figure 2: Cumulative Distributions, by Treatment Group

(a) Scores
1 0 0 .2 Cumulative Distribution .4 .6 .8

.2

.4 Score Treatment

.6 Control

.8

(b) Requested Loan Amounts

1 0 0 .2 Cumulative Distribution .4 .6 .8

2000

4000 Score Treatment

6000

8000 Control

10000

Cumulative Distribution of the scores and requested loan amounts of the loan applications in the randomized pilot program. In the treatment applications, the credit review committee received the score before making nal decisions. Scores and requested amounts are pre-determined at the time of the randomization.

25

Table 1: Loan Application Characteristics, Committee Decisions, and Approved Loan Performance, by Treatment Group
(1) Control (n = 335) Mean SD (2) Treatments (n = 1086) Mean SD (3) p-value (1) = (2)

Requested Amount (USD) Credit Risk Score

Panel A. Ex Ante Loan Characteristics 1551.5 1321.4 1552.7 0.151 0.068 0.156 Outcomes 4.68 3.28 0.887 0.318 0.003 0.055 0.110 0.314

1335.5 0.077

0.978 0.253

Time to Dummy Dummy Dummy

Panel B. Decision (minutes) Decision: Approve Loan = 1 Decision: Reject Loan = 1 No Decision (More Info, Pass Up) = 1

8.91 0.926 0.014 0.060

69.95 0.261 0.117 0.237

0.271 0.025 0.101 0.002

Panel C. Outcomes, Conditional on Approval Approved Amount/Requested Amount 0.975 0.419 0.969 Dummy In Default after 6 Months = 1 0.0329 0.1788 0.0398

0.312 0.1955

0.773 0.627

The last column presents the p-value of a t test of equality of means between the treatment and control applications. The requested amounts in dollars are calculated at prevailing exchange rate of 1,779 pesos per dollar. The credit risk score is a number between zero and one assigned by the credit risk model estimated using BanaMias historical data on borrower characteristics and repayment performance. The time to decision was calculated from begin and end time of each applications discussion, recorded by the studys research assistants in the eld. The No Decision Dummy is equal to one when the committee does not approve or reject the loan, and either sends the application to a manager at a higher hierarchical level of the bank, or sends the loan ocer to collect additional information.

26

Table 2: Committee Output

(1) OLS Dummy Decide (2) OLS Decision Time (minutes) 0.4104** (0.199) 0.5733*** (0.141) 1.1042 (1.279) Yes 1,405 0.205 (3) OLS ln(Loan Amount) 0.0235 (0.028) 0.4922*** (0.016) -0.4823*** (0.164) Yes 1,348 0.691 (4) OLS In Detault after 6 months 0.0018 (0.014) -0.0067** (0.003) 0.3624*** (0.096) Yes 1,046 0.037

Estimation:

Treatment: Score Available Requested Amount (/1000) Credit Risk Score Trend and Branch Dummies Observations R-squared

0.0395** (0.017) -0.0362*** (0.007) -0.1318 (0.112) Yes 1,414 0.067

OLS estimates of the eect of treatment on committee outcomes. Columns (1) and (2) are estimated on all applications, and columns (3) and (4) on the subsample of approved applications. Robust standard errors in parenthesis. ***, **, and * indicate signicance at the 1%, 5% and 10% levels.

27

Table 3: Information Collection and Problem Solving

Estimation Decide (Omitted) Treatment: Score Available Requested Amount (/1000) Credit Risk Score Trend and Branch Dummies Observations Pseudo R-squared Fraction in Control Subsample Marginal Eects: Treatment Yes 1,413 0.237 0.8896 0.0372*** (0.0133) 0.0627 -0.0113 (0.0102) 0.0478 -0.0259*** (0.0086) Multinomial Logit More Information -0.3428 (0.295) 0.4400*** (0.083) 0.9421 (1.859)

Send to Manager -1.1451*** (0.368) 0.4589*** (0.095) 3.3556* (2.029)

Multinomial Logistic Regression estimates of the eect of treatment on nal committee actions: make a decision on an application (approve or reject), postpone until the loan ocer collects additional information, or send the application to the manager. The rst action, make a decision, is the omitted category. The bottom rows present the proportion of each action in the control group and the estimated marginal eect of treatment on the probability that the committee takes an action. Robust standard errors in parenthesis. ***, **, and * indicate signicance at the 1%, 5% and 10% levels.

28

Table 4: Information and Incentives

(1) OLS Dummy Decide 0.0324* (0.019) 0.0353* (0.019) -0.0349*** (0.007) -0.0929 (0.112) Yes 1,414 0.061 (2) OLS ln(Loan Amount) 0.0129 (0.031) 0.0352 (0.031) 0.4924*** (0.016) -0.4835*** (0.164) Yes 1,348 0.691 0.223 0.8896 0.0311** (0.0158) 0.0365** (0.0162) 0.0627 -0.0107 (0.0117) -0.0120 (0.0119) 0.0478 -0.0205* (0.0107) -0.0246** (0.0110) (3) Multinomial Logit Decide More Send to (Omitted) Information Manager -0.3190 (0.341) -0.3594 (0.346) 0.4374*** (0.083) 0.9142 (1.857) Yes 1,413 -0.7768* (0.402) -0.9302** (0.406) 0.3792*** (0.097) 2.0631 (2.459) Yes 1,413

Estimation

Treatment 1: Score Available Treatment 2: Score Expected Requested Amount (/1000) Credit Risk Score Trend and Branch Dummies Observations R-squared Pseudo R-squared Fraction in Control Subsample Marginal Eects: Treatment 1 Treatment 2

Estimated eect of treatment on nal (Treatment 1) and interim (Treatment 2) committee decisions. In treatment T 2 committees make interim decisions before observing the score. Columns (1) and (3) are estimated on all applications, and column (2) on the subsample of approved applications. Robust standard errors in parenthesis. ***, **, and * indicate signicance at the 1%, 5% and 10% levels.

29

Table 5: Interim and Final Decisions in Treatment T 2

Final Decision (after Observing Score): Accept Loan Reject Loan Obtain More Send Decision Information to Manager Interim Decision: Accept Loan Reject Loan Obtain More Information Send Decision to Boss Total

Total

482 0 0 7 489

0 8 0 0 8

0 0 20 0 20

1 0 0 5 6

483 8 20 12 523

Each observation in the matrix represents the two sequential decision made by a committee regarding the same application in treatment T 2. Interim decisions (rows) are the decisions made before observing the score and nal decisions (columns) are the revised decisions after observing the score.

30

Table 6: Aggregate Eects on Branch Outcomes

(1) Score (2) Loan Amount (3) In Default after 6 months

Panel A. Experiment Branches Only Experiment Week 0.0023 -0.0370 -0.0011 (0.002) (0.024) (0.005) Branch Dummies Yes Yes Yes Week Dummies Yes Yes Yes Branch Trends Yes Yes Yes Observations R-squared 9,607 0.029 9,607 0.017 9,591 0.012

Panel B. Experiment and Propensity Score Matched Branches Experiment Week -0.0014 0.0026 -0.0014 (0.002) (0.020) (0.004) Branch Dummies Yes Yes Yes Week Dummies Yes Yes Yes Branch Trends Yes Yes Yes Observations R-squared 18,327 0.026 18,327 0.019 18,296 0.010

OLS regression of committee outcomes on a dummy equal to one if the application was evaluated during a week in which the randomized pilot study was taking place in the branch. Sample contains only approved loans, and the sample period is from week 41 of 2010 to week 26 of 2011 (four weeks before and after the pilot program began and ended). Panel A: sample includes only loans approved in the eight pilot BancaMia branches. Panel B: sample includes eight BancaMia branches and eight propensity-score matched branches. The matching was based on branch size (number and total amount of loans approved), average approved loan size and borrower score during October 2010. Robust standard errors in parenthesis. ***, **, and * indicate signicance at the 1%, 5% and 10% levels.

31

Table A.1: Study Sample: Number of Applications per branch and per Treatment Status
Control Branch #: 1 2 3 4 5 6 7 8 Total 44 89 26 69 18 22 20 47 335 T1 67 153 51 88 28 26 45 105 563 T2 62 132 66 87 27 14 38 98 524 Total 173 374 143 244 73 62 103 250 1,422

Control: the committee makes decision without observing the score. T 1: the borrowers score is made available at the beginning of the application evaluation. T 2: the committee makes an interim decision before the score is made available, and the allowed to revise the decision after observing the score.

32

Table A.2: Committee Output, No Controls

(1) OLS Dummy Decide (2) OLS Decision Time 0.5962** (0.242) No 1,412 0.003 (3) OLS ln(Loan Amount) 0.0183 (0.050) No 1,348 0.000 (4) OLS In Detault after 6 months 0.0068 (0.013) No 1,048 0.000

Estimation

Treatment: Score Available Controls Observations R-squared

0.0506*** (0.019) No 1,421 0.007

OLS estimates of the eect of treatment on nal committee outcomes. Columns (1) and (2) are estimated on all applications, and columns (3) and (4) on the subsample of approved applications. Robust standard errors in parenthesis. ***, **, and * indicate signicance at the 1%, 5% and 10% levels.

33

Table A.3: Committee Final Output, Treatments T 1 and T 2

(1) OLS Dummy Decide (2) OLS Decision Time 0.3256 (0.239) 0.5038** (0.252) 0.5743*** (0.140) 1.0928 (1.282) Yes 1,405 0.205 (3) OLS ln(Loan Amount) 0.0129 (0.031) 0.0352 (0.031) 0.4924*** (0.016) -0.4835*** (0.164) Yes 1,348 0.691 (4) OLS In Detault after 6 months 0.0058 (0.015) -0.0028 (0.015) -0.0067** (0.003) 0.3624*** (0.096) Yes 1,046 0.038

Estimation

Treatment 1 Treatment 2 Requested Amount (/1000) Credit Risk Score Trend and Branch Dummies Observations R-squared

0.0326* (0.019) 0.0472*** (0.018) -0.0362*** (0.007) -0.1328 (0.112) Yes 1,414 0.067

OLS estimates of the eect of treatment on nal committee outcomes. In treatment T 1 the committee makes the decision after observing the score, and in T 2 the committee makes an interim decision before observing the score and revises it after observing the score. Estimates are obtained using the nal decisions only. Columns (1) and (2) are estimated on all applications, and columns (3) and (4) on the subsample of approved applications. Robust standard errors in parenthesis. ***, **, and * indicate signicance at the 1%, 5% and 10% levels.

34