K. Ponnambalamt V.H. Quintana* A. Vannelli* t Department of Systems Design Engineering * Department of Electrical & Computer Engineering University of Waterloo Waterloo, Ontario Canada ABSTRACT. Linear Programming (LP) is a widely used tool for solving many linear/non-linear power-system optimi- zation problems such as transmission planning, constrained- security dispatch, optimal power flow, emergency control, etc. Variants of simplex-based methodologies are generally used to solve the underlying LP problems. In this paper an implementation of the newly developed Dual Affine (DA) algorithm (a variant of Karmarkars interior point method) is described in detail and some computational results are presented. This algorithm is particularly suitable for prob- lems with a large number of constraints, and isapplicable to linear and nonlinear optimization problems. In contrast with the simplex method, the number of iterations required by the DA algorithm to solve large-scale problems is relatively small, generally between 20 - 60, irrespective of problem size; this feature makes the DA method very fast in practical applications. The DA algorithm has been implemented con- sidering the sparsity of the constraint matrix. The normal equation that is required to be solved in every iteration is solved using a preconditioned conjugate gradient method. An application of the proposed optimization technique to hydro-scheduling is presented; the largest problem is comprised of 880 variables and 3680 constraints, and is solved over 9 ti mes faster than an efficient simplex (MINOS) code. Moreover, as the problem size grows, the speedup ratio over the simplex method also increases. We have also implemented a new heuristic basis recovery procedure to provide primal and dual optimal basic solutions which are not generally available if interior point methods are used. The tested examples indicate that this new approach requires less than ten percent of the original iterations of the simplex method to find the optimal basis. This feature is very useful in accelerating simplex-based linear/nonlinear optimization procedures. KEYWORDS. Optimization, Linear Programming, Karmarkars Interior Point Methods, Hydro Scheduling N2L 3G1 1. INTRODUCTION A large number of current techniques used to solve linear/non-linear power-system optimization problems depend greatly on primal/dual simplex methods because of the underlying linear programming structure, (1-31. A new interior-point method for solving L P problems was developed in 1984 by Karmarkar 141 as an alternative to the simplex method. I t became apparent that with an efficient imple- mentation of the Karmarkars algorithm it was possible to attain considerable speedups over the simplex method for some very large, sparse problems. Karmarkar and Ramak- rishnan [5] reported significant speedups over the simplex method of MINOS (61, sometimes over 100 times, for a variety of problems which included sparse and dense con- straint matrices, and very large problems of many thousands of variables and constraints. In this paper we describe a preliminary implementation of the Dual Affine (DA) algorithm for solving large-scale engineering problems, such as the hydro-scheduling problem that isincluded as an application The dual affine algorithm isan interior point method that uses a linear transformation, while the original Karmarkar algorithm used a nonlinear pro- jective transformation of the problem. The original Karmar- kar algorithm [4] has mathematically provable better compu- tational complexity but, in practice, the affine algorithms (which could be derived as simplifications of the Karmarkar algorithm (5]), are better for computational speed. Although an affine algorithm was first proposed by Dikin [7], it was Karmarkar ( Kozlov, [SI) who had any significant computa- tional results until 1986 when Adler et al. [9], implementing the dual affine algorithm with the help of Karmarkar, also reported manifold speedups over the simplex method. Many others (see Bazaraa et al., [lo], page 417, for a historical note on projective and affine scaling algorithms) also had pro- posed affine scaling algorithms following the publication of Karmarkars original algorithm in 1984. In the dual affine algorithm, the problem solved is usu- ally the dual of the original Linear Programming (LP) prob- lem. In the case of degeneracy, as in most real world prob- lems, determining the primal basic solution is a non-trivial exercise. The determination of the primal basis is critical for warm starting the simplex iterations for modified LP prob- lems, especially if Successive Linear Programming (SLP) iterations are used to solve non-linear optimization problems. Determining basic solution from a non-basic solution is an old problem (see Charnes et al. 1111); and in the context of recent interior point methods, basis recovery procedures were presented by Ye and Kojima 1121 and Megiddo [13] and others (see [lo] for a short review. The work by Ye and Kojima 112) had an assumption of non degeneracy and in CH2948-8/9 1/0ooO-0393$1.000 1991 IEEE Megiddo [ 131 there were no reports of tests or results on the proposed basis recovery procedure. Ponnambalam [ 141 independently presented a method for basis recovery based on complimentary slackness, and also used the method as a stopping criterion. The method was also presented in I151 and will be briefly described in a latter section of this paper. For brevity, we will consider only the LP aspects in this paper; SLP aspects for nonlinear problems are reported else- where (161. Therefore, the main objective of this paper is to report results corresponding to an application of dual affine algorithm to a specific problem, namely, the hydro- scheduling problem. Our implementation is simple and easily applicable but sophisticated enough to demonstrate the greater potential of the dual affine algorithm over the traditionally-used simplex method for solving problem similar to the hydro-scheduling problem. For a more advanced implementation of this algorithm, suitable for a wider variety of problems, the reader isreferred to [5] and [9]. Before we describe the dual affine algorithm, we want to state the major differences between the simplex method and the dual affine algorithm, especially those that are important to engineering applications. I t is noted that, the simplex method is used to solve problems generally with more vari- ables than constraints -- and most commercial packages such as MINOS [6] and MPSX [17] assume such a form. However, in many power-system optimization problems there are usu- ally a greater number of constraints ( m) than variables (n). In such problems, we usually resort to in-house implementa- tions of the dual simplex method that is suitable for such problems (for instance, [l]); the number of simplex iterations (pivots) may depend linearly on the number of constraints plus variables, that is, on ( n+m) and each iteration costs approximately in the order of n2, [18]. However, in practice, because of the accumulation of numerical roundoff errors, especfally in large problems where the constraint matrix is not of purely network structure, there is a frequent need (approximately once in 50-200 simplex pivots) for reinverts (that is, the entire basis has to be refactorized) which costs in the order of na operations. Therefore, for large problems consisting of over 1000 to 2000 variables, the simplex method may need upwards of 20 to 50 LU factorizations, each cost- ing in the order of ns operations. The dual affine method isdirectly applicable to problems with many more constraints than variables. In fact, the computer time to solve the LP problem depends directly on n8, and only indirectly on the number of constraints. This point will become clear in the section where the dual affine algorithm is described. In addition, as found in practice, the number of iterations to solve linear problems of any size is only about 20 to 60 iterations, [5,9 and 15, among many oth- ers]. Therefore, for large problems consisting of over 1000 to 2000 constraints and variables, it is clear that the simplex method will be easily outrun by the dual affine algorithm; moreover, the larger the problem size, the larger will be the speed up in favour of the dual affine algorithm. Only in problems where the constraint matrix is purely of network structure, a special purpose network simplex code such as in [19] can match the dual affine algorithm even up to problems consisting of tens of 1000's of variables and constraints. This is mainly due to the non-requirement of frequent basis rein- verts. Lastly, the dual affine algorithm can take advantage of indirect methods of solving simultaneous equations such as the conjugate gradient methpd, unlike the simplex method where the direct factorization methods isessential. 2. LP MODEL OF HYDRO-SCHEDULING OPTIMIZATION For simplicity, let us assume a system of two reservoirs seri- ally connected and a linear objective function [23], T 2 max {C C cijdij 1 (2.0) j -1 i-1 where T is the number of periods in the horizon, cij is the linear objective function coefficient and di, is the release from the reservoir i in period j . The constraints are basi- cally the volume conservation constraints, the lower and upper bounds on the reservoir storage volumes and releases. Let us assume a system with no spills and evaporation losses, and define si, as the i th reservoir storage volume at the beginning of period j , q;, the natural inflow, all for the i th reservoir during period j . The lower and upper bounds for releases and storage volumes for the i th reservoir in period j are day, day, s a y , and s, ?, respectively. The ini- tial and final storage volume for the i th reservoir is given by sfl and siT+1, respectively. The complete model can be formulated as follows: max{--cllsl2-~2~sz +C Cci; (si j -Si ,j +J +C,,T~~,T +c2. ~ 1 subject to (2.1) T-1 2 j=2 i=1 (2.12) (2.13) (2.14) (2.15) (2.16) (2.17) (2.18) (2.19) 394 The reasons for the above form are (i) the constraint set isof the form Ax 5 b where the equal- ity constraints with respect to the initial and final storage volumes, that is, the volume conservation con- straints, are eliminated; and (ii) as it is, the matrix A contains the staircase structure, that is, blocks along the diagonal and zeros everywhere else, essentially requiring no reordering of rows and columns to get a tight banded structure. In this serially connected two-reservoirs problem, the maximum number of nonzeros in any one row is4. The number of nonzeros in a row isa function of the number of reservoirs and the systems configuration. For example, series of reservoirs produce more nonzeros per row than parallel system of reservoirs. As the number of reservoirs and the number of periods in the horizon grows, the matrix A becomes larger and sparser but retaining the staircase structure. The sparsity and staircase structure facilitates fast solu- tion of projection calculations in the dual affine algo- rithm and, in fact, can be done more efficiently using parallel processors. However, in certain reservoir configurations, mainly, in systems consisting of large number of reservoirs in series, the constraint matrix as formed above may become unneces- sarily dense. In such problems, the following formulation is more favourable to retaining sparsity without increasing the size of the problem. For example, the problem considered has the form where the number of constraints are larger than the number of variables: Primal: max {cTx } (2.20) Alx - bl (2.21) A'2x I b2 (2.22) subject to Because the dual affine algorithm is suitable for problems with only inequality constraints, the above problem is transformed to the following form: F'rimal: max {cTx f p T (A,x - b,) } (2.23) subject to (2.24) (2.25) where the equality constraint set (2.21) has been converted to inequality as shown in (2.24); pT =[P, P, . . . P,]. If a suitable large value for p , is chosen -- for instance, lo' -- then the solution of problems in (2.20>(2.22) and (2.23)-(2.25) will be identical, usually up to 4 to 6 decimal places. 3. THE DUAL AFFINE ALGORITHM The LP form considered in the dual affine algorithm isgiven by (2.23)-(2.25). I t is assumed that the problem has an inte- rior point xo and isbounded. These assumptions are reason- able for solving real-world problems; if the problem fails to adhere to these assumptions, then usually there is no solu- tion or the solution isunbounded. The results from the algo- rithm will indicate the problem's failure to satisfy the above assumptions. With xo as the starting point, the algorithm generates a sequence of feasible interior points for LP problem, (XI, x2, . . . , xk, ' . . ) such that The algorithm terminates at the satisfaction of the stopping criterion. Let v k =b - Axk (3.2) be an m vector of nonnegative slack variables at each itera- tion k. Let the diagonal matrix be used to do the affine transformation of the constraint matrix A. The body of the algorithm is described below. The reader isreferred to [9] for the derivation details. Algorithm Let xo and 7 be given such that Axo <b and 7 =0.99 Set k :=0 Whi l e stopping criterion not satisfied, do y k :=b - h k d, :=-Ad, y k :=-Did, U; (dv)i ' a: =vxmax{- . (d,); <0, i =l , . . * ,m } xk+l :=xk - ad, set k :=k+l end where y k is the tentative dual solution. The key point is that the directional vector d,, at each iteration k, can be approximately calculated but maintaining feasibility of xk+'. As in gradient projection methods, a feasible direction vector along which the objective function increases is found and then an appropriate step length is determined to find the new feasible solution which is strictly better than the previ- ous solution. We see the differences between the simplex approach and the new interior point approach in the steps to find the next solution. A new corner point solution isdeter- mined in each iteration of the simplex method, whereas in the interior point method, a new feasible solution is deter- mined by projecting the current interior point onto the boundary of the largest ellipsoid contained in the feasible region. 3.1 Calculation of the Direction Vector d, Almost all the calculations required in each iteration of the algorithm are spent in finding the direction vector d, along which the objective function increases inside the feasible PolYtoPe, d, =(ATDiA)-'c (3.4) For a dense constraint matrix A, the number of calculations required to find the direction vector d, isO(ns) since the fol- lowing normal equations are solved directly, Bkd, =c (3.5) B~ - ( A~ D~ A) (3.6) where and is a symmetric and positive-definite matrix (this is because A is a full rank matrix due to the nature of the 395 problem constraints). As the iterations proceeds, only the diagonal matrix Dk changes and Bk remains a symmetric positive definite matrix and retains the same structure. 3.2 I ni ti al Sol uti on Adler et al. [9] propose the following Phase I procedure for determining the initial feasible interior point. Let (3.7) If vo >b-Axo, then xo is the initial solution. If not, an artificial variable xP - -2 x mini up; i-I, ..., m} (3.8) Max {cTx-Mx, } (3.9) is added and (xo, x t ) is an initial point for the following LP problem of the Phase I step: subject to where eT =(l ,l ,...,l )T and A4 is estimated as follows: Ax-eTx, 5 b (3.10) (3.11) where p isa large constant, for example 10'. The dual affine algorithm described above is applied on the above problem and is terminated when x: <0. Provided p is chosen large enough, this Phase I procedure will find either an initial feasible point or the optimal solution itself. 3.3 Stoppi ng Cri teri on and Basis Recovery Adler et al. [9] used the following stopping criterion, a cri- terion often used in non-linear optimization algorithms. The iterations are stopped when the relative improvement of the objective function issmall, that is, (3.12) where E is a small positive value, for example, lo-* used in their implementation. Adler et al. [9] also suggested an alternative stopping criterion based on the complementary slackness. The complimentary slackness criterion can also be used to verify that xk and y k are really the approximate optimal primal and dual solutions, respectively. The above alternative stopping criterion is to iterate until (3.13) and are satisfied for given small values of t 1 and t z. However, we have used a different stopping criterion that, in practice, saves significant number of iterations towards the end. The proposed stopping criterion 1141 is based on the dual and slack values and, therefore, is similar to the criterion based on the complimentary slackness. The advantage of the latter two stopping criterion is that the solution obtained is indeed optimal or in the neighborhood of optimality, as is in the last one. The stopping criterion check is activated only when either the dual solution bTy or the primal solution cTx has a growth of less than t , similar to that defined in 191. However, herein the t can be much larger than the 10" used in [9]. A conservative value for 6 herein is lo". If the value of c istoo large then there is the danger of the premature stopping of iterations. The following steps are used in the application of the proposed stopping criterion. St ep 1. Find the indices of the first n constraints whose slack vari- ables given by vf =b; - ~ a , ; x ~ , i=l, ..., m are sorted in an ascending order. Step 2 . Check 1 . If the dual values yf belonging to set of n con- straints determined above is positive then stop the itera- tions. The n constraints determined above form the basic constraint set. If Check 1 fails and there are only n--7 con- straints whose dual values are positive then perform Check 2 . Check 2. If the lgfl belonging to the m- n+r constraints are near zero, for example =lo-", then stop the iteration but inform that there are multiple optimal solutions. If Check 2 fails continue the iterations until either Check 1 or Check 2 becomes true or the maximum allowed number of iterations is exceeded. It is always possible to use the first stopping criterion suggested in 191as an additional stopping criterion in case both Checks 1 and 2 fail to stop and the number of iterations are already prohibitively large. The proposed stopping criterion may find most of the basic constraints but not all of them, especially in degen- erate problems. The degeneracy could be due to: (i ) the existence of multiple optimal solutions for the dual problem; that is, the optimal objective function plane passes through a constraint plane or through the plane intersected by many constraint planes resulting in a non-basic solution, (note that the dual affine method will always tend to end up in a non- basic optimal solution if multiple optimal solutions exist); and (ii) an active constraint is linearly dependent on one or more of the active constraints. Due to multiple optimal solu- tions and linear dependencies (degenerate problems have either one or both), the basis identified may be singular. Currently, we have used MINOS [6] to remove the singular- ity where the singularity is removed by substituting with an appropriate slack column. Once a basis is identified, the Phase I and I1 of the simplex method is used to the identify the optimal basis. For many degenerate problems, we have tested the above procedure and had considerably fewer itera- tions in the simplex method for optimal solution than with a cold start; that is, with no starting basic feasible solution (Ponnambalam et al. [20]). In summary, the advantages of the proposed stopping criterion are (i ) early stopping saving considerable number of iterations towards the end of optimization; and (i i ) identification of the basic constraint set which can be used in a standard simplex package to perform post- optimality and sensitivity analyses. n ; =1 4. COMPUTATI ONAL I SSUES The two of the most computatlonally expensive steps in the algorithm are (I ) formulating the ATDzAT matrix, and (11) solving for d, The computer implementation should vary depending upon whether the problem is sparse wlth any spe- cial structure or dense The number of calculations required to determine the direction vector d, is O(ns) for dense 396 matrices since the following nori.:al equations are solved directly: Bkd, - c where Bk =(ATDEA) is an n x n symmetric and positive- definite matrix. As the iterations proceed the diagonal matrix Dk changes and Bk remains a symmetric positive definite matrix retaining the same nonzero structure. In our implementation, a minimum-degree ordering algorithm is used to order Bk such that minimum fill-in occurs during the factorization of Bk. However, the fact that the sparsity structure of Bk remains the same through all iterations is taken advantage of while solving the normal equations in each iteration. For dense problems, formulation of the ATD2A is much more expensive (z 0.5n2m multiplications) than solving for d, ( x 0. 16n3 multiplications for the Cholesky factorization). Moreover, we have used the Cholesky factorization which is a few times faster than the QR factorization, maintaining, however, a factorization accuracy of 10-". For dense prob- lems, at most iterations, we update the factorization for solv- ing d,; the normal equations are explicitly formulated only at the first time and at most a couple of times in a total of about 20 iterations. The computational advantage is clear. Updating the factorization involves O( n2) calculations. If only m' (m' <m) of the elements in the diagonal matrix D2 changes significantly, then it is more efficient to update the factorization of ATD2A matrix than formulating the entire matrix once every iteration for solving d,. That is, O( n2m' ) calculations instead of the O(n2m) provided that m' <m. Moreover, if an element, say Di , issmall (or, equivantly, the slack of constraint i islar e), then D, has little effect on the condition number of A D2A but may have tremendous effect on the fill-in [24]. As an illustrative example, let us consider B 1 0 0 A = [i , D - 1 0 10' 0 0 1 0 0 los Using these matrices, we obtain 1 1 +10' 1 1 1 +10' A ~ D ~ A = Since D 1 - 1 is small compared to other elements, and has little effect in the solution of ATDzA d, =c, we can set D, =0. Thus, we can now define a more sparse D, D'- 1 $ Using D' yields which ismuch more sparse than ATD2A. For large sparse implementations, the formulation of ATDzA is usually done as ~j m, l D, a~aj . Because only Dj's change from one iteration to another the details (sparsity structure and values) of aTaj for each J' isstored in memory once at the beginning of the iterations and used in every iteration saving considerable computing time. Because the projection needed to calculate the direction vector d, in the dual affine algorithm need not be accurate, fast algorithms such as the conjugate gradient technique can be used instead of the direct method as described herein. The reader is referred to [5] for further implementation details on the conjugate gradient method. In our experi- ments one of the most useful step is not considering rows (constraints) for formulation of the normal equations if the d; of the corresponding row is significantly smaller than other dis. This results in increased sparsity and in better condi- tioning of the normal equations; such a method iscalled the row reduction technique. The preconditioned method we use is that of Munksgaard [21] and the accompanying general purpose rou- tine MA31AD from HARWELL software library. The conju- gate gradient method (CGM) isan efficient and fast solver of large sparse system of equations i f the condition number of system solved is small. If the system is ill-conditioned, as is in our case, the number of iterations in the CGM will be too large and often there may not be convergence. In CGM, the CPU time required is in the order of CGM iterations X O( n2) . In order to overcome the ill-conditioning and to reduce the number of CGM iterations, the preconditioned conjugate gradient method (PCCGM) is used. An approxi- mate but easily factorizable matrix M isfound such that the precon litioned system M-'A~D;A x I ( 44 The preconditioning is expected to improve the condition number of the system of normal equations solved and, as a result, much fewer CGM iterations are required to determine the direction vector d,. However, one of the major expense in terms of CPU time and memory requirement in PCCGM goes towards finding the preconditioner M-'. As mentioned before, the row reduction technique aids in sparsifying and better conditioning of the normal equations solved. Because of the row reduction technique, the structure of the normal equations with respect to the non-zeroes changes every itera- tion and hence the minimum-degree ordering routine is used in every iteration. A major improvement possible in our code is that if the preconditioner found in one iteration -- that is, M-' -- can be reused for a few subsequent iterations as in [5]. Such an improvement will greatly benefit problems where the normal equations are somewhat denser than in the example studied herein. There are other preconditioning techniques available that may be suitable for certain prob- lems and In addition the PCCGM is very suitable for solving the normal equations using parallel processors. The reader is referred to Ortega [22], an excellent source for studying such applications. 5. TESTING RESULTS The proposed Dual Affine (DA) algorithm has been code in FORTRAN, and its performance compared to the sparse simplex solver MINOS 5.0, which is also coded in FOR- TRAN. While the DA algorithm solves trhe primal problem (2.23)-(2.25). which is in the form max {cTx 1 s. t . A x 5 b where A is an mx n matrix, m >n (i.e., there more con- straints than variables), the simplex MINOS algorithm is used to solve the more appropriate dual simplex problem, 391 min {bTy } Problem MINOS 5.0 DA nxm sec Iters. sec Iters. 22x92 0.75 26 2.41 15 220x920 57.7 260 36.75 24 440x1840 236.0 520 64.48 23 880x3680 986.2 1040 106.04 18 8. t . A T y = c , y>O where AT is an n x m matrix. That is, the dual simplex problem has fewer constraints ( n) than variables ( m) . The DA algorithm has been tested on a hydro-scheduling optimization problem for 4 different horizon lengths, that is, for 1 period, 10 periods, 20 periods and 40 periods, respec- tively. The largest problem comprises 880 variables and 3680 constraints. Table 1 presents some results without basis recovery in the DA algorithm. As observed by all other researchers who implemented the DA method, the number of iterations in the DA method is very small and in the simplex method it grows linearly as the size of the problem increases. The last column of Tables 1 and 2 indicates the speed-up of the DA algorithm versus the simplex MINOS; the speed-up isevaluated by calculating the ratio of the execution times of MINOS and DA algorithms. Table 2 presents results corresponding to the same prob- lems with basis recovery in the DA algorithm. MINOS DA 0.31 1.57 3.66 9.30 I Table 1. LP Multi-reservoir Optimization results (Without Basis Recovery) Problem Size Growth 10 20 40 SIMPLEX STATISTICS Dual Affine Iterations CPU Time (Col 1)' CPU Time Growth Growth Growth 10 76 100 15.25 20 314.67 400 26.76 40 13 14.67 1600 44 Problem nxm 22x92 220x920 440x1840 880x3680 A comparison of the CPU time required by MINOS 5.0 and the dual affine techniques, as a function of the horizon time,is presented in Fig. 1. Table 3 presents results corresponding to problem size growth (both in terms of number of constraints and variables and in the number of nonzeroes) versus CPU time growth for the two methods on these problems. Table 4 presents results similar to Table 3 but with the inclusion of the CPU time for basis recovery. MINOS 5.0 DA Pivots MINOS sec (Iters) sec (Iters) BRect DA 0.75 (26) 2.53 (15) 4 0.30 57.7 (260) 44.52 (24) 35 1.30 236.0 (520) 87.17 (23) 50 2.71 for / 986.2 (1040) 225.52 (18) 126 4.37 It isclear that for these problems, the CPU times of the inte- rior point method grows only linearly (column 5 in Table 3) and in the simplex method it is approximately quadratic (column 3 in Table 3) as size of the problem grows (column Problem Size Growth 750 w E 3 c 0 F: 500 250 SIMPLEX STATISTICS Dual Affine Iterations CPU Time (Col 1)' CPU Time Growth Growth Growth U MI NOS 5.0 Results P 0 Dual Affine Results 10 20 30 40 Time Horizon (years I Fig. 1 CPU time vs horizon time 10 10 76 100 17.6 40 I 1 1 in Table 3). This fact is also reflected in Table 4 although with a somewhat higher CPU time because of the basis recovery procedure. In addition, the convergence of the objective function in the DA method was found to be monotonic all through the itera- tions and was rapid in the beginning and was slow in the end. Therefore, i f only an approximate solution is required 398 then the iterations could be stopped earlier saving a few iterations, unlike in the simplex method where the conver- gence is not necessarily monotonic; that is, in the simplex method, at 90% of the total CPU time the solution deter- mined need not be 90% of the optimal solution. Also, with the basis recovery procedure in conjunction with the dual affine method, the iterations in the dual affine method could be stopped earlier and a very accurate optimal basic solution can be determined. With the basis recovered, the solutions from both the simplex and dual affine methods are qualita- tively the same since the final solution is primal and dual optimal basic solutions. When using SLP approaches to solve nonlinear problems, the first one or two SLP iterations requires a large number of simplex iterations [3] whereas, by using the previous basis to initialize or start the subsequent SLP iterations, the required simplex iterations to solve subse- quent LP problems isgreatly reduced. With the dual affine method and with the basis recovered the same procedure could be followed, where, for the first one or two SLP itera- tions the dual affine algorithm could be used instead of the simplex method with a possibility for significant reduction in computing time. In addition, if the problem solved is in the original dual affine form and that there is no need for accu- rate solutions then it ispossible to stop the dual affine algo- rithm much earlier, that is only with about half the number of originally required iterations, saving a large slice of CPU time. 6. CONCLUSIONS A preliminary implementation of the dual affine algorithm demonstrates its potential for solving large-scale specially structured LP problems. The superiority of the DA method over the simplex method for staircase structured multi- period hydro-scheduling problems is now well established. However, with the basis recovery procedure, it is possible to use both the DA and the simplex method in their respective places for solving nonlinear optimization problems with the successive linear programming method. This results in only a minor change to the current simplex-based codes used in the industry but can lead to a considerable speedup in solv- ing some very large scheduling problems. ACKNOWLEDGEMENTS The authors acknowledge the Natural Sciences and Engineering Research Council (NSERC) of Canada for the financial support. REFERENCES BStott, J .L. Marinho and 0. Alsac, "Review of Linear Programming Applied Power System Rescheduling", Proc. of 1979 PICA Conf., pp. 142- 154, Minnesota, MN, 1979. J .S. Horton and L. L. Grigsby, "Voltage Optimization Using Combined Linear Programming and Gradient Techniques", IEEE Tran. on Power Apparatus and Sys- tems, Vol. PAS-102,No.7, J uly 1984. H. Habibollahzadeh, G.X. Luo and A. Semlyen, "Hydroth- ermal Optimal Power Flow Based on A Combined Linear and Nonlinear Programming Methodology", IEEE Tran. on Power Systems, vo1.4, NO.^., May 1989. N. Karmarkar, "A New Polynomial-time Algorithm for Linear Programming", Combinatorica, 4, pp. 373-395, 1984. N. Karmarkar and K.G. Ramakrishnan, "Implementation and Computational Results of the Karmarkar Algorithm for Linear Programming, Using an Iterative Method for Computing Projections", 13th International Math. Prog. Symposium, Tokyo, J apan, August 1988. B.A. Murtaugh and M.A. Saunders, "MINOS 5.0 User's Guide", Tech. Rep. SOL 83-20, Stanford Univ., CA, 1983. 1.1. Dikin, "Iterative Solution of Problems of Linear and Quadratic Programming", Soviet Math. Doklady, 8, pp. A. Kozlov, "The Karmarkar Algorithm: Is It for Real?, SIAM News, 18(6), pp. 1-4, 1985. I. Adler, N. Karmarkar, M.G.C. Resende and G. Veiga, "An Implementation of Karmarkar's Algorithm for Linear Programming", Working Paper, Operations Research Center, Univ. of California, Berkeley, C.A, 1986, (also in Math. Prog., 44, pp. 297-335, 1989). 674-675, 1967. [lo] M.S. Bazaraa, J .J . J arvis and H.D. Sherali, "Linear Pro- gramming and Network Flows", Second Edition, J ohn Wiley and Sons, Toronto, 1990. [l l ] A. Charnes, K.O. Kortanek and W. Raike, "Extreme Point Solutions in Mathematical Programming: An Oppo- site Sign Algorithm", Systems Research Memorandum No. 129, Northwestern University, Evanston, 1965. (121 Y. Ye, and M. Kojima, "Recovering Optimal Dual Solu- tions in Karmarkar's Polynomial-time Algorithm for Linear Programming", Math. Prog., 39, pp. 305-318, 1987. [13] N. Megiddo, "On Finding Primal- and Dual-Optimal Bases", Research Report, RJ 6328 (61997), IBM York- town Heights, New York, 1988. [14] K. Ponnambalam, "New Starting and Stopping Pro- cedures for the Dual Affine Method", Working Paper, Dept. of Civil Engg., University of Waterloo, 1988. [15] K. Ponnambalam, A. Vannelli, E.A. McBean and T.E. Unny, "Solving Large-scale Electric Energy Production Planning and Pollution Control Problems with Karmar- kar Algorithms", Proc. of Workshop on Resource Plan- ning Under Uncertainty for Electric Power Systems, Stanford Univeristy, Eds. G.B. Dantzig and P. Glynn, 179-196, J an 1989. [ 161 K. Ponnambalam, "Large Scale Nonlinear Programming Using Interior Point and Successive Linear Programming Methods, SIAM Annual Meeting, J uly 16-20, Chicago, 1990. [ 171 Mathematical Programming System Extended (MPSX), IBM Computers Manual, 1979. [18] M.J . Best and K. Ritter, "Linear Programming: Active Set Analysis and Computer Programs", Prentice Hall Inc., NJ ., 1985. [19] J .L. Kennington and R Tr Tlelgason, "Algorithm for Net- work Programming", J ohn Wiley and Sons, N.Y., 1981. [20] K. Ponnambalam, A. Vannelli, and S. Woo, "An Interior Point Implementation for Solving Large Planning Prob- lems in the Oil Refinebi v Industry", submitted to Cana- dian J . of Chem. Engg., J uly 1990. 399 1211 N. Munksgaard, "Solving Sparse Symmetric Sets of Linear Equations by Preconditioned Conjugate Gra- dients", ACM Trans. on Math. Soft., 6, pp. 206-219, 1980. [22] J .M. Ortega, "Introduction to Parallel and Vector Solu- tion of Linear Systems", Plenum Press, N.Y., 1988. (231 D.P. Loucks, J .R. Stedinger and D.A. Haith, "Water Resource Systems Planning and Analysis", Prentice-Hall Inc., N.J ., 1981. [24] Anthony Vannelli, "An Adaptation of the Interior Point Method for Solving the Glabal Routine Problem," IEEE Trans. on Computer-Aided Design, Vol. 10, No. 2, Feb. 1991, pp. 193-203. K. Ponnambalam received the B.E. degree from Madras University in 1979, MSc. degree from the National Univer- sity of Ireland in 1981 and Ph.D. degree from the University of Toronto in 1987. He iscurrently an Assistant prof ;sor in the Department of Systems Design Engineering, University of Waterloo. His main research interests include large scale optimization, interior point methods and stochastic modeling as applicable to hydrc-power optimization. V.H. Quintana (IEEE SM'73) received the Dipl. Ing. degree from the State Technical University of Chile in 1959, and the MSc. and Ph.D. degrees in Electrical Engineering from the University of Wisconsin, Madison in 1965, and University of Toronto, Ontario, in 1970, respectively. Since 1973 he has been at the University of Waterloo, Department of Electrical Engineering, where he is currently a full professor and Associaf,e Chairman for Graduate Stu- dies. His main research interests are in the areas of numerical optimization techniques, state estimation and control theory as applied to power systems. Dr. Quintana is an Associate Editor of the International J ournal of Energy Systems, and a member of the Association of Professional Engineers of the Province of Ontario. A. Vannelli received the Ph.D. degree from the Department of Electrical Engineering at the University of Waterloo, Waterloo, Ontario, Canada, in 1983. From 1983-84, he was an IBM Post-Doctoral Fellow in the Mathematical Sciences Department at IBM Thomas J. Watson Research Center. He joined the Department of Industrial Engineering at the University of Toronto in 1984. Since 1987 he has been with the Department of Electrical and Computer Engineering at the University of Waterloo, where he's currently an Associ- ate Professor. He has been a Natural Sciences and Engineer- ing Council of Canada University Research Fellow since 1984. His main research focuses on the development of effi- cient linear, nonlinear, and combinatorial optimization tech- niques to solve VLSI circuit layout and design problems.