A041 Production Optimization under Constraints Using Adjoint Gradients

A041 Production Optimization under Constraints Using Adjoint Gradients P. de Montleau* (ENI SpA - E&P Div.), A. Cominelli (ENI E&P), K. Neylon (Schlumberger), D. Rowan (Schlumberger), I. Pallister (Schlumberger), O. Tesaker (Statoil ASA) & I. Nygard (statoil ASA) SUMMARY The introduction of controllable downhole devices has greatly improved the ability of the reservoir engineer to implement complex well control strategies to optimize hydrocarbon recovery. The determination of these optimal control strategies, subject to limitations imposed by production and injection constraints, is an area of much active research and generally involves coupling some form of control logic to a reservoir simulator. Some of these strategies are reactive: interventions are made when conditions are met at particular wells or valves, with no account taken for the effect on the future lifetime of the reservoir. Moreover, it may be too late to prevent unwanted breakthrough when the intervention is applied. Alternative proactive strategies may be applied to the lifetime of the field and fluid flow controlled early enough to delay breakthrough. This paper presents a proactive, gradient based method to optimize production throughout the field life. This method requires the formulation of a constrained optimization problem, where bottomhole pressure or target flow rates of wells, or flow rates of groups, represent the controllable parameters. To control a large number of wells or groups at a reasonably high frequency, efficient calculation of accurate well sensitivities (gradients) is required. Hence, the adjoint method has been implemented in a commercial reservoir simulator to compute these gradients. Once these have been calculated, the simulator can be run in optimization mode to find a locally optimal objective function (e.g., cumulative production). This optimization procedure usually involves progressively activating constraints, with each new constraint representing a significant improvement in the objective. Proper management of degrees of freedom of the parameters is essential when calculating the constrained optimization search direction. Adjoint methods have already been used for production optimization within reservoir simulation; however, an accurate analysis of optimal management of active and inactive constraints for different type of recovery processes in field like cases has not been discussed to our knowledge.

Introduction In reservoir engineering, improvement of field production is a continuous challenge. The desired improvement may simply be maximization of oil production over a given time period, but it can be a more complex objective that incorporates the need to reduce produced gas and injected water and that includes a financial discount coefficient. Moreover, each component of the objective may be subjected to physical and/or economic constraints; typically, for example, production and injection are constrained by surface handling capacities. Traditionally, reservoir engineers have made optimization decisions based on observed local conditions around each well, or on fluid and pressure distributions predicted by a reservoir simulator. This reactive approach to field management means that decisions made at a certain stage of the field lifetime can induce detrimental effects later on. For example, a potential-based allocation of well target rates will prioritize production from highpermeability zones, which may lead to early water or gas breakthrough. In addition, an action of the type shut the well when breakthrough occurs is a local control that cannot influence the period preceding breakthrough. Careful control of the early life of well production could improve recovery and delay breakthrough. Work is required by the reservoir engineer to manually modify field, group, and well controls to construct a set of production scenarios from which a field development plan can be selected. This effort may be huge in the case of fields of complex geology, uncertainty in the reservoir model, and large numbers or wells. Maximizing oil or gas production using a numerical reservoir simulator is an optimal control problem. The reservoir simulator plays the role of a dynamic system, able to compute a response function (e.g., cumulative oil production) for some controllable parameters (well flowing rates, bottomhole pressures), and subject to nonlinear constraints (both equality and inequality). The optimization used in this study is a gradient-based algorithm. Compared with genetic algorithms, e.g., (Goldberg, 1989), a gradient-based method is ideal for constrained optimization problems. The gradient method is commonly applied to solve history-matching problems, e.g., (Schlumberger, 2006b, Gosselin et al., 2003), but while it is not yet widely used for industrial dynamic optimization, a number of studies have shown promise (Brouwer et al., 2001, 2002b, Sarma et al., 2005). Once an objective (or cost) function is defined, the gradients of this function with respect to the set of control parameters are computed and used in a maximization algorithm. Because the number of controllable parameters can be extremely large, the expense of the gradient computation is an issue. For instance, if well control intervention is desired every month, then in the case of a field with N controllable wells to be simulated for 30 years, the total number of controllable parameters will be 360 N. So, there are thousands of controllable parameters for most reservoir models. Because of the size of the system, gradients are calculated using the adjoint method (Ramirez et al., 1984, Zakirov et al., 1996). The method is computationally efficient, but there is a requirement that the reservoir simulation is fully implicit (Brouwer and Jansen, 2002a). The role of constraints in the successful application of an optimization method is crucial. Constraints in reservoir simulation may be simple bounds on the control parameters or complex nonlinear functions of reservoir and well operating conditions. Such nonlinear constraints (e.g., that a well or group may not flow above a certain rate limit) typically become active as the optimizer attempts to maximize production. The addition of each new active constraint to the system removes one degree of freedom (and hence one controllable free parameter) from the optimization; this is because that dependent parameter has to be used to honor the constraint. The details of how the optimization proceeds as constraints become active or inactive is discussed in the next section. Great care must be taken when the constraints are not directly influenced by the control parameters (insensitive constraints), and a method to deal with this is proposed and demonstrated. After a short discussion on the treatment of constraints in a Lagrangian formulation of the optimization problem, three example cases are discussed. The first synthetic example highlights the insensitive constraint problem and the need for improved constraint management. This new treatment of insensitive constraints is applied with success. A second optimization is proposed for a highly channeled layer that is produced using a repeated five-

spot pattern. This optimization combines control of both injectors and producers and investigates how different objective functions influence the optimization in addition to the constraints. The final application concerns the optimization of the water injection in part of a real field, for which vertical heterogeneities make the water distribution an issue. Optimization Method The primary influence that can be exerted on the recovery from a reservoir is the setting of the well and group operating conditions (e.g., flow rates, bottomhole pressure, valve settings, etc.) for both the injection and production systems. These are the control parameters. The problem considered in this paper is to determine the set of control parameters, P, that maximize an objective function, J, over the lifetime of the field. This objective function can be as simple as cumulative oil or gas production, or a more sophisticated measure of the recovery performance such as the net present value of the hydrocarbon production taking into account costs incurred in both injection and in disposal of unwanted produced fluids. While attempting to maximize the objective function, certain physical and economic constraints, C, must be obeyed. For example, the water cut in the producers must not exceed a certain limit, or the production from a particular group must not exceed the capacity for the surface facilities of that group. These (generally nonlinear) optimization constraints can always be written in the form C 0. These constraints can be divided into those for which currently C = 0 (called the active constraints, C A ) and those for which currently C < 0 (called the inactive constraints). A commercial compositional reservoir simulator (Schlumberger, 2006a) is used to solve this constrained maximization problem. The simulator equations themselves are constraints upon the optimization process, indicating that the simulator must obey laws for conservation of mass, fluid flow, etc. These simulator equations can be represented by R(X, P) = 0, where X is the vector of simulator solution variables and includes the well model equation set. To solve this constrained optimization problem, the Kuhn-Tucker conditions (Kuhn and Tucker, 1951) for optimality are invoked. This leads to the formation of the Lagrangian function, L, which is the objective function, J, augmented with the reservoir equation constraints, R, and the active optimization constraints, C A. L(X, P) = J(X, P) + Ψ R R(X, P) + Ψ C C A (X, P) (Eq. 1) Here Ψ R and Ψ C are reservoir and constraint Lagrange multiplier vectors respectively (also termed the adjoint or state vectors) and are determined by the Kuhn-Tucker conditions such that at the solution all active constraints are met and the objective function is an optimum in all feasible directions. Because of the nonlinear nature of the components of the Lagrangian, and the determinacy of the problem, this requires an iterative method. Following (Zakirov et al., 1996), an active set method and a projected gradient technique are used to ensure that the gradient search direction obeys the simulator and optimization constraints. For a gradient based optimization method, the derivative of the Lagrangian with respect to the control parameters (Eq. 2) is needed, which in turn requires partial derivatives described in (Eq. 3) and (Eq. 4): dl dp L = L + (Eq. 2) L L J R C A = + Ψ R + Ψ C (Eq. 3) J R C A = + Ψ R + Ψ C. (Eq. 4)

The expensive computation of the partial derivative of all the solution variables with respect to all the control parameters (/) can be avoided by choosing the Lagrange multiplier vectors such that L/ is zero in Eq. 2. If there are N A active constraints, then there are only enough degrees of freedom to set N A of the L/ equations to zero, that is, each active constraint removes a freely controllable parameter. The system of equations (Eq. 5 and Eq. 6) is solved for the Lagrange multiplier vectors Ψ R and Ψ C, and then L/ i is calculated for i > N A. This is called the free parameter search direction. J R C A + Ψ R + Ψ C = 0 (Eq. 5) J i + Ψ R R i + Ψ C C A i = 0 i N A (Eq. 6) This solution approach is called the adjoint method. Because of the coupled time structure of the reservoir simulator equations, this step essentially means solving these equations backwards in time. The state of the simulation is saved to file at various (report) times in the conventional forward simulation, then the simulation is stepped backward from the end to the start, reading the saved state to allow the reconstruction of the Jacobian matrix and the well model quantities that are needed for the backward solve. Following (Zakirov et al., 1996), linearization of the simulator and optimization constraints allows the calculation of the remaining L/ i for i N A (the dependent parameter search direction) by stepping forward through the saved states, giving the entire dl/dp search direction (and largest magnitude) which obeys the constraints. This search direction is then used in a steepest descent, or conjugate gradient, maximization algorithm, in conjunction with a line-search algorithm, which requires a forward simulation at each step. Choice of free parameters and handling of insensitive constraints The choice of which N A of the N control parameters to use as the free parameters is apparently arbitrary. However, there are some algorithmic considerations that dictate the choice of the most suitable ones to make the solution of the system of equations (Eq. 5 and Eq. 6) stable. The most efficient way to solve the system is to eliminate Ψ C using Eq. 6, and then solve for Ψ R using Eq. 5. This requires the inversion of the C A / i matrix. With this solution approach, the free parameters are chosen to be those that make the inversion of the C A / i matrix stable. However, care must be taken in the case of an insensitive constraint, where C A / i = 0 (as would occur, for example, for a constraint on a production well with the control parameters only on the injection wells) as the algorithm would break down. In this case, the more expensive approach (in terms of both operations and storage) of eliminating Ψ R using Eq. 6 and then solving for Ψ C using Eq. 5 must be used. This requires the inversion of a matrix of the form C A / ( R/) R/ i. Again, when this solution approach is used, the free parameters are chosen to make the inversion of this matrix stable. The implementation used switches from the efficient algorithm to the expensive one dynamically during the calculation of the gradients (and only when necessary). Applications A five-layer problem: Constraints insensitive to the control parameters This example is a synthetic reservoir composed of five independent, one-dimensional, homogeneous layers, with a different permeability in each layer (see Figure 1 top). On the

right side, five injectors are completed independently in each layer, while a single producer is completed in each of the five layers on the left. The objective is to maximize the oil production under a water production rate constraint. The control parameters are the water injection rates in the five injectors. The production period is divided into 16 steps of 12.5 days such that there are 90 (= 16 5) parameters to optimize. The producer is operated so that it maintains a bottomhole pressure (BHP) target of 99 bar. The optimization starts from a point where the BHP of the five injectors is set to 160 bar, leading to water displacements proportional to the permeability of each layer (Figure 1 middle). K=600 md K=400 md K=400 md K=200 md K=100 md Figure 1. Top: Permeability. Middle: Oil saturation in fixed BHP case. Bottom: Oil saturation in optimized simulation. In the first step of the optimization, no constraints are active, so each well tries to inject as much water as possible to increase recovery. The water in the high-permeability (top) layer travels to the producer fastest, and soon the water cut constraint is violated. Since the control parameters are on the injectors and the constraint is on the producer, then this is an example of an insensitive constraint situation. Without any special handling of the insensitive constraints, the optimization algorithm breaks down when the first constraint is encountered at a time period because of water breakthrough in the top layer, and the recovery is only 82% of the original oil in place (OIP). However, with proper consideration given to the insensitive constraints, the optimization can proceed and converges after seven iterations, with the constraint being active for three time periods. The optimizer reduces injection in the top layer to allow recovery in the other layers, which improves the overall objective while not violating the constraints. The recovery is 90.3% of the original OIP. A piston-like displacement (Figure 1 bottom) is achieved starting from an unfavourable starting point. A Multiple Five-Spot Pattern This example concerns a synthetic plane layer taken from a real channeled reservoir. The permeability field (Figure 2) presents an interesting high/low permeability profile, ranging from 50 md (blue) to 2,400 md (red). The field is produced by means of 3 five-spot patterns (3 water injectors and 8 producers). Figure 2. The multiple five-spot pattern: 3 injectors and 8 producers. Map of the permeability. In this first part of this study, the objective is to maximize the oil cumulative production (FOPT): J = FOPT. To compare the effect of the constraints, the field production is subject to two different sets of rates: (1) only on the water production rate (FWPR) and (2) on both FWPR and oil production rate (FOPR). The injection pressure is fixed at 200 bar. The control parameters are the liquid production rates (LPR) of the eight producers with a lower limit on the BHP set to 10 bar.

The production life is divided into 100 report steps of 4 months, so there is a total of 800 (= 8 100) control parameters. For both cases, the optimized solution (opt) is compared with a base case (base) for which the producers are under a group control: An oil rate target is set at the value of the oil rate constraint (or an arbitrary high value if there is no oil rate constraint) and switched to the water rate constraint once it is overshot. Characteristics and results for the two cases are presented in Table 1. Objective function: FOPT, Optimizer parameters: 0 < LPR < 10,000 m 3 /d Case Constraints (m 3 /d) Active constraints steps: FOPR, FWPR FOPT opt/base ( 10 6 m 3 ) A FWPR < 1,500 -, 63 50.2/43.1 (+ 16.5%) A-CO FOPR < 3500, FWPR < 1,500 91, 27 40.6/37.3 (+ 8.4%) Table 1. Results for the multiple five-spot pattern: objective function FOPT. For both cases, the constraints are active for a significant number of steps and the optimizer increases the cumulative oil production. The results of the case A-CO are displayed in Figure 3. In particular, Figure 3 (left) shows the constraints (red = oil, blue = water) for the optimized and the base simulation (thick and thin lines respectively). The water constraint forces a reduction of the oil production earlier in the optimized case; subsequently the oil plateau is maintained for a longer period, gaining 8% cumulative oil production overall. This improvement is achieved because the two constraints (water and oil rates) are active a) for a longer period, and b) during the same period, roughly from year 2024 to year 2030. This is also illustrated in Figure 3 (right) where the objective function, and the number of steps for which constraints are active, are plotted against the number of simulations performed. 100 4.5E+07 80 3.6E+07 60 40 const FOPR const FWPR FOPT 2.7E+07 1.8E+07 20 9.0E+06 Figure 3. Optimization case A-CO. 0 0.0E+00 0 10 20 30 40 50 60 Another optimization is performed with the objective of maximizing the total oil production while reducing the water injected (FWIT): the objective function is J = FOPT 0.5 FWIT. The water injection rate (WIR) on the three injectors is added to the previous set of control parameters, so there is a total of (8 + 3) 100 = 1,100 control parameters. The upper limit on the BHP of the injectors is set to 200 bar. The base cases are built in the same way as in the previous section. The cumulative oil and water produced, as well as water injected, are shown in Table 2, and compared to the base case. Figure 4 shows results of the optimized case, B_CO (thick line), compared to the corresponding base case (thin line). On the left are the oil (red) and water (blue) constraints and on the right are the reduction of the water injection total (blue) and water production total (yellow).

Objective function: FOPT - 0.5 FWIT; Optimizer parameters: 0 < LPR, WIR < 10,000 m 3 /d Case Constraints (m 3 /d) Active constraints: FWPR, FOPR B FWPR<2,000 0,- B_CO FWPR<2,000 FOPR<3,500 0,27 FOPT opt/base ( 10 6 m 3 ) 44.1/46.6 ( 5%) 37.5/39.3 ( 5%) FWPT opt/base ( 10 6 m 3 ) 3.1/19.0 ( 83%) 2.6/10.7 ( 76%) FWIT opt/base ( 10 6 m 3 ) 46.4/66.1 ( 30%) 39.4/50.1 ( 21%) Table 2. Results for the multiple five-spot pattern: objective function FOPT - 0.5 FWIT Figure 4. Optimization case B_CO. By negatively weighting the injected water, the oil produced is reduced (by 5%), while the injected water is reduced significantly and produced water is reduced still further (80%). In this example, by penalizing the water injected, the water constraint is not even reached in the optimized simulation. The optimizations carried out on this multiple five-spot pattern field give satisfactory results. Besides increasing cumulative oil production, injected/produced water can be reduced by penalizing it in the objective function. Increased recovery using downhole valves in water injectors This example uses part of a real field model. In the region of interest, there are two producers and two water injectors. The reservoir was originally thought to have two layers, Z1 and Z2, but layer Z1 is now known to contain an internal shale barrier in the vicinity of the injectors and so can itself be represented by layers UZ1 and LZ1 (see Figure 5). Hence this is really a three-layer system. One of the main issues is to evaluate the improvement in the ultimate recovery that can be achieved with separate downhole valves in the injectors in each sand lobe in Z1 (i.e., in layers UZ1 and LZ1), as opposed to a single downhole valve in Z1. In this work, only layer Z1 (represented by UZ1 and LZ1) is optimized, the rest of the reservoir is not modeled. Figure 5. Log example showing the internal shale barrier in the Z1 layer

Figure 6. The simulation model showing the original oil saturation. Each injection well is represented by a pair of simulation wells. Because of the lack of availability of downhole valve control parameters on injection wells, the individual downhole valves are simulated by using separate wells. Hence the model contains four injection wells (representing the separate downhole valve in each layer for the two real injectors) and two production wells (see Figure 6). An upper limit is set on the BHP for all the wells. Each pair of simulated injection wells is treated as a group representing the actual injection well. The control parameters are the water rates in each injection group. There is a constraint of 0.95 on the water cut in the producers. The objective is to maximize the cumulative oil produced at the end of the simulation. An experienced engineer performed a manual optimization. Independently, and from the same starting point, an adjoint optimization was completed on a simulation spanning the first half of the production period. In Figure 7, both these approaches are compared to the original plan where water injection is controlled by only one downhole valve covering the Z1 layer. Optimized by using the OPTIMIZE function Optimized by using the OPTIMIZE function Manually optimized Manually optimized FOPR Single valve case FOPT Single valve case YEARS YEARS Figure 7. Field oil production rate and cumulative field oil production for the original plan and for the optimized models. This result shows that the adjoint optimization performs as well as the manual optimization on this case, indicating that it is possible to allow the optimizer to do this field operations work, thereby freeing the experienced engineer for other tasks. Conclusion In this paper, a small synthetic five-layer problem has demonstrated that insensitive constraints can be successfully managed with the careful calculation of constrained gradients in a reservoir simulator. In the second example, different optimization scenarios have been investigated, simultaneously combining the desire to increase the oil production and reduce water injection. Successful optimal field operation was obtained using the optimizer within the reservoir simulator, a task that would have taken much reservoir engineering effort. Moreover, this example shows that the constrained gradient is the key to field optimization. In particular, in the case A-CO, it allowed the field to maintain an oil production plateau while a maximum water production rate was reached; this is not possible with a simple group

production control. This case also highlights the power of this method, as the controls for a large number of wells can be optimized. In summary, optimization has been shown to work for a range of reservoir problems. This approach is obviously preferable to reactive manual interventions, because field integrity can be preserved and an optimum production schedule rapidly achieved by harnessing cheap computational power and a modern reservoir simulator. The savings in human effort may then be redirected to other interesting problems, such as completion design and well location strategies. Acknowledgments The authors are grateful to ENI, Schlumberger, and Statoil for permission to publish this paper. Thanks are also due to Total and Norsk Hydro for contributing to the project during which much of the adjoint gradient development was carried out in the commercial simulator. References Brouwer, D.R. et al.: Recovery Increase through Water Flooding with Smart Well Technology, SPE 68979, SPE European Formation Damage Conference, The Hague, The Netherlands, (May 21 22, 2001). Brouwer, D.R. and Jansen, J.D.: Dynamic Optimization of Water Flooding with Smart Wells Using Optimal Control Theory: Stability of the Adjoint Equation, 8 th European Conference on the Mathematics of Oil Recovery, Freiberg, Germany, (September 3-6, 2002 a). Brouwer, D.R. and Jansen J.D.: Dynamic Optimization of water Flooding with Smart Wells Using Optimal Control Theory, SPE 78278, 13 th SPE European Conference, Aberdeen, Scotland, (October 29-31, 2002 b). Goldberg, D.E.: Genetic Algorithms in Search, Optimization, and Machine Learning, Addisson-Wesley Professional, New York, (1989). Gosselin, O. et al.: History matching Using Time-lapse Seismic (HUTS), SPE 84464 presented at the SPE Annual Technical Conference and Exhibition, Denver, Colorado, USA, (October 5-8, 2003). Kuhn, H.W. and Tucker, A.W.: Nonlinear Programming, proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, (Ed. J. Neyman), University of Carolina Press, (1951). Ramirez, W.F. et al.: Optimal Injection Policies for Enhanced Oil Recovery: Part 1 Theory and Computational Strategies, SPEJ 328-332, (1984). Sarma, P. et al.: Implementation of Adjoint Solution for Optimal Control of Smart Wells, SPE 92864, SPE Reservoir Simulation Symposium, Houston, Texas, (Jan. 31 Feb. 2 2005). Schlumberger: ECLIPSE 2006.1 Technical Description, (2006 a). Schlumberger: SimOpt 2006.1 User Guide, (2006 b). Zakirov, I.S. et al.: Optimizing Reservoir Performance by Automatic Allocation of Well Rates, 5 th European Conference on the Mathematics of Oil Recovery, Leoben, Austria, (September 3-5, 1996).