Efficient Scheduling of Constant Pressure Stratified Water Injection Flow Rate: A Deep Reinforcement Learning Method

Abstract

Layered water injection is an important means of oilfield development. However, the process is fraught with challenges due to the complexity of the subsurface environment and the variability in water absorption properties across different strata. The water injection flow rate of each layer is affected by various factors, resulting in a typical incomplete system and extremely complex measurement and adjustment of layered water injection. The reasons include development plans, water absorption properties of each layer, and differences in the structure of underground water injection devices. The combined effect of these factors leads to the typical incomplete system and complexity of the measurement and regulation of the layer section of layered water injection. In this paper, a reinforcement learning-based stratified water injection control algorithm was proposed to solve the problem of stratum flow scheduling in the complex environment of wellbore during stratified water injection. The reinforcement learning algorithm is combined with the differential pressure stratified water injection control algorithm, and a reasonable reward function is set according to the injection error to improve the algorithm control accuracy and efficiency. In order to evaluate the performance of the algorithm, a gym-based simulation environment is established to simulate the nonlinear stratigraphic environment under stochastic conditions, so that the model has a better generalization performance. Compared with other algorithms, the proposed stratified water injection control algorithm saves 41.94% of training time, the average injection error is less than 5% in the water injection environment with different number of stratum segments, the average success rate is more than 90%, and there is 85% probability to reach the injection target within 1-5 steps, which provides a more excellent performance in terms of control accuracy and adjustment speed. The algorithm has an important guiding role in the flow scheduling control of injection wells and realizing the automation of layered water injection process. Our code will be available online at: https://github.com/HJZ-hub/ASWI.

Keywords

• Deep reinforcement learning
• layered water injection
• SAC
• flow scheduling