Optimal Dynamic Regime for CO Oxidation Reaction Discovered by Policy-Gradient Reinforcement Learning Algorithm

Authors: Lifar M. S., Tereshchenko A. A., Bulgakov A. N., Guda S. A., Guda A. A., Soldatov A. V.

Abstract:

Metal nanoparticles are widely used as heterogeneous catalysts to activate adsorbed molecules and reduce the energy barrier of the reaction. Reaction product yield depends on the interplay between elementary processes - adsorption, activation, reaction, and desorption. These processes, in turn, depend on the inlet feed concentrations, temperature, and pressure. At stationary conditions, the active surface sites may be poisoned by reaction byproducts or blocked by thermodynamically adsorbed gaseous reagents. Thus, the yield of reaction products can significantly drop. On the contrary, the dynamic control accounts for the changes in the surface properties and adjusts reaction parameters accordingly. Therefore dynamic control may be more efficient than stationary control. In this work, a reinforcement learning algorithm has been applied to control the simulation of CO oxidation on a catalyst. The policy gradient algorithm is learned to maximize the CO₂ production rate based on the CO and O₂ flows at a given time step. Nonstationary solutions were found for the regime with surface deactivation. The maximal product yield was achieved for periodic variations of the gas flows, ensuring a balance between available adsorption sites and the concentration of activated intermediates. This methodology opens a perspective for the optimization of catalytic reactions under nonstationary conditions.

Keywords: artificial intelligence, catalyst, co oxidation, reinforcement learning, dynamic control