Mathematical models of optimal execution of large orders under stochastic liquidity and price impact with a balance between slippage and risk

Abstract

The study addresses the problem of optimal execution of large exchange orders under stochastic liquidity and price impact, which constitute a significant share of transaction costs for institutional investors. The aim of the work is to construct a mathematical model that enables the identification of a dynamic balance between minimizing slippage and controlling execution risk, taking into account random fluctuations in market depth, spread, and volatility. The empirical basis relies on high-frequency data on the most liquid technology sector stocks traded on a major U.S. exchange over a multi-year period, ensuring parameter estimates that are both representative and stable. Liquidity is modeled using a stationary mean-reverting process, while price impact is decomposed into temporary and permanent components with a nonlinear dependence on trade size. The optimal execution trajectory is derived from a stochastic control problem based on the Hamilton–Jacobi–Bellman equation, where the objective function aggregates the expected cost and variance of execution value, accounting for the investor’s risk aversion coefficient. The effectiveness of the proposed model is evaluated through numerical simulations and compared with common benchmarks, including time-weighted average execution, volume-profile-based strategies, and the classical static model based on deterministic liquidity. The results show a significant reduction in average slippage and execution risk, particularly during periods of elevated market volatility, as well as a pronounced dependence of the optimal trading horizon and aggressiveness on the risk aversion parameter. In addition, the model’s behavior across different market regimes, classified by volatility levels of key indicators, is analyzed, revealing that the advantage of the adaptive approach increases with rising uncertainty and decreasing available liquidity. The study also compares the cost structure under temporary and permanent price impacts, demonstrating the model’s ability to concentrate trading volume within favorable liquidity windows and reduce pressure on the order book. The practical applicability of the findings is discussed in the context of tuning algorithmic strategies for institutional investors, integrating the model into risk management systems, developing stress-testing protocols, and selecting parameters based on market regime and mandate characteristics. Directions for further research include enriching the stochastic structure of liquidity, employing machine learning methods to classify market regimes, and accounting for asymmetry in price impact as well as higher-order microstructural effects.