Speakers
Description
In reinforcement learning, an agent interacts sequentially with an environment to maximize a reward, receiving only partial, probabilistic feedback. This creates a fundamental exploration-exploitation trade-off: the agent must explore to learn the hidden dynamics while exploiting this knowledge to maximize its target objective. While this framework has been extensively studied classically, applying it to quantum systems requires dealing with hidden quantum states that evolve under unknown dynamics. We formalize this problem via a framework in which the environment maintains a hidden quantum memory that evolves via unknown quantum channels, and the agent intervenes sequentially via quantum instruments. For this setting, we adapt an optimistic maximum-likelihood estimation algorithm. We extend the analysis to continuous action spaces, enabling us to model general positive-operator-valued measures (POVMs). By controlling the propagation of estimation errors through quantum channels and instruments, we prove that the cumulative regret of our strategy scales as $\widetilde{\mathcal{O}}(\sqrt{K})$ over $K$ episodes. Furthermore, via reduction to the multi-armed quantum bandit problem, we establish information-theoretic lower bounds demonstrating that this scaling is optimal up to polylogarithmic factors. As a physical application, we consider state-agnostic work extraction. When extracting free energy from a sequence of non-i.i.d.\ quantum states correlated by a hidden memory, any lack of knowledge about the source leads to thermodynamic dissipation. In our setting, the mathematical regret exactly quantifies this cumulative dissipation. Using our adaptive algorithm, the agent uses past energy outcomes to improve its extraction protocol on the fly, achieving sublinear cumulative dissipation and an asymptotically zero dissipation rate.
| Topic | Theoretical foundations of quantum learning |
|---|---|
| Additional Topic | Quantum state and process tomography with learning-based approaches |