Efficiency Bound of Learning with Coarse Graining

Minghao Li(李明昊), Shihao Xia(夏世豪), Youlin Wang(王有林),\\ Minglong Lv(律明龙), Jincan Chen(陈金灿), and Shanhe Su(苏山河)$^{\hyperlink{s}{*}}$

doi:10.1088/0256-307X/40/11/110501

⇦Chinese Physics Letters, 2023, Vol. 40, No. 11, Article code 110501⇨ Efficiency Bound of Learning with Coarse Graining Minghao Li (李明昊), Shihao Xia (夏世豪), Youlin Wang (王有林), Minglong Lv (律明龙), Jincan Chen (陈金灿), and Shanhe Su (苏山河)* Affiliations Department of Physics, Xiamen University, Xiamen 361005, China Received 6 August 2023; accepted manuscript online 25 September 2023; published online 25 October 2023 ^*Corresponding author. Email: sushanhe@xmu.edu.cn Citation Text: Li M H, Xia S H, Wang Y L et al. 2023 Chin. Phys. Lett. 40 110501 Abstract A thermodynamic formalism describing the efficiency of information learning is proposed, which is applicable to stochastic thermodynamic systems with multiple internal degrees of freedom. The learning rate, entropy production rate and entropy flow from the system to the environment under coarse-grained dynamics are derived. The Cauchy–Schwarz inequality is applied to demonstrate the lower bound on the entropy production rate of an internal state. The inequality of the entropy production rate is tighter than the Clausius inequality, leading to a derivative of the upper bound on the efficiency of learning. The results are verified in cellular networks with information processes.

DOI:10.1088/0256-307X/40/11/110501 © 2023 Chinese Physics Society Article Text Thermodynamic inequalities and uncertainties have a wide range of applications in many fields.[1-11] The role of information in thermodynamics is currently an active research topic. The Landauer principle has stated that information erasure requires heat dissipation,[12] which has been experimentally verified.[13-15] The Markov approximation of dynamics provides a versatile tool for characterizing information in stochastic thermodynamic systems.[16-26] The efficiency of information exchange was defined as the information transfer over the total entropy production for a pair of Brownian particles.[27] The rate of conditional Shannon entropy reduction describing the learning of the internal process about the external process has been proved to be bounded by thermodynamic entropy production.[28-30] Despite significant progress in informational thermodynamics, one of unresolved question is whether a fundamental bound of irreversible entropy production exists in thermodynamic processes involving information. Providing such a relationship would be beneficial for a better understanding of the trade-off between the learning rate and the energy dissipation. In this work, the lower limit of the irreversible entropy production rate of thermodynamic systems with multiple internal degrees of freedom is demonstrated and applied to reveal the learning efficiency of cellular networks. Using the Markovian master equation under coarse graining and the Cauchy–Schwartz inequality, we derive an inequality associated with the entropy flow from the system to the environment and the entropy production rate. A tighter upper bound for the learning efficiency for one of the internal states is then obtained. Through a single receptor model and an adaptive cellular network, the learning efficiency of the biological information processing and its upper bound are verified. Finally, we draw our conclusions. Stochastic Thermodynamics under Coarse Graining and Learning Efficiency. We first briefly introduce the stochastic thermodynamics of an information processing system inspired by the Escherichia coli sensory network. The evolution of the system is modeled by the Markovian master equation \begin{align} \dot{p}(y,x)=\sum_{x^{\prime},y^{\prime}}\Big[w_{x^{\prime}x}^{y^{\prime}y}p(y^{\prime},x^{\prime}) -w_{xx^{\prime}}^{yy^{\prime}}p(y,x)\Big], \tag {1} \end{align} where $p(y,x)$ is the probability of the system in the discrete state $(y,x)$, $x$ is an external state unaffected by the internal state $y$, $w_{x^{\prime}x}^{y^{\prime}y}$ represents the transition rate from state $(y^{\prime},x^{\prime})$ to state $(y,x)$. Note that the overdot notation in this work is used to emphasize that the quantity is a rate. By assuming that the external and internal states never jump simultaneously, $w_{x^{\prime}x}^{y^{\prime}y}$ is simplified to \begin{align} w_{x^{\prime}x}^{y^{\prime}y}=\begin{cases} w_{x'x}^{y},&x\neq x',~y=y',\\ w_{y'y}^{x},&x=x',~y\neq y',\\ 0,&x\neq x',~y\neq y', \end{cases} \tag {2} \end{align} where $w_{x'x}^{y}$ is the transition rate of the external state from $x'$ to $x$ given that the internal state is $y$, and similar to $w_{y'y}^{x}$. Considering an internal process comprising two variables $y=(y_{1},y_{2})$, we have the transition rate $w_{y'y}^{x}=w_{(y_{1}',y_{2}')(y_{1},y_{2})}^{x}$. The internal state $y_{1}$ may be indirectly connected to the external state $x$ via the other internal state $y_{2}$. Using Eqs. (1) and (2), we can express the dynamics of the internal state $y_{1}$ as follows: \begin{align} \dot{p}(y_{1},x)=\sum_{y_{2}}\dot{p}(y_{1},y_{2},x)=\sum_{y_{1}'}J_{y_{1}'y_{1}}^{x}, \tag {3} \end{align} where the marginal probability $p(y_{1},x)$ is obtained by summing $p(y_{1},y_{2},x)$ over the variable $y_{2}$, $J_{y_{1}'y_{1}}^{x}=W_{y_{1}'y_{1}}^{x}p(y_{1}',x)-W_{y_{1}y_{1}'}^{x}p(y_{1},x)$, and \begin{align} W_{y_{1}y_{1}'}^{x}=\sum_{y_{2},y_{2}'}w_{(y_{1},y_{2})(y_{1}',y_{2}')}^{x}\frac{p(y_{1},y_{2},x)}{p(y_{1},x)} \tag {4} \end{align} denotes the coarse-grained transition rate from state $(y_{1},x)$ to state $(y_{1}^{\prime},x)$. With the help of Eq. (3), the time derivative of the Shannon entropy $\dot{S}^{y_{1}}=-\sum_{y_{1}}\dot{p}(y_{1})\mathrm{{\ln}}p(y_{1})$ of the internal state $y_{1}$ can be decomposed into three terms,[31] i.e., \begin{align} \dot{S}^{y_{1}}=\dot{\sigma}^{y_{1}}-\dot{S}_{r}^{y_{1}}+\dot{l}^{y_{1}}, \tag {5} \end{align} where $\dot{\sigma}^{y_{1}}$ and $\dot{S}_{r}^{y_{1}}$ are, respectively, the thermodynamic entropy production rate and the entropy flow from the system to the environment associated with the coarse-grained dynamics of state $y_{1}$, which are defined as \begin{align} \tag {6} \dot{\sigma}^{y_{1}}=\,&\frac{1}{2}\sum_{y_{1},y_{1}',x}J_{y_{1}'y_{1}}^{x}\mathrm{{\ln}} \frac{W_{y_{1}'y_{1}}^{x}p(y_{1}',x)}{W_{y_{1}y_{1}'}^{x}p(y_{1},x)},\\ \tag {7} \dot{S}_{r}^{y_{1}}=\,&\frac{1}{2}\sum_{y_{1},y_{1}',x}J_{y_{1}'y_{1}}^{x}\mathrm{{\ln}}\frac{W_{y_{1}'y_{1}}^{x}}{W_{y_{1}y_{1}'}^{x}}; \end{align} $\dot{\sigma}^{y_{1}}$ is always positive because of $W_{y_{1}'y_{1}}^{x}p(y_{1}',x)\geq0$ and $J_{y_{1}'y_{1}}^{x}\mathrm{{\ln}}\frac{W_{y_{1}'y_{1}}^{x}p(y_{1}',x)}{W_{y_{1}y_{1}'}^{x}p(y_{1},x)}\geq0$;[15] \begin{align} \dot{l}^{y_{1}}=\frac{1}{2}\sum_{y_{1},y'_{1},x}J_{y'_{1}y_{1}}^{x}\mathrm{{\ln}}\frac{p(x|y_{1})}{p(x|y_{1}')} \tag {8} \end{align} quantifies the rate of the coarse-grained internal process $y_{1}$ learning about $x$ with $p(x|y_{1})=\frac{p(x,y_{1})}{p(y_{1})}=\frac{p(x,y_{1})}{\sum_{x}p(x,y_{1})}$ being the conditional probability. Similar to Eq. (6), the thermodynamic entropy production rate, related to the total internal state $y=(y_{1},y_{2})$, is defined as \begin{align} \dot{\sigma}^{y}=\frac{1}{2}\sum_{y,y',x}J_{y'y}^{x}\mathrm{{\ln}} \frac{p(y',x)w_{y'y}^{x}}{p(y,x)w_{yy'}^{x}}, \tag {9} \end{align} with $J_{y'y}^{x}=w_{y'y}^{x}p(y',x)-w_{yy'}^{x}p(y,x)$. The log-sum inequality $\sum_{i}u_{i}\mathrm{{\ln}}\frac{u_{i}}{v_{i}}\geqslant\sum_{i}u_{i}\mathrm{{\ln}}\frac{\sum_{i}u_{i}}{\sum_{i}v_{i}}(\forall u_{i},v_{i}\geq0)$[32] implies \begin{align} \dot{\sigma}^{y}\geq\dot{\sigma}^{y_{1}}. \tag {10} \end{align} This result can also be understood as the process of the change of state $y_{1}$ is a part of the internal process. In the cellular network, when $\dot{l}^{y_{1}}\geq0$, the cell creates information by perceiving the change of the external state $x$. The information learned will be consumed by the cell for driving the internal process. Following Refs. [25,29,33], we define a learning efficiency $\eta^{y_{1}}$ for biological information processing, which is given by \begin{align} \eta^{y_{1}}=\frac{\dot{l}^{y_{1}}}{\dot{S}_{r}^{y_{1}}}. \tag {11} \end{align} Under the condition of a steady state, the change rate of the probability $\dot{p}(y_{1},x)=0$, leading to the time derivative of the Shannon entropy $\dot{S}^{y_{1}}=0$. Then, the entropy production rate $\dot{\sigma}^{y_{1}}=\dot{S_{r}^{y_{1}}}-\dot{l}^{y_{1}}\geq0$ indicates that $\eta^{y_{1}}=\frac{\dot{l}^{y_{1}}}{\dot{S}_{r}^{y_{1}}}\leq1$. By using the Cauchy–Schwartz inequality $\sum_{i}a_{i}^{2}\sum_{i}b_{i}^{2}\geqslant(\sum_{i}a_{i}b_{i})^{2}$ and the logarithmic inequality $\frac{(x-y)^{2}}{x+y}\leqslant\frac{x-y}{2}{\log}\frac{x}{y}$,[26] we derive the following inequality associated with $\dot{S}_{r}^{y_{1}}$ and $\dot{\sigma}^{y_{1}}$, \begin{align} |\dot{S}_{r}^{y_{1}}|\leq\sqrt{\theta^{y_{1}}\dot{\sigma}^{y_{1}}}, \tag {12} \end{align} where \begin{align}\theta^{y_{1}}=\frac{1}{2}\sum_{y_{1},y'_{1},x} \Big(\mathrm{{\ln}}\frac{W_{y_{1}'y_{1}}^{x}}{W_{y_{1}y_{1}'}^{x}}\Big)^{2}W_{y_{1}'y_{1}}^{x}p (y_{1}',x).\nonumber \end{align} In the case of $\dot{S}_{r}^{y_{1}}\geq0$ and $\dot{l}^{y_{1}}\geq0$, a tighter upper bound for learning efficiency is obtained, i.e., \begin{align} \eta^{y_{1}}=\frac{\dot{l}^{y_{1}}}{\dot{S}_{r}^{y_{1}}}\leqslant1-\frac{\dot{S}_{r}^{y_{1}}}{\theta^{y_{1}}}. \tag {13} \end{align} This bound, which is the main result of this work, is applicable to many systems as long as the principle of detailed balance is satisfied. In the following, the above result will be applied to analyze cellular networks. By using the coarse-graining process, the learning efficiency for biological information processing and its upper bound can be determined. The costs and thermodynamic efficiencies for various cellular networks learning about an external random environment will be well characterized. The upper bound of the learning efficiency is universal for the evolution of a system governed by the Markovian master equation. This is because the derivation of the bound is generally taken from the Cauchy–Schwartz inequality and the logarithmic inequality. It was proved in Ref. [31] that the upper bound can be applied to quantum-dot systems as well. The Learning Efficiencies of Cellular Networks—Single Receptor Model. In the cell, E. coli receptors are placed at the membrane and have a ligand-binding site. The kinase is connected to the receptor and its activity depends on the binding of external ligands. The kinase in its active form acts as an enzyme for the phosphorylation reaction of the protein. Here, we consider a single receptor accounting for the indirect regulation of the kinase activity by the binding events. The equivalent eight-state network of the single receptor model is illustrated in Fig. 1(a). Each state of the system is described by $(a,b,c)$. The internal process due to the change of the internal state $y=(a,b)$ corresponds to a four-state network, as shown in Fig. 1(b). The receptor is occupied by an external ligand at $b=1$ or unoccupied at $b=0$. For a given external ligand concentration $c$, the free energy difference between the occupied and unoccupied states is given by \begin{align} F(a,1,c)-F(a,0,c)=\ln(K_{a}/c), \tag {14} \end{align} where $F(a,b,c)$ represents the free energy of state $(a,b,c)$, $K_{a}$ is the dissociation constant, $\ln K_{a}$ is associated with a change in the free energy of the receptor and $\ln c$ denotes the chemical potential of taking a particle from the solution in a binding event.[29] The subscript in $K_{a}$ reflects the interaction between the receptor and the kinase. The activity increases the dissociation constant by considering $K_{1}>K_{0}$. The state of the kinase attached to the receptor may be inactive $a=0$ or active $a=1$, resulting in a conformational change in the receptor. The free-energy difference between the active and inactive states with fixed values of $b$ and $c$ is assumed to be $\Delta E$, i.e., \begin{align} F(1,b,c)-F(0,b,c)=\Delta E. \tag {15} \end{align} Combining Eq. (14) with Eq. (15), we can define the free energy of state $(a,b,c)$ as \begin{align} F(a,b,c)=a\Delta E+b\ln(K_{a}/c). \tag {16} \end{align} The transition rate $w_{(a,b)(a',b^{\prime})}^{c}$ from state $(a,b,c)$ to state $(a^{\prime},b^{\prime},c)$ given in Fig. 1(b) satisfies the detailed balance condition \begin{align} \ln[w_{(a,b)(a^{\prime},b^{\prime})}^{c}/w_{(a^{\prime},b^{\prime})(a,b)}^{c}]=F(a,b,c)-F(a^{\prime},b^{\prime},c). \tag {17} \end{align} In Fig. 1(b), $\gamma_{a}$ and $\gamma_{b}$ are, respectively, the timescales of the conformational change and the binding event. The timescale of the binding event is assumed to be much smaller than that of the conformational changes, i.e., $\gamma_{b}\gg\gamma_{a}$. In addition to the four-state subsystem in Fig. 1(b), the full model includes the transition of the external ligand concentrations between concentrations $c_{1}$ and $c_{2}$ at rate $\gamma_{c}$ [green dashed arrows in Fig. 1(a)].

Fig. 1. (a) Schematic diagram of a single receptor model. (b) For a given external ligand concentration of $c$, the transition rates correspond to the internal processes (left), and schematic diagram of a four-state coarse-graining model by summing out the variable $b$ (right).

By summing out the variable $b$, a coarse-grained model with four states is shown by in Fig. 1(b) (right). The coarse-grained transition rate from state $(a,c)$ to $(a^{\prime},c)$ under the external ligand concentration $c$ is given by \begin{align} W_{aa^{\prime}}^{c}=\sum_{b,b^{\prime}}w_{(a,b)(a^{\prime},b^{\prime})}^{c} \frac{p(a,b,c)}{p(a,c)}, \tag {18} \end{align} where the probability $p(a,c)$ of coarse-grained state $(a,c)$ is calculated by the summation $\sum_{b}p(a,b,c)$. In the limit $\gamma_{a}/\gamma_{b}\rightarrow0$, Eq. (18) is simplified to \begin{align} W_{aa^{\prime}}^{c}=\sum_{b,b^{\prime}}w_{(a,b)(a^{\prime},b^{\prime})}^{c} \frac{(c/K_{a})^{b}}{1+c/K_{a}}, \tag {19} \end{align} with $(c/K_{a})^{b}/(1+c/K_{a})$ corresponding to the stationary condition probability $p(b|a,c)=\frac{p(a,b,c)}{p(a,c)}$. The free-energy difference between state $(1,c)$ and state $(0,c)$ due to the conformational change is then given by \begin{align} F(1,c)-F(0,c)=\ln\frac{W_{10}^{c}}{W_{01}^{c}}=\Delta E+\ln\Big(\frac{1+\frac{c}{K_{0}}}{1+\frac{c}{K_{1}}}\Big). \tag {20} \end{align} According to Eqs. (6) and (7) and under the stationarity condition, the entropy production rate associated with the coarse-grained dynamics of state $a$ is given by \begin{align} \dot{\sigma}^{a} & =\sum_{a,a^{\prime},c}W_{a^{\prime}a}^{c}p(a^{\prime},c)\mathrm{{\ln}} \frac{W_{a^{\prime}a}^{c}p(a^{\prime},c)}{W_{aa^{\prime}}^{c}p(a,c)} \notag\\ &=\dot{S}_{r}^{a}-\dot{l}^{a}, \tag {21} \end{align} where the entropy flow from the cell to the environment related to the change of $a$ reads \begin{align} \dot{S}_{r}^{a}=\sum_{a,a^{\prime},c}W_{a^{\prime}a}^{c}p(a^{\prime},c)\mathrm{{\ln}} \frac{W_{a^{\prime}a}^{c}}{W_{aa^{\prime}}^{c}}, \tag {22} \end{align} and the rate of the coarse-grained internal process $a$ learning about the change of the external ligand concentration $c$ reads \begin{align} \dot{l}^{a} & =-\sum_{a,a^{\prime},c}W_{a^{\prime}a}^{c}p(a^{\prime},c)\mathrm{{\ln}}\frac{p(a^{\prime},c)}{p(a,c)} \notag\\ &=\gamma_{c}\sum_{a}\big[p(a,c_{2})-p(a,c_{1})\big]\mathrm{{\ln}}\frac{p(a,c_{2})}{p(a,c_{1})}. \tag {23} \end{align} By applying Eqs. (11)-(13), the efficiency $\eta^{a}$ of the dynamics of the kinase in learning about the change of the external ligand concentration is \begin{align} \eta^{a}=\frac{\dot{l}^{a}}{\dot{S}_{r}^{a}}\leqslant1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}, \tag {24} \end{align} where the coefficient \begin{align} \theta^{a}=\frac{1}{2}\sum_{a,a',c}\Big(\mathrm{{\ln}}\frac{W_{a'a}^{c}}{W_{aa'}^{c}}\Big)^{2}W_{a'a}^{c}p(a',c). \tag {25} \end{align} By taking the limit $\gamma_{a}/\gamma_{b}\rightarrow0$ and using Eq. (19), the entropy flow in Eq. (22) is simplified to \begin{align} \!\dot{S}_{r}^{a}=\gamma_{c}[p(0,c_{2})-p(0,c_{1})]\mathrm{{\ln}} \Big[\Big(\frac{1+\frac{c_{1}}{K_{0}}}{1+\frac{c_{1}}{K_{1}}}\Big) \Big(\frac{1+\frac{c_{2}}{K_{1}}}{1+\frac{c_{2}}{K_{0}}}\Big)\Big], \tag {26} \end{align} and the coefficient in Eq. (25) becomes \begin{align} \theta^{a} =\,&\frac{1}{2}\sum_{c}\gamma_{a}\Big[\Delta E+\ln\Big(\frac{1+\frac{c}{K_{0}}}{1+\frac{c}{K_{1}}}\Big)\Big]^{2}\Big(1+\frac{c}{\sqrt{K_{0}K_{1}}}\Big) \notag\\ &\cdot\Big(\frac{e^{\Delta E/2}p(1,c)}{1+c/K_{1}}+\frac{e^{-\Delta E/2}p(0,c)}{1+c/K_{0}}\Big). \tag {27} \end{align} Using Eqs. (22)-(27), we plot the learning efficiency $\eta^{a}$ versus $\gamma_{c}$ and $\Delta E$ in Fig. 2(a). It is shown that the relation $\eta^{a}\leqslant1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}$ always holds, which demonstrates that the bound in Eq. (24) is tighter than 1. When $\Delta E$ is fixed, $\eta^{a}$ decreases monotonically with increase in $\gamma_{c}$. If $\gamma_{c}\gg\gamma_{a}$, the external concentration jumps so quickly that the intracellular kinase activity cannot accurately track the change in the external environment, leading to the learning rate $\dot{l}^{a}\rightarrow0$ [Fig. 2(b)]. Consequently, $\eta^{a}$ is very small and deviates from the upper bound $1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}$. When $\Delta E$ is larger than $-2.2$ or smaller than $-12.5$, the entropy flow $\dot{S}_{r}^{a}$ [Fig. 2(c)] and the coefficient $\theta^{a}$ [Fig. 2(d)] are not very sensitive to the variation of $\gamma_{c}$, such that $1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}$ does not change significantly with respective to $\gamma_{c}$. For a $\Delta E$ between $-12.5$ and $-2.2$, the increase of $\gamma_{c}$ results in dramatic changes in the coefficient $\theta^{a}$ [Fig. 2(d)] and $1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}$ [Fig. 2(a)]. If $\gamma_{c}$ is much smaller than $\gamma_{a}$, both the actual efficiency $\eta^{a}$ and the upper bound $1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}$ approach unity, as the learning rate $\dot{l}^{a}$ [Fig. 2(b)] and the entropy flow $\dot{S}_{r}^{a}$ [Fig. 2(c)] simultaneously tend to zero. For a given value of $\gamma_{c}$, the values of the learning efficiency $\eta^{a}$ and the upper bound $1-\dot{S}_{r}^{a}/\theta^{a}$ first decrease and then increase with the increase of $\Delta E$ [Fig. 2(a)]. Compared with $\eta^{a}$, $1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}$ is more sensitive to the change of $\Delta E$, because $\theta^{a}$ as a function of $\Delta E$ displays a bimodal structure with large fluctuation [Fig. 2(d)]. There exists a point of $\Delta E$ where $1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}$ reaches its minimum value close to $\eta^{a}$ due to the fact that $\theta^{a}$ approaches the lower limit [Fig. 2(b)] and $\dot{S}_{r}^{a}$ is relatively large [Fig. 2(c)]. By varying with $\Delta E$, Figs. 2(b) and 2(c) indicate that both $\dot{l}^{a}$ and $\dot{S}_{r}^{a}$ will have their respective maximum values.

Fig. 2. (a) The learning efficiency $\eta^{a}$ and the upper limit $1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}$ of the single receptor model varying with the transition rate $\gamma_{c}$ and the free energy difference $\Delta E$ between the active and inactive states of CheA. (b) The learning rate $\dot{l}^{a}$ varying with $\gamma_{c}$ and $\Delta E$. (c) The entropy flow $\dot{S}_{r}^{a}$ from the cell to the environment related to the change of $a$ varying with $\gamma_{c}$ and $\Delta E$. (d) The coefficient $\theta^{a}$ varying with $\gamma_{c}$ and $\Delta E$. The other parameters are $K_{0}=1/400$, $K_{1}=400$, $\gamma_{a}=1$, $\gamma_{b}=1000$, $c_{1}=1/3$, and $c_{2}=3$.

Model with Adaptation. Adaptation, in a biological sense, refers to a characteristic of an organism that makes it fit for its environment. We consider an adaptive model (Fig. 3) that incorporates the methylation level $m$,[29] which is a self-regulation factor for the activity $a$ of the kinase. In this model, the activity $a=1$ if the kinase is active and $a=0$ if it is inactive, as shown in Fig. 3. Without the existence of external ligands, the average value of $a$ is assumed to be equal to $1/2$. A change in the external ligand concentration $c$ quickly changes $a$ at a timescale $\gamma_{a}^{-1}$, while the methylation level $m$ provides the adaption effect by adjusting the average of $a$ back to $1/2$ at a timescale $\gamma_{m}^{-1}\gg\gamma_{a}^{-1}$. More specifically, a decrease in $c$ leads to a fast growth of $a$. The change in $a$ leads to a slow decrease in the methylation level $m$, which acts back on the activity by slowly reducing $a$ to $1/2$. In the internal process of the cell, the methylation level $m$ and the kinase activity $a$ form a negative feedback network that maintains $a$ at a stable level.

Fig. 3. (a) Schematic diagram of the internal process in the adaptive model. (b) The full model containing the jump of the external ligand concentration.

By considering that the binding event is faster than the conformational change and integrating out the variable $b$, the free energy difference between the active state $(1,m,c)$ and the inactive state $(0,m,c)$ is obtained from Eq. (20), i.e., \begin{align} F(1,m,c)-F(0,m,c)=\Delta E(m)+\ln\Big(\frac{1+\frac{c}{K_{0}}}{1+\frac{c}{K_{1}}}\Big), \tag {28} \end{align} where the energy dependence on the methylation level \begin{align} \Delta E(m)\equiv-\frac{m}{4}\ln\frac{K_{1}}{K_{0}}. \tag {29} \end{align} For simplicity, it is assumed that the free energy of state $(a,m,c)$ \begin{align} F(a,m,c)=a\Delta E(m)+a\ln\Big(\frac{1+\frac{c}{K_{0}}}{1+\frac{c}{K_{1}}}\Big). \tag {30} \end{align} The methylation level $m$ is controlled by chemical reactions. If $a=0$, the receptor may be methylated due to the reaction \begin{align} [m]_{0}+\mathrm{SAM}\rightleftharpoons[m+1]_{0}+\mathrm{SAH}, \tag {31} \end{align} where SAM is a shortened form of the S-adenosyl methionine molecule and SAH represents an abbreviation of the S-adenosyl-L-homocysteine molecule, and the subscript denotes the value of $a$. The free-energy difference of the reaction is given by $\mu_{\mathrm{SAM}}-\mu_{\mathrm{SAH}}$. However, if $a=1$, the receptor may be demethylated with the reaction \begin{align} [m+1]_{1}+\mathrm{H}_{2}\mathrm{O}\rightleftharpoons[m]_{1}+\mathrm{CH}_{3}\mathrm{OH}, \tag {32} \end{align} where the free-energy difference \begin{align} &\mu_{\mathrm{H}_{2}\mathrm{O}}-\mu_{\mathrm{CH}_{3}\mathrm{OH}}+F(1,m+1,c)-F(1,m,c) \notag\\ =\,&\mu_{\mathrm{H}_{2}\mathrm{O}}-\mu_{\mathrm{CH}_{3}\mathrm{OH}} -\frac{1}{4}\ln\frac{K_{1}}{K_{0}}. \tag {33} \end{align} The chemical potential difference \begin{align} \Delta\mu=\mu_{\mathrm{SAM}}+\mu_{\mathrm{H}_{2}\mathrm{O}}-\mu_{\mathrm{SAH}}-\mu_{\mathrm{CH}_{3}\mathrm{OH}} \tag {34} \end{align} provides the affinity for driving the internal process. The variable $(a,m)$ defines the internal process. Schematic diagrams of the internal processes in the adaptive model and each transition rate between difference states are given in Fig. 3(a). The full model of the jump of the external ligand concentration with rate $\gamma_{c}$ between $c_{1}$ and $c_{2}$ is presented in Fig. 3(b). In Fig. 3(a), the transition rate $w_{(1,m)(0,m)}^{c}$ from the active state $(1,m,c)$ and the inactive state $(0,m,c)$ is calculated by Eq. (19), where the only difference is that the energy dependence on the methylation level in Eq. (29) is taken into account. In addition, the transition rates between state $(1,m,c)$ and state $(0,m,c)$ are related to the free-energy difference in Eq. (28) as \begin{align} F(1,m,c)-F(0,m,c)=\ln\frac{w_{(1,m)(0,m)}^{c}}{w_{(0,m)(1,m)}^{c}}. \tag {35} \end{align} The length of the arrow in Fig. 3(a) indicates the relative magnitude of $w_{(1,m)(0,m)}^{c}$ and $w_{(0,m)(1,m)}^{c}$. The formulas of the transition rates between state $(a,m,c)$ and state $(a,m+1,c)$ are presented in Fig. 3(a), ensuring that the adaptation effect happens if $\Delta\mu$ overcomes the affinity in the internal cycle $\Delta\mu^{*}=\frac{1}{4}\ln\frac{K_{1}}{K_{0}}$. In other words, the increase of $a$ reduces the methylation level $m$, which in turn tends to reduce the value of $a$.

Fig. 4. For the maximum methylation level $m_{\max}=2,\,3,\,4,\,5$, the average $\langle a\rangle $ of state $a$ varying with the transition rate $\gamma_{c}$ and the chemical potential difference $\Delta\mu$. For the model with adaptation, the parameters are $K_{0}=1/400$, $K_{1}=400$, $\gamma_{a}=1$, $\gamma_{m}=10^{-2}$, $c_{1}=1/3$, and $c_{2}=3$.

By using Eqs. (21)-(23), the coarse-graining entropy production rate $\dot{\sigma}^{a}$, the entropy flow $\dot{S}_{r}^{a}$ from the cell to the environment, and the learning rate $\dot{l}^{a}$ associated with the change of the internal state $a$ in the adaptive model can be computed. The only difference is that the coarse-grained transition rate from state $(a,c)$ to $(a^{\prime},c)$ under the external ligand concentration $c$ is given by \begin{align} W_{aa^{\prime}}^{c}=\sum_{m,m^{\prime}}w_{(a,m)(a^{\prime},m^{\prime})}^{c}\frac{p(a,m,c)}{p(a,c)}, \tag {36} \end{align} where the probability $p(a,c)$ of coarse-grained state $(a,c)$ is determined by the summation $\sum_{m}p(a,m,c)$. In the same way, the efficiency $\eta^{a}$ of the dynamics of the internal state $a$ in learning about the change of external ligand concentration and its upper limit are computed by Eqs. (24) and (25). In the adaptive model, the intracellular kinase activity $a$ will respond to the change in the external ligand concentration $c$. This process is indirectly regulated by altering the methylation level $m$. For the maximum methylation level $m_{\max}=2,\,3,\,4,\,5$, Fig. 4 plots the average $\langle a\rangle $ of state $a$ varying with the transition rate $\gamma_{c}$ and the chemical potential difference $\Delta\mu$. For a given level of methylation level $m$, both $\gamma_{c}$ and $\Delta\mu$ affect the activity of the kinase. When $m_{\max}=4$, the average $\langle a\rangle $ of state $a$ is more inclined to maintain around $1/2$ even with large variations of $\gamma_{c}$ and $\Delta\mu$. That is to say, the feedback network in Fig. 3 is capable of maintaining $a$ at a stable level by optimizing the methylation level $m$.

Fig. 5. (a) The learning efficiency $\eta^{a}$ and the upper limit $1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}$ of the model with adaptation versus the transition rate $\gamma_{c}$ and chemical potential difference $\Delta\mu$. (b) The learning rate $\dot{l}^{a}$ versus $\gamma_{c}$ and $\Delta\mu$. (c) The entropy flow $\dot{S}_{r}^{a}$ from the cell to the environment related to the change of $a$ versus $\gamma_{c}$ and $\Delta\mu$. (d) The coefficient $\theta^{a}$ versus $\gamma_{c}$ and $\Delta\mu$.

By applying Eqs. (22)-(25) and choosing $m_{\max}=4$, Fig. 5 plots the learning efficiency $\eta^{a}$ and the upper bound $1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}$ for the adaptive model versus the transition rate $\gamma_{c}$ and the chemical potential difference $\Delta\mu$. As illustrated in Fig. 5(a), for a fixed value of $\Delta\mu$, $\eta^{a}$ and its upper bound $1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}$ decrease monotonically as $\gamma_{c}$ increases, while the learning rate $\dot{l}^{a}$, entropy flow $\dot{S}_{r}^{a}$, and coefficient $\theta^{a}$ appear to be nonmonotonic functions of $\gamma_{c}$. When $\gamma_{c}$ is fixed, $\eta^{a}$ and $1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}$ are not sensitive to the change of $\Delta\mu$, as $\dot{l}^{a}$, $\dot{S}_{r}^{a}$, and $\theta^{a}$ monotonically increase with the increase of $\Delta\mu$ at the same time. The increase of $\Delta\mu$ will accelerate the transition rates between different methylation levels, which makes the response of $m$ to the change in the external concentration $c$ more sensitive. This enables the internal state $a$ learning more external information per unit time, but also makes the network generate more dissipation. It is important to observe that $\eta^{a}$ is always smaller than $1-\frac{\dot{S}_{r}^{a}}{\theta^{a}}$, which provides evidence for the validity of Eq. (24). In summary, we have developed a coarse-graining approach to quantify the upper bound of the efficiency of learning in multivariable systems, which is tighter than that given by the conventional second law of thermodynamics. By applying this approach to thermodynamic processes in cellular networks, the general applicability of our method is demonstrated. Our findings have important implications for understanding the fundamental principles governing the dynamics of complex systems. Acknowledgments. This work was supported by the National Natural Science Foundation of China (Grant No. 12075197), the Fundamental Research Fund for the Central Universities (Grant No. 20720210024), and the Natural Science Foundation of Fujian Province (Grant No. 2023J01006). References Geometric thermodynamic uncertainty relation in a periodically driven thermoelectric heat engineThermodynamic uncertainty relation for first-passage times on Markov chainsThermodynamic Uncertainty Relation for Biomolecular ProcessesUniversal Trade-Off between Power, Efficiency, and Constancy in Steady-State Heat EnginesFinite-Time Landauer PrincipleThermodynamic uncertainty relations constrain non-equilibrium fluctuationsOperationally accessible uncertainty relations for thermodynamically consistent semi-Markov processesThermodynamics of Statistical Inference by CellsUncertainty relations in stochastic processes: An information inequality approachThermodynamic Uncertainty Relations from Exchange Fluctuation TheoremsMapping of uncertainty relations between continuous and discrete timeIrreversibility and Heat Generation in the Computing ProcessExperimental demonstration of information-to-energy conversion and validation of the generalized Jarzynski equalityExperimental verification of Landauer’s principle linking information and thermodynamicsSingle-Atom Demonstration of the Quantum Landauer PrincipleFundamental Bounds on First Passage Time Fluctuations for CurrentsEmergence of local irreversibility in complex interacting systemsDissipation Bounds All Steady-State Current FluctuationsDecomposing the Local Arrow of Time in Interacting SystemsThree-terminal refrigerator based on resonant-tunneling quantum wellsThe free-energy cost of accurate biochemical oscillationsGeometrical Bounds of the Irreversibility in Markovian SystemsThermodynamics with Continuous Information FlowStochastic thermodynamics under coarse grainingStochastic thermodynamics of bipartite systems: transfer entropy inequalities and a Maxwell’s demon interpretationFundamental Relation Between Entropy Production and Heat CurrentThermodynamic efficiency of information and heat flowA unifying picture of generalized thermodynamic uncertainty relationsEfficiency of cellular information processingInformation-theoretic versus thermodynamic entropy production in autonomous sensory networksTheoretical bound of the efficiency of learningUniversal Bound on Energy Cost of Bit Reset in Finite TimeStochastic Thermodynamics of Learning

[1]	Lu J C, Wang Z, Peng J B, Wang C, Jiang J H, and Ren J 2022 Phys. Rev. B 105 115428
[2]	Pal A, Reuveni S, and Rahav S 2021 Phys. Rev. Res. 3 L032034
[3]	Barato A C and Seifert U 2015 Phys. Rev. Lett. 114 158101
[4]	Pietzonka P and Seifert U 2018 Phys. Rev. Lett. 120 190602
[5]	Proesmans K, Ehrich J, and Bechhoefer J 2020 Phys. Rev. Lett. 125 100602
[6]	Horowitz J M and Gingrich T R 2020 Nat. Phys. 16 15
[7]	Ertel B, van der Meer J, and Seifert U 2022 Phys. Rev. E 105 044113
[8]	Lang A H, Fisher C K, Mora T, and Mehta P 2014 Phys. Rev. Lett. 113 148103
[9]	Hasegawa Y and Van Vu T 2019 Phys. Rev. E 99 062126
[10]	Timpanaro A M, Guarnieri G, Goold J, and Landi G T 2019 Phys. Rev. Lett. 123 090604
[11]	Chiuchiù D and Pigolotti S 2018 Phys. Rev. E 97 032109
[12]	Landauer R 1961 IBM J. Res. Dev. 5 183
[13]	Toyabe S, Sagawa T, Ueda M, Muneyuki E, and Sano M 2010 Nat. Phys. 6 988
[14]	Bérut A, Arakelyan A, Petrosyan A, Ciliberto S, Dillenschneider R, and Lutz E 2012 Nature 483 187
[15]	Yan L, Xiong T, Rehan K, Zhou F, Liang D, Chen L, Zhang J, Yang W, Ma Z, and Feng M 2018 Phys. Rev. Lett. 120 210601
[16]	Gingrich T R and Horowitz J M 2017 Phys. Rev. Lett. 119 170601
[17]	Lynn C W, Holmes C M, Bialek W, and Schwab D J 2022 Phys. Rev. E 106 034102
[18]	Gingrich T R, Horowitz J M, Perunov N, and England J L 2016 Phys. Rev. Lett. 116 120601
[19]	Lynn C W, Holmes C M, Bialek W, and Schwab D J 2022 Phys. Rev. Lett. 129 118101
[20]	Lin Z B, Yang Y Y, Li W, Wang J H, and He J Z 2020 Phys. Rev. E 101 022117
[21]	Cao Y S, Wang H L, Ouyang Q, and Tu Y H 2015 Nat. Phys. 11 772
[22]	Van Vu T and Hasegawa Y 2021 Phys. Rev. Lett. 126 010601
[23]	Horowitz J M and Esposito M 2014 Phys. Rev. X 4 031015
[24]	Esposito M 2012 Phys. Rev. E 85 041125
[25]	Hartich D, Barato A C, and Seifert U 2014 J. Stat. Mech.: Theory Exp. 2014 P02016
[26]	Shiraishi N and Saito K 2019 J. Stat. Phys. 174 433
[27]	Allahverdyan A E, Janzing D, and Mahler G 2009 J. Stat. Mech.: Theory Exp. 2009 P09011
[28]	Barato A, Chetrite R, Faggionato A, and Gabrielli D 2019 J. Stat. Mech.: Theory Exp. 2019 084017
[29]	Barato A C, Hartich D, and Seifert U 2014 New J. Phys. 16 103024
[30]	Barato A, Hartich D, and Seifert U 2013 Phys. Rev. E 87 042104
[31]	Su S, Chen J, Wang Y, Chen J, and Uchiyama C 2022 arXiv:2209.08096 [cond-mat.stat-mech]
[32]	Zhen Y Z, Egloff D, Modi K, and Dahlsten O 2021 Phys. Rev. Lett. 127 190602
[33]	Goldt S and Seifert U 2017 Phys. Rev. Lett. 118 010601