⇦Chinese Physics Letters, 2020, Vol. 37, No. 9, Article code 090501 Rescaled Range Permutation Entropy: A Method for Quantifying the Dynamical Complexity of Extreme Volatility in Chaotic Time Series Jia-Chen Zhang (张嘉晨), Wei-Kai Ren (任卫凯), and Ning-De Jin (金宁德)* Affiliations School of electrical and information engineering, Tianjin University, Tianjin 300072, China Received 13 May 2020; accepted 24 July 2020; published online 1 September 2020 Supported by the National Natural Science Foundation of China (Grant Nos. 51527805 and 11572220).
*Corresponding author. Email: ndjin@tju.edu.cn Citation Text: Zhang J C, Ren W K and Jin N D 2020 Chin. Phys. Lett. 37 090501    Abstract Information entropy, as a quantitative measure of complexity in nonlinear systems, has been widely researched in a variety of contexts. With the development of a nonlinear dynamic, the entropy is faced with severe challenges in dealing with those signals exhibiting extreme volatility. In order to address this problem of weighted permutation entropy, which may result in the inaccurate estimation of extreme volatility, we propose a rescaled range permutation entropy, which selects the ratio of range and standard deviation as the weight of different fragments in the time series, thereby effectively extracting the maximum volatility. By analyzing typical nonlinear systems, we investigate the sensitivities of four methods in chaotic time series where extreme volatility occurs. Compared with sample entropy, fuzzy entropy, and weighted permutation entropy, this rescaled range permutation entropy leads to a significant discernibility, which provides a new method for distinguishing the complexity of nonlinear systems with extreme volatility. DOI:10.1088/0256-307X/37/9/090501 PACS:05.45.-a, 05.45.Tp, 89.70.Cf © 2020 Chinese Physics Society Article Text Quantifying the complexity of natural systems has always been an important task in the field of nonlinear dynamics.[1–3] Various measures of complexity have been developed to compare time series, including the Lyapunov exponent,[4,5] entropy, Kolmogorov complexity,[6] and fractal dimensions.[7,8] As a measure of the average uncertainty of information, entropy enables a sufficient indication of nonlinear dynamics for complex chaotic systems. Since Shannon summarized previous studies and proposed the concept of information entropy in 1948, a variety of quantitative methods of characterizing system complexity have been proposed, such as approximate entropy,[9,10] sample entropy (SE),[11] multivariate sample entropy (MSE), permutation entropy,[12] multi-scale entropy,[13] and multivariate fuzzy entropy (MFE),[14] which have been widely employed in the fields of biological and pharmaceutical signal analysis.[15–19] Permutation entropy (PE), proposed by Bandt and Pompe, has demonstrated greater robustness and more prominent classification capability than other entropies in terms of analyses of nonlinear signals.[20–22] However, PE ignores the amplitude information in the analysis of nonlinear time series. Bilal et al.[23] proposed the concept of weighted permutation entropy (WPE) for preserving the amplitude information of time series, which addressed the problems of weak resolution and robustness inherent in PE in respect of changes in sequence amplitude, and achieved good results in terms of practical application.[24–27] It was found, however, that adopting the variance as weight still experiences major limitations in terms of analyses of nonlinear signals with extreme volatility. Nonlinear systems with extreme volatility exist in a wide variety of fields, such as finance,[28,29] meteorology,[30–32] multiphase flow,[33,34] among others. Improving WPE has become an urgent problem to be solved. In this Letter, we use the ratio of the range and the standard deviation of the sequence fragment as the weight, so as to obtain a rescaled range permutation entropy (RSPE). Signal resolution performances of MSE, MFE, RSPE and WPE are investigated and compared with the numerical analyses of typical nonlinear chaotic systems. The results suggest that RSPE demonstrates the potential capacity to characterize nonlinear systems with extreme volatility. Firstly, we introduce the principles of the RSPE, and take $n$-variate time series defined as $$ \left\{ {x^{k}(t),x^{k}(t+T_{0}),\ldots,x^{k}\left[ {t+\left({L-1} \right)T_{0} } \right]} \right\},~~ \tag {1} $$ where $L$ represents the length, $T_{0}$ the sampling interval, and $k$ the $k$th channel in the $n$-variate time series. Each channel is then decomposed into fragments of length $m$. A fragment of any channel can be expressed as $$ X^{k}=\left\{ x^{k}(t),x^{k}(t+\tau),x^{k}(t+2\tau),\ldots,x^{k}[t+(m-1)\tau] \right\},~~ \tag {2} $$ in which $m$ is the embedding dimension, and $\tau$ denotes the delay time; $n[L-(m-1)\tau ]$ fragments are sorted from the original time series. Each fragment can be encoded according to the value of each element. An example is given as follows: When $m$ equals 6, with $\tau =1$, if a fragment is expressed as {3,6,7,4,9,2}, it could be encoded as {2,4,5,3,6,1}. Code 1 therefore indicates that the signal amplitude in this position is the smallest of all elements in this fragment, code 2 indicates that the signal amplitude in this position is the second smallest of all elements in this fragment, and so on, until we arrive at code $m$, which contains the largest signal amplitude in this fragment. Each fragment has a code, which means that there should be $m!$ codes at most. All fragments could be classified into $m!$ permutations. It is possible to number each permutation as $i$ and to express the number of the occurrence of fragments in each permutation as $N_{i}$. The probability of the occurrence of each permutation for all fragments can then be expressed as $$ P(i)=\frac{N_{i} }{n [ {L-({m-1})\tau }]},~~~1\leqslant i\leqslant m!.~~ \tag {3} $$ In order to reflect information relating to amplitude in the signal, the weight of each permutation of the sequence is accumulated. One may introduce the probability of each permutation containing information of amplitude appearing in the fragments of all channels as $$ P_{\omega } ({k,i})=\frac{\sum\nolimits_{j=1}^{N_{i} } {\omega_{i,j}^{k} } }{\sum\nolimits_{k=1}^n {\sum\nolimits_{i=1}^{m!} {\sum\nolimits_{j=1}^{N_{i} } {\omega_{i,j}^{k} } } } },~~ \tag {4} $$ where $$ \sum\limits_{k=1}^n {\sum\limits_{i=1}^{m!} {P_{\omega } ({k,i})} } =1.~~ \tag {5} $$ In formula (4), $\omega^{k}_{i,j}$ represents the weight of the $j$th fragment on the $i$th permutation in the $k$th channel. The WPE for a certain permutation of the signal primarily considers the variance as the weight.[23] This variance mainly shows the average property of each element in each fragment on the same permutation, but the volatility will be weakened by the property of the variance. Therefore, for signals with extreme volatility, WPE cannot achieve sufficient resolution. In this study, we characterize the volatility of the fragments with the ratio of the range and standard deviation, rather than using variance as the weight in the WPE. Meanwhile, in order to perform unified processing for different fragments of the signal, the difference in the range is expected to be rescaled, i.e.: $$ \omega_{i,j}^{k} =\frac{{\rm Range}\left({X_{i,j}^{k} } \right)}{{\rm Std}(X_{i,j}^{k})}.~~ \tag {6} $$ Range($X^{k}_{i,j}$) is the range of the fragment $X^{k}_{i,j}$ and Std($X^{k}_{i,j}$) denotes the standard deviation of the fragment $X^{k}_{i,j}$: $$ {\rm Range}\left({X_{i,j}^{k} } \right)=\max \left({X_{i,j}^{k} } \right)-\min \left({X_{i,j}^{k} } \right),~~ \tag {7} $$ $$ {\rm Std}\left({X_{i,j}^{k} } \right)=\sqrt {\frac{1}{m}\sum\limits_{l=1}^m {\left({X_{i,j}^{k,l} -\bar{{X}}_{i,j}^{k} } \right)^{2}} }.~~ \tag {8} $$ The length of the signal, $L$, should be large enough to ensure the accuracy of the calculation. The delay time $\tau$ and the embedding dimension $m$ are two parameters to be determined, and are selected in accordance with the literature.[12] The probability of each permutation containing amplitude information in the fragment can then be expressed as: $$ P_{\omega } ({k,i})=\frac{\sum\nolimits_{j=1}^{N_{i} } {{{\rm Range}\left({X_{i,j}^{k} } \right)} / {{\rm Std}\left({X_{i,j}^{k} } \right)}} }{\sum\nolimits_{i=1}^{m!} {\sum\nolimits_{j=1}^{N_{i} } {{{\rm Range}\left({X_{i,j}^{k} } \right)} / {{\rm Std}\left({X_{i,j}^{k} } \right)}} } }.~~ \tag {9} $$ RSPE can then be defined as $$ {\rm RSPE}=-\sum\limits_{k=1}^n {\sum\limits_{i=1}^{m!} {P_{\omega } ({k,i})\ln } P_{\omega }({k,i})}.~~ \tag {10} $$ The multi-scale RSPE can be obtained by combining Eq. (10) with the coarse-grained method,[13] and the value of time series under different coarse-grained scales can be expressed as: $$ \left\{ {y^{s,k}(t),y^{s,k}(t+\tau),\ldots,y^{s,k}\left[ {t+({m-1})\tau } \right]} \right\},~~ \tag {11} $$ $$ y^{s,k}(t)=\frac{1}{s}\sum\limits_{i=\left({t-1} \right)s+1}^{\rm ts} {x^{k}(t)},~~~1\leqslant t\leqslant \frac{L}{s}.~~ \tag {12} $$ Substituting Eq. (12) into Eqs. (9) and (10) yields the multi-variate RSPE under scale $s$, as follows: $$ {\rm RSPE}^{s}=-\sum\limits_{k=1}^n {\sum\limits_{i=1}^{m!} {P_{\omega }^{s} \left({k,i} \right)\ln } P_{\omega }^{s} \left({k,i} \right)}.~~ \tag {13} $$ Next, we apply the above method to typical chaotic signals. In particular, we compute the MSE, MFE, WPE and RSPE of Schuster maps and Rössler maps, which exhibit extreme volatility. We study Schuster maps, i.e., a class of maps generating fluctuant signals with chaotic bursts: $$ x_{n+1} =x_{n} +x_{n}^{z},~~\bmod~1~~ \tag {14} $$ where the length of each time series is $2^{13}$ points and the initial value is $x_{0} = 0.05$. Signals with different $z$ values are shown in Fig. 1. In order to clearly show the details, Fig. 1 illustrates the volatility of the nine sequences, ranging from $n=0$ to $n$=1000. The sequence generated by the Schuster equation due to its different $z$ values are taken as the signal source. The resolution of the complexity in different systems is analyzed using four entropies: MSE, MFE, WPE and RSPE, along with the coarse-grained method, as shown in Fig. 2. The Schuster sequences exhibit characteristics of extreme fluctuation, as shown in Fig. 1. The signal complexity decreases significantly with the increase of the $z$ value. Figure 2 shows results for four types of multi-scale entropy (MSE, MFE, WPE and RSPE) in relation to nine Schuster sequences with different $z$ values. Figures 2(a) and 2(b) show the results for MSE and MFE with the multi-scale method. While the classification in the low scale is good to some extent, there is still a confined space for classification, as the range of entropy values is between 0–2.5 for MSE and 0–1 for MFE. Moreover, when the scale increases, the different sequences in MSE and MFE exhibit poor resolution, and the curves start to tangle at low scale. Figure 2(c) shows the multi-scale entropy of the Schuster sequences obtained by WPE, which first increases and then decreases. The WPE entropy relating to the Schuster sequences is in the range 3.5–6.5, and the curves decrease hierarchically at low scales. When the scale exceeds 10, the difference between the different sequences is inconspicuous. Furthermore, the six sets of curves are entangled, and cannot be clearly distinguished. Figure 2(d) shows the results for multi-scale entropy of the Schuster sequence using RSPE. The entropy is in the range 1.5–6.5. This is clearly greater than MSE, MFE, and WPE, indicative of a larger space for classification. Between $z=1.1$–1.5 and $z=1.6$–1.9, differences appear in relation to the complexity of the sequences, which is the same rule as for the complexity of the original sequence changes with $z$ in Fig. 1, i.e., in Figs. 1(e) and 1(f), the complexity of Schuster sequences decreases, and continuous low-amplitude points appear. There is a direct relation between the information structure and the complexity of the sequence. Therefore, RSPE demonstrates several advantages in terms of processing Schuster sequences, as regards the range, the direct reflection of sequence complexity, and the complexity differentiation.
cpl-37-9-090501-fig1.png
Fig. 1. Schuster time series with different $z$ values: (a) $z=1.1$, (b) $z=1.2$, (c) $z=1.3$, (d) $z=1.4$, (e) $z=1.5$, (f) $z=1.6$, (g) $z=1.7$, (h) $z=1.8$, (i) $z=1.9$.
cpl-37-9-090501-fig2.png
Fig. 2. Four kinds of multi-scale entropy values obtained by processing nine types of Schuster sequence with different $z$ values: (a) MSE, (b) MFE, (c) WPE, (d) RSPE (this study).
Next, we study the Rössler maps, i.e., a class of maps which generate multi-variate signals with chaotic bursts. Here, we adopt a variety of different Rössler family[35] systems, having different parameters and initial values, so as to construct mutational multivariate sequences with different information structures: $$ \dot{{x}}=-\left({y+z} \right);~\dot{{y}}=x+ay;~\dot{{z}}=bx+xz-cz.~~ \tag {15} $$ Sequence A1: the sequence generated by Eq. (15), where the parameters are configured as $a=0.36$, $b=0.4$, $c=4.5$ with the initial value $(x_{0},y_{0},z_{0})=(0,3,0)$. Sequence A2: the sequence generated by Eq. (15), where the parameters are configured as $a=0.5$, $b=0.4$, $c=4.5$ with the initial value $(x_{0},y_{0},z_{0})=(0,3,0)$. $$ \dot{{x}}=x-xy-z;~\dot{{y}}=x^{2}-ay;~\dot{{z}}=b\left({cx-z} \right).~~ \tag {16} $$ Sequence B: the sequence generated by Eq. (16), where the parameters are configured as $a=0.1$, $b=0.08$, $c=0.125$ with the initial value $(x_{0},y_{0},z_{0})=(0,0.2,-10^{-6})$. $$ \dot{{x}}=-\left({y+x} \right);~\dot{{y}}=x;~\dot{{z}}=a\left({y-y^{2}} \right)-bz.~~ \tag {17} $$ Sequence C1: the sequence generated by Eq. (17), where the parameters are configured as $a=0.2$, $b=0$ with the initial value $(x_{0},y_{0},z_{0})=(0.3,0,-0.25)$. Sequence C2: the sequence generated by Eq. (17), where the parameters are configured as $a=0.4$, $b=0$ with the initial value $(x_{0},y_{0},z_{0})= (0.3,0,-0.25)$. Sequence C3: the sequence generated by Eq. (17), where the parameters are configured as $a=0.386$, $b=0.2$ with the initial value $(x_{0},y_{0},z_{0})=(0.4,-0.4,-0.7)$. $$\begin{alignat}{1} &\dot{{x}}=-y+ax-bz;~\dot{{y}}=x+1.1;\\ &\dot{{z}}=\left({1-z^{2}} \right)\left({x+z} \right)-\varepsilon z.~~ \tag {18} \end{alignat} $$ Sequence D: the sequence generated by Eq. (18), where the parameters are configured as: $a=0.2$, $b=0$, $\varepsilon =0.03$ with the initial value $(x_{0},y_{0},z_{0})=(0,0,-1)$. There are seven Rössler group systems with multi-variate sequences in this study. A1 and A2 are both generated by Eq. (15), but with different complexities. Similarly, C1, C2 and C3 are multi-variate chaotic sequences with different information structures, generated by Eq. (17), as shown in Fig. 3.
cpl-37-9-090501-fig3.png
Fig. 3. Graph showing multiple sequences produced by Rössler systems in three dimensions: (a) series A1, (b) series A2, (c) series B, (d) series C1, (e) series C2, (f) series C3, (g) series D.
The multi-variate Rössler sequences of seven systems are combined with the multi-scale method to analyze the entropy (MSE, MFE, WPE and RSPE), as shown in Fig. 4. The multi-variate sequences produced by Rössler systems have distinct characteristics of extreme volatility: In Figs. 3(a)–3(g), sequences D and A2 are more complex than either sequences C3, C1, A1 or sequences C2, D. Figures 4(a) and 4(b) show the MSE and MFE methods of processing the different Rössler systems; the resolution of these two systems is poor compared to the RSPE method shown in Fig. 4(d), particularly in relation to the higher scales. Eventually the curves of MSE and MFE come across, making it difficult to show the complexity of the different systems. In Fig. 4(c), with an increase in scale, all the curves of WPE gradually increase with the scale. However, the WPE curves for the sequences A1, A2, B, C1, C2 and C3 show slope instability. From low to high scale, the trend of the curves changes repeatedly, leading to the sequences A1, C1, C3 being indistinguishable. Figure 4(d) shows the RSPE curves for each sequence. The slopes of the RSPE curves of sequences A1, B, C1, C2 and C3 are stable. With the increasing scale, the RSPE curves of sequences A1, C1 and C3 remain clearly separated. The slopes of sequences C2 and B are also preserved. The cross between the curves of sequences A2 and D in RSPE appears later than for the WPE curves. RSPE accurately reflects the complexity of the two sequences. In conclusion, the entropy resolution on multi-variate Rössler sequences of RSPE sequences performs far better than MSE, MFE or WPE.
cpl-37-9-090501-fig4.png
Fig. 4. Four varieties of multivariate multi-scale entropy obtained by processing seven types of Rössler series: (a) MSE, (b) MFE, (c) WPE, (d) RSPE (this study).
In summary, the maximum volatility of the sequence is extracted using the ratio of range and standard deviation as the weight. RSPE can effectively improve on the capacity of WPE to identify and distinguish signals with extreme amplitude volatility. RSPE also improves the sensitivity of nonlinear complex recognition, and exhibits great adaptability to both single channel and multi-channel signals. In the simulation experiments relating to typical chaotic time series with extreme fluctuations, RSPE shows excellent sensitivity: it can effectively classify the internal information structures of both Schuster and Rössler sequences. Its sensitivity is superior to MSE, MFE and WPE at both low scales and high scales. RSPE is compatible with WPE in terms of its capacity for recognition and discrimination of ordinary signal processing. It extends the adaptability of WPE in terms of volatility signals, and is of great significance in revealing the internal structure of information in dynamics systems which are nonlinear and exhibit extreme volatility. References A comparative classification of complexity measuresComplex systems, complexity measures, grammars and model-inferringErgodic theory of chaos and strange attractorsDetermining Lyapunov exponents from a time seriesThree approaches to the quantitative definition of information *How Long Is the Coast of Britain? Statistical Self-Similarity and Fractional DimensionDIMENSION THEORY IN DYNAMICAL SYSTEMS: CONTEMPORARY VIEWS AND APPLICATIONS By YAKOV B. PESIN Chicago Lectures in Mathematics, University of Chicago Press, 312 pp. Price: hardback $56, paperback $19.95. ISBN 0 226 66222 5Approximate entropy as a measure of system complexity.Approximate entropy (ApEn) as a complexity measurePhysiological time-series analysis using approximate entropy and sample entropyPermutation Entropy: A Natural Complexity Measure for Time SeriesMultiscale Entropy Analysis of Complex Physiologic Time SeriesComplexity analysis of the biomedical signal using fuzzy entropy measurementQuantification of hormone pulsatility via an approximate entropy algorithmApplication of equiprobable symbolization sample entropy to electroencephalography analysisSample entropy analysis of electroencephalogram based on the two-dimensional feature of amplitude and periodRecognition of denatured biological tissue based on variational mode decomposition and multi-scale permutation entropyStudy of multiscale entropy model to evaluate the cognitive behavior of healthy elderly people based on resting state functional magnetic resonance imagingEntropy Measures of Heart Rate Variability for Short ECG Datasets in Patients with Congestive Heart Failure An investigation of oil–water two-phase flow instability using multivariate multi-scale weighted permutation entropyEEG entropy measures in anesthesiaWeighted-permutation entropy: A complexity measure for time series incorporating amplitude informationComplexity extraction of electroencephalograms in Alzheimer's disease with weighted-permutation entropyThe experimental signals analysis for bubbly oil-in-water flow using multi-scale weighted-permutation entropyMultivariate weighted multiscale permutation entropy for complex time seriesRefined composite multiscale weighted-permutation entropy of financial time seriesAbrupt climate change detection based on heuristic segmentation algorithmThe research of dynamic structure abrupt change of nonlinear time seriesExtremely low temperature and its composite index based on stochastically re-sorting detrended fluctuation analysisUncovering dynamic behaviors underlying experimental oil–water two-phase flow based on dynamic segmentation algorithmDetecting the Flow Pattern Transition in the Gas-Liquid Two-Phase Flow Using Multivariate Multi-Scale Entropy AnalysisCONTINUOUS CHAOS-FOUR PROTOTYPE EQUATIONS
[1]Lindgren K and Nordahl M G 1988 Complex Syst. 2 409
[2] Wackerbauer R, Witt A, Atmanspacher H, Kurths J and Scheingeraber H 1994 Chaos, Solitons & Fractals 4 133
[3] Günther R, Schapiro B and Wagner P 1994 Chaos, Solitons & Fractals 4 635
[4] Eckmann J P and Ruelle D 1985 Rev. Mod. Phys. 57 617
[5] Wolf A, Swift J B, Swinney H L and Vastano J A 1985 Physica D 16 285
[6] Kolmogorov A N 1968 Int. J. Comput. Math. 2 157
[7] Mandelbrot B B 1967 Science 156 636
[8] Pesin Y B 1998 Ergod. Theory Dyn. Syst. 18 1043
[9] Pincus S M 1991 Proc. Natl. Acad. Sci. USA 88 2297
[10] Pincus S M 1995 Chaos 5 110
[11] Richman J S and Moorman J R 2000 Am. J. Physiol. - Heart Circ. Physiol. 278 H2039
[12] Bandt C and Pompe B 2002 Phys. Rev. Lett. 88 174102
[13] Costa M, Goldberger A L and Peng C K 2002 Phys. Rev. Lett. 89 068102
[14] Xie H B, Chen W T, He W X and Liu H 2011 Appl. Soft Comput. 11 2871
[15] Pincus S M and Keefe D L 1992 Am. J. Physiol. - Endocrinol. Metab. 262 E741
[16] Huang X L, Huo C Y, Si J F and Liu H X 2014 Acta Phys. Sin. 63 100503 (in Chinese)
[17] Guo J L, Zhong N, Ma X M, Zhang M H and Zhou H Y 2016 Acta Phys. Sin. 65 190501 (in Chinese)
[18] Liu B, Hu W P, Zou X, Ding Y J and Qian S Y 2019 Acta Phys. Sin. 68 028702 (in Chinese)
[19] Zhang F Y, Ge M L, Guo Z T, Xie C, Yang Z K and Song Z B 2020 Acta Phys. Sin. 69 108703 (in Chinese)
[20] Graff B, Graff G and Kaczkowska A 2012 Acta Phys. Pol. B 5 153
[21] Han Y F, Jin N D, Zhai L S, Ren Y Y and He Y S 2019 Physica A 518 131
[22] Liang Z H, Wang Y H, Sun X, Li D, Voss L J, Sleigh J W, Hagihira S and Li X L 2015 Front. Comput. Neurosci. 9 16
[23] Fadlallah B, Chen B D, Keil A and Príncipe J 2013 Phys. Rev. E 87 022911
[24] Deng B, Liang L, Li S N, Wang R F, Yu H T, Wang J and Wei X L 2015 Chaos 25 043105
[25] Chen X, Jin N D, Zhao A, Gao Z K, Zhai L S and Sun B 2015 Physica A 417 230
[26] Yin Y, Shang P J 2017 Nonlinear Dyn. 88 1707
[27] Zhang Y P and Shang P J 2018 Physica A 496 189
[28] Liu H J, Gu L and Hu X B 2014 J. Appl. Stat. Management 33 158 (in Chinese)
[29]Yuan Y, Yu X T and Zhang T H 2019 Shanghai Manag. Sci. 41 81 (in Chinese)
[30] Feng G L, Gong Z Q, Dong W J and Li J P 2005 Acta Phys. Sin. 54 5494 (in Chinese)
[31] Gong Z Q, Feng G L, Dong W J and Li J P 2006 Acta Phys. Sin. 55 3180 (in Chinese)
[32] Hou W, Zhang D Q, Qian Z H and Feng G L 2011 Acta Phys. Sin. 60 109203 (in Chinese)
[33] Gao Z K, Jin N D 2013 Physica A 392 1180
[34] Liu Y D, Wang D Y, Ren Y Y and Jin N D 2019 Z. Naturforsch. 74 837
[35] Rössler O E 1979 Ann. N. Y. Acad. Sci. 316 376