Computing the entropy rate of information source with methods of statistical physics

Shuang Ping Chen, Hao Ran Zheng, Meng Ma, Zhen Ya Zhang, Xu Fa Wang

Research output: Contribution to journalArticlepeer-review

3 Scopus citations


From the mathematical point of view, information sources can be 1-to-1 mapped to stochastic processes. Known from the theory of chaos, multi-fractal of stochastic process is a key characteristic of its dynamics, of which entropy rate is a special fractal dimension named information dimension. The paper introduces methods of statistical physics to compute the multi-fractal of stochastic process so that the entropy rate of source can be obtained at once. Take binary hidden Markov processes as example, the paper demonstrate how this approach works. The results shows that the methods is applicable to numerically approximate the entropy rate of binary hidden Markov processes (BHMPs) in practical applications, and it can be applied in more generalized kinds of information sources.

Original languageEnglish
Pages (from-to)129-132
Number of pages4
JournalDianzi Yu Xinxi Xuebao/Journal of Electronics and Information Technology
Issue number1
StatePublished - Jan 2007
Externally publishedYes


  • Entropy rate
  • Hidden Markov processes
  • Information source
  • Multi-fractal


Dive into the research topics of 'Computing the entropy rate of information source with methods of statistical physics'. Together they form a unique fingerprint.

Cite this