Abstract
Random fields are useful mathematical objects in the characterization of non-deterministic complex systems. A fundamental issue in the evolution of dynamical systems is how intrinsic properties of such structures change in time. In this paper, we propose to quantify how changes in the spatial dependence structure affect the Riemannian metric tensor that equips the model's parametric space. Defining Fisher curves, we measure the variations in each component of the metric tensor when visiting different entropic states of the system. Simulations show that the geometric deformations induced by the metric tensor in case of a decrease in the inverse temperature are not reversible for an increase of the same amount, provided there is significant variation in the system's entropy: the process of taking a system from a lower entropy state A to a higher entropy state B and then bringing it back to A, induces a natural intrinsic one-way direction of evolution. In this context, Fisher curves resemble mathematical models of hysteresis in which the natural orientation is pointed by an arrow of time.
References
1 R. Albert and A. L. Barabási, Statistical mechanics of complex networks, Rev. Modern Phys. 74 (2002), 47–97. 10.1103/RevModPhys.74.47Search in Google Scholar
2 S. Albeverio and R. Hoegh-Krohn, Homogeneous random fields and statistical mechanics, J. Funct. Anal. 19 (1975), 3, 242–272. 10.1016/0022-1236(75)90058-0Search in Google Scholar
3 S. I. Amari, Differential-Geometrical Methods in Statistics, Lecture Notes in Statist. 28, Springer, Berlin, 1985. 10.1007/978-1-4612-5056-2Search in Google Scholar
4 S. I. Amari, Methods of Information Geometry, Transl. Math. Monogr. 191, American Mathematical Society, Providence, 2000. Search in Google Scholar
5 S. I. Amari, Information geometry on hierarchy of probability distributions, IEEE Trans. Inform. Theory 47 (2001), 5, 1701–1711. 10.1109/18.930911Search in Google Scholar
6 S. I. Amari, K. Kurata and H. Nagaoka, Information geometry of Boltzmann machines, IEEE Trans. Neural Netw. 3 (1992), 2, 260–271. 10.1109/72.125867Search in Google Scholar PubMed
7 K. A. Arwini and C. T. J. Dodson, Information Geometry: Near Randomness and Near Independence, Lecture Notes in Math. 1953, Springer, Berlin, 2008. 10.1007/978-3-540-69393-2Search in Google Scholar
8 O. E. Barndorff-Nielsen, Information and Exponential Families in Statistical Theory, John Wiley & Sons, Chichester, 1978. Search in Google Scholar
9 J. Besag, Spatial interaction and the statistical analysis of Lattice systems, J. R. Stat. Soc. Ser. B 36 (1974), 192–236. 10.1111/j.2517-6161.1974.tb00999.xSearch in Google Scholar
10 S. Boccaletti, V. Latora, Y. Moreno, M. Chavez and D. U. Hwang, Complex networks: Structure and dynamics, Phys. Rep. 424 (2006), 4–5, 175–308. 10.1016/j.physrep.2005.10.009Search in Google Scholar
11 L. Boltzmann, On certain questions of the theory of gases, Nature 51 (1895), 413–415. 10.1038/051413b0Search in Google Scholar
12 X. Calmet and J. Calmet, Dynamics of the fisher information metric, Phys. Rev. E (3) 71 (2005), Article ID 056109. 10.1103/PhysRevE.71.056109Search in Google Scholar PubMed
13 M. Campisi and P. Hanggi, Fluctuation, dissipation and the arrow of time, Entropy 13 (2011), 12, 2024–2035. 10.3390/e13122024Search in Google Scholar
14 G. Casella and R. L. Berger, Statistical Inference, 2nd ed., Duxbury Press, Pacific Grove, 2002. Search in Google Scholar
15 S. Chib, Markov Chain Monte Carlo technology, Handbook of Computational Statistics, Springer, Berlin (2011), 73–104. 10.1007/978-3-642-21551-3_4Search in Google Scholar
16 D. Chu, R. Strand and R. Fjelland, Theories of complexity, Complexity 8 (2003), 3, 19–30. 10.1002/cplx.10059Search in Google Scholar
17 S. I. R. Costa, S. A. Santos and J. E. Strapasson, Fisher information distance: A geometrical reading, Discrete Appl. Math. 197 (2014), 59–69. 10.1016/j.dam.2014.10.004Search in Google Scholar
18 R. A. Fisher, Theory of statistical estimation, Math. Proc. Cambridge Philos. Soc. 22 (1925), 5, 700–725. 10.1017/S0305004100009580Search in Google Scholar
19 B. R. Frieden, Science from Fisher Information: A Unification, Cambridge University Press, Cambridge, 2004. 10.1017/CBO9780511616907Search in Google Scholar
20 S. Geman and D. Geman, Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Mach. Intelligence 6 (1984), 6, 721–741. 10.1016/B978-0-08-051581-6.50057-XSearch in Google Scholar
21 J. W. Gibbs, Elementary Principles in Statistical Mechanics, Charles Scribner's Sons, New York, 1902. Search in Google Scholar
22 W. R. Gilks, D. G. Clayton, D. J. Spiegelhalter, N. G. Best, A. J. McNeil and L. D. Sharples, Modeling complexity: Applications of Gibbs sampling in medicine, J. R. Stat. Soc. Ser. B 55 (1993), 1, 39–52. 10.1111/j.2517-6161.1993.tb01468.xSearch in Google Scholar
23 W. M. Haddad, Temporal asymmetry, entropic irreversibility, and finite-time thermodynamics: From Parmenides–Einstein time-reversal symmetry to the Heraclitan entropic arrow of time, Entropy 14 (2012), 3, 407–455. 10.3390/e14030407Search in Google Scholar
24 J. M. Hammersley and P. Clifford, Markov field on finite graphs and lattices, preprint 1971, www.statslab.cam.ac.uk/~grg/books/hammfest/hamm-cliff.pdf. Search in Google Scholar
25 A. E. Hassanien, A. Taher Azar, V. Snasel, J. Kacprzyk and J. H. E. Abawajy, Big Data in Complex Systems: Challenges and Opportunities, Springer, New York, 2015. 10.1007/978-3-319-11056-1Search in Google Scholar
26 T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., Springer, New York, 2009. 10.1007/978-0-387-84858-7Search in Google Scholar
27 W. K. Hastings, Monte Carlo sampling methods using Markov chains and their applications, Biometrika 57 (1970), 97–109. 10.1093/biomet/57.1.97Search in Google Scholar
28 W. Heisenberg, Zur Theorie des Ferromagnetismus, Z. Phys. 49 (1928), 9–10, 619–636. 10.1007/978-3-642-61659-4_35Search in Google Scholar
29 E. Ising, Beitrag zur Theorie des Ferromagnetismus, Z. Phys. 39 (1925), 253–258. 10.1007/BF02980577Search in Google Scholar
30 L. Isserlis, On a formula for the product-moment coefficient of any order of a normal frequency distribution in any number of variables, Biometrika 12 (1918), 134–139. 10.1093/biomet/12.1-2.134Search in Google Scholar
31 W. Janke, D. A. Johnston and R. Kenna, Information geometry and phase transitions, Phys. A 336 (2004), 181–186. 10.1016/j.physa.2004.01.023Search in Google Scholar
32 N. W. Jankowski, Exploring e-science: An introduction, J. Computer-Mediated Commun. 12 (2007), 2, 549–562. 10.1111/j.1083-6101.2007.00337.xSearch in Google Scholar
33 E. Jaynes, Information theory and statistical mechanics, Phys. Rev. 106 (1957), 620–630. 10.1103/PhysRev.106.620Search in Google Scholar
34 V. Jejjala, M. Kavic, D. Minic and C.-H. Tze, Modeling Time's Arrow, Entropy 14 (2012), 4, 614–629. 10.3390/e14040614Search in Google Scholar
35 J. L. Jensen and H. R. Künsh, On asymptotic normality of pseudo likelihood estimates for pairwise interaction processes, Ann. Inst. Statist. Math. 46 (1994), 3, 475–486. 10.1007/BF00773511Search in Google Scholar
36 R. E. Kass, The geometry of asymptotic inference, Statist. Sci. 4 (1989), 3, 188–234. 10.1214/ss/1177012480Search in Google Scholar
37 D. P. Landau and K. Binder, A Guide to Monte Carlo Simulations in Statistical Physics, Cambridge University Press, Cambridge, 2000. Search in Google Scholar
38 A. L. M. Levada, Learning from complex systems: On the roles of entropy and Fisher Information in pairwise isotropic Gaussian Markov random fields, Entropy 16 (2014), 1002–1036. 10.3390/e16021002Search in Google Scholar
39 Z. Liu, J. Luo and C. Shao, Potts model for exaggeration of a simple rumor transmitted by recreant rumormongers, Phys. Rev. E (3) 64 (2001), 6, Article ID 046134. 10.1103/PhysRevE.64.046134Search in Google Scholar PubMed
40 J. Machicao, A. G. Marco and O. M. Bruno, Chaotic encryption method based on life-like cellular automata, Expert Syst. Appl. 39 (2012), 16, 12626–12635. 10.1016/j.eswa.2012.05.020Search in Google Scholar
41 I. D. Mayergoyz, Mathematical Models of Hysteresis and Their Applications, Academic Press, New York, 2003. 10.1016/B978-012480873-7/50005-0Search in Google Scholar
42 R. M. H. Merks and J. A. Glazier, A cell-centered approach to developmental biology, Phys. Rev. A (3) 352 (2005), 1, 113–130. 10.1016/j.physa.2004.12.028Search in Google Scholar
43 N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller and E. Teller, Equation of state calculations by fast computing machines, J. Chem. Phys. 21 (1953), 6, 1087–1092. 10.2172/4390578Search in Google Scholar
44 E. Montroll, Statistical mechanics of nearest neighbor systems, J. Chem. Phys. 9 (1941), 706–706. 10.1063/1.1750981Search in Google Scholar
45 K. P. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, Cambridge, 2012. Search in Google Scholar
46 M. E. J. Newman, The structure and function of complex networks, SIAM Rev. 45 (2003), 2, 167–256. 10.1137/S003614450342480Search in Google Scholar
47 L. Onsager, Crystal statistics. 1. A two-dimensional model with an order-disorder transition, Phys. Rev. 65 (1944), 3–4, 117–149. 10.1103/PhysRev.65.117Search in Google Scholar
48 G. Pistone and M. P. Rogantin, The exponential statistical manifold: Mean parameters, orthogonality, and space transformation, Bernoulli 5 (1999), 721–760. 10.2307/3318699Search in Google Scholar
49 G. O. Roberts, Markov chain concepts related to sampling algorithms, Markov Chain Monte Carlo in Practice, Chapman & Hall, London (1996), 45–57. Search in Google Scholar
50 K. K. Sabelfeld, Random Fields and Stochastic Lagrangian Models. Analysis and Applications in Turbulence and Porous Media, De Gruyter, Berlin, 2012. 10.1515/9783110296815Search in Google Scholar
51 C. Shannon and W. Weaver, The Mathematical Theory of Communication, University of Illinois Press, Champaign, 1949. Search in Google Scholar
52 P. Sibani and H. J. Jensen, Stochastic Dynamics of Complex Systems, World Scientific, Hackensack, 2013. 10.1142/p877Search in Google Scholar
53 S. D. Silvey, Statistical Inference, Monogr. Statist. Appl. Probab., Chapman & Hall, London, 1975. Search in Google Scholar
54 A. F. M. Smith and G. O. Robert, Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods, J. R. Stat. Soc. Ser. B 55 (1993), 1, 3–23. 10.1111/j.2517-6161.1993.tb01466.xSearch in Google Scholar
55 F. Spitzer, Markov random fields and Gibbs ensembles, Amer. Math. Monthly 78 (1971), 142–154. 10.1080/00029890.1971.11992710Search in Google Scholar
56 S. H. Strogatz, Exploring complex networks, Nature 410 (2001), 268–276. 10.1038/35065725Search in Google Scholar PubMed
57 R. Swendsen and J. Wang, Nonuniversal critical dynamics in Monte Carlo simulations, Phys. Rev. Lett. 58 (1987), 86–88. 10.1103/PhysRevLett.58.86Search in Google Scholar PubMed
58 E. Vanmarcke, Random Fields: Analysis and Synthesis, World Scientific, Hackensack, 2010. 10.1142/5807Search in Google Scholar
59 G. Winkler, Image Analysis, Random Fields and Markov Chain Monte Carlo Methods: A Mathematical Introduction, Springer, New York, 2006. Search in Google Scholar
60 U. Wolff, Collective Monte Carlo updating for spin systems, Phys. Rev. Lett. 62 (1989), 361–364. 10.1103/PhysRevLett.62.361Search in Google Scholar PubMed
61 F. Y. Wu, The Potts model, Rev. Modern Phys. 54 (1982), 1, 235–268. 10.1142/9789812813893_0005Search in Google Scholar
62 F. Y. Wu, Jones polynomial as a Potts model partition function, J. Knot Theory Ramifications 1 (1992), 1, 47–57. 10.1142/S0218216592000057Search in Google Scholar
63 X. Wu, X. Zhu, G. Q. Wu and W. Ding, Data mining with big data, IEEE Trans. Knowledge Data Engrg. 26 (2014), 1, 97–107. 10.1109/TKDE.2013.109Search in Google Scholar
64 P. Zanardi, P. Giorda and M. Cozzini, Information-theoretic differential geometry of quantum phase transitions, Phys. Rev. Lett. 99 (2007), Article ID 100603. 10.1103/PhysRevLett.99.100603Search in Google Scholar PubMed
© 2016 by De Gruyter
