Abstract
This paper deals with the relatively new field of sequence-based estimation which involves utilizing both the information in the observations and in their sequence of appearance. Our intention is to obtain Maximum Likelihood estimates by “extracting” the information contained in the observations when perceived as a sequence rather than as a set. The results of [15] introduced the concepts of Sequence Based Estimation (SBE) for the Binomial distribution. This current paper generalizes these results for the multinomial “two-at-a-time” scenario. We invoke a novel phenomenon called “Occlusion” that can be described as follows: By “concealing” certain observations, we map the estimation problem onto a lower-dimensional binomial space. Once these occluded SBEs have been computed, we demonstrate how the overall Multinomial SBE (MSBE) can be obtained by mapping several lower-dimensional estimates onto the original higher-dimensional space. We formally prove and experimentally demonstrate the convergence of the corresponding estimates.
The first author is a Fellow: IEEE and Fellow: IAPR. The work was done while he was visiting at Myongji University, Yongin, Korea. He also holds an Adjunct Professorship with the Department of Information and Communication Technology, University of Agder, Grimstad, Norway. The work was partially supported by NSERC, the Natural Sciences and Engineering Research Council of Canada and a grant from the National Research Foundation of Korea. This work was also generously supported by the National Research Foundation of Korea funded by the Korean Government (NRF-2012R1A1A2041661).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
This information is, of course, traditionally used when we want to consider dependence information, as in the case of Markov models and n-gram statistics.
- 2.
- 3.
We apologize for this cumbersome notation, but this is unavoidable considering the complexity of the problem and the ensuing analysis.
- 4.
For the present, we consider non-overlapping subsequences. We shall later extend this to overlapping sequences when we report the experimental results.
- 5.
The reader must take pains to differentiate between the q’s and the s’s, because the former refer to the BSBEs and the latter to the MSBEs.
- 6.
How BSBEs are obtained for specific instantiations of \(\pi (a,b)\) is discussed later.
- 7.
The fact that c is a dummy variable will not be repeated in future invocations.
- 8.
This, of course, makes sense only if \(\forall c, \widehat{q}_{a}\Big |_ {{\pi (a,c)}} ^{ac} \ne 0\).
- 9.
Observe that it would be statistically advantageous (since the number of occurrences obtained would be almost doubled) if all the overlapping \(N_{ab}-1\) subsequences of length 2 were considered. The computational consequences of this are given in [16].
- 10.
In the tables, values of unity/zero represent the cases when the roots are complex or when the number of occurrences of the event concerned are zero.
References
Bickel, P., Doksum, K.: Mathematical Statistics: Basic Ideas and Selected Topics, vol. 1, 2nd edn. Prentice Hall, Upper Saddle River (2000)
Bunke, H.: Structural and syntactic pattern recognition. In: Chen, C.H., Pau, L.F., Wang, P.S.P. (eds.) Handbook of Pattern Recognition and Computer Vision, pp. 163–209. World Scientific-25, River Edge (1993)
Casella, G., Berger, R.: Statistical Inference, 2nd edn. Brooks/Cole Publisher Company, Pacific Grove (2001)
Duda, R., Hart, P., Stork, D.: Pattern Classification, 2nd edn. John Wiley and Sons, Inc., New York (2000)
El-Gendy, M.A., Bose, A., Shin, K.G.: Evolution of the internet QoS and support for soft real-time applications. Proc. IEEE 91, 1086–1104 (2003)
Friedman, M., Kandel, A.: Introduction to Pattern Recognition - Statistical, Structural, Neural and Fuzzy Logic Approaches. World Scientific, New Jersey (1999)
Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press, San Diego (1990)
Goldberg, S.: Probability: An Introduction. Prentice-Hall, Englewood Cliffs (1960)
Herbrich, R.: Learning Kernel Classifiers: Theory and Algorithms. MIT Press, Cambridge (2001)
Jones, B., Garthwaite, P., Jolliffe, I.: Statistical Inference, 2nd edn. Oxford University Press, New York (2002)
Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–20, 226–239 (1998)
Kreyszig, E.: Advanced Engineering Mathematics, 8th edn. John Wiley & Sons, New York (1999)
Kuncheva, L.I., Bezdek, J.C., Duin, R.P.W.: Decision templates for multiple classifier fusion: an experimental comparison. Pattern Recogn. 34, 299–414 (2001)
Kuncheva, L.I.: A theoretical study on six classifier fusion strategies. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–24, 281–286 (2002)
Oommen, B.J., Kim, S.-W., Horn, G.: On the estimation of independent binomial random variables using occurrence and sequential information. Pattern Recogn. 40(11), 3263–3276 (2007)
Oommen, B.J., Kim, S-W.: Occlusion-based estimation of independent multinomial random variables using occurrence and sequential information. To be submitted for Publication
Ross, S.: Introduction to Probability Models, 2nd edn. Academic Press, Orlando (2002)
Shao, J.: Mathematical Statistics, 2nd edn. Springer, Heidelberg (2003)
Sprinthall, R.: Basic Statistical Analysis. Allyn and Bacon, Boston (2002)
van der Heijden, F., Duin, R.P.W., de Ridder, D., Tax, D.M.J.: Classification, Parameter Estimation and State Estimation: An Engineering Approach using MATLAB. John Wiley and Sons Ltd, England (2004)
Webb, A.: Statistical Pattern Recognition, 2nd edn. John Wiley & Sons, New York (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Oommen, B.J., Kim, SW. (2016). On the Foundations of Multinomial Sequence Based Estimation. In: Nguyen, NT., Iliadis, L., Manolopoulos, Y., Trawiński, B. (eds) Computational Collective Intelligence. ICCCI 2016. Lecture Notes in Computer Science(), vol 9875. Springer, Cham. https://doi.org/10.1007/978-3-319-45243-2_20
Download citation
DOI: https://doi.org/10.1007/978-3-319-45243-2_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-45242-5
Online ISBN: 978-3-319-45243-2
eBook Packages: Computer ScienceComputer Science (R0)