Abstract
The paper deals with the following problem: is returning to wrong conjectures necessary to achieve full power of learning? Returning to wrong conjectures complements the paradigm of U-shaped learning [2,6,8,20,24] when a learner returns to old correct conjectures. We explore our problem for classical models of learning in the limit: TxtEx-learning – when a learner stabilizes on a correct conjecture, and TxtBc-learning – when a learner stabilizes on a sequence of grammars representing the target concept. In all cases, we show that, surprisingly, returning to wrong conjectures is sometimes necessary to achieve full power of learning. On the other hand it is not necessary to return to old “overgeneralizing” conjectures containing elements not belonging to the target language. We also consider our problem in the context of so-called vacillatory learning when a learner stabilizes to a finite number of correct grammars. In this case we show that both returning to old wrong conjectures and returning to old “overgeneralizing” conjectures is necessary for full learning power. We also show that, surprisingly, learners consistent with the input seen so far can be made decisive [2,21] – they do not have to return to any old conjectures – wrong or right.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Angluin, D.: Inductive inference of formal languages from positive data. Information and Control 45, 117–135 (1980)
Baliga, G., Case, J., Merkle, W., Stephan, F., Wiehagen, R.: When unlearning helps (Manuscript 2005); Preliminary version of the paper appeared in ICALP (2000), http://www.cis.udel.edu/~case/papers/decisive.ps
Bārzdiņš, J.: Inductive inference of automata, functions and programs. In: Int. Math. Congress, Vancouver, pp. 771–776 (1974)
Blum, L., Blum, M.: Toward a mathematical theory of inductive inference. Information and Control 28, 125–155 (1975)
Blum, M.: A machine-independent theory of the complexity of recursive functions. Journal of the ACM 14, 322–336 (1967)
Bowerman, M.: Starting to talk worse: Clues to language acquisition from children’s late speech errors. In: Strauss, S., Stavy, R. (eds.) U-Shaped Behavioral Growth. Developmental Psychology Series, Academic Press, New York (1982)
Carey, S.: An analysis of a learning paradigm. In: Strauss, S., Stavy, R. (eds.) U-Shaped Behavioral Growth. Developmental Psychology Series, Academic Press, New York (1982)
Carlucci, L., Case, J., Jain, S., Stephan, F.: U-shaped learning may be necessary. Technical Report TRA11/04, School of Computing, National University of Singapore (November 2004)
Case, J.: The power of vacillation in language learning. SIAM Journal on Computing 28(6), 1941–1969 (1999)
Case, J., Lynes, C.: Machine inductive inference and language identification. In: Nielsen, M., Schmidt, E.M. (eds.) ICALP 1982. LNCS, vol. 140, pp. 107–115. Springer, Heidelberg (1982)
Case, J., Smith, C.: Comparison of identification criteria for machine inductive inference. Theoretical Computer Science 25, 193–220 (1983)
Fulk, M.: Prudence and other conditions on formal language learning. Information and Computation 85, 1–11 (1990)
Fulk, M., Jain, S., Osherson, D.: Open problems in systems that learn. Journal of Computer and System Sciences 49(3), 589–604 (1994)
Gold, E.M.: Language identification in the limit. Information and Control 10, 447–474 (1967)
Hopcroft, J., Ullman, J.: Introduction to Automata Theory, Languages, and Computation. Addison-Wesley, Reading (1979)
Jantke, K., Beick, H.: Combining postulates of naturalness in inductive inference. Journal of Information Processing and Cybernetics (EIK) 17, 465–484 (1981)
Kurtz, S., Royer, J.: Prudence in language learning. In: Haussler, D., Pitt, L. (eds.) Proceedings of the Workshop on Computational Learning Theory, pp. 143–156. Morgan Kaufmann, San Francisco (1988)
Lange, S., Wiehagen, R.: Polynomial time inference of arbitrary pattern languages. New Generation Computing 8, 361–370 (1991)
Machtey, M., Young, P.: An Introduction to the General Theory of Algorithms. North-Holland, New York (1978)
Marcus, G., Pinker, S., Ullman, M., Hollander, M., Rosen, T., Xu, F.: Overregularization in Language Acquisition. In: Monographs of the Society for Research in Child Development, vol. 57(4), University of Chicago Press, Chicago (1992); Includes commentary by Harold Clahsen
Osherson, D., Stob, M., Weinstein, S.: Systems that Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists. MIT Press, Cambridge (1986)
Plunkett, K., Marchman, V.: U-shaped learning and frequency effects in a multi-layered perceptron: implications for child language acquisition. Cognition 38(1), 43–102 (1991)
Rogers, H.: Theory of Recursive Functions and Effective Computability. MIT Press, Cambridge (1987); Reprinted by MIT Press in 1987
Strauss, S., Stavy, R.: U-Shaped Behavioral Growth. In: Developmental Psychology Series, Academic Press, New York (1982)
Strauss, S., Stavy, R., Orpaz, N.: The child’s development of the concept of temperature. Manuscript, Tel-Aviv University (1977)
Taatgen, N.A., Anderson, J.R.: Why do children learn to say broke? a model of learning the past tense without feedback. Cognition 86(2), 123–155 (2002)
Wiehagen, R., Liepe, W.: Charakteristische Eigenschaften von erkennbaren Klassen rekursiver Funktionen. Journal of Information Processing and Cybernetics (EIK) 12, 421–438 (1976)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Carlucci, L., Jain, S., Kinber, E., Stephan, F. (2005). Variations on U-Shaped Learning. In: Auer, P., Meir, R. (eds) Learning Theory. COLT 2005. Lecture Notes in Computer Science(), vol 3559. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11503415_26
Download citation
DOI: https://doi.org/10.1007/11503415_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26556-6
Online ISBN: 978-3-540-31892-7
eBook Packages: Computer ScienceComputer Science (R0)