Vulnerability to emotional disorders including depression derives from interactions between genes and environment, especially during sensitive developmental periods. Adverse early-life experiences provoke the release and modify the expression of several stress mediators and neurotransmitters within specific brain regions. The interaction of these mediators with developing neurons and neuronal networks may lead to long-lasting structural and functional alterations associated with cognitive and emotional consequences. Although a vast body of work has linked quantitative and qualitative aspects of stress to adolescent and adult outcomes, a number of questions are unclear. What distinguishes 'normal' from pathologic or toxic stress? How are the effects of stress transformed into structural and functional changes in individual neurons and neuronal networks? Which ones are affected? We review these questions in the context of established and emerging studies. We introduce a novel concept regarding the origin of toxic early-life stress, stating that it may derive from specific patterns of environmental signals, especially those derived from the mother or caretaker. Fragmented and unpredictable patterns of maternal care behaviors induce a profound chronic stress. The aberrant patterns and rhythms of early-life sensory input might also directly and adversely influence the maturation of cognitive and emotional brain circuits, in analogy to visual and auditory brain systems. Thus, unpredictable, stress-provoking early-life experiences may influence adolescent cognitive and emotional outcomes by disrupting the maturation of the underlying brain networks. Comprehensive approaches and multiple levels of analysis are required to probe the protean consequences of early-life adversity on the developing brain. These involve integrated human and animal-model studies, and approaches ranging from in vivo imaging to novel neuroanatomical, molecular, epigenomic, and computational methodologies. Because early-life adversity is a powerful determinant of subsequent vulnerabilities to emotional and cognitive pathologies, understanding the underlying processes will have profound implications for the world's current and future children.