A linear discriminant analysis model of imbalanced associative learning in the mushroom body compartment

David Lipshutz; Aneesh Kashalikar; Shiva Farashahi; Dmitri B. Chklovskii

doi:10.1371/journal.pcbi.1010864

Abstract

To adapt to their environments, animals learn associations between sensory stimuli and unconditioned stimuli. In invertebrates, olfactory associative learning primarily occurs in the mushroom body, which is segregated into separate compartments. Within each compartment, Kenyon cells (KCs) encoding sparse odor representations project onto mushroom body output neurons (MBONs) whose outputs guide behavior. Associated with each compartment is a dopamine neuron (DAN) that modulates plasticity of the KC-MBON synapses within the compartment. Interestingly, DAN-induced plasticity of the KC-MBON synapse is imbalanced in the sense that it only weakens the synapse and is temporally sparse. We propose a normative mechanistic model of the MBON as a linear discriminant analysis (LDA) classifier that predicts the presence of an unconditioned stimulus (class identity) given a KC odor representation (feature vector). Starting from a principled LDA objective function and under the assumption of temporally sparse DAN activity, we derive an online algorithm which maps onto the mushroom body compartment. Our model accounts for the imbalanced learning at the KC-MBON synapse and makes testable predictions that provide clear contrasts with existing models.

Author summary

To adapt to their environments, animals learn associations between sensory stimuli (e.g., odors) and unconditioned stimuli (e.g., sugar or heat). In flies and other insects, olfactory associative learning primarily occurs in a brain region called the mushroom body, which is partitioned into multiple compartments. Within a compartment, neurons that represent odors synapse onto neurons that guide behavior. The strength of these synapses is modulated by a dopamine neuron that responds to one type of unconditioned stimuli (e.g., sugar), which implicates these synapses as a biological substrate for associative learning in insects. Modifications of these synapses is imbalanced in the sense that dopamine-induced modifications only weaken the synapses and are temporally sparse. In this work, we propose a simple mechanistic model of learning in the mushroom body that accounts for this imbalanced learning. Our model is interpretable as implementing an algorithm for linear discriminant analysis, a classical statistical method for linearly separating feature vectors that belong to different classes. Our model makes testable predictions that provide clear contrasts with existing models.

Citation: Lipshutz D, Kashalikar A, Farashahi S, Chklovskii DB (2023) A linear discriminant analysis model of imbalanced associative learning in the mushroom body compartment. PLoS Comput Biol 19(2): e1010864. https://doi.org/10.1371/journal.pcbi.1010864

Editor: Michele Migliore, National Research Council, ITALY

Received: September 26, 2022; Accepted: January 10, 2023; Published: February 6, 2023

Copyright: © 2023 Lipshutz et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Code for reproducing the numerical experiments on the synthetic datasets may be found on GitHub (https://github.com/flatironinstitute/bio-lda).

Funding: The authors received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Behavioral responses of animals are shaped in part by learned associations between sensory stimuli (e.g., odors) and unconditioned stimuli (e.g., sugar, heat or electric shock). A challenge in neuroscience is to understand the neural mechanisms that underlie associative learning. In invertebrates, the mushroom body is a well-studied brain region that plays a central role in olfactory associative learning [1–4]. The goal of this work is to propose a normative, mechanistic model of associative learning in the mushroom body that accounts for experimental observations and provides clear contrasts with existing models.

The mushroom body is segregated into functionally independent compartments [5], Fig 1. Within each compartment, Kenyon cells (KCs), which encode sparse odor representations [6], form synapses with the dendrites of mushroom body output neurons (MBONs), whose outputs guide learned behavior [7]. Associated with each compartment is a single Dopamine neuron (DAN) that responds to an unconditioned stimulus [8, 9], and projects its axon into the mushroom body compartment where it innervates the KC-MBON synapses to modulate plasticity, implicating the KC-MBON synapse as the synaptic substrate for associative learning in invertebrates.

Download:

Fig 1. A simplified schematic of a mushroom body compartment.

In both the left and right panels, the mushroom body compartment, indicated by the shaded box, is innervated by the axons of multiple KCs, the dendrites from one MBON and the axon terminals of one DAN. The bi-colored circles at the intersections of the KC axons and the MBON dendrites denote the KC-MBON synapses. Faintly shaded cell bodies indicate inactive neurons and boldly shaded cell bodies indicate active neurons. In the left panel, the DAN is inactive. In the right panel, the DAN is active and co-activation of the KCs and the DAN weakens the associated KC-MBON synapses (as illustrated by the smaller synapses).

https://doi.org/10.1371/journal.pcbi.1010864.g001

Experimental evidence suggests that learning at the KC-MBON synapse is imbalanced in the sense that DAN-induced plasticity is one-sided and temporally sparse. In particular, co-activation of a KC and the DAN weakens the KC-MBON synapse (see Fig 1, right) and DAN-induced plasticity is independent of the MBON activity [5]. This suggests that DAN-induced plasticity is one-sided and another mechanism such as homeostatic plasticity is responsible for strengthening the KC-MBON synapse. Furthermore, since each DAN responds to one type of unconditioned stimulus [2], which only constitutes a small fraction of all stimuli, the DAN activity is temporally sparse.

In this work, we propose a normative, mechanistic model of associative learning in the mushroom body that accounts for the imbalanced learning. We model each MBON as a linear discriminant analysis (LDA) classifier, which predicts if an associated unconditioned stimulus is present (the class label) given a KC odor representation (the feature vector). Under this interpretation, the KC-MBON synapses and an MBON bias term define a hyperplane in the high-dimensional space of KC odor representations that separates odor representations associated with the unconditioned stimulus from all other odor representations, Fig 2.

Download:

Fig 2. Linear separation of odors in the space of KC activities.

Left: Illustration of the hyperplane (dashed teal line) in the space of KC activities that separates conditioned odor responses from neutral odor responses. Each light red (resp. blue) dot denotes the KC response to a conditioned (resp. neutral) odor. The teal arrow denotes the vector of KC-MBON synaptic weights w, which is translated to show that it is orthogonal to the hyperplane . Right: Co-activation of the KCs and the DAN weakens the synaptic weights w. The KC activities x_t are denoted by the dark red dot with black border. The change of the synaptic weights Δw is in the direction −x_t. The hyperplane rotates to remain orthogonal to w. The change in bias Δb, which translates the hyperplane, is not depicted.

https://doi.org/10.1371/journal.pcbi.1010864.g002

Here, ‘normative’ refers to the fact that our mechanistic model is interpretable as an algorithm for optimizing an LDA objective. The normative approach is top-down in the sense that first the circuit objective is proposed and then an optimization algorithm is derived and compared with known physiology. There are several advantages to this approach. First, it directly relates the circuit objective to its mechanism; for example, neural activities and synaptic weight updates are interpretable as steps in an algorithm for solving a relevant circuit objective. Second, the approach distills down what aspects of the physiological are essential for optimizing the circuit objective and what aspects are not captured by the objective. Third, normative models are often analytically tractable, which allows them to be analyzed for any input statistics without resorting to exhaustive numerical simulation.

To derive our algorithm, we start with a convex objective for LDA (in terms of the KC-MBON synaptic weights). The objective can be optimized in the offline setting by taking gradient descent steps with respect to the KC-MBON synaptic weights. To obtain an online algorithm that accounts for the imbalanced learning, we take advantage of the fact that DAN activity is temporally sparse to obtain online approximations of the input statistics. Finally, we show numerically that our algorithm performs well even when DAN activity is not temporally sparse.

Our model makes testable predictions that are a direct result of the learning imbalance. First, our model predicts that DAN-induced plasticity at the KC-MBON synapse is sensitive to the time elapsed since the DAN was last active. Second, our model predicts that if the DAN is never active, then the KC-MBON synapses adapt to align with the mean KC activity (normalized by the covariance of the KC activity).

Results

LDA model of the mushroom body compartment

We consider a simplified mushroom body compartment that consists of n KC axons, the axon terminals from one DAN and the dendrites of one MBON, Fig 1. At each time t = 1, 2, …, the vector encodes the KC activities and the scalar y_t ∈ {0, 1} indicates whether the DAN is active (y_t = 1) or inactive (y_t = 0). If the DAN is active, we refer to x_t as a conditioned odor response, whereas if the DAN is inactive, we refer to x_t as a neutral odor response. We assume the DAN activity is temporally sparse, which can be expressed mathematically as π₁ ≪ 1, where π₁ ≔ 〈y_t〉_t is the fraction of time that the DAN is active.

In our model, the MBON is a linear classifier that predicts the DAN activity y_t (class label) given the KC activities x_t (feature vector). Let be a synaptic weight vector whose i^th component represents the strength of the synapse between the i^th KC and the MBON. At each time t, the KC activities x_t are multiplied by the synaptic weight vector w to generate the total input to the MBON, denoted c_t ≔ w · x_t. The output (firing rate) of the MBON is given by where b represents the ‘bias’ of the MBON; that is, the threshold below which the MBON does not fire. Under this interpretation, the KC-MBON synapses w and MBON bias b define a hyperplane in the n-dimensional space of KC activities that separates conditioned odor responses from neutral odor responses, Fig 2. In this case, z_t > 0 (resp. z_t = 0) corresponds to the prediction y_t = 0 (resp. y_t = 1). In other words, the MBON is a linear classifier that is active when predicting there is no unconditioned stimulus and inactive when predicting there is an unconditioned stimulus, which is consistent with experimental observations [2].

We derive learning rules for the KC-MBON synaptic weights w (and bias b) that solve an LDA objective and are consistent with experimental observations [2, 5]. LDA is popular linear classification method that is optimal under the assumption that the neutral odor responses and conditioned odor responses are Gaussian with common covariance matrix, but works well in practice even when these assumptions do not hold [10].

Our starting point is the convex LDA objective (1) where μ₀ and μ₁ denote the means of the neutral odor responses and conditioned odor responses, respectively, and Σ denotes the covariance of the neutral odor response. In the offline setting, we can minimize L(w) by taking gradient steps with respect to w: (2) where η > 0 is the step size. However, computing the means μ₀, μ₁ and covariance Σ requires the MBON to have access to the entire sequence of inputs, which is an unrealistic assumption.

To derive our online algorithm, we replace the averages μ₀, μ₁ and Σ in Eq 2 with online estimates. When the DAN is inactive (y_t = 0), we update the KC-MBON weights w according to the homeostatic plasticity rule (3) where μ_0,t denotes the running estimate of the mean neutral odor response and ζ_t denotes the running estimate of the mean total MBON input c_t conditioned on the DAN being inactive. Here, μ_0,t and (c_t − ζ_t) (x_t − μ_0,t) are online estimates of μ₀ and Σ, respectively (see Methods section). The running means μ_0,t and ζ_t can be represented by biophysical quantities such as calcium concentrations at the pre- and postsynaptic terminals of the KC-MBON synapses.

When the DAN is active, we update the KC-MBON weights w according to the following DAN-induced plasticity rule (4) where ℓ_t−1 denotes the time elapsed since the last time the DAN was active; see Fig 2 (right) for a geometric interpretation of the plasticity rule. The update in Eq 4 is in line with experimental evidence showing that DAN-induced plasticity is independent of the MBON activity z_t and co-activation of the KCs and the DAN reduces the strength of the synapses between the KCs and the MBON [5]. Biologically, the scalar ℓ_t−1 can be represented as the sensitivity of the KC-MBON synapses to DAN-induced plasticity. Assuming the conditioned odor response is independent of the time elapsed between DAN activations, then on average, the update in Eq 4 is approximately equal to −ημ₁, see Methods section. Therefore, the updates in Eqs 3–4 together account for all three terms in the offline update in Eq 2. The full model, including the bias updates (see Methods section), is summarized in Algorithm 1.

Algorithm 1 LDA in the mushroom body compartment

input: (x₁,y₁), … ,(x_T,y_T)

initialize: w = (w₁, … ,w_n), b = 0, ℓ₀ = 1, η > 0

for t = 1, 2, …, T do

c_t ← w · x_t

z_t ← max(c_t − b, 0)

if y_t = 0 then

w ← w + η(μ_0,t − (c_t − ζ_t)(x_t − μ_0,t))

ℓ_t ← ℓ_t−1 + 1

else if y_t = 1 then

w ← w − ηℓ_t−1x_t

ℓ_t ← 1

end if

end for

Algorithm 1 only has one hyper-parameter—the learning rate η > 0—which corresponds to timescale for learning in the mushroom body compartment. Hige et al. [5] showed that mushroom body compartments have distinct timescales for learning, which can be modeled by choosing different learning rates η > 0.

Numerical experiments

Next, we test Algorithm 1 on synthetic and real datasets. We test our algorithm on inputs when our assumption π₁ ≪ 1 holds, but also on inputs when π₁ ≈ 0.5. To evaluate our algorithm, we measure the running accuracy of the projections z_t over the previous min(100, t) iterations, where the algorithm is accurate at the t^th iterate if z_t = 0 and y_t = 1 or if z_t > 0 and y_t = 0.

Synthetic dataset.

We begin by evaluating Algorithm 1 on a synthetic dataset generated by a mixture of 2 overlapping Gaussian distributions, so that the optimal accuracy is less than 1. The data points of the 2 classes are each drawn from a 2-dimensional mean with common covariance. We simulate datasets of 10⁵ data points using the same mean and covariance in both classes but vary the frequency of class 1 samples encountered. We consider the cases π₁ = 0.1, 0.2, 0.3, 0.4, 0.5. In Fig 3 (left) we plot the error and the accuracy of our model for varying π₁. Remarkably, while the derivation of Algorithm 1 relied on the fact that π₁ ≪ 1, the algorithm still performs well even when π₁ = 0.5.

Download:

Fig 3. Performance of Algorithm 1.

Accuracy of Algorithm 1 on the synthetic datasets (left) and the KC dataset (right). Each line denotes the mean accuracy over 10 runs. Each shaded region indicates the area between the minimum and maximum accuracy over 10 runs.

https://doi.org/10.1371/journal.pcbi.1010864.g003

KC activities dataset.

We test our model on KC activities reported in [11]. Campbell et al. recorded odor-evoked KC responses in the fly mushroom body. The dataset we tested on contains the responses of 124 KCs in a single fly to the presentation of 7 odors, see [11, Figure 1]. To ensure the KC responses are well conditioned, we add Gaussian noise with covariance ϵ I₁₂₄, where ϵ = 0.01. We apply Algorithm 1 to the KC dataset. We first consider the case that odor 1 denotes the class 1 odor and odors 2–7 denote the class 0 odors, so . We then consider the cases that odors 1–2 (resp. 1–3) odors denote the class 1 odors and the remaining odors denote the class 0 odors, so (resp. ). In Fig 3 (right) we plot the error and accuracy of our model for varying π₁. Impressively, the modified algorithm performs well (approximately 85% accuracy) even when the assumption π₁ ≪ 1 is violated.

Competing MBONs.

Using the KC activities dataset, we model 2 MBONs with competing valences by running 2 instances of Algorithm 1 in parallel with different class assignments for the odors. We consider the case that odor 1 is aversive, odor 7 is attractive and the remaining odors are neutral. For MBON 1 (resp. MBON 2), we assume that odor 1 (resp. odor 7) denotes the class 1 odor and odors 2–7 (resp. odors 1–6) denote the class 0 odors, so that MBON 1 (resp. MBON 2) activity promotes approach (resp. avoid) behavior. Let z_i,t denote the output of MBON i ∈ {1, 2}. At each iterate t, if odor 1 (resp. odor 7, odors 2–6) is presented, then the model is accurate if z_1,t > 0 and z_2,t = 0 (resp. z_1,t = 0 and z_2,t > 0, resp. z_1,t > 0 and z_2,t > 0), and inaccurate otherwise. We then repeat the experiment two more times, but with odor 2 (resp. odor 3) labeled as aversive and odor 6 (resp. odor 5) as attractive. In Fig 4, we plot the performance of the competing MBONs.

Download:

Fig 4. Performance of competing MBONs.

Accuracy of 2 parallel runs of Algorithm 1 on the KC dataset to classify odors as aversive, attractive or neutral. Each line denotes the mean accuracy over 10 runs. Each shaded region indicates the area between the minimum and maximum accuracy over 10 runs.

https://doi.org/10.1371/journal.pcbi.1010864.g004

Discussion

Summary

In this work, we proposed a normative model of the mushroom body compartment that accounts for imbalanced learning at the KC-MBON synapse. Testing our model on synthetic and real datasets shows that it performs well under a variety of conditions. In our model, DAN-induced plasticity at the KC-MBON synapse does not depend on the MBON activity, but rather on the time elapsed since the last time the DAN was active. This aspect of our model suggests testable predictions that provide clear contrasts with existing models of associative learning in the mushroom body.

Model predictions

Prediction 1—In the absence of DAN activity, the KC-MBON synapses will align with the mean KC activity normalized by the covariance of their activities. When presented with neutral odors, the synapses adapt according to the homeostatic update in Eq 3. Since this update is equal to η(μ₀ − Σw) on average (see Methods section), the KC-MBON synaptic weights equilibrate at w = Σ⁻¹μ₀. Experimentally, this prediction could be tested by first presenting a fly with neutral odors and simultaneously recording from multiple KCs and an MBON. The weights can be estimated from the neural activities (using, e.g., [12]) and compared with our prediction w = Σ⁻¹μ₀.

Prediction 2—DAN-induced plasticity is proportional to the time elapsed since the DAN was last active. According to update in Eq 4, DAN-induced plasticity is proportional to the time elapsed since the DAN was last active. Experimentally, this prediction could be tested by presenting a fly with conditioned odors with different time intervals between presentations and estimating the resulting change in the synaptic weights.

Relation to existing models

There are a number of existing computational models of associative learning in the mushroom body [13–17], many of which are faithful to biophysical details and successfully capture important computational principles underlying associative learning in the mushroom body (see, e.g., [15]). Through extensive numerical simulations, these computational models can explain a number of phenomena. For example, Heurta and Nowotny [15] show that the organization of the mushroom body supports fast and robust associative learning, Bazhenov et al. [16] show that interactions between unsupervised and supervised forms of learning can explain how the timescale of associative learning depends on experimental conditions, and Peng and Chittka [17] show how complex forms of learning (e.g., peak shift) depend on different mechanistic aspects of learning in the mushroom body. In this work, we propose a top-down normative model of learning at the KC-MBON synapse, which contrasts with the bottom-up approach in these works that build models closely tied to physiological evidence. In this way, the our model is interpretable as an algorithm for optimizing a circuit objective and the output can be predicted analytically for any environmental condition without needing to resort to numerical simulation. In addition, our normative model makes testable predictions that are in clear contrast with these models, providing a method for validating or invalidating our model.

In addition to these models, Bennett et al. [18] propose a reinforcement learning model of the KC-MBON synapses as minimizing reinforcement prediction errors. They first consider a model in which the reinforcement signal is computed as the difference between DAN activities, so their plasticity rule requires 2 DANs to innervate a single mushroom body compartment, which is in contrast to experiment evidence showing that most compartments only receive inputs from a single DAN [9]. To account for this experimental observation, they propose a heuristic modification that adds a constant source of synaptic potentiation, which can be viewed as a form of homeostatic plasticity and is in line with experimental evidence. However, the modification is not normative and can fail to minimize prediction errors.

A significant difference between our model and these existing models is that DAN-induced plasticity depends on ℓ_t−1, the time elapsed since the DAN was last active. In our model, the variable ℓ_t−1 is critical for balancing homeostatic plasticity and DAN-induced plasticity. In S1 Appendix, we consider a modification of our algorithm in which ℓ_t−1 is replaced by a fixed constant ℓ_*.

Comparison of LDA to other linear classification methods

LDA is a linear classifier that is optimal under strict assumptions on the inputs, so it is worth considering other linear classification methods such as logistic regression and support vector machines (SVMs). Logistic regression is classical method for estimating the probability of one class versus another class. In terms of performance, there is evidence that there is not a substantial difference in the performance of logistic regression and LDA even when the assumptions for LDA are not met [19]. As a model of the insect mushroom body, we are unaware of an online algorithm for logistic regression that maps onto the mushroom body compartment and matches the experimental observations in [2, 5].

SVMs are flexible linear classifiers that do not make assumptions about the underlying data distribution. Huerta et al. [13, 15] proposed models of the mushroom body that are closely related to SVMs [20, 21]; however, the DAN-induced synaptic update rules depend on the MBON activity, which is in contrast to recent experimental evidence [5].

Limitations

Our model is a dramatic simplification of the mushroom body focused on providing a normative account of learning at the KC-MBON synapse that can account for how balance between DAN-induced plasticity and homeostatic plasticity is optimally maintained. Consequently, our model does not account for a number of the physiological details. For example, in order to implement an LDA algorithm, we do not sign-constrain the synaptic weight vector w, which violates Dale’s law. In addition, we assume that the DAN activity is binary. In reality the DAN may fire at different rates depending on the strength of the unconditioned stimulus and the firing rate may affect the DAN-induced plasticity. We can modify our model to allow y_t to be any nonnegative scalar and replace the update in Eq 4 with Δw = −ηℓ_t−1y_t x_t. However, in this case the algorithm is not derived from an objective function for LDA and so it is more challenging to understand the output. In addition to such simplifications, there are other features such as feedback connections in the mushroom body that have been recently discovered and are relevant for associative learning [7, 22], which are also not captured by our model.

Methods

Linear discriminant analysis

LDA is a statistical method for linear classification [23, section 4.3], which makes the following simplifying assumption: the conditional probability distributions p(x|y = 0) and p(x|y = 1) are both Gaussian with common full-rank n × n covariance matrix Σ; that is where μ₀ and μ₁ denote the means of the class 0 and class 1 feature vectors. In this case, the optimal decision criteria for assigning class 0 (resp. class 1) to feature vector x is w · x > b (resp. w · x < b), where (5) and π_i denotes the probability that a samples belongs to class i, for i = 0, 1. In particular, the hyperplane defines the optimal separation boundary for predicting whether a feature vector belongs to class 0 or class 1. While LDA assumes a specific generative model, it performs well in practice even when the assumptions do not hold [10].

The optimal weights w_opt can be expressed as the solution of the convex minimization problem in Eq 1, which we can solve for by taking the gradient descent steps (Eq 2). Formally, taking the step size η to zero in Eq 2 yields the linear gradient flow whose solution is given by In particular, we see that the solution w(t) converges exponentially to the optimal solution w_opt.

An online algorithm for imbalanced learning

In the online setting, the class means μ₀, μ₁ and the covariance Σ are not available. Instead, at each time t the algorithm has access to the feature vector x_t and class label y_t. To derive our online algorithm, we make online approximations of the offline quantities μ₀, μ₁ and Σ that are based on the fact that the unconditioned stimuli are sparse in time, i.e., π₁ ≪ 1, where we recall that π₁ denotes the proportion of conditioned odors. First, we note that we can rewrite the sample class means where π₀ ≔ 〈1 − y_t〉_t ≈ 1 is the fraction of odors that are neutral, and the sample covariance

Estimating the mean response to a neutral odor and the covariance.

Since π₀ ≈ 1, we approximate (6) Therefore, in the online setting, we can keep a running estimate of μ₀ and ζ ≔ w · μ₀ ≈ 〈(1 − y_t)c_t〉_t, where we recall that c_t = w · x_t, by performing the updates (7) In view of Eq 6 and the definitions of c_t and ζ, we can replace Σw with the online approximation (8) We replace the first and third terms in the offline update in Eq 2, η(μ₀ − Σw), with the online estimate η(1 − y_t)(μ_0,t − (c_t − ζ_t)(x_t − μ_0,t)).

Estimating the mean response to a conditioned odor.

To obtain an online approximation of μ₁, we first note that is approximately equal to the average time elapsed between class 1 samples. To see this, let t₁, t₂, … denote the subset of times such that y_t = 1. Then, letting t₀ = 0, we have Thus, in the online setting, when the j^th class 1 sample is presented (i.e., y_t = 1), we can use the time elapsed since the last class 1 sample, t_j − t_j−1, as an online estimate of . Setting ℓ₀ = 1 and we see that at time t such that y_t = 1, ℓ_t−1 denotes the time elapsed since the last class 1 sample, so . Assuming that the variables ℓ_t−1 and x_t are independent given y_t = 1—i.e., the KC representation x_t of a conditioned odor is independent of the time elapsed since the last conditioned odor ℓ_t−1—we see that μ₁ = 〈ℓ_t−1|y_t = 1〉_t 〈x_t|y_t = 1〉_t = 〈ℓ_t−1x_t|y_t = 1〉_t. We replace the second term in the offline update in Eq 2, −ημ₁, with the online approximation −ηy_tℓ_t-1x_t.

Estimating the bias.

To estimate the bias b, we note that because π₀ ≈ 1 and where the final inequality follows from the fact that log is concave and Jensen’s inequality (with equality holding when the variance of ℓ_t−1 given y_t = 1 is zero). Thus, assuming the variance of the time elapsed between conditioned odors is small, we can estimate the bias b in the online setting with the updates: Substituting these approximations into the offline update rules in Eq 2 yields our online algorithm (Algorithm 1).

In view of Jensen’s inequality, if the variance of the time elapsed between conditioned odors is large, then the bias b will be overestimated, meaning that the MBON will be less active than optimal. In other words, irregular intervals between DAN activity biases the MBON to be less active (i.e., predict that the unconditioned stimulus is present more often).

Details of numerical experiments

The experiments were performed on an Apple iMac with a 3.2 GHz 8-Core Intel Xeon W processor. For each experiment, we used a learning rate of the form We chose the parameters η₀ and γ by performing a grid search over η₀ ∈ {1, 10⁻¹, 10⁻², 10⁻³, 10⁻⁴} and γ ∈ {10⁻², 10⁻³, 10⁻⁴, 10⁻⁵, 10⁻⁶}. The optimal parameters for the synthetic dataset (resp. KC dataset) are η₀ = 10⁻¹ and γ = 10⁻³ (resp. η₀ = 10⁻¹ and γ = 10⁻⁴).

Supporting information

S1 Appendix. Comparison with a modified algorithm.

We consider a modification of Algorithm 1 in which the DAN-induced plasticity of the KC-MBON synapses does not depend on the time elapsed since the last time the DAN was active.

https://doi.org/10.1371/journal.pcbi.1010864.s001

(PDF)

Acknowledgments

We are grateful to Lucy Reading-Ikkanda for creating Fig 1. We thank Yanis Bahroun, Siavash Golkar, Jason Moore and Tiberiu Teşileanu for helpful feedback on an earlier draft of this work.

References

1. Heisenberg M. Mushroom body memoir: from maps to models. Nature Reviews Neuroscience. 2003;4(4):266–275. pmid:12671643
- View Article
- PubMed/NCBI
- Google Scholar
2. Owald D, Waddell S. Olfactory learning skews mushroom body output pathways to steer behavioral choice in Drosophila. Current Opinion in Neurobiology. 2015;35:178–184. pmid:26496148
- View Article
- PubMed/NCBI
- Google Scholar
3. Eichler K, Li F, Litwin-Kumar A, Park Y, Andrade I, Schneider-Mizell CM, et al. The complete connectome of a learning and memory centre in an insect brain. Nature. 2017;548(7666):175. pmid:28796202
- View Article
- PubMed/NCBI
- Google Scholar
4. Modi MN, Shuai Y, Turner GC. The Drosophila mushroom body: from architecture to algorithm in a learning circuit. Annual Review of Neuroscience. 2020;43:465–484. pmid:32283995
- View Article
- PubMed/NCBI
- Google Scholar
5. Hige T, Aso Y, Modi MN, Rubin GM, Turner GC. Heterosynaptic plasticity underlies aversive olfactory learning in Drosophila. Neuron. 2015;88(5):985–998. pmid:26637800
- View Article
- PubMed/NCBI
- Google Scholar
6. Honegger KS, Campbell RA, Turner GC. Cellular-resolution population imaging reveals robust sparse coding in the Drosophila mushroom body. Journal of Neuroscience. 2011;31(33):11772–11785. pmid:21849538
- View Article
- PubMed/NCBI
- Google Scholar
7. Eschbach C, Fushiki A, Winding M, Schneider-Mizell CM, Shao M, Arruda R, et al. Multilevel feedback architecture for adaptive regulation of learning in the insect brain. bioRxiv. 2019; p. 649731.
- View Article
- Google Scholar
8. Waddell S. Reinforcement signalling in Drosophila; dopamine does it all after all. Current Opinion in Neurobiology. 2013;23(3):324–329. pmid:23391527
- View Article
- PubMed/NCBI
- Google Scholar
9. Aso Y, Hattori D, Yu Y, Johnston RM, Iyer NA, Ngo TT, et al. The neuronal architecture of the mushroom body provides a logic for associative learning. Elife. 2014;3:e04577. pmid:25535793
- View Article
- PubMed/NCBI
- Google Scholar
10. Michie D, Spiegelhalter DJ, Taylor CC, editors. Machine Learning, Neural and Statistical Classification. Ellis Horwood; 1994.
11. Campbell RA, Honegger KS, Qin H, Li W, Demir E, Turner GC. Imaging a population code for odor identity in the Drosophila mushroom body. Journal of Neuroscience. 2013;33(25):10568–10581. pmid:23785169
- View Article
- PubMed/NCBI
- Google Scholar
12. Linderman S, Stock CH, Adams RP. A framework for studying synaptic plasticity with neural spike train data. Advances in Neural Information Processing Systems. 2014;27.
- View Article
- Google Scholar
13. Huerta R, Nowotny T, García-Sanchez M, Abarbanel HDI, Rabinovich MI. Learning classification in the olfactory system of insects. Neural Computation. 2004;16(8):1601–1640. pmid:15228747
- View Article
- PubMed/NCBI
- Google Scholar
14. Smith D, Wessnitzer J, Webb B. A model of associative learning in the mushroom body. Biological Cybernetics. 2008;99(2):89–103. pmid:18607623
- View Article
- PubMed/NCBI
- Google Scholar
15. Huerta R, Nowotny T. Fast and robust learning by reinforcement signals: explorations in the insect brain. Neural Computation. 2009;21(8):2123–2151. pmid:19538091
- View Article
- PubMed/NCBI
- Google Scholar
16. Bazhenov M, Huerta R, Smith BH. A computational framework for understanding decision making through integration of basic learning rules. Journal of Neuroscience. 2013;33(13):5686–5697. pmid:23536082
- View Article
- PubMed/NCBI
- Google Scholar
17. Peng F, Chittka L. A simple computational model of the bee mushroom body can explain seemingly complex forms of olfactory learning and memory. Current Biology. 2017;27(2):224–230. pmid:28017607
- View Article
- PubMed/NCBI
- Google Scholar
18. Bennett JE, Philippides A, Nowotny T. Learning with reinforcement prediction errors in a model of the Drosophila mushroom body. Nature Communications. 2021;12(1):1–14.
- View Article
- Google Scholar
19. Lei PW, Koehly LM. Linear discriminant analysis versus logistic regression: A comparison of classification errors in the two-group case. The Journal of Experimental Education. 2003;72(1):25–49.
- View Article
- Google Scholar
20. Huerta R, Vembu S, Amigó JM, Nowotny T, Elkan C. Inhibition in multiclass classification. Neural Computation. 2012;24(9):2473–2507. pmid:22594829
- View Article
- PubMed/NCBI
- Google Scholar
21. Huerta R. Learning pattern recognition and decision making in the insect brain. In: AIP Conference Proceedings. vol. 1510. American Institute of Physics; 2013. p. 101–119.
22. Li F, Lindsey JW, Marin EC, Otto N, Dreher M, Dempsey G, et al. The connectome of the adult Drosophila mushroom body provides insights into function. Elife. 2020;9:e62576. pmid:33315010
- View Article
- PubMed/NCBI
- Google Scholar
23. Hastie T, Tibshirani R, Friedman JH. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. vol. 2. Springer; 2009.

[ref1] 1. Heisenberg M. Mushroom body memoir: from maps to models. Nature Reviews Neuroscience. 2003;4(4):266–275. pmid:12671643
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. Owald D, Waddell S. Olfactory learning skews mushroom body output pathways to steer behavioral choice in Drosophila. Current Opinion in Neurobiology. 2015;35:178–184. pmid:26496148
View Article
PubMed/NCBI
Google Scholar

[6] View Article

[7] PubMed/NCBI

[8] Google Scholar

[ref3] 3. Eichler K, Li F, Litwin-Kumar A, Park Y, Andrade I, Schneider-Mizell CM, et al. The complete connectome of a learning and memory centre in an insect brain. Nature. 2017;548(7666):175. pmid:28796202
View Article
PubMed/NCBI
Google Scholar

[10] View Article

[11] PubMed/NCBI

[12] Google Scholar

[ref4] 4. Modi MN, Shuai Y, Turner GC. The Drosophila mushroom body: from architecture to algorithm in a learning circuit. Annual Review of Neuroscience. 2020;43:465–484. pmid:32283995
View Article
PubMed/NCBI
Google Scholar

[14] View Article

[15] PubMed/NCBI

[16] Google Scholar

[ref5] 5. Hige T, Aso Y, Modi MN, Rubin GM, Turner GC. Heterosynaptic plasticity underlies aversive olfactory learning in Drosophila. Neuron. 2015;88(5):985–998. pmid:26637800
View Article
PubMed/NCBI
Google Scholar

[18] View Article

[19] PubMed/NCBI

[20] Google Scholar

[ref6] 6. Honegger KS, Campbell RA, Turner GC. Cellular-resolution population imaging reveals robust sparse coding in the Drosophila mushroom body. Journal of Neuroscience. 2011;31(33):11772–11785. pmid:21849538
View Article
PubMed/NCBI
Google Scholar

[22] View Article

[23] PubMed/NCBI

[24] Google Scholar

[ref7] 7. Eschbach C, Fushiki A, Winding M, Schneider-Mizell CM, Shao M, Arruda R, et al. Multilevel feedback architecture for adaptive regulation of learning in the insect brain. bioRxiv. 2019; p. 649731.
View Article
Google Scholar

[26] View Article

[27] Google Scholar

[ref8] 8. Waddell S. Reinforcement signalling in Drosophila; dopamine does it all after all. Current Opinion in Neurobiology. 2013;23(3):324–329. pmid:23391527
View Article
PubMed/NCBI
Google Scholar

[29] View Article

[30] PubMed/NCBI

[31] Google Scholar

[ref9] 9. Aso Y, Hattori D, Yu Y, Johnston RM, Iyer NA, Ngo TT, et al. The neuronal architecture of the mushroom body provides a logic for associative learning. Elife. 2014;3:e04577. pmid:25535793
View Article
PubMed/NCBI
Google Scholar

[33] View Article

[34] PubMed/NCBI

[35] Google Scholar

[ref10] 10. Michie D, Spiegelhalter DJ, Taylor CC, editors. Machine Learning, Neural and Statistical Classification. Ellis Horwood; 1994.

[ref11] 11. Campbell RA, Honegger KS, Qin H, Li W, Demir E, Turner GC. Imaging a population code for odor identity in the Drosophila mushroom body. Journal of Neuroscience. 2013;33(25):10568–10581. pmid:23785169
View Article
PubMed/NCBI
Google Scholar

[38] View Article

[39] PubMed/NCBI

[40] Google Scholar

[ref12] 12. Linderman S, Stock CH, Adams RP. A framework for studying synaptic plasticity with neural spike train data. Advances in Neural Information Processing Systems. 2014;27.
View Article
Google Scholar

[42] View Article

[43] Google Scholar

[ref13] 13. Huerta R, Nowotny T, García-Sanchez M, Abarbanel HDI, Rabinovich MI. Learning classification in the olfactory system of insects. Neural Computation. 2004;16(8):1601–1640. pmid:15228747
View Article
PubMed/NCBI
Google Scholar

[45] View Article

[46] PubMed/NCBI

[47] Google Scholar

[ref14] 14. Smith D, Wessnitzer J, Webb B. A model of associative learning in the mushroom body. Biological Cybernetics. 2008;99(2):89–103. pmid:18607623
View Article
PubMed/NCBI
Google Scholar

[49] View Article

[50] PubMed/NCBI

[51] Google Scholar

[ref15] 15. Huerta R, Nowotny T. Fast and robust learning by reinforcement signals: explorations in the insect brain. Neural Computation. 2009;21(8):2123–2151. pmid:19538091
View Article
PubMed/NCBI
Google Scholar

[53] View Article

[54] PubMed/NCBI

[55] Google Scholar

[ref16] 16. Bazhenov M, Huerta R, Smith BH. A computational framework for understanding decision making through integration of basic learning rules. Journal of Neuroscience. 2013;33(13):5686–5697. pmid:23536082
View Article
PubMed/NCBI
Google Scholar

[57] View Article

[58] PubMed/NCBI

[59] Google Scholar

[ref17] 17. Peng F, Chittka L. A simple computational model of the bee mushroom body can explain seemingly complex forms of olfactory learning and memory. Current Biology. 2017;27(2):224–230. pmid:28017607
View Article
PubMed/NCBI
Google Scholar

[61] View Article

[62] PubMed/NCBI

[63] Google Scholar

[ref18] 18. Bennett JE, Philippides A, Nowotny T. Learning with reinforcement prediction errors in a model of the Drosophila mushroom body. Nature Communications. 2021;12(1):1–14.
View Article
Google Scholar

[65] View Article

[66] Google Scholar

[ref19] 19. Lei PW, Koehly LM. Linear discriminant analysis versus logistic regression: A comparison of classification errors in the two-group case. The Journal of Experimental Education. 2003;72(1):25–49.
View Article
Google Scholar

[68] View Article

[69] Google Scholar

[ref20] 20. Huerta R, Vembu S, Amigó JM, Nowotny T, Elkan C. Inhibition in multiclass classification. Neural Computation. 2012;24(9):2473–2507. pmid:22594829
View Article
PubMed/NCBI
Google Scholar

[71] View Article

[72] PubMed/NCBI

[73] Google Scholar

[ref21] 21. Huerta R. Learning pattern recognition and decision making in the insect brain. In: AIP Conference Proceedings. vol. 1510. American Institute of Physics; 2013. p. 101–119.

[ref22] 22. Li F, Lindsey JW, Marin EC, Otto N, Dreher M, Dempsey G, et al. The connectome of the adult Drosophila mushroom body provides insights into function. Elife. 2020;9:e62576. pmid:33315010
View Article
PubMed/NCBI
Google Scholar

[76] View Article

[77] PubMed/NCBI

[78] Google Scholar

[ref23] 23. Hastie T, Tibshirani R, Friedman JH. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. vol. 2. Springer; 2009.

Figures

Abstract

Author summary

Introduction

Results

LDA model of the mushroom body compartment

Numerical experiments

Synthetic dataset.

KC activities dataset.

Competing MBONs.

Discussion

Summary

Model predictions

Relation to existing models

Comparison of LDA to other linear classification methods

Limitations

Methods

Linear discriminant analysis

An online algorithm for imbalanced learning

Estimating the mean response to a neutral odor and the covariance.

Estimating the mean response to a conditioned odor.

Estimating the bias.

Details of numerical experiments

Supporting information

S1 Appendix. Comparison with a modified algorithm.

Acknowledgments

References