Existence of Stationary Markov Perfect Equilibria in Stochastic Altruistic Growth Economies

Abstract

We study an altruistic growth model with production uncertainty viewed as an intergenerational stochastic game. The existence of stationary Markov perfect equilibria is proved under general assumptions on utility functions for the generations and for non-atomic transition probabilities. This paper answers some issues that arose from the literature in the 1980s decade.

1 Introduction

In many real-life situations, the preferences of an economic agent change over time. Rational behaviour of such agents was studied by Strotz [1], Pollak [2], Bernheim and Ray [3] who considered so-called ‘consistent plans’. Phelps and Pollak [4] introduced the notion of ‘quasi-hyperbolic discounting’, which is a modification of the classical discounting proposed in 1937 by Samuelson [5]. Within such a framework, an economic agent is represented by a sequence of ‘selves’ who play a non-cooperative discrete-time dynamic game with appropriately defined payoff functions, see [6] and references cited therein. Alternatively, one can look at the model of Phelps and Pollak [4] as an intergenerational game with altruism between generations. In such a game, it is assumed that each generation lives over just one period and consumes a fixed good. The part left after consumption constitutes an investment for following generations. Therefore, each generation derives utility from its own consumption and those of its descendants. The next generation’s endowment is determined by investment and certain production function.

The existence of a Markov perfect equilibrium in an intergenerational dynamic game is a fixed point problem in an appropriately defined function space. The set of possible endowments is usually an interval in the real line. This problem was successfully treated by Bernheim and Ray [7, 8] and Leininger [9] for certain classes of deterministic bequest games, where utility of each generation depends only on its own consumption and that of an immediate descendant (see also pp. 507–509 in [10]). Related results on subgame Markov perfect equilibria were also obtained in [11, 12]. In particular, Ray [12] dealt with games with general utility functions depending on partial utilities of countably many descendants. These models are termed ‘nonpaternalistic intergenerational altruism models’. Unfortunately, Lemma 6 in [12] is false. Certain difficulties arising in the study of deterministic intergenerational games are discussed in Sect. 5. The erroneously stated results from [12] were later applied by Bernheim and Ray [13] to a general intergenerational game, known also as an ‘altruistic growth model’ with production uncertainty.

In this paper, we examine a general version of Bernheim and Ray’s model [13] using a completely different proof’s technique based on a certain extension of Fatou’s Lemma for varying probability measures [14]. Similarly as in [6, 13], we assume that the transition probabilities are non-atomic. As shown in Sect. 5, this assumption plays an important role in our approach. A common representation of the transition probability (stochastic production function) in various models in economic growth theory is given by a difference equation involving a sequence of i.i.d. random shocks; see for instance [3, 6, 13, 15, 16] and references cited therein. For example, Harris and Laibson [6] considered a model with a linear production function with additive shocks and proved the existence theorem on Markov perfect equilibria for so-called ‘hyperbolic consumers’. Other results on intergenerational stochastic games are obtained under different and more specific assumptions on the transition probability functions. A detailed discussion of the literature is given in Remarks 3.2 and 3.3.

2 The Model

Let $\mathbb {R}$ be the set of all real numbers, $\mathbb {N}$ denotes the set of all positive integers. Put $S:=[0,\overline{s}]$ for some fixed $\overline{s}>0.$ Let $S^\infty := S\times S\times \dots$ be endowed with the product topology and the Borel $\sigma$-algebra. Define

$$\begin{aligned} A(s):=[0,s] \text{ for } \text{ every } s\in S\quad \text{ and }\quad D:=\{(s,a):\;s\in S,\; a\in A(s)\}. \end{aligned}$$

Consider an infinite sequence of generations labelled by $t\in \mathbb {N}.$ There is one commodity, which may be consumed or invested. Every generation lives one period and derives utility from its own consumption and consumptions of some or all of its descendants. Generation $t$ receives the endowment $s_t\in S$ and chooses consumption at a level $a_t\in A(s_t).$ The investment of $y_t:=s_t-a_t$ determines the endowment of its successor according to the transition probability $q$ from $S$ to $S,$ which depends on $y_t\in A(s_t)$. Furthermore, for any $t\in \mathbb {N}$ by $H^t$, we denote the set of all sequences

$$\begin{aligned} h^t= (a_t,s_{t+1},a_{t+1},s_{t+2},\dots ) \in S^\infty \quad \text{ such } \text{ that } (s_k,a_k)\in D \text{ for } \text{ all } k\ge t. \end{aligned}$$

$H^t$ is the set of feasible future histories of the consumption-investment process from period $t$ onwards. Following Bernheim and Ray [13], we assume that generation $t$’s preferences are represented by the utility function of the form

$$\begin{aligned} U_t(h^t):= u(a_t) + w(a_{t+1},a_{t+2},\dots ) \end{aligned}$$

(1)

where $u:S \mapsto \mathbb {R}$ and $w: S^\infty \mapsto \mathbb {R}$ are bounded Borel measurable functions.

Let $\Phi$ be the set of Borel measurable functions $\varphi : S\mapsto S$ such that $\varphi (s)\in A(s)$ for each $s\in S.$ A strategy for generation $t$ is a function $c_t\in \Phi$. If $c_t= c$ for all $t\in \mathbb {N}$ and some $c\in \Phi ,$ then we say that the generations employ a stationary strategy. The transition probability induced by $q$ and any $c\in \Phi$ is $q(\cdot |i(s))$ where $i(s):=s-c(s)$ is the investment or saving in state $s\in S.$

We endow $H^t$ with the product $\sigma$-algebra. Assume that every generation $\tau$ ($\tau \ge t$) is going to employ a stationary strategy $c\in \Phi .$ By the Ionescu-Tulcea theorem (see Proposition V.1.1 in [17] or Chapter 7 in [18]), for each $s_t \in S$, there exists a unique probability measure $P^{c}_{s_t}$ on $H^t$ induced by the stationary strategy $c\in \Phi$ used by each generation $\tau$ ($\tau \ge t$) and the transition probability $q.$ By $E^{c}_{s_t}$, we denote the expectation operator corresponding to the measure $P^{c}_{s_t}.$ Then the expected utility of generation $t$ is

$$\begin{aligned} W_t(c)(s_t): = E^c_{s_t}U_t(h^t) = u(c(s_t)) + E^c_{s_t}[w(a_{t+1},a_{t+2},\dots )]. \end{aligned}$$

For any $c\in \Phi ,$ $j\ge 2$ and $s_j\in S,$ put

$$\begin{aligned} J(c)(s_j):= E^c_{s_j}[w(a_{j},a_{j+1},\dots )]. \end{aligned}$$

(2)

In the sequel, we assume that $w$ is continuous. This fact will imply that $w$ is the uniform limit of a sequence $(w_n)_{n\in \mathbb {N}}$ of continuous functions, where $w_n$ depends on first $n$ coordinates in $S^\infty .$ Clearly, every $w_n$ can also be viewed as a function on $S^\infty .$ Then, we have that

$$\begin{aligned}&J_n(c)(s_{t+1}) = E^c_{s_{t+1}}[w_n(a_{t+1},a_{t+2},\dots ,a_{t+n})]\\&\quad = \int _S \!\cdots \! \int _S w_n(c(s_{t+1}),c(s_{t+2}),\dots ,c(s_{t+n}))q(\mathrm{d}s_{t+n}|i(s_{t+n-1}))\cdots q(\mathrm{d}s_{t+2}|i(s_{t+1})) \end{aligned}$$

and by the dominated convergence theorem

$$\begin{aligned} W_t(c)(s_t) = u(c(s_t)) +\lim _{n\rightarrow \infty }\int _S J_n(c)(s_{t+1})q(\mathrm{d}s_{t+1}|i(s_t)). \end{aligned}$$

(3)

Let us define

$$\begin{aligned} W(c)(s) := u(c(s)) +\lim _{n\rightarrow \infty }\int _S J_n(c)(s')q(\mathrm{d}s'|i(s)) \end{aligned}$$

and

$$\begin{aligned} P(a,c)(s):=u(a)+\int _S J(c)(s')q(\mathrm{d}s'|s-a), \end{aligned}$$

where $s\in S$, $a \in A(s)$ and $c\in \Phi$. If $s_t=s,$ then $P(a,c)(s)$ is the utility for generation $t$ choosing $a \in A(s_t)$ in this state under the assumption that all future generations will employ a stationary strategy $c\in \Phi .$

Definition 2.1

A Stationary Markov Perfect Equilibrium ($SMPE$) is a function $c^*\in \Phi$ such that for every $s\in S$ we have

$$\begin{aligned} \sup _{a\in A(s)}P(a,c^*)(s)=P(c^*(s),c^*)(s)=W(c^*)(s). \end{aligned}$$

(4)

Note that equality (4) says that, if all descendants of generation $t$ are going to employ $c^*$, then the best choice for the fresh generation in state $s=s_t\in S$ is $c^*(s_t)$. Arguments for studying a SMPE are based on its simplicity and the property that behaviour of the players is independent of irrelevant payoff histories [19].

Utility functions of the form (1) were considered by Bernheim and Ray [13]. However, two special cases, strongly exploited in the literature, are worth mentioning. Namely, in a number of papers, it is assumed that

$$\begin{aligned} U_t(h^t):= u(a_t) + \widetilde{w}(a_{t+1}), \end{aligned}$$

(5)

where $\widetilde{w}$ is a function of one variable. This model is called an intergenerational game with paternalistic altruism or a bequest game. The reader is referred to the following papers [8–10, 20–22], where this case was described in details. The second model, intensively studied by numerous authors, is concerned with a generalisation of Samuelson’s discounted utility [5] and was suggested by Phelps and Pollak [4]. Then, utility (1) takes the following form

$$\begin{aligned} U_t(h^t):= u(a_t)+ \alpha \beta \sum _{\tau =t+1}^{\infty } \beta ^{\tau -t-1}u(a_\tau ), \end{aligned}$$

(6)

where $\alpha > 0$ is a short-run discount factor, $\beta \in ]0,1[$ is a long-run discount factor; see [6]. Sometimes, $\alpha$ is also called as an altruism factor towards future generations. This intergenerational game can be viewed as a dynamic choice model with quasi-hyperbolic discounting. Such a formulation of the problem is due to Phelps and Pollak [4] and was applied by Harris and Laibson [6] to a study of a consumption and savings model under uncertainty with ‘hyperbolic consumers’. Within such a framework, an individual decision maker is represented by a sequence of ‘selves’ indexed by discrete-time parameter $t\in \mathbb {R},$ who play a sequential game. The utility of self $t$ is defined as in (6) and corresponds to the situation, where preferences of the decision maker change over time. For more detailed discussion, the reader is referred to [6] and references cited therein.

3 The Main Result

Let Y be a compact metric space. By $C(Y)$, we denote the set of all real-valued continuous functions on $Y.$ Clearly, $C(Y)$ is a Banach space when endowed with the supremum norm. By $P(Y)$, we denote the set of all probability measures on the Borel subsets of $Y.$ It is assumed that $P(Y)$ is endowed with the topology of weak convergence. A sequence $(\mu _n)_{n\in \mathbb {N}}$ of probability measures on $Y$ converges weakly to some $\mu \in P(Y)$ (which is denoted by $\mu _n\Rightarrow \mu$) iff $\lim _{n\rightarrow \infty }\int _Yg(y)\mu _n(\mathrm{d}y) =\int _Yg(y)\mu (\mathrm{d}y)$ for every $g\in C(Y).$

We now formulate our basic assumptions.

(A1) $u\in C(S)$ is strictly concave and increasing and $w\in C(S^\infty );$

(A2) $q$ is weakly continuous on $S$, that is, if $y_m\rightarrow y_0$ in $S$ (as $m\rightarrow \infty$), then $q(\cdot |y_m) \Rightarrow q(\cdot |y_0).$ Moreover, for each $y\in S\setminus \{0\},$ the probability measure $q(\cdot |y)$ is non-atomic, $q(\cdot |0)$ is non-atomic or $q(\{0\}|0)=1.$

Remark 3.1

Assumption (A1) implies that the functions $u$ and $w$ are bounded. If we wish to deal with unbounded utility functions, e.g. the logarithmic function, then some integrability conditions must be met. Moreover, certain requirements on consumption strategies can be unavoidable, either. For instance, let us consider the model with ‘hyperbolic consumers’ [4, 6], i.e. when the utility of generation $t$ is the expected value of (6). Assume that $S=[0,1]$ and $u(a)=\ln a.$ Further, suppose that each generation uses the same strategy: $c(s)=\exp {(-1/s)}$ for $s>0$ and $c(0)=0.$ Suppose also that the transition probability at each state enjoys the uniform distribution on $[0,1].$ Then, $\int _S u(c(s))\mathrm{d}s= -\int _0^1 \frac{1}{s}\mathrm{d}s= -\infty .$ Hence, it follows that the expected utility of each generation $t$ equals $-\infty .$ This fact implies that $c$ constitutes an equilibrium that is rather senseless. In order to obtain the existence of reasonable equilibria in games with unbounded from below utility functions $u$, one has to impose additional and rather technical assumptions. Such problems do not arise in the deterministic case. If $w$ is bounded, then our results can also be stated for unbounded, e.g. isoelastic utility functions $u.$

Remark 3.2

A typical representation of the transition probability in economic growth theory is

$$\begin{aligned} s_{t+1}= \bar{f}(y_t,z_t), \end{aligned}$$

where $y_t=s_t - a_t$ is the investment in state $s_t$, $(z_t)_{t\in \mathbb {N}}$ is a sequence of i.i.d. random ‘shocks’ having a probability distribution $\pi$; see [3, 13, 15, 16] for example. The function $\bar{f}$ is continuous and for any Borel set $B$ in $S$ and investment $y\in S$, we have

$$\begin{aligned} q(B|y)= \int _S 1_B(\bar{f}(y,z)) \pi (\mathrm{d}z). \end{aligned}$$

We would like to point out three special cases:

(C1) $\bar{f}(y_t,z_t)= z_tf_1(y_t) +(1-z_t)f_2(y_t)$ where $f_1,\ f_2:S \mapsto S$ are continuous increasing functions such that $f_1(y)<f_2(y)$ for each $y\in ]0,\bar{s}[$ and $f_1(0)=f_2(0)=0.$ In addition, $\pi$ is a non-atomic probability measure on $[0,1].$

(C2) The model with additive shocks: $\bar{f}(y_t,z_t)= f(y_t) + z_t$ where $f:S\mapsto S$ is a continuous increasing function. In addition, there is a state $\hat{s}\in ]0,\bar{s}[$ such that $f(y)>y$ for $y\in ]0,\hat{s}[$ and $f(y)< y$ for $y\in ]\hat{s}, \bar{s}[.$ The probability measure $\pi$ is non-atomic with support included in $[0, \bar{s}-f(\bar{s})].$

(C3) The model with multiplicative shocks: $\bar{f}(y_t,z_t)= f(y_t)z_t$ where $f$ is as in (C2) and the probability measure $\pi$ is non-atomic with support included in $[0, \bar{s}/f(\bar{s})].$

A more specific transition probability is of the following form:

(C4) $q(B|y) = \sum _{j=1}^l g_j(y)\mu _j(B)$ where $B$ is a Borel subset of $S,$ $\mu _1,\dots ,\mu _l$ are non-negative measures on $S$ and the functions $g_1,\dots ,g_l \ge 0$ are continuous.

In all cases (C1)–(C3), the transition probability $q$ satisfies assumption (A2). If the measures $\mu _1,\dots ,\mu _l$ are non-atomic, then $q$ given in (C4) also satisfies (A2). Therefore, our model embraces all these particular aforementioned transition probabilities.

Let $I$ denotes the set of non-decreasing lower semicontinuous functions $\phi : S\mapsto \mathbb {R}$ such that $\phi (s)\in A(s)$ for each $s\in S.$ Note that every $\phi \in I$ is continuous from the left and has at most a countable number of discontinuity points. Put

$$\begin{aligned} F:=\{c\in \Phi :\; c(s)=s-i(s),\; i\in I\}. \end{aligned}$$

Every $c\in F$ is upper semicontinuous and continuous from the left.

Our main result is the following.

Theorem 3.1

If (A1)–(A2) hold, then there exists a $SMPE$ $c^* \in F.$

Remark 3.3

The existence of a SMPE in deterministic bequest games [with utility function given in (5)] was proved by Bernheim and Ray [8] and Leininger [9]. The extensions to stochastic bequest games were examined in [20–23]. For instance, Amir [20] considered games with some stochastic ‘convex production function’, but his assumptions on the transition probability are different from the ones discussed in (C1)–(C4). However, the results reported in [23], and later extended in [21, 22], were obtained for the transition probabilities as in (C4). Additionally, it was assumed that functions $g_j$ and measures $\mu _j$ in (C4) meet certain requirements. The results in [22] are stated for Borel state space games and the proof makes use of the Dvoretzky–Wald–Wolfowitz purification theorem from the statistical decision theory. Intergenerational stochastic games, where each generation has countably many descendants and the utility is of the form as in (6), were studied in [24] under condition that the state space is denumerable and in [6, 22, 25] within a framework of consumption-savings problems with an uncountable state space. Underlying assumptions on the transition structure in [22, 25] are similar to the ones described in (C4). The model considered by Harris and Laibson [6] was based on assumption (C2) with the state space $S=[0,\infty [$ and the linear function $f$. Their paper provides the existence theorem for a SMPE but for a specific model (‘hyperbolic consumers’) and under relatively stronger assumptions. Therefore, Theorem 3.1 in this paper fills a serious gap in the existence of a SMPE in the theory of stochastic intergenerational games with general utility functions. The result reported by Bernheim and Ray on page 199 in [13] is a special case of our result. As already mentioned, the proof given in [13] is based on incorrect Lemma 6 in [12].

Remark 3.4

The idea of using the class $F$ of strategies for analysing equilibria in deterministic bequest games comes from Bernheim and Ray [8]. Further, it was successfully applied to the study of other classes of dynamic games with simultaneous moves; see [26, 27].

The function $w$ is bounded because $S^\infty$ is a compact metric space. Let $a=(a_1,a_2,\dots ) \in S^\infty .$ Define $a(n) :=(a_1,a_2,\dots ,a_n)$. Let $b^{n+1} :=(b_{n+1}, b_{n+2},\dots )$ be an arbitrary element of $S^\infty .$ Write $(a(n),b^{n+1})$ for the sequence $(a_1,\ldots ,a_n,b_{n+1},b_{n+2},\ldots ) \in S^\infty .$ Let

$$\begin{aligned} w_n(a):=\min _{b^{n+1}\in S^\infty }w(a(n),b^{n+1}). \end{aligned}$$

Clearly, $w_n$ can be recognized as a function on $S^\infty$ that depends on the first $n$ coordinates only.

Lemma 3.1

Every $w_n$ is continuous on $S^\infty$ and

$$\begin{aligned} \lim _{n\rightarrow \infty }\sup _{a\in S^\infty }|w_n(a)-w(a)|=0. \end{aligned}$$

Proof

The continuity of every $w_n$ is obvious. Let $a=(a_1,a_2,\dots )\in S^\infty .$ For any $n\in \mathbb {N}$, choose $\widetilde{b}^{n+1}$ such that $w_n(a)=w(a(n),\widetilde{b}^{n+1}).$ Observe that $k_n := (a(n),\widetilde{b}^{n+1})$ converges in the product topology on $S^\infty$ to $a.$ Hence $w_n(a)= w(k_n) \rightarrow w(a)$ as $n\rightarrow \infty .$ Because $w_n \le w_{n+1}$ on the compact space $S^\infty$, by Dini’s Theorem, the convergence is uniform. $\square$

We recall that the function $u$ is strictly concave and continuous on $S$. Let $v:S \mapsto \mathbb {R}$ be continuous. Put

$$\begin{aligned} V(s,y):= u(s-y)+ v(y), \quad y\in A(s). \end{aligned}$$

Define

$$\begin{aligned} A_0(s):=\mathrm{arg}\max _{y\in A(s)} V(s,y) \quad \mathrm{and}\quad i_0(s):= \min A_0(s), \quad s\in S. \end{aligned}$$

Obviously, $A_0(0)=\{0\}$ and $A_0(s)$ is non-empty and compact for each $s\in S.$ Thus, the function $i_0$ is well defined.

The following result is a simple modification of Theorem 6.3 in [28].

Lemma 3.2

Let the above assumptions on $u$ and $v$ hold. Then the correspondence $s\mapsto A_0(s)$ has a closed graph and is strongly ascending, i.e. if $s_1<s_2$ and $y_1 \in A_0(s_1),$ $y_2 \in A_0(s_2)$, then $y_1\le y_2.$ Moreover, the function $i_0$ is lower semicontinuous and non-decreasing.

Proof

Suppose that $s\mapsto A_0(s)$ is not strongly ascending. Then there exist $s_1 < s_2$ and $y_1 \in A_0(s_1),$ $y_2 \in A_0(s_2)$ such that $y_1 > y_2.$ Clearly, $D$ is a lattice with the usual component-wise order on the plane $R^2.$ Thus, $(s_2,y_1)$ and $(s_1,y_2)$ belong to $D$. Since $u$ is strictly concave, from the proof of Lemma 2 in [23], we conclude that

$$\begin{aligned} u(s_2-y_1)-u(s_2-y_2) > u(s_1-y_1)-u(s_1-y_2). \end{aligned}$$

(7)

Adding $v(y_1)-v(y_2)$ to both sides of (7) and remembering that $y_1 \in A_0(s_1)$ and $y_2 \in A_0(s_2)$, we obtain

$$\begin{aligned} 0\ge V(s_2,y_1)-V(s_2,y_2) > V(s_1,y_1)-V(s_1,y_2)\ge 0. \end{aligned}$$

This contradiction implies that the correspondence $s\mapsto A_0(s)$ is strongly ascending. Obviously, it has a closed graph. Thus, the function $i_0$ is non-decreasing and continuous from the left. Consequently, $i_0$ is lower semicontinuous. $\square$

Remark 3.5

Lemma 3.2 does not follow from Theorem 6.3 in [28], because in general, the function $V$ cannot be extended from its domain $D$ to the product space $S\times S$ so that this extension has increasing differences. For example, consider $u(s-y) =\sqrt{s-y},$ $(s,y)\in D.$

Lemma 3.3

Let the assumptions of Lemma 3.2 hold. Let $\varphi :S\mapsto S$ be a non-decreasing function such that $\varphi (s) \in A_0(s)$ for each $s\in S.$ If $s_0$ is a continuity point of $\varphi$, then $A_0(s_0)$ is a singleton.

Proof

Suppose that $y_1$ and $y_2$ belong to $A_0(s_0)$ and $y_1 < y_2$. Since $s\mapsto A_0(s)$ is strongly ascending, we conclude that $\lim _{s\rightarrow s_0^-}\varphi (s) \le y_1 <y_2\le \lim _{s\rightarrow s_0^+}\varphi (s).$ This contradicts our assumption that $\varphi$ is continuous at $s_0\in S.$ $\square$

Example 3.1

Let $S=[0,1]$, $u(s-y)=2(s-y) -(s-y)^2$ and $v(y)=3y^2$ for $y\in A(s),$ $s\in S.$ It is easy to check that $A_0(s)= \{0\}$ for $s\in [0,1/2[$, $A_0(s)= \{s\}$ for $s\in ]1/2,1]$ and $A_0(1/2)= \{0, 1/2\}$. Hence $i_0(s)= 0$ for $s\in [0,1/2]$ and $i_0(s)= s$ for $s\in ]1/2,1].$ Note that there is no continuous function $\varphi : S\mapsto S$ such that $\varphi (s) \in A_0(s)$ for each $s\in S.$

Let $X$ be the vector space of real-valued functions of bounded variation on $S$ which are continuous from the left. Let $(\eta _m)_{m\in \mathbb {N}}$ be a sequence of functions in $X.$ We say that $(\eta _m)_{m\in \mathbb {N}}$ converges weakly to some $\eta \in X$ iff $\lim _{m\rightarrow \infty } \eta _m(s)=\eta (s)$ for every continuity point of $\eta$ in $]0,\bar{s}[.$ The weak convergence of $(\eta _m)_{m\in \mathbb {N}}$ to $\eta$ is denoted by $\eta _m \mathop {\rightarrow }\limits ^{\omega } \eta .$

We endow $I\subset X$ with the topology of weak convergence. Let $\mathcal{M}$ be the space of all regular signed measures on $S$ with bounded variation. It is well known that $\mathcal{M}$ is the dual of $C(S)$ (see Theorem 14.14 in [29]). Moreover, $\mathcal{M}$ is a linear metrisable topological space when equipped with the weak-star topology. In addition, there is a homeomorphism between $I$ and the set $\mathcal {M}_S$ of non-negative measures $\mu \in \mathcal{M}$ such that $\mu (S)\le \overline{s}.$ By the Banach–Alaoglu Theorem, we infer that $\mathcal {M}_S$ is compact in the weak-star topology. Thus, $I$ is also compact. This fact can also be deduced from Helly’s Theorem; see [30]. It is obvious that $F\subset X$ is convex and is obtained by a continuous transformation of $I$, namely $c(s)= s- i(s),$ $s\in S,$ $i\in I.$ Hence, we arrive at the following auxiliary result.

Lemma 3.4

$F$ is a convex sequentially compact subset of the space $X$ endowed with the topology of weak convergence.

Let $S_c$ denote the union of the set of continuity points of $c\in F$ contained in $]0,\bar{s}[$ and $\{0\}.$

Lemma 3.5

Let $c^m \mathop {\rightarrow }\limits ^{\omega } c$ in $F$ and let $i^m(s):= s-c^m(s),$ $i(s):=s-c(s),$ $s\in S.$ If $x_m\rightarrow x\in S_c\setminus \{0\}$ and $x_m\in S$ for every $m\in \mathbb {N}$, then $\lim _{m\rightarrow \infty }c^m(x_m)=c(x)$ and $\lim _{m\rightarrow \infty }i^m(x_m)=i(x).$ Moreover, $\lim _{m\rightarrow \infty }c^m(x_m)=c(0)$ and $\lim _{m\rightarrow \infty }i^m(x_m)=i(0)$ for every $x_m\searrow 0.$

Proof

It is more convenient to work with the functions $i^m$ and $i,$ because they are non-decreasing. Take any sequence $x_m \searrow 0$ as $m\rightarrow \infty .$ We have $0\le i^m(x_m) \le x_m$ for each $m\in \mathbb {N}.$ Hence, $i^m(x_m)\rightarrow i(0)=0$ as $m\rightarrow \infty .$ Next note that the set of continuity points for the functions $c$ and $i$ coincide. Let $x \in S_c\setminus \{0\}.$ Assume that $x_m\rightarrow x$ as $m\rightarrow \infty .$ Choose $x',\ x'' \in S_c$ such that $x' <x<x''.$ Then for sufficiently large $m$ we have

$$\begin{aligned} i^m(x')\le i^m(x_m) \le i^m(x''). \end{aligned}$$

Hence,

$$\begin{aligned} i(x')\le \liminf _{m\rightarrow \infty }i^m(x_m) \le \limsup _{m\rightarrow \infty }i^m(x_m) \le i(x''). \end{aligned}$$

Since $x'$ and $x''$ can be chosen arbitrarily close to $x$, we infer that $i(x)= \lim _{m\rightarrow \infty }i^m(x_m).$ Now the lemma follows from the fact that $c^m(x_m)=x_m-i^m(x_m)$ and $c(x)=x-i(x).$ $\square$

Next result is due to Serfozo; see Lemma 3.2 in [14]. Let $(\phi _m)_{m\in \mathbb {N}}$ be a bounded sequence of Borel measurable real-valued functions on $S.$ For each $x\in S,$ define

$$\begin{aligned}&\phi _*(x) := \inf \{\liminf _{m\rightarrow \infty } \phi _m(x_m):\;x_m\rightarrow x\} \quad \text{ and }\\&\quad \phi ^*(x) := \sup \{\limsup _{m\rightarrow \infty } \phi _m(x_m):\;x_m\rightarrow x\}. \end{aligned}$$

Lemma 3.6

Assume that $\mu _m\Rightarrow \mu$ as $m\rightarrow \infty .$ Then

$$\begin{aligned}&\int _S \phi _*(s)\mu (\mathrm{d}s) \le \liminf _{m\rightarrow \infty } \int _S \phi _m(s) \mu _m(\mathrm{d}s) \quad \mathrm{and}\\&\quad \int _S \phi ^*(s)\mu (\mathrm{d}s) \ge \limsup _{m\rightarrow \infty } \int _S \phi _m(s) \mu _m(\mathrm{d}s). \end{aligned}$$

Lemma 3.7

Fix any $c\in F.$ Assume that $\mu _m\Rightarrow \mu$ as $m\rightarrow \infty$ and $\mu$ has no atoms in $]0,\overline{s}].$ Let $\phi$ and $\phi _m$ $(m\in \mathbb {N})$ be bounded Borel measurable functions on $S$. Assume that for any $x\in S_c$ and $x_m\rightarrow x$ as $m\rightarrow \infty$, we have $\phi _m(x_m) \rightarrow \phi (x).$ Then

$$\begin{aligned} \int _S\phi _m(s)\mu _m(\mathrm{d}s) \rightarrow \int _S\phi (s)\mu (\mathrm{d}s). \end{aligned}$$

(8)

Proof

Clearly, $\phi (x) =\phi _*(x) =\phi ^*(x)$ for each $x\in S_c.$ (Recall that $0\in S_c$). Since $\mu$ has no atoms in $]0,\overline{s}]$, we have

$$\begin{aligned} \int _S\phi (s)\mu (\mathrm{d}s) = \int _S\phi _*(s) \mu (\mathrm{d}s)=\int _S\phi ^*(s)\mu (\mathrm{d}s). \end{aligned}$$

This fact and Lemma 3.6 imply that

$$\begin{aligned} \limsup _{m\rightarrow \infty }\int _S\phi _m(s) \mu _m(\mathrm{d}s)\le \int _S\phi (s)\mu (\mathrm{d}s)\le \liminf _{m\rightarrow \infty }\int _S \phi _m(s)\mu _m(\mathrm{d}s). \end{aligned}$$

Hence (8) follows. $\square$

Lemma 3.8

Assume that (A1)–(A2) hold. Let $c^m \mathop {\rightarrow }\limits ^{\omega } c$ in $F,$ $x^1\in S_c$ and $x^1_m \rightarrow x^1$ as $m\rightarrow \infty .$ Then, $J(c^m)(x^1_m)\rightarrow J(c)(x^1)$ as $m\rightarrow \infty .$

Proof

Step 1. Let $g\in C(S^\infty )$ and let $g$ depend on its first $n$ coordinates. We claim that

$$\begin{aligned} E^{c^m}_{x^1_m}[g(a_{1},\ldots , a_{n})] \rightarrow E^{c}_{x^1}[g(a_{1},\ldots , a_{n})] \end{aligned}$$

(9)

as $m\rightarrow \infty .$ If $n=1,$ then using Lemma 3.5, it is obvious that

$$\begin{aligned} E^{c^m}_{x^1_m}[g(a_{1})]=g(c^m(x^1_m))\rightarrow g(c(x^1))=E^{c}_{x^1}[g(a_{1})]. \end{aligned}$$

Observe that if $n\ge 2,$ then we have

$$\begin{aligned}&{E^{c^m}_{x^1_m}[g(a_{1},\ldots , a_{n})]=} \\&\int _S \ldots \int _Sg(c^m(x^1_m),c^m(s^2),\ldots ,c^m(s^{n}))q(\mathrm{d}s^{n}|i^m(s^{n-1})) \ldots q(\mathrm{d}s^{2}|i^m(x^1_m)), \end{aligned}$$

where $i^m(s):=s-c^m(s),$ $s\in S.$ Recall that $i(s)=s-c(s),$ $x\in S.$ Assume that $x^2,\ldots ,x^{n}$ are any points from the set $S_c.$ Next take any sequences $(x_m^{k})_{m\in \mathbb {N}}$ such that $x_m^{k}\rightarrow x^k$ for $2\le k\le n,$ $m\rightarrow \infty .$ We have already chosen a sequence $(x_m^{1})_{m\in \mathbb {N}}$ converging to $x^1\in S_c.$ By Lemma 3.5 and assumption (A2), we get that

$$\begin{aligned} q(\cdot |i^m(x_m^{k}))\Rightarrow q(\cdot |i(x^{k})) \quad \text{ as } m\rightarrow \infty \text{ and } \text{ for } 1\le k<n. \end{aligned}$$

(10)

Define

$$\begin{aligned} \phi _m(\cdot ):=g(c^m(x_m^{1}),\ldots ,c^m(x_m^{n-1}),c^m(\cdot )), \quad \phi (\cdot ):=g(c(x^{1}),\ldots ,c(x^{n-1}),c(\cdot )). \end{aligned}$$

By Lemma 3.5 and the continuity of $g$, we conclude that

$$\begin{aligned} g(c^m(x_m^{1}),\ldots ,c^m(x_m^{n-1}),c^m(x_m^{n}))\rightarrow g(c(x^1),\ldots ,c(x^{n-1}),c(x^{n})),\quad m\rightarrow \infty . \end{aligned}$$

Thus, the functions $\phi _m$ and $\phi$ satisfy assumptions of Lemma 3.7. Hence, by Lemma 3.7 and (10) for $k=n-1$, we deduce that

$$\begin{aligned}&{\int _Sg(c^m(x^1_m), \ldots ,c^m(x_m^{n-2}),c^m(x_m^{n-1}),c^m(s^{n}))q(\mathrm{d}s^n|i^m(x_m^{n-1}))\rightarrow }\nonumber \\&\int _Sg(c(x^1),\ldots ,c(x^{n-2}),c(x^{n-1}), c(s^{n}))q(\mathrm{d}s^n|i(x^{n-1})). \end{aligned}$$

(11)

We now continue the above procedure. We consider

$$\begin{aligned}&\displaystyle \phi _m(\cdot ):= \int _Sg(c^m(x_m^1),\ldots ,c^m(x_m^{n-2}),c^m(\cdot ),c^m(s^{n}))q(\mathrm{d}s^n|i^m(\cdot )),\\&\displaystyle \phi (\cdot ):= \int _Sg(c(x^1),\ldots ,c(x^{n-2}),c(\cdot ), c(s^{n}))q(\mathrm{d}s^n|i(\cdot )) \end{aligned}$$

and using (11), we observe that these functions also satisfy the assumptions of Lemma 3.7. Therefore, making use of (10) for $k=n-2,$ Lemma 3.7 we infer that

$$\begin{aligned}&{\int _S\int _Sg(c^m(x^1_m),\ldots ,c^m(x_m^{n-2}),c^m(s^{n-1}),c^m(s^{n})) q(\mathrm{d}s^n|i^m(s^{n-1})) q(\mathrm{d}s^{n-1}|i^m(x_m^{n-2}))\rightarrow }\\&\quad \int _S\int _Sg(c(x^1),\ldots ,c(x^{n-2}),c(s^{n-1}), c(s^{n}))q(\mathrm{d}s^n|i(s^{n-1}))q(\mathrm{d}s^{n-1}|i(x^{n-2})). \end{aligned}$$

Continuing the above reasoning we finally arrive at the conclusion that (9) holds.

Step 2. Consider the sequence of functions $(w_n)_{n\in \mathbb {N}}$ introduced in Lemma 3.1. For any state $x\in S$ and $\tilde{c}\in F$, put $J_n(\tilde{c})(x)= E^{\tilde{c}}_{x}[w_n(a_{1},\ldots , a_{n})].$ By Lemma 3.1, we conclude that

$$\begin{aligned} \lim _{n\rightarrow \infty }\sup _{(x,\tilde{c})\in S\times F}|J_n(\tilde{c})(x) - J(\tilde{c})(x)|=0. \end{aligned}$$

(12)

From (9) with $g$ replaced by $w_n$, we conclude that for every $n\in \mathbb {N}$

$$\begin{aligned} J_n(c^m)(x^{1}_m)=E^{c^m}_{x^{1}_m}[w_n(a_{1},\ldots , a_{n})] \rightarrow J_n(c)(x^{1})=E^{c}_{x^{1}}[w_n(a_{1},\ldots , a_{n})] \end{aligned}$$

as $m\rightarrow \infty .$ Making use of this convergence and (12), we can easily show that

$$\begin{aligned} \lim _{m\rightarrow \infty }J(c^m)(x^{1}_m)= J(c)(x^{1}), \end{aligned}$$

which completes the proof. $\square$

Proof of Theorem 3.1 Let $c\in F$. Put

$$\begin{aligned} A_0(c)(s):=\text{ arg }\max _{y\in A(s)}\widetilde{P}(y,c)(s), \end{aligned}$$

where

$$\begin{aligned} \widetilde{P}(y,c)(s)=u(s-y)+\int _S J(c)(s')q(\mathrm{d}s'|y) \end{aligned}$$

and note that $A_0(c)(s)$ is non-empty and compact for each $s\in S$ and $c\in F.$ Indeed, by Lemma 3.8, we infer that $J(c)(x_m^1)\rightarrow J(c)(x^1)$ for $x^1\in S_c$ and $x_m^1\rightarrow x^1$ as $m\rightarrow \infty .$ Thus, from Lemma 3.7, we deduce that the function $y\mapsto \int _S J(c)(s')q(\mathrm{d}s'|y)$ is continuous. Define

$$\begin{aligned} i_0(s):=\min A_0(c)(s). \end{aligned}$$

(13)

By Lemma 3.2, $i_0\in I.$ Set $c_0(s):=s-i_0(s).$ Then, $c_0\in F.$ We define the mapping $L:F \mapsto F$ by

$$\begin{aligned} Lc(s):=c_0(s). \end{aligned}$$

We now show that $L$ is continuous when $F$ is given the topology of weak convergence. Assume that $c^m\mathop {\rightarrow }\limits ^{\omega } c$ in $F$ and consider $c_0(s)=Lc(s)$ and $c_0^m(s)=Lc^m(s)$ for $m\in \mathbb {N}.$ By Lemma 3.4, the set $\{ c_0^m\}_{m\in \mathbb {N}}$ is relatively compact in $F.$ Let $\widetilde{c}_0$ be any accumulation point of the sequence $(c_0^m)_{m\in \mathbb {N}}$ in $F.$ Denote by $S_0$, the set of continuity points of $\widetilde{c}_0.$ Let $s\in S_0.$ By our assumptions (A1)–(A2) and Lemmas 3.7 and 3.8, we deduce that

$$\begin{aligned} u(c_0^m(s))+\int _S J(c^m)(s')q(\mathrm{d}s'|s-c_0^m(s))\rightarrow u(\widetilde{c}_0(s))+\int _S J(c)(s')q(\mathrm{d}s'|s-\widetilde{c}_0(s)) \end{aligned}$$

(14)

as $m\rightarrow \infty .$ On the other hand, $c_0^m(s)=Lc^m(s)$, so we have that

$$\begin{aligned} P(c_0^m(s),c^m)(s)\ge u(a)+\int _S J(c^m)(s')q(\mathrm{d}s'|s-a). \end{aligned}$$

(15)

for every $(s,a)\in D$ and $s\in S_0.$ Letting $m\rightarrow \infty$ in (15) and making use of the Dominated Convergence Theorem, Lemma 3.8 and (14), we obtain that

$$\begin{aligned} u(\widetilde{c}_0(s))+\int _S J(c)(s')q(\mathrm{d}s'|s-\widetilde{c}_0(s))\ge u(a)+\int _S J(c)(s')q(\mathrm{d}s'|s-a) \end{aligned}$$

for every $(s,a)\in D$ and $s\in S_0.$ Therefore,

$$\begin{aligned} P(\widetilde{c}_0(s),c)(s)=\max _{a\in A(s)}\left[ u(a)+\int _S J(c)(s')q(\mathrm{d}s'|s-a)\right] . \end{aligned}$$

Hence, $\widetilde{c}_0(s)\in \text{ arg }\max _{a\in A(s)}P(a,c)(s).$ Consequently, $\widetilde{i}_0(s):=s-\widetilde{c}_0(s)\in A_0(c)(s).$ Moreover, $s$ is a continuity point of $\widetilde{i}_0.$ From Lemma 3.3, we infer that $A_0(c)(s)$ is a singleton. Therefore, we have

$$\begin{aligned} \widetilde{c}_0(s)=c_0(s)=Lc(s). \end{aligned}$$

If, on the other hand, $s\in S\setminus (S_0\cup \{0\})$, then we may take a sequence $(x_m)_{m\in \mathbb {N}}$ such that $x_m<s,$ $x_m\in S_0\setminus \{0\}$ for each $m\in \mathbb {N}$ and $x_m \rightarrow s$ as $m\rightarrow \infty .$ Obviously, we have $\widetilde{c}_0(x_m)=c_0(x_m)$ for each $m\in \mathbb {N}.$ Since both $\widetilde{c}_0$ and $c_0$ are continuous from the left at $s,$ we get

$$\begin{aligned} \widetilde{c}_0(s)=\lim _{m\rightarrow \infty }\widetilde{c}_0(x_m)=\lim _{m\rightarrow \infty }c_0(x_m)=c_0(s) =Lc(s). \end{aligned}$$

Observe that $\widetilde{c}_0(0)=Lc(0).$ Thus, we have shown that $\widetilde{c}_0=Lc$ and therefore $L$ is a continuous mapping from $F$ into itself. By the Schauder–Tychonoff Fixed Point Theorem, there exists $c^*\in F$ such that $c^*=Lc^*$ and this fact completes the proof.

Remark 3.6

One can prove Theorem 3.1 for concave function $u.$ Then, it is sufficient to take a sequence $(u_n)_{n\in \mathbb {N}}$ of strictly concave functions converging uniformly on $S$ to $u.$ For every $n\in \mathbb {N}$, there exists a SMPE, say $c_n^* \in F.$ Without loss of generality, this sequence has a weak limit, say $c_0^*.$ Using Lemma 3.8 and the fact that $c_0^*$ is continuous from the left, it is easy to prove that $c_0^*$ is a SMPE in the limiting case.

4 Invariant Distributions

One of the major interests in economic applications is to examine stability of the Markov process induced by a SMPE [16]. Therefore, in this section, we provide conditions that guarantee the existence of an invariant distribution. We start with a description of essential notions and auxiliary results.

Let $(\Sigma ,\le _{o})$ be a partially ordered space (poset). An element $\sigma _0\in \Sigma$ is an upper bound of $\Sigma _0\subset \Sigma$ iff $\sigma \le _{o} \sigma _0$ for every $\sigma \in \Sigma _0.$ We define supremum ($\sup \Sigma _0$) as the least upper bound of $\Sigma _0.$ A subset $\Sigma _0$ of $\Sigma$ is a chain iff for each two elements $\sigma _1,\ \sigma _2 \in \Sigma _0$ we have $\sigma _1 \le _{o} \sigma _2$ or $\sigma _2 \le _{o} \sigma _1.$ The set $\Sigma$ is said to be chain-complete poset iff for each chain $\Sigma _0\subset \Sigma$ there exists $\sup \Sigma _0$ in $\Sigma .$ Theorem 9 in [31] is as follows.

Lemma 4.1

Let $\Sigma$ be a chain-complete poset. If $\Psi : \Sigma \mapsto \Sigma$ is a non-decreasing mapping, then the set of fixed points of $\Psi$ is a chain-complete poset in the induced order and has a least element.

Let us set $\Sigma := P(S)$ and define $\le _o$ as the stochastic order relation on $P(S).$ Recall that $\sigma _1 \le _o \sigma _2$ iff for any non-decreasing Borel measurable function $g:S\mapsto \mathbb {R}$, $\int _Sg(s)\sigma _1(\mathrm{d}s) \le \int _Sg(s)\sigma _2(\mathrm{d}s).$ By Proposition 1 in [15], $\Sigma = P(S)$ is a chain-complete poset.

Let us now turn to the game model studied in the previous sections. We make an additional assumption:

(A3) If $y_1< y_2$, then $q(\cdot |y_1)\le _o q(\cdot |y_2).$

Let (A1)–(A3) be satisfied. By Theorem 3.1, there exists a SMPE $c^*\in F.$ Then $s\mapsto i^*(s) = s-c^*(s)$ is non-decreasing on $S.$ Put $q^*(B|s):= q(B|i^*(s))$ where $B$ is a Borel subset of $S,$ $s\in S.$ From (A3), it follows that $s\mapsto q^*(\cdot |s)$ is non-decreasing. Define the mapping $\Psi :P(S)\mapsto P(S)$ by

$$\begin{aligned} \Psi \sigma (B):= \int _S q^*(B|s)\sigma (\mathrm{d}s) \end{aligned}$$

where $B$ is a Borel subset of $S.$ An invariant distribution for the Markov process induced by the transition probability $q^*$ determined by $i^*$ (and thus by a SMPE $c^*$) is any fixed point of $\Psi .$ It is easy to see that if $\sigma _1\le _o \sigma _2$, then $\Psi \sigma _1\le _o \Psi \sigma _2.$ In other words, $\Psi$ is stochastically non-decreasing. Therefore, by Corollary 4 in [15], the set of invariant distributions $\Delta (q^*)$ for the process induced by $q^*$ is non-empty. Summing up the aforementioned discussion and using Lemma 4.1, we obtain the following result.

Theorem 4.1

Assume (A1)–(A3). Let $c^*\in F$ be any SMPE. Then the set of invariant distributions $\Delta (q^*)$ is a chain-complete poset in the induced order and has a least element.

Our next result is as follows.

Theorem 4.2

Under assumptions of Theorem 4.1, the set of invariant distributions $\Delta (q^*)$ is compact in the weak topology on $P(S).$

Proof

Note that, if $q(\{0\}|0)=1$, then the Dirac measure $\delta _0 \in \Delta (q^*).$ Every other distribution $\sigma \in \Delta (q^*)$ is non-atomic. If every $q(\cdot |y)$ is non-atomic ($y\in S$), then each $\sigma \in \Delta (q^*)$ is non-atomic too. In any case, $\sigma \in \Delta (q^*)$ has no atoms in $]0,\overline{s}].$ Choose any sequence $(\sigma _n)_{n\in \mathbb {N}}$ in $\Delta (q^*).$ Without loss of generality, we can assume that this sequence has a limit $\sigma$ in $P(S)$ endowed with the weak topology. For every $n\in \mathbb {N}$ and $g\in C(S)$, we have

$$\begin{aligned} \int _S g(s)\sigma _n(\mathrm{d}s) = \int _S\int _S g(s')q(\mathrm{d}s'|i^*(s))\sigma _n(\mathrm{d}s). \end{aligned}$$

(16)

By (A2), the mapping $s\mapsto \int _Sg(s')q(\mathrm{d}s'|s)$ is continuous. Adapting the arguments used in Lemma 3.7, we can easily conclude from (16) that

$$\begin{aligned} \int _S g(s)\sigma (\mathrm{d}s) = \int _S\int _S g(s')q(\mathrm{d}s'|i^*(s))\sigma (\mathrm{d}s), \end{aligned}$$

that is, $\sigma \in \Delta (q^*).$ $\square$

Remark 4.1

For each $\sigma \in \Delta (q^*)$, $M(\sigma ):=\int _Ss\sigma (\mathrm{d}s)$ is the mean of distribution $\sigma$. Theorem 4.1 implies that there exists $\sigma _* \in \Delta (q^*)$ such that $M(\sigma _*)\le M(\sigma )$ for every $\sigma \in \Delta (q^*).$ Moreover, for each chain $\Sigma _0$ in $\Delta (q^*)$ there is a distribution $\sigma ^* \in \Delta (q^*)$ with highest mean, i.e. $M(\sigma ^*)\ge M(\sigma )$ for each $\sigma \in \Sigma _0.$ By Theorem 4.2, there exists $\sigma ^{**}$ with the highest mean over the set $\Delta (q^*).$

Remark 4.2

Theorem 4.1 can be applied to the models with the transition probabilities described in (C1)–(C3). Note that then assumption (A3) holds automatically. Invariant distributions under condition (C4) are studied in [22]. However, one may ask whether the obtained invariant distribution is unique or under which additional requirements we may expect its uniqueness. Let us mention here few cases. If $f(0)=0$ in case (C3) with the the multiplicative shocks, then the Dirac measure at the point zero is a ‘trivial invariant distribution’. If we put $\alpha =1$ in (6), then the problem reduces to standard dynamic programming and we know that under certain additional assumptions on $f$ and $u$ there exists also a non-trivial invariant distribution with support included in $]0,\overline{s}]$; see for example, pp. 1401–1402 in [15]. As pointed out in [32] (see p.1029), in some circumstances, it is natural to assume that $f(0)>0.$ Below we shall show that the model with transition probabilities defined in (C3), with the function $f$ satisfying the above condition, possesses a unique invariant distribution. For this purpose, we introduce the ‘monotone mixing condition’ used in [15]; compare also [16].

(M) There exists $s^*\in ]0,\overline{s}[$ such that

$$\begin{aligned} q^*([0,s^*]|\bar{s})>0\quad \text{ and } \quad q^*([s^*,\bar{s}]|0)>0. \end{aligned}$$

Theorem 4.3

Assume (A1) and (C3). Let $\underline{z}\ge 0$ be such that $f(\overline{s})\underline{z} < f(0)\overline{z},$ where $\overline{z}:= \overline{s}/f(\overline{s}).$ Assume that the whole interval $[\underline{z},\overline{z}]$ is included in the support of the non-atomic measure $\pi .$ Then for any SMPE $c^* \in F$, there exists a unique invariant distribution $\sigma ^*$ for the Markov process induced by $q^*.$

Proof

Clearly, since $f$ in (C3) is increasing, the mapping $s\mapsto q^*(\cdot |s)$ from $S$ into $P(S)$ is non-decreasing. Moreover, (C3) implies (A2). By Theorem 3.1, there exists a SMPE $c^*\in F.$ Let us pick any $s^*$ in the interval $]f(\overline{s})\underline{z}, f(0)\overline{z}[.$ Our assumptions imply that $f(0)>0$ and both real numbers $s^*/f(0)$ and $s^*/f(\overline{s})$ belong to the open interval $]\underline{z},\overline{z}[.$ Let $G_{\pi }$ be the distribution function of $\pi .$ Since $\pi$ is non-atomic with support including $[\underline{z},\overline{z}]$, we have that $G_{\pi }(s)\in ]0,1[,$ whenever $s\in ]\underline{z},\overline{z}[$. Thus, by (C3) we infer that

$$\begin{aligned} q^*([0,s^*]|\bar{s})&= \pi \left( \left\{ z:f(i^*(\bar{s}))z\le s^*\right\} \right) =\pi \left( \left\{ z:z\le s^*/f(i^*(\bar{s}))\right\} \right) \\&\ge \pi \left( \left\{ z:z\le s^*/f(\bar{s})\right\} \right) =G_{\pi }\left( s^*/f(\bar{s})\right) >0. \end{aligned}$$

On the other hand, we obtain that

$$\begin{aligned} q^*([s^*,\bar{s}]|0)&= \pi \left( \left\{ z:f(i^*(0))z\ge s^*\right\} \right) =\pi \left( \left\{ z:z\ge s^*/f(0)\right\} \right) \\&= 1-G_{\pi }(s^*/f(0))>0. \end{aligned}$$

Hence, $q^*(\cdot |s)$ satisfies (M). By Theorem 2 in [15], the Markov process induced by $q^*$ has a unique invariant distribution. $\square$

Theorem 4.3 remains also true, if we replace $c^*$ by any strategy $c\in F,$ because the mapping $s\mapsto q(\cdot |s-c(s))$ is still stochastically non-decreasing. We close this section with an example illustrating that there is no counterpart of this result in the model with additive shocks, that is, when the transition probabilities are defined as in (C2).

Example 4.1

Let $S=[0,2]$ and $f(s)=\sqrt{s}+\epsilon$ for $s\in S$ with $0\le \epsilon < 2-\sqrt{2}.$ Suppose that $\pi$ is the uniform distribution on $[0,2-\sqrt{2}-\epsilon ]$. Consider the investment strategy

$$\begin{aligned} i^*(s):= \left\{ \begin{array}{c@{\quad }c@{\quad }c} 0,&{}\text{ if }&{} s\le 0.7,\\ s,&{}\text{ if }&{} 0.7< s\le 0.8,\\ 0.8,&{}\text{ if }&{}0.8 < s\le 2.\\ \end{array} \right. \end{aligned}$$

(17)

Define $q^*$ as above. We prove that there are at least two invariant distributions and they are supported on disjoint sets. First, we claim that $q^*([0,2-\sqrt{2}]|s)=1,$ whenever $s\in [0,2-\sqrt{2}]$. Indeed, we obtain that

$$\begin{aligned} q^*([0,2-\sqrt{2}]|s)&= \pi \left( \left\{ z:f(i^*(s))+z\le 2-\sqrt{2}\right\} \right) \\&= \pi \left( \left\{ z:z\le 2-\sqrt{2}-\epsilon \right\} \right) =1. \end{aligned}$$

Note that the support of $q^*(\cdot |s)$ is $[0,2-\sqrt{2}],$ if $s\in [0,2-\sqrt{2}].$ Hence, $[0,2-\sqrt{2}]$ is an absorbing set. By Theorem 2 in [15], $q^*$ restricted to this interval has a unique invariant distribution, because $q^*$ satisfies (M) with, for instance, $s^*:=(2-\sqrt{2})/2$ and $\overline{s}:=2-\sqrt{2}.$ On the other hand, $q^*([\sqrt{0.8}+\epsilon ,2]|s)=1$ whenever $s\in [\sqrt{0.8}+\epsilon ,2].$ Indeed, for $s\in [\sqrt{0.8}+\epsilon ,2],$ we get that $f(i^*(s))=\sqrt{0.8}+\epsilon$. Thus,

$$\begin{aligned} q^*([\sqrt{0.8}+\epsilon ,2]|s)&= \pi \left( \left\{ z:\sqrt{0.8}+\epsilon \le \sqrt{0.8}+\epsilon +z\le 2\right\} \right) \\&= \pi \left( \left\{ z:z\le 2-\sqrt{0.8} -\epsilon \right\} \right) =1.\\ \end{aligned}$$

Observe that the support of $q^*(\cdot |s)$ is $[\sqrt{0.8}+\epsilon ,2]$ for $s\in [\sqrt{0.8}+\epsilon ,2],$ and consequently this set is absorbing. By Theorem 2 in [15], $q^*$ restricted to this interval has a unique invariant distribution. Thus, we have obtained two distinct invariant distributions.

5 Problems in Games With Deterministic Transitions

In this section, we assume that $S=[0,1]$ and the production function is $f(y) =\sqrt{y}.$ Below we give an example showing that the approach of the previous sections cannot be applied to games with deterministic transitions.

Example 5.1

Consider

$$\begin{aligned} U_t(h^t)=u(a_t)+ \widetilde{w}(a_{t+2}). \end{aligned}$$

This is the utility of the form (1) where $w =\widetilde{w}$ depends only on $a_{t+2}.$ Assume that $u(a)=4\root 8 \of {a}$ and $\widetilde{w}(a)=\root 4 \of {8}\sqrt{a}.$ Consider $c(y)= y$ for $y\in [0,0.5]$ and $c(y)= 0.5y$ for $y\in ]0.5,1].$ Note that $i(s)= s-c(s)$ is a non-decreasing lower semicontinuous investment function. Observe that

$$\begin{aligned} P(s-y,c)(s):=\widetilde{P}(y,c)(s)=u(s-y)+\widetilde{w}(c(f(f(y)-c(f(y))))) \end{aligned}$$

where $y=s-a \in A(s).$ Fix any $s\in ]0.5,1].$ Put $\theta (y):= \widetilde{P}(y,c)(s)$ with $y\in A(s).$ It is easy to check that

$$\begin{aligned} \theta (y)= 4\root 8 \of {s-y},\quad \text{ if }\quad y\in [0,0.25], \quad \theta (y)= 4\root 8 \of {s-y} + \root 8 \of {y},\quad \text{ if }\quad y\in ]0.25,s]. \end{aligned}$$

Note that the function $\theta$ is decreasing on $[0,0.25]$ and

$$\begin{aligned} \lim _{y\rightarrow 0.25^-}\theta (y)=\theta \left( 0.25\right) = 4\root 8 \of {s-0.25} < \lim _{y\rightarrow 0.25^+}\theta (y)=4\root 8 \of {s- 0.25}+\root 8 \of {0.25}. \end{aligned}$$

Moreover, we have that

$$\begin{aligned} \theta (0)=4\root 8 \of {s}< 4\root 8 \of {s- 0.25}+\root 8 \of {0.25}\quad \text{ for } \text{ any } s\in ]0.5,1]. \end{aligned}$$

The function $\theta$ is also decreasing on $]0.25,s]$ since

$$\begin{aligned} \theta '(y)=-0.5(s-y)^{-\frac{7}{8}}+0.125y^{-\frac{7}{8}}<0. \end{aligned}$$

This enables us to conclude that $\sup _{y\in A(s)}\theta (y)$ is not attainable and therefore $A_0(s)=\emptyset$ for $s\in ]0.5,1].$

Ray [12] and Bernheim and Ray [13] looked for equilibria in the class of upper semicontinuous non-decreasing investment (savings) functions. Example 5.2 illustrates that this set is useless in our approach, since the best response correspondence may have empty values. In other words, the operator examined in Lemma 6 in [12] adjusted to utility functions depending on consumptions of following generations is not well defined. Therefore, a direct application of methods from [12] to a study of the stochastic model, as it is done on page 201 in [13], is not correct. Here, instead of the aforementioned class of investment strategies we consider the set of lower semicontinuous non-decreasing saving functions. Such a set was already introduced in [7, 8] for bequest games. However, Example 5.1 shows that this class of functions does not solve the issue of existence of a SMPE, when the utility of generation $t$ depends on $a_t$ and $a_{t+2}$ and the transition probabilities allow for atoms.

Example 5.2

Consider

$$\begin{aligned} U_t(h^t)=u(a_t)+ \hat{w}(a_{t+1}). \end{aligned}$$

This is the utility of the form (1) where $w =\hat{w}$ depends only on $a_{t+1}.$ Let $u(a)=\root 4 \of {a}$ and $\hat{w}(a)=\sqrt{a}.$ Consider $c(y)= y$ for $y\in [0,0.5[$ and $c(y)=0$ for $y\in [0.5,1].$ Observe that the investment function $\hat{i}(s)= s-c(s)$ is non-decreasing and upper semicontinuous. We have that

$$\begin{aligned} P(s-y,c)(s)=\widetilde{P}(y,c)(s)=u(s-y)+\hat{w}(c(f(y))) \end{aligned}$$

where $y=s-a \in A(s).$ Fix any $s\in ]0.5,1].$ Put $\xi (y):= \widetilde{P}(y,c)(s)$ with $y\in A(s).$ It is easy to check that

$$\begin{aligned} \xi (y)= \root 4 \of {s-y} + \root 4 \of {y},\quad \text{ if }\quad y\in [0,0.25[, \quad \xi (y)= \root 4 \of {s-y}, \quad \text{ if }\quad y\in [0.25,s]. \end{aligned}$$

The function $\xi$ is increasing on $[0,0.25[$ and

$$\begin{aligned} \lim _{y\rightarrow \frac{1}{4}^-}\xi (y)= \root 4 \of {s-0.25}+ \root 4 \of {0.25} > \xi \left( 0.25\right) = \root 4 \of {s-0.25}. \end{aligned}$$

Clearly, the function $\xi$ is decreasing on $[0.25,s].$ From the above discussion, we conclude that for each $s\in ]0.5,1],$ $B_0(s):=\mathrm{arg}\max _{y\in A(s)}P(s-y,c)(s)=\emptyset .$ Here, $B_0(s)$ is the set of best replies to the upper semicontinuous investment function $\hat{i}$ given above.

6 Conclusions

The existence of a Markov perfect equilibrium in an altruistic growth model is a fixed point problem in an appropriately chosen function space. Bernheim and Ray [7] noted that the class of consumption strategies, for which corresponding investment policies are non-decreasing and continuous from the left, is adequate to deal with models with one descendant. Another family of functions that can be considered as strategies are Lipschitz functions. Nonetheless, Example 3.1 shows that the best replies in the set of Lipschitz functions need not exist.

Example 5.2, on the other hand, illustrates that the replacement of strategies that are continuous from the left by ones that are continuous from the right results in non-existence of best reply functions for the current generation even in the model with one descendant. This problem was not noticed in [12, 13], where the games with countably many successors were considered. Example 5.1 shows that even for the model with two descendants, we have to deal with compositions of lower and upper semicontinuous functions. As a result, the current generation may face the problem of maximising a function that is neither lower nor upper semicontinuous. Therefore, the best reply of the current generation to strategies of the followers may not exist. Such situation occurs, when the transition functions allow one to possess atoms, for instance, when the transition functions are deterministic. Hence, the methods derived from [12] and then applied to stochastic models by Bernheim and Ray [13] did not solve the issue of the existence of a SMPE. In this paper, we consider the weak convergence of non-atomic measures that leads to a generalised Fatou’s Lemma [14]. Since the strategies used by generations have countably many discontinuity points, we are able to prove, with the aid of the Serfozo Lemma, a helpful result on iterative integrals, that allow to compute the expected payoff for each generation (Lemma 3.8). This fact, in turn, plays an essential role in proving the existence of a SMPE. In this manner, we conveniently omit technical issues arising in [13], and moreover, we are able to deal with more general transition probability functions. The transition probabilities, described by some difference equations and including i.i.d. random variables, are only special cases in our approach. In particular, our class of transitions embraces also the transition function of the linear form considered in [6].

Finally, we would like to mention that, except for papers concerning pure equilibria, there are works on stochastic altruistic growth models, that prove the existence of an equilibrium in the class of randomised strategies. For example, Alj and Haurie [24] and Nowak [33] considered such models with a countable state space and a Borel state space, respectively. However, from the economic applications point of view randomised equilibria do not enjoy great popularity and are treated as a remedy in the models, where pure equilibria cannot be obtained.

Other models on stochastic intergenerational games, with one descendant for each generation and with completely different types of transition probabilities, were studied in [20] and [23]. As already mentioned in Remark 3.3, they are convex combinations of finitely many probability measures on the state space and they do not embrace the transition probabilities described by some difference equation with additive or multiplicative noise. Particularly, Amir [20] studied transition probabilities with convex cumulative distributions and had to assume that the set of states is unbounded from above. However, the class of such transitions is relatively narrow; e.g. in Example 1 in [20], the conditional expectation of capital for every generation given any positive investment of its predecessor is infinite. Some recent results on equilibria in stochastic altruistic growth models with many descendants and additive transition probability functions can be found in [21, 22, 34].