NON-CONSTANT POSITIVE SOLUTIONS OF A GENERAL GAUSE-TYPE PREDATOR-PREY SYSTEM WITH SELF- AND CROSS-DIFFUSIONS∗

In this paper, we investigate the non-constant stationary solutions of a general Gause-type predator-prey system with selfand cross-diffusions subject to the homogeneous Neumann boundary condition. In the system, the cross-diffusions are introduced in such a way that the prey runs away from the predator, while the predator moves away from a large group of preys. Firstly, we establish a priori estimate for the positive solutions. Secondly, we study the non-existence results of non-constant positive solutions. Finally, we consider the existence of non-constant positive solutions and discuss the Turing instability of the positive constant solution. Mathematics Subject Classification. 35K51, 35K57, 35A01, 35A09. Received August 16, 2020. Accepted March 22, 2021.


Introduction
In this paper, we investigate the existence and non-existence of the non-constant positive stationary solutions for the following general Gause-type predator-prey system with constant self-and cross-diffusions where Ω ⊂ R N (N ≥ 1 be an integer) is a bounded domain with smooth boundary ∂Ω, ν is the outward unit normal vector on the boundary ∂Ω with ∂ ν = ∂ ∂ν , and the homogeneous Neumann boundary condition means that the individuals do not cross the habitat boundary, the variables u and v represent the densities of prey and predator, respectively, the coefficient c > 0 refers to the conversion rate of prey captured by the predator and d > 0 represents the predator's death rate.
Throughout this paper, the function g(u) ∈ C 1 ([0, +∞)) describes the growth rate of prey when the predator is absent, i.e., the intrinsic growth rare of prey and it also measures the availability of using the natural resources. The function p(u) ∈ C 1 ([0, +∞)) ∩ C 2 ((0, +∞)) represents the consumption rate of prey by predator as a result of a change in the prey density. In order to describe the biological significance of the system, g(u) and p(u) are assumed to be satisfying the following two assumptions.
In fact, the growth rate of the predator is improved in the presence of prey by an amount proportional to the number of prey. Therefore, the functional response p(u) can be explained as the consumption rate at which the prey is consumed by an individual predator. In terms of the functional response p(u), it can be divided into many forms, for instance, the classical Holling-type functional responses (see [1,6,[20][21][22] for examples), the Ivlev and the inverted Ivlev functional responses, and so on (see [5,23] for examples). We note that among these functional responses, the Holling I, II and the Ivlev types all satisfy the assumption (A2). Meanwhile, we point out that system (1.1) with assumption (A2) does not contain the systems with non-monotonic functional responses, such as p(u) = u a+u 2 with a being a positive constant. In system (1.1), the coefficients d 1 and d 2 are both positive constants and d 3 , d 4 are non-negative constants. In our system, the self-and cross-diffusions satisfy the following two identities, The diffusion terms d i , i = 1, 2 stand for the natural dispersive force of movement of an individual prey or predator. While the cross-diffusions pressure d 3 and d 4 (or d 1 d 3 and d 2 d 4 ), which can not be described in classical predator-prey systems, plays an important role in generating various spatial patterns in biological population dynamics (see [12,18]).
Precisely speaking, we regard the term d 1 u + d 1 d 3 uv as the survival conditions of prey, meaning the diffusion is toward the places of better survival conditions. The diffusion d 1 ∆[(1 + d 3 v)u] of population can be explained as follows. Denoting ∆u = div( u), we may consider u as the flux. Then, the prey u diffuses with flux We perceive that, as d 1 d 3 u > 0 , the part −d 1 d 3 u v of flux is directed toward the decreasing population density of the predator. Particularly, when d 3 is large, the term −d 1 d 3 u v represents the preys running away from a large group of predators.
Similarly, the predator v diffuses with flux We notice that, since d 2 d 4 v ≥ 0, the part −d 2 d 4 v u of flux is directed toward the decreasing population density of the prey. Particularly, when the diffusion term d 4 is large enough, −d 2 d 4 v u represents the predator far away from a large group of preys. Generally speaking, if a prey species has a pretty high populations density or size, it is usually difficult for other prey or predator species to invade its habitat. For more information and background, see [7,13,16,24] for detailed discussion and explanation on relevant and homologous biological systems.
Over the past few years, there are many valuable results and progresses on the effects of self-diffusion and cross-diffusion on the dynamics of different systems. In mathematical biological ecology, the classical predatorprey system, independently, introduced by Lotka and Volterra in the 1920s, only reflects the population changes caused by predation when the density of predator and prey is not spatially dependent. It usually does not consider the fact that the population distribution is uneven, nor does it consider the fact that predator and prey naturally form survival strategies.
Both of these considerations involve diffusion processes, which can explain that different concentrations of predators and preys can bring about different movements. These movements can be determined by the concentration (diffusion) of the same species and the concentration (cross-diffusion) of other species. Additionally to the classical diffusion, self-and cross-diffusions allow to take into account the effect of pressure between species. For example, at a point where the term uv is at local maximum, ∆u is negative, which stands for the fact that the prey tends to run away from there. Roughly speaking, the population growth systems including cross-diffusions can produce more complex and abundant population dynamics than the population growth model without considering these factors.
The role of self-and cross-diffusions in modeling of various biological, physical, and chemical processes has been widely investigated. For example, in [2,15], the authors investigated the existence of steady state in a linear cross-diffusion predator-prey system with Beddington-DeAngelis functional response and the proofs mainly depend on analytical techniques and fixed point index theory.
Paper [25] considered a strongly coupled predator-prey system arising from the classical Lotke-Volterra system, in which the cross-diffusions are included in such a way that the predator moves away from a large group of preys and the prey flees away from the predator. The author established the non-existence and existence of its non-constant positive solutions for suitable large values of the cross-diffusion or small enough cross coefficients. In [9], without taking into account the cross-diffusions, the authors studied the qualitative behavior of non-constant positive solutions on a general Gause-type predator-prey system subject to the homogeneous Neumann boundary condition, in which with diffusion rates are constants. The results show that the non-existence and existence of the non-constant positive steady-state solutions are affected by the self-and cross-diffusion rates. In addition, they also investigated the local existence of periodic solutions, the asymptotic behavior of spatially inhomogeneous solutions, and the diffusion-driven instability. Recently, a special case of system (1.1) was considered in [8], specifically, the cross-diffusion rate d 4 is absent, in which, the cross-diffusion d 1 d 3 u is included in the meaning of the prey escapes from the predator. The paper investigated the existence of nonconstant positive steady states and discuss the stability of constant equilibrium point. And the result points out the conditions which can provide the system admits a non-constant positive steady state. In fact, there also have abundant and important results and conclusions on the mathematical developments of self-and cross-diffusion systems produced in various research fields, see [14,17], for example.
As far as we know, there are few works related system (1.1) with the cross-diffusions for both predator and prey. Inspired by these works, we investigate the existence of the non-constant positive solutions of system (1.1) by using the well-known Leray-Schauder degree theory. We justify the choice of nonlinear reaction terms by referring to the classical expression in use in literature and usual references. Since the cross-diffusion terms d 3 and d 4 are introduced, the corresponding computation and analysis in the subsequence are more difficult and require a bit of techniques than systems without considering the cross-diffusions term. The main aim of this paper is to investigate the effect of the self-and cross-diffusions pressures on the non-constant positive solutions of system (1.1), that is, we study the non-existence and the existence of the non-constant positive solutions to the following elliptic system (1. 2) The organization of this paper is as follows. In Section 2, we establish a priori estimate for positive solutions of the system (1.2) by employing the Harnack inequality and the maximum principle. In Section 3, we study the non-existence results of the non-constant positive solutions by using the energy integral method. We mainly consider the influence of the change of diffusion coefficients on the existence of non-constant positive solutions. In Section 4, we mainly consider the existence of the non-constant positive solutions by making use of the Leray-Schauder degree theory. Moreover, we also discuss the Turing instability of system (1.1) mainly by considering the influence of the diffusion terms.

The priori estimate of non-constant positive solutions
The main purpose of this section is to establish the priori upper and lower estimates with respect to the diffusion coefficients d i for the non-constant positive solutions of system (1.2). In order to guarantee the result, we first give a brief account of the following two preliminaries, that is, the Harnack inequality and the Maximum principle. These two results are usually used to prove the priori estimates of positive solutions.
Then there exists a positive constant Proof. Suppose that (u, v) be a positive solution of system (1.2). We denote φ = (1 + d 3 v)u, ϕ = (1 + d 4 u)v and then system (1.2) can be rewritten as x ∈ ∂Ω.
Simple mathematical analysis shows that 0 < inf u∈(0,S) u p(u) ≤ sup u∈(0,S) u p(u) < ∞ by the assumption (A2) and p(u) ∈ C 1 ([0, +∞)) ∩ C 2 ((0, +∞)), even if p(u) → 0. This indicates that g(0)u(x1) p(u(x1)) is bounded, we denote an upper bound by M . Thus, there holds We multiply the first equation of system (1.2) by constant c, then add it to the other equation. Finally, we integrate it over Ω and we can achieve According to the divergence theorem, we obtain Thus, we get Also, we notice that system (1.2) can also be rewritten as for some suitable large positive constant C * 0 . Thus, we can obtain Denote C = max{C 1 , C 2 }. Then the result follows.
Proof. We integrate the first equation of system (1.2) over Ω. Then we get Therefore, there must exist a point x 2 ∈ Ω such that By the assumptions (A1) and (A2), we get 0 < u( We get a contradiction, then v has a positive lower bound when Next, we mainly prove that u also has a positive lower bound. According to assumption (A2), p(u) ∈ C([0, +∞)) ∩ C 2 ((0, +∞)) and lim we can assert that there must exist a positive constantp such that p(u) u ≤p for some small u, 0 < u ≤ S. Obviously, we obtain Hence, the Harnack inequality holds for φ, namely On the other hand, assume that there exists a sequence {(d 1n , d 2n , d 3n , d 4n )}, n = 1, 2, · · · , satisfying d 1n ≥ D 1 , d 2n ≥ D 2 , d 3n ≤ D 3 for some D 3 > 0, and d 4n ≤ D 4 , such that the corresponding positive solutions (u n , v n ) of system (1.2) with (d 1 , d 2 , d 3 , d 4 ) = (d 1n , d 2n , d 3n , d 4n ) satisfy min Ω u n → 0 as n → ∞. Combining with (2.6), we have max Ω u n → 0 as n → ∞. By the regularity theory for elliptic equations [3], we have u n ∈ C 2,α (Ω). Therefore, we know that there exists a subsequence of {(u n , v n )}, which will be also denoted by {(u n , v n )}, such that u n → 0 converges uniformly as n → ∞. We integrate the second equation of system (1.2) with (u, v) = (u n , v n ) and we can obtain Ω v n (−d + cp(u n ))dx = 0. (2.7) Since u n → 0 as n → ∞, we have −d + cp(u n ) < 0 on Ω for any large enough integer n. This contradicts to the identity (2.7) by considering v n > 0. In summary, there is C such that C < u(x), v(x). This finishes the proof.

Non-existence of the non-constant positive solutions
In this section, we analysis the results of non-existence of the non-constant positive solution of system (1.2) mainly by using the well-known Poincaré inequality and the energy integral method. We mainly consider the influence of the change of diffusion coefficients on the existence of non-constant positive solutions. To this end, we first introduce the following notations.
Let µ 1 be the smallest positive eigenvalue of the operator −∆ subject to the homogeneous Neumann boundary condition. Denoteũ The following non-existence conclusion is obtained by mainly considering the self-and cross-diffusion rates. Through a lot of analysis and calculation, we give the following theorem.
Theorem 3.1. Let C 1 and M be the same as that of in (2.2) and d 1 , d 2 satisfy Denote where ξ, η, ζ are positive constants satisfying 0 < ξ, η, ζ ≤ C 1 and K 0 = sup Ωṽ p (u). By the Cauchy inequality and Theorem 2.3, we have with K be an arbitrary positive constant. We obtain the inequality from the fact Particularly, taking K = µ 1 and using the well-known Poincaré inequality, we have Recall that C 1 = (1 + d 3 M )S is given in (2.2) and, by direct computation, we know that system (1.2) has no non-constant positive solutions when i.e., Thus, the proof is completed.

Existence and stability of the non-constant positive solutions
In this part, we mainly discuss the existence of non-constant positive solutions to system (1.2) by using the self-and cross-diffusion coefficients as parameters. In particular, we focus on the cases that the cross-diffusion is small enough and the self-diffusion is large enough. The key method used here in proving the existence of non-constant positive solutions is the well-known Leray-Schauder degree theory [10], which has been extensively used in many different articles. Moreover, we also discuss the Turing instability of system (1.1) mainly by considering the influence of the diffusion terms.

Local analysis at the constant positive solution
In this subsection, we discuss the linearization of system (1.2) at the constant positive equilibrium point W * = (u * , v * ) T . In order to facilitate, we need introduce some frequently-used definitions and notations in advance.
Let {µ i , ψ i } ∞ i=0 be a complete set of eigenpairs for the operator −∆ in Ω under homogeneous Neumann boundary condition. Set We decompose X = ⊕ ∞ i=0 X i , where X i is the eigenspace corresponding to the eigenvalue µ i . Let By direct computation, we know that system (1.2) only has a unique constant positive equilibrium point W * = (u * , v * ) T when the assumptions (A1) and (A2) hold, where with p −1 (u) being the inverse of the function p(u).
is positive. Therefore, we can rewrite system (1.2) as the form It is easy to know that X i is invariant under D W F (Λ; W * ) for each integer i ≥ 0. Furthermore, λ is an eigenvalue of the operator D W F (Λ; W * ) on X i if and only if λ is an eigenvalue of the matrix Let Since the sign of the depends on the number of negative eigenvalue of the matrix If H(Λ, W * ; µ i ) = 0 for all integer i ≥ 0, then 0 is not an eigenvalue of the operator D W F (Λ; W * ). This implies that D W F (Λ; W * ) is a homeomorphism operator from the space X to X. Then the implicit function theorem shows that the equilibrium point W = W * is an isolated solution of equation F (Λ; W ) = 0. In conclusion, according to Leray-Schauder degree theory, we give the following results (One can refer to [19]).
Furthermore, combining with (4.2), we obtain the following lemma.

Turing instability of the positive constant solution
In this subsection, we focus on the stability of the positive constant solution W * = (u * , v * ) T . Using the notations in subsection 4.1, it is easy to know that system (1.1) can be written as Thus, the corresponding spatially homogeneous counterpart of system (4.7) is (4.8) The Turing instability means diffusion drives instability. Since the introduction of the self-and cross-diffusion terms, the stability of the positive constant solutions W * may change from the stable, for the ODE system (4.8), to unstable, for the PDE system (4.7). In the following, we discuss the stability of the constant positive solution W * = (u * , v * ) T of system (4.8). The linearization of system (4.8) at W * can be expressed by where G W (W * ) is the same as that of given in Subsection 4.1. The character polynomial equation of G W (W * ) is In conclusion, we present the following lemma.
Next, we investigate the stability of the positive constant solution W * of the PDE dynamics system (4.7). In order to find the criterion for the Turing instability, we first give some calculations. The linearization of system (4.7) at W * can be written as We have the following lemma.
Lemma 4.7. If the matrix −µΦ W (W * ) + G W (W * ) has an eigenvalue with positive real part, then the constant positive solution W * of system (4.7) is unstable.
By direct computation, we obtain  Remark 4.9. The result shows that, due to the introduction of diffusion terms, the stability of the positive constant solution W * of the PDE system (4.7) (also (1.1)) may change from stable, for the counterpart ODE system, to unstable under certain conditions.

Comments and conclusions
This paper investigates the existence and non-existence of non-constant positive solutions for a generalized Gause-type predator-prey system with self-and cross-diffusions under homogeneous Neumann boundary condition, in which the cross-diffusions are included in such away that the prey runs away from the predator and the predator moves away from a large group of preys. On the basis of Leray-Schauder degree theory and mathematical analysis, we prove that cross-diffusions can create coexistence for the prey and predator under suitable conditions. Our main results indicate that system (1.2) admits non-constant positive solutions if one of the self-diffusions d 1 or the cross-diffusions d 3 is large under certain conditions. Moreover, we also discuss the Turing instability of system (1.1) mainly by considering the influence of the diffusion terms. The results show that the diffusion terms can influence the stability of the system.
The theoretical analysis indicates that the self-and cross-diffusions phenomena have the potential to play an important role for the symbiosis and coexistence of the species. The main distinction between this article and [8] is the introduction of the cross-diffusion factor d 4 . Since we introduce the cross-diffusion d 4 into the system, compared with [8], the conclusion shows that the coexistence is mainly affected by the self-diffusion rate d 1 or the cross-diffusion rate d 3 .