Discrete and continuum phenotype-structured models for the evolution of cancer cell populations under chemotherapy^*

¹ School of Mathematics and Statistics, University of St Andrews, North Haugh, St Andrews, Fife KY16 9SS, UK.
² Institute of Applied Mathematics, Im Neuenheimer Feld 205, Heidelberg University, 69120 Heidelberg, Germany.
³ School of Mathematics and Statistics, University of St Andrews, North Haugh, St Andrews, Fife KY16 9SS, UK.
⁴ Institute of Applied Mathematics, BIOQUANT and IWR, Im Neuenheimer Feld 205, Heidelberg University, 69120 Heidelberg, Germany.
⁵ School of Mathematics and Statistics, University of St Andrews, North Haugh, St Andrews, Fife KY16 9SS, UK.

^** Corresponding author: tl47@st-andrews.ac.uk

Received: 31 January 2019
Accepted: 17 June 2019

Abstract

We present a stochastic individual-based model for the phenotypic evolution of cancer cell populations under chemotherapy. In particular, we consider the case of combination cancer therapy whereby a chemotherapeutic agent is administered as the primary treatment and an epigenetic drug is used as an adjuvant treatment. The cell population is structured by the expression level of a gene that controls cell proliferation and chemoresistance. In order to obtain an analytical description of evolutionary dynamics, we formally derive a deterministic continuum counterpart of this discrete model, which is given by a nonlocal parabolic equation for the cell population density function. Integrating computational simulations of the individual-based model with analysis of the corresponding continuum model, we perform a complete exploration of the model parameter space. We show that harsher environmental conditions and higher probabilities of spontaneous epimutation can lead to more effective chemotherapy, and we demonstrate the existence of an inverse relationship between the efficacy of the epigenetic drug and the probability of spontaneous epimutation. Taken together, the outcomes of the model provide theoretical ground for the development of anticancer protocols that use lower concentrations of chemotherapeutic agents in combination with epigenetic drugs capable of promoting the re-expression of epigenetically regulated genes.