Issue 
Math. Model. Nat. Phenom.
Volume 15, 2020
Coronavirus: Scientific insights and societal aspects



Article Number  40  
Number of page(s)  13  
DOI  https://doi.org/10.1051/mmnp/2020031  
Published online  17 August 2020 
Lifting the COVID19 lockdown: different scenarios for France
University of Bordeaux, GREThA, 16 avenue Léon Duguit,CS 50057,
33608
Pessac Cedex, France.
^{*} Corresponding author: emmanuelle.augeraud@ubordeaux.fr
Received:
7
April
2020
Accepted:
26
July
2020
A mathematical model has been developed to study the spread of the COVID19 epidemic in France. To calibrate the model, data from the French Public Health Agency were examined. The spread of the epidemic greatly depends on lockdown measures (referred to in France as ‘confinement’). The aim of this paper is to predict the expected evolution of the epidemic based on the various possible scenarios for ending the lockdown. The spread of the disease, and its reemergence, will be determined by these scenarios.
Mathematics Subject Classification: 92.10 / 92D30
Key words: COVID19 / confinement / public health policy
© The authors. Published by EDP Sciences, 2020
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
1 Introduction
In December 2019, the World Health Organization warned of several cases of pneumonia in Wuhan, China. On January 7th 2020, China confirmed that a new virus, of the coronavirus family, referred to as COVID19 was responsible for this pneumonia. On March 11th, the World Health Organization officially declared a pandemic, and by March 22nd, there were 81,500 confirmed cases in China, with 3,267 deaths. At this time, some 188 countries around the world had been affected. However, prior to this, on March 18th 2020, China had already stated that no new cases of the virus had been reported for the first time since the beginning of the epidemic. Stopping the spread of this virus in China was made possible by lockdown. In fact, by the end of January 2020, due to the extent of the spread of the virus across China, its government had set drastic lockdown measures in place.
In France, the coronavirus was identified on January 24th 2020, when the first three cases were recorded. As of February 29th, 100 individuals were already reported to have the disease, with 2 deaths being recorded. Therefore, Stage 2 of the ORSAN plan was activated with the aim of slowing down the spread of the virus. By March 14th, with some 4500 reported cases and 91 deaths, the epidemic was raised to Stage 3, with all nonessential services being closed to the general public. Since March 17th, all citizens have been confined to their homes, except for certain justifiable reasons, and movement has been limited to a strict minimum.
By March 25th, it was estimated that more than 100,000 fines had been issued for noncompliance with the lockdown measures (Mignon Colombet and Floreancig [13]). In fact, since March 17th, the means for dealing with those who contravene the lockdown measures have become more stringent, with a fine of 135 euros being introduced for any violations of nonjustified movement. Two days later, five people who had been fined several times for nonrespect of the lockdown measures were taken into police custody for endangering the lives of others. The government has had to take more coercive measures in the face of noncompliance with these lockdown measures, and the Courts have had to impose a prison sentence on the basis of endangerment of life (Article 2231 of French Penal Code).
Lockdown ended in France on May 11th, with many restrictions (school closures, movement restrictions ....). This partial lockdown lift (called phase 1) has enable to contain the epidemics. The decree of May 31st (Decret 2020663) organizing the entry into phase 2 of the lift of lockdown, published in the Official Journal specifies all measures concerning travel and public transport, shops, establishments and places open or closed after June 2nd 2020 until June 21st. From June 22nd started phase three, with obligation to go to school for all students.
The issue that needs to be considered is the future evolution of the virus when this lockdown is lifted. Different scenarios, depending on the application of advices for stopping the virus spread, will be considered.
2 Model and data
We consider an SLIAR compartmental model with susceptible (S), latent infectious (L), symptomatic infectious (I) and asymptomatic (or mild symptomatic) infectious (A). This SLIAR model has been considered by Arino et al. [2] to describe diseases such as influenza. Susceptible individuals S(t) can be contaminated by all infectious individuals and become latent infectious asymptomatic (L(t)) then, according to their symptoms, they can become either infectious asymptomatic (or mild symptomatic) individuals (A(t)) or infectious severe symptomatic (I(t)). We assume that infectious asymptomatic develop severe symptoms at rate fν with f ∈]0, 1[, and they have mild symptoms at rate (1 − f)ν, where ν^{−1} is the average duration of the asymptomatic period. Asymptomatic individuals recovers at rate η, and severe symptomatic individuals may either recover at rate η or die at rate γ. The spread of the disease across the population is considered in the following system (2.1)
Let us denote τ the transmission rate at the early stage of the epidemics. It depends on infectiousness of the disease times the number of contact by time unit at time t_{0}. At time t > t_{0}, susceptible individuals may be contaminated by latent infectious L(t), asymptomatic infectious A(t) or infectious severe symptomatic individuals I(t). On the contrary to Liu et al. [11] we assume that these reported individuals are still active in the transmission of the disease, because many cases of health care workers have been contaminated while they were taking care of infectious severe symptomatic cases. However, we would assume that their transmission rate τε_{2} is very small compared to the one of latent or asymptomatic individuals by assuming 0 < ε_{2} ≪ 1, to take into account that they are isolated as soon as their symptoms are detected. For latent or asymptomatic individuals, the transmission rate is τε_{1}, where ε_{1} is by definition equal to 1 before public health measures had been taken. Parameter ε_{1} ∈ [0, 1] is time dependent in order to take into account the impact of physical distancing. It is a central parameter in our modeling: ε_{1} (t) represents the number of contacts by time units at time t over the number of contacts people had before the epidemic outbreak. Parameter ε_{1} will be called the intensity of the contact. This parameter depends on health policy, but also on a behavioral component (protective measures applied by individuals). We consider this parameter as a control parameter in order to study the spread of the epidemic for different values of ε_{1}.
Current modeling is based on Liu et al. [10, 11] who modeled the spread of the epidemic in both Wuhan and in China, and on Magal and Webb [12] who calibrated the epidemic with early data, on several countries including France. However, there are three main differences in our modeling and theirs. Firstly, they do not consider contamination by reported symptomatic cases. It appears that in France, many cases of contamination occur in hospitals (on March 22nd, it was announced that the first hospital doctor had died as a result of his infection with the virus). Secondly, they do not consider overmortality caused by the virus. Finally, Liu et al. [10] chose an exponential function to model the reduction of the number of contacts during lockdown. However, as lockdown lasted a long time, exponential function would lead to a too low impact on the last period of the lockdown. Thus we would prefer the original function used by Chowell et al. ([6]).
We have used the calibration defined in Magal and Webb [12] to describe the epidemic before the implementation of lockdown measures. The impact of lockdown has then been calibrated. The objective of our paper is to focus on the impact of strategies for lifting the lockdown.
To describe the evolution of the epidemics, we consider parameters given in Table 1.
Table of parameters.
3 Results
Let us first consider the reproduction number on the epidemics, based on the dynamical system (2.1). Results of Van den Driessche and Watmough [14] are applied. The linearized dynamics of the infective classes (that is latent L, symptomatic I and asymptomatic A) of System (2.1) around the disease free equilibrium (DFE) given by (S^{*}, 0, 0, 0) with S^{*} ≈ S_{0}, is considered. This linearized dynamics is given as follows. (3.1)
The reproduction number is the expected number of cases generated by one infected individual in a population where all individuals are susceptible to infection. Since the model includes parameter ε_{1}, that is influenced by public policy, we would call the reproduction number the control reproduction number, and denote it . Formally, the reproduction number is the spectral radius of FV^{−1}, where matrix V and F are defined as follows.
Matrix F − V is the Jacobian matrix of System (3.1). F is linked with the terms involved in the transmission of the disease, V in the terms linkedto the change of compartment for other reasons that disease transmit.
Thus is explicitly computed as
It can be noticed that for , then , there is no epidemic peak. It can also be seen that is a linear function of the intensity of the contacts ε_{1}.
To understand the course of the epidemics, we are also interested in characterizing the relationship between the final size of the epidemics, that is the number of susceptible at the end of the epidemics, and the intensity of the contacts. Our approach relies on a method developed by Arino et al. [3] in a general framework applied to various cases. Brauer [4] considers the final size of an epidemic computed with early data, and stresses the importance of taking into account how behavioral components modify the final size of the epidemics.
Let us consider Y a weighted number of infective which is defined by
Then, taking the derivative of Y according totime yields (3.2)
Using equation (3.2) and the first equation of System (2.1) it yields (3.3)
From the dynamics of susceptible individuals S in System (2.1), it is straightforward that S is non negative strictly decreasing function of t. Then exists. We would denote . S_{∞} is called the final size of the epidemics and measures the number of individuals that have not been contaminated during the epidemics.
If , then, according to equation (3.3), the weighted number of infective cases first increases up to a maximum value, achieved for . We would denote t_{max} the time such that S(t_{max}) = S_{max}. Thus for all t > t_{max}. If , we obtain that for all t > t_{0}.
Moreover, it can be proved (see appendix) that exists and satisfies Y_{∞} = 0.
The relationship between the final size of the epidemics, initial conditions and parameters is given as follows.
Using the definition of Y_{0} gives (3.4)
The final size of the epidemics, defined by implicit equation (3.4), strongly depends on the parameters of the model, and in particular on ε_{1}. Formally, it yields
As we have seen that , the more the intensity of the contact, the less the final size of the epidemics. To quality this impact, we would prefer to compute the elasticity of S_{∞} according toε_{1} defined as:
It indicates the percentage of reduction of the final size of the epidemics, when the intensity of the contact increases by 1%.
Figure 1 gives quantitative value of elasticity using calibration for COVID19 specified in Table 1. Not surprisingly, in Figure 1, it can be seen that . It means the greater the intensityof the contact among individuals, the smaller the final size of the epidemics. However, Figure 1 also shows that elasticity is not a monotonous function of ε_{1}. Reducing the intensity of contact of one percent compare to businessasusual situation ε_{1} = 1 enables to increase the final size of the epidemics of 3.57%. When ε_{1} is close to 0.4, reducing the intensity of the contact of 1% increases the final size of the epidemics of 2.18%. When ε_{1} is close to , the marginal effort of reducing contact intensity has a huge impact on the size of the epidemics.
Let us consider the number of individuals that have been contaminated during the epidemics, denoted C_{∞}. They correspond to those who have recovered or who are dead from the disease. According to System (2.1), C_{∞} is obtained as C_{∞} = S_{0} − S_{∞}. Then
and elasticity , which is defined as
Non surprisingly, it can be seen in Figure 2 that . It means that the total number of individuals who have recovered or who are dead from the epidemics is increasing is ε_{1} ; the more the intensity of the contact the more infected individuals. However, elasticity is a decreasing function of ε_{1}. If the intensity of the contact is close to its level before the epidemic outbreak (that is ɛ_{1} close to 1), then a reduction of 1% of the intensity of the contact yields to a reduction of 0.18% of C_{∞}. If the intensity of the contact is half the one it was before the epidemic outbreak, a reduction of 1% of the intensity of the contact yields to a reduction of 1.36% of C_{∞}.
Figure 1 Elasticity of final size S_{∞} according to contact intensity ε_{1} for various values of contact intensity ε_{1}. 
Figure 2 Elasticity of number of individuals that have been contaminated at the end of the epidemics C_{∞} according to contact intensity ε_{1} for various values of contact intensity ε_{1}. 
4 Numerical experiments
Data from the French Public Health Agency from February 25th to the 1st of June have been used.
Calibration of the epidemics in the early phase of the epidemics is obtained using the results of Liu et al. [10] and are given in Table 1. The starting time of our simulations is taken as t_{0} being February the 3rd.
During lockdown.
The effect of lockdown is to reduce the number of the contacts by time unit. The impact of this reduction is modeled through parameter ε_{1}, which was set to 1 before public health measures had been taken. The impact of these interventions is not instantaneous, as it was stressed by Chowell et al. [6]. This impact can be modeled using exponential form, which is the case used by Althaus [1] for 2014 Ebola virus outbreak or by Liu et al. [11] concerning the early stage of COVID19 epidemics. However, as the lockdown period lasted a long time, we would prefer to use a form similar to Chowell et al. [6], for which ε_{1} (t) decreases gradually from 1 to , as given by equation (4.1), as follows. (4.1)
Parameters , μ and γ have been calibrated using leastsquares fitting. The following results have been obtained: , μ = 0.1424125 and γ = 0.0203774. Thus contacts have reduced to 5–6 fold in France (compared to 7–8 fold in Wuhan and Shangai [16]). Figure 3 presents our simulations compared to data of the Public Health Agency from February 25th to the 1st of June on daily number of symptomatic cases (fνL(t)) (Fig. 3a), on the daily number of deaths (Fig. 3b).
The comparison of our simulations to data of the Public Health Agency as far as cumulative number of reported cases and cumulative number of deaths is concerned is given in Figure 4.
Lockdown lifting.
At time t_{2} taken as May 11th, the lockdown ended in France, but a number of restrictive measures still apply: movement restriction and schools, restaurants and bars remained closed. Different scenarios for the end of the lockdown which are compatible with data available on June 1st are now presented. On the contrary to Ferguson et al. [8], we only consider one specific time for the start and end of lockdown. We assume that the intensity of the contact ε_{1} can be modeled by equation (4.2). (4.2)
Function ε_{1}(t) is continuous, and for t > t_{2}, ε_{1} (t) satisfies the following differential equation.
Thus, after the lift of lockdown, intensity of contact is modeled by a logistic function which reflects that physical distancing habits acquired during lockdown period would be progressively lost as time goes on. Modeling health habit with logistic function has been used by Fournier et al. [9]. In our modeling c is the speed of the increase contact intensity and G^{−1} is the long run contact intensity.
We would first consider three scenarios modeling contact intensity after the lift of lockdown. These scenarios would depend on parameters c and G. A first scenario is a fast loss of physical distancing habit acquired during lockdown period and convergence to contact intensity as it was before lockdown. However, to take into account the three phases of the lift of lockdown in France, immediate convergence to contact intensity as it was before the epidemic outbreak is excluded. Thus, this first scenario will be obtained by considering (c, G) = (0.08, 1). The second scenario is a slow loss of physical distancing habits and in the long run, contact intensity is the same as the one before the epidemic outbreak. This second scenario corresponds to (c, G) = (0.05, 1). In these two scenarios, we would call businessasusual the situation of contact intensity in the long run. A third scenario is also considered. It corresponds to a slow loss of social distancing habits with individuals continuing to respect advices to stop the virus spread and physical distancing even in the long run. In this scenario, we would assume that contact intensity is a third of what it was before the epidemic outbreak by assuming (c, G) = (0.05, 3).
The value of contact intensity ε_{1} for these three scenarios are plotted in Figure 5.
Simulations obtained from System (2.1) using functional form of contact intensity given by equation (4.2) are now considered. Parameters are given in Table 1, together with , μ =0.1424125 and γ = 0.0203774 (as explained previously). Various choices of values for (c, G) will generate various scenarios. We will first consider the three scenarios described previously corresponding to (c, G) = (0.08, 1) (scenario 1), (c, G) = (0.05, 1) (scenario 2) and (c, G) = (0.05, 3) (scenario 3). A fourth scenario corresponding to (c, G) = (0.05, 3.33333) will be also considered.
The trajectories of daily symptomatic reported cases and daily deaths are presented in Figure 6 based on the various scenarios. It can be seen that with the businessasusual scenario (with either a fast or slow end to the lockdown) a major peak of the epidemic appears in summer (Fig. 6a and b). The only differences between a fast and slow end to lockdown are that for the fast end scenario, the timing of the peak would be reached sooner (end of July with in Fig. 6a and end of August in Fig. 6b) and its magnitude would be greater with fast end to lockdown. Even if individuals reduce their contact habits to a third of what they were before the epidemic outbreak, this is still not enough to prevent a major epidemic outbreak that would happen by the end of the year.
In Figure 6d a fourth scenario is considered. It corresponds to a slow loss of physical distancing habits, with contact habits in the long run being reduced to 70% of what they were, and it is obtained by considering (c, G) = (0.05, 0.3^{−1}). With this scenario, there is no reemergence of the epidemic. Beyond this threshold, there is reemergence of the epidemic. This threshold corresponds 1.8 times the contact intensity achieved by May 11th.
Figure 3 Daily number of reported symptomatic cases (a) and daily number of COVID19 deaths (b) for France. 
Figure 4 Cumulative number of reported symptomatic cases and deaths for France. 
Figure 5 Contact intensity function ε_{1} according the various scenarios. 
Figure 6 Daily number of reported cases and daily number of deaths according to the various scenarios. (c, G) = (0.08, 1) (scenario 1) panel (a). (c, G) = (0.05, 1) (scenario 2) panel (b). (c, G) = (0.05, 3) (scenario 3) panel (c). (c, G) = (0.05, 0.3^{−1}) (scenario 4) panel (d). 
5 Discussion
France was the 7th country to be affected by the COVID19 virus and delayed the strict lockdown application. By that time, the number of symptomatic reported cases was already large (6633 reported cases on March the 17th) and lockdown has not prevented deaths and contamination of a large part of the population. However, it has drastically slowed down the spread of the virus. The economic loss is huge (production has decreased by one third) and lockdown can only be a temporary measure. This is the reason for studying the endoflockdown measures in this paper. We have seen that only scenarios with drastic reduction of physical distancing compared to the one that prevailed before the epidemic outbreak prevent from the reemergence of the disease.
To avoid the reemergence of the epidemics after the end of lockdown, public policies still need to be applied, as for example a more intensive screening process. Figure 7 shows scenario 3 ((c, G) = (0.05, 3)) with the addition of a more intense screening process. It can be seen that this would avoid the reemergence of the disease.
Another possible health policy measure might be to lift the lockdown for only one part of the population. We would consider a realistic scenario with slow loss of intensity of contact habits acquired during the lockdown and reduction of one half of the intensity of contact in the long run from what it was before the epidemic outbreak. This corresponds to (c, G) = (0.05, 2). Figure 8 presents the evolution of the epidemics under various scenarios which differ the one from another by the number of individuals concerned by the lift of lockdown. It is assumed in Figure 8a that the end of lockdown applies to all the population. The peak of the epidemics appears in November 2020. In Figure 8b we consider the case in which 30% of the population is contained and its contact intensity is similar to its value at the end of the lockdown period. The peak of the epidemics appears in February 2021. In Figure 8c that it is assumed that 60% of the population is maintained under a lockdown regime. It can be seen that there is no reemergence of the disease.
Figure 7 Daily number of reported cases and daily number of deaths with (c, G) = (0.05, 3) (scenario 3) and f = 0.6 (instead of f = 0.4 in the previous figures). 
Figure 8 Daily number of reported cases and daily number of deaths with (c, G) = (0.05, 2) under various scenarios of partial lockdown. Lift of lockdown for all the population (a). Lift of lockdown for 70% of the population (b). Lift of lockdown for 40% of the population (c). 
6 Supplementary material
6.1 Proof of Y_{∞} = 0
We have proved that there exists t_{m} ≥ 0 such that Y (t) is a decreasing function of t, for t > t_{m}. Moreover, from equation (2.1), L(t), A(t) and I(t) are non negative functions, which implies that Y (t) ≥ 0.
Let us call D(t) the number of deaths. From equation (2.1),
Thus S + L + I + A + D is a decreasing function and S(t) + L(t) + I(t) + A(t) + D(t) ≤ N(0), where N(0) = S(0) + L(0) + A(0). Moreover
thus , which implies that .
As A′(t) = (1 − f)νL(t) − ηA(t), then
which implies that .
Thus Y (t) = 0.
References
 C.L. Althaus, Estimating the reproduction number of Ebola virus (EBOV) duringthe 2014 outbreak in West Africa. PLoS Curr. 6 (2014). [Google Scholar]
 J. Arino, F. Brauer, P. van den Driessche, J. Watmough and J. Wu, Simple models for containment of a pandemic. J. R. Soc. Interface 3 (2006) 453–457. [CrossRef] [PubMed] [Google Scholar]
 J. Arino, F. Brauer, P. Van Den Driessche, J. Watmough and J. Wu, A final size relation for epidemic models. Math. Biosci. Eng. 4 (2007) 159. [CrossRef] [PubMed] [Google Scholar]
 F. Brauer, Early estimates of epidemic final sizes. J. Biol. Dyn. 13 (2019) 23–30. [CrossRef] [PubMed] [Google Scholar]
 F. Brauer, The final size of a serious epidemic. Bull. Math. Biol. 81 (2019) 869–877. [Google Scholar]
 G. Chowell, N.W. Hengartner, C. CastilloChavez, P.W. Fenimore, J.M. Hyman, The basic reproductive number of Ebola and the effects of public health measures: the cases of Congo and Uganda. J. Theor. Biol. 229 (2004) 119–126. [CrossRef] [PubMed] [Google Scholar]
 A.B. Gumel, S. Ruan, T. Day, J. Watmough, F. Brauer, P. Van den Driessche and J. Wu, Modelling strategies for controlling SARS outbreaks. Proc. R. Soc. Lond. Ser. B 271 (2004) 2223–2232. [CrossRef] [Google Scholar]
 N. Ferguson, D. Laydon, G. Nedjati Gilani, N. Imai, K. Ainslie, M. Baguelin and A. Dighe, Report 9: Impact of nonpharmaceutical interventions (NPIs) to reduce COVID19 mortality and healthcaredemand (2020). [Google Scholar]
 M. Fournier, F. d'ArripeLongueville, C. Rovere, C.S. Easthope, L. Schwabe, J. El Methni and R. Radel, Effects of circadian cortisol on the development of a health habit. Health Psychol. 36 (2017) 1059. [CrossRef] [PubMed] [Google Scholar]
 Z. Liu, P. Magal, O. Seydi and G. Webb, Understanding unreported cases in the 2019nCov epidemic outbreak in Wuhan, China, and the importance of major public health interventions. MPDI Biol. 9 (2020) 50. [Google Scholar]
 Z. Liu, P. Magal, O. Seydi and G. Webb, Predicting the cumulative number of cases for the COVID19 epidemic in China from early data. arXiv preprint arXiv:2002.12298 (2020). [Google Scholar]
 P. Magal and G. Webb, Predicting the number of reported and unreported cases for the COVID19 epidemic in South Korea, Italy, France and Germany. Italy, France and Germany, March 19, 2020. [Google Scholar]
 A. Mignon Colombet and D. Floreancig, L'infraction de “mise en danger d'autrui” est à manier avec précaution, Tribune, Le Monde, 25 mars 2020. [Google Scholar]
 P. Van den Driessche and J. Watmough, Reproduction numbers and subthreshold endemic equilibria for compartmental models of disease transmission. Math. Biosci. 180 (2002) 29–48. [Google Scholar]
 https://en.wikipedia.org/wiki/2020_coronavirus_outbreak_in_France. [Google Scholar]
 J. Zhang, M. Litvinova, Y. Liang, Y Wang, W. Wang, S. Zhao and M. Ajelli, Changes in contact patterns shape the dynamics of the COVID19 outbreak in China. Science 368 (2020) 1481–1486. [Google Scholar]
All Tables
All Figures
Figure 1 Elasticity of final size S_{∞} according to contact intensity ε_{1} for various values of contact intensity ε_{1}. 

In the text 
Figure 2 Elasticity of number of individuals that have been contaminated at the end of the epidemics C_{∞} according to contact intensity ε_{1} for various values of contact intensity ε_{1}. 

In the text 
Figure 3 Daily number of reported symptomatic cases (a) and daily number of COVID19 deaths (b) for France. 

In the text 
Figure 4 Cumulative number of reported symptomatic cases and deaths for France. 

In the text 
Figure 5 Contact intensity function ε_{1} according the various scenarios. 

In the text 
Figure 6 Daily number of reported cases and daily number of deaths according to the various scenarios. (c, G) = (0.08, 1) (scenario 1) panel (a). (c, G) = (0.05, 1) (scenario 2) panel (b). (c, G) = (0.05, 3) (scenario 3) panel (c). (c, G) = (0.05, 0.3^{−1}) (scenario 4) panel (d). 

In the text 
Figure 7 Daily number of reported cases and daily number of deaths with (c, G) = (0.05, 3) (scenario 3) and f = 0.6 (instead of f = 0.4 in the previous figures). 

In the text 
Figure 8 Daily number of reported cases and daily number of deaths with (c, G) = (0.05, 2) under various scenarios of partial lockdown. Lift of lockdown for all the population (a). Lift of lockdown for 70% of the population (b). Lift of lockdown for 40% of the population (c). 

In the text 
Current usage metrics show cumulative count of Article Views (fulltext article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 4896 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.