USC Marshall

In the summer of 2020, in collaboration with the Greek government, we designed and deployed Eva – the first national scale, reinforcement learning system for targeted COVID-19 testing. In this paper, we detail the rationale for three major design/algorithmic elements: Eva’s testing supply chain, estimating COVID-19 prevalence, and test allocation. Specifically, we describe the design of Eva’s supply chain to collect and process thousands of biological samples per day with special emphasis on capacity procurement. Then, we propose a novel, empirical Bayes estimation strategy to estimate COVID-19 prevalence among different passenger types with limited data and showcase how these estimates were instrumental for a variety of downstream decision-making. Finally, we propose a novel, multi-armed bandit algorithm that dynamically allocates tests to arriving passengers in a non-stationary environment with delayed feedback and batched decisions. All of our design and algorithmic choices emphasize the need for transparent reasoning to enable human-in-the-loop analytics. Such transparency was crucial to building trust and buy-in among policymakers and public health experts in a period of global crisis.

Throughout the coronavirus disease 2019 (COVID-19) pandemic, countries have relied on a variety of ad hoc border control protocols to allow for non-essential travel while safeguarding public health, from quarantining all travellers to restricting entry from select nations on the basis of population-level epidemiological metrics such as cases, deaths or testing positivity rates^1,2. Here we report the design and performance of a reinforcement learning system, nicknamed Eva. In the summer of 2020, Eva was deployed across all Greek borders to limit the influx of asymptomatic travellers infected with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and to inform border policies through real-time estimates of COVID-19 prevalence. In contrast to country-wide protocols, Eva allocated Greece’s limited testing resources on the basis of incoming travellers’ demographic information and testing results from previous travellers. By comparing Eva’s performance against modelled counterfactual scenarios, we show that Eva identified 1.85 times as many asymptomatic, infected travellers as random surveillance testing, with up to 2–4 times as many during peak travel, and 1.25–1.45 times as many asymptomatic, infected travellers as testing policies that utilize only epidemiological metrics. We demonstrate that this latter benefit arises, at least partially, because population-level epidemiological metrics had limited predictive value for the actual prevalence of SARS-CoV-2 among asymptomatic travellers and exhibited strong country-specific idiosyncrasies in the summer of 2020. Our results raise serious concerns on the effectiveness of country-agnostic internationally proposed border control policies³ that are based on population-level epidemiological metrics. Instead, our work represents a successful example of the potential of reinforcement learning and real-time data for safeguarding public health.

Efficient and targeted COVID-19 border testing via reinforcement learning

In this paper, we study a model of information consumption in which consumers sequentially interact with a platform that offers a menu of signals (posts) about an underlying state of the world (fact). At each time, incapable of consuming all posts, consumers screen the posts and only select (and consume) one from the offered menu. We show that, in the presence of uncertainty about the accuracy of these posts and as the number of posts increases, adverse effects, such as slow learning and polarization, arise. Specifically, we establish that, in this setting, bias emerges as a consequence of the consumer’s screening process. Namely, consumers, in their quest to choose the post that reduces their uncertainty about the state of the world, choose to consume the post that is closest to their own beliefs. We study the evolution of beliefs, and we show that such a screening bias slows down the learning process and that the speed of learning decreases with the menu size. Further, we show that the society becomes polarized during the prolonged learning process even in situations in which the society’s belief distribution was not a priori polarized.

Information products provide agents with additional information that can be used to update actions. In many situations, access to such products can be quite limited. For instance, in epidemics, there tends to be a limited supply of medical testing kits, or tests. These tests are information products because their output of a positive or a negative answer informs individuals and authorities on the underlying state and the appropriate course of action. In this paper, using an analytical model, we show how the accuracy of a test in detecting the underlying state affects the demand for the information product differentially across heterogeneous agents. Correspondingly, the test accuracy can serve as a rationing device to ensure that the limited supply of information products is appropriately allocated to the heterogeneous agents. When test availability is low and the social planner is unable to allocate tests in a targeted manner to the agents, we find that moderately good tests can outperform perfect tests in terms of social outcome.