The application of randomised controlled trials for urban policy evaluation in smart cities: a systematic literature review

The policy evaluation is mandatory in developing and implementing urban planning instruments. The literature shows that Randomized Controlled Trials (RCTs), being the most rigorous experimental method for obtaining high-quality evidence, are most suitable for policy evaluation (Neves, de Castro Neto, & Aparicio, 2020; Pearce & Raman, 2014). However, in an urban context can be challenging to implement this type of experiment (Athey & Imbens, 2017). The advent of smart cities brings new types of data and information that allows the use of these challenging experiments to evaluate the creation and implementation of a policy in the different stages (Barbero et al., 2016). Nevertheless, there is still a lack of application of this technique to planning policies evaluation.This paper aims to create a clear vision of the application of the RCT in urban planning policies evaluation. Building on the question “How has the RCT been used in urban planning policy evaluation in smart cities context?”, a research query with related keywords was created to use on the different databases, such as ISI Web of Knowledge, Scopus and Science Direct. The results were organized and screened following the PRISMA methodology. The references were organized by intervention goal, study sample, unit of analysis, and randomization method.The results suggest different types of interventions having their major difference related to the randomization unit, and also show a lack of evaluation of the effect of policies on urban interventions. This literature revision systemizes the research that uses RCT for urban planning policies evaluation. The increase of available data and analytical tools allows applying RCT experiments and improving decision-making. Therefore, this provides better insights into the whole policy process opening an opportunity to design data-driven public policies. This can be especially powerful in identifying the adverse effects of a policy, allowing proper adjustments to be made (Ubaldi, Van Ooijen, & Welby, 2019).The increase of the data available makes possible the use of RCT experiments for different urban dimensions controlling the required factors on the experiment, permitting the rigorous quantification of the effect of policy on the evaluated dimension. This methodology is increasing in interest for planning programs and has exciting results, but it is harder to implement (Pearce & Raman, 2014). This paper brings the overall vision of applying this method in a smart city’s context, giving a helpful roadmap to the implementation of this methodology.