Neal R Haddaway | Stockholm Environment Institute (original) (raw)

Papers by Neal R Haddaway

Background: Mining activities, including prospecting, exploration, construction, operation, maint... more Background: Mining activities, including prospecting, exploration, construction, operation, maintenance, expansion, abandonment, decommissioning and repurposing of a mine can impact social and environmental systems in a range of positive and negative, and direct and indirect ways. Mining can yield a range of benefits to societies, but it may also cause conflict, not least in relation to above-ground and sub-surface land use. Similarly, mining can alter environments, but remediation and mitigation can restore systems. Boreal and Arctic regions are sensitive to impacts from development, both on social and environmental systems. Native ecosystems and aboriginal human communities are typically affected by multiple stressors, including climate change and pollution, for example. Methods: We will search a suite of bibliographic databases, online search engines and organisational websites for relevant research literature using a tested search strategy. We will also make a call for evidence ...

Synthesis of evidence from the totality of relevant research is essential to inform and improve p... more Synthesis of evidence from the totality of relevant research is essential to inform and improve prevention efforts and policy. Given the large and usually heterogeneous evidence available,reaching a thorough understanding of what works, for whom, and in what contexts, can only be achieved through a systematic and comprehensive synthesis of evidence. Many barriers impede comprehensive evidence synthesis, which leads to uncertainty about the generalizability of intervention effectiveness, including: inaccurate terminology titles/abstracts/keywords (hampering literature search efforts); ambiguous reporting of study methods (resulting ininaccurate assessments of study rigor); and poorly reported participant characteristics, outcomes, and key variables (obstructing the calculation of an overall effect or the examination of effect modifiers). To address these issues and improve the reach of primary studies through theirinclusion in evidence syntheses, we provide a set of practical guideli...

Journal of Clinical Epidemiology

Environment International, 2016

Systematic reviews and maps should be based on the best available evidence, and reviewers should ... more Systematic reviews and maps should be based on the best available evidence, and reviewers should make all reasonable efforts to source and include potentially relevant studies. However, reviewers may not be able to consider all existing evidence, since some data and studies may not be publicly available. Including non-public studies in reviews provides a valuable opportunity to increase systematic review/map comprehensiveness, potentially mitigating negative impacts of publication bias. Studies may be non-public for many reasons: some may still be in the process of being published (publication can take a long time); some may not be published due to author/publisher restrictions; publication bias may make it difficult to publish non-significant or negative results. Here, we consider what forms these non-public studies may take and the implications of including them in systematic reviews and maps. Reviewers should carefully consider the advantages and disadvantages of including non-public studies, weighing risks of bias against benefits of increased comprehensiveness. As with all systematic reviews and maps, reviewers must be transparent about methods used to obtain data and avoid risks of bias in their synthesis. We make tentative suggestions for reviewers in situations where non-public data may be present in an evidence base.

The North American signal crayfish (Pacifastacus leniusculus) is native to the Pacific North West... more The North American signal crayfish (Pacifastacus leniusculus) is native to the Pacific North West, with introduced populations extending to southern California. The species has also been widely introduced across Europe, with multiple detrimental impacts on native biota and abiotic environments. Understanding the differences between native and invasive populations of invading species may provide an insight into why some species are successful invaders, and may also aid in predicting future invaders and their impacts. Previous work shows that native British crayfish are morphologically adapted to local environments. Here we present evidence that invasive populations of North American signal crayfish in the UK show the same morphological adaptations; carapaces are narrower in running water and wider in still water, and areolae show the opposite correlation. This relationship may reflect differences in dissolved oxygen concentrations between lentic and lotic environments. No such relati...

Nature Climate Change, 2021

Assessing global progress on human adaptation to climate change is an urgent priority. Although t... more Assessing global progress on human adaptation to climate change is an urgent priority. Although the literature on adaptation to climate change is rapidly expanding, little is known about the actual extent of implementation. We systematically screened >48,000 articles using machine learning methods and a global network of 126 researchers. Our synthesis of the resulting 1,682 articles presents a systematic and comprehensive global stocktake of implemented human adaptation to climate change. Documented adaptations were largely fragmented, local and incremental, with limited evidence of transformational adaptation and negligible evidence of risk reduction outcomes. We identify eight priorities for global adaptation research: assess the effectiveness of adaptation responses, enhance the understanding of limits to adaptation, enable individuals and civil society to adapt, include missing places, scholars and scholarship, understand private sector responses, improve methods for synthesizing different forms of evidence, assess the adaptation at different temperature thresholds, and improve the inclusion of timescale and the dynamics of responses.

Nature Energy, 2021

Despite the importance of evaluating all mitigation options to inform policy decisions addressing... more Despite the importance of evaluating all mitigation options to inform policy decisions addressing climate change, a comprehensive analysis of household-scale interventions and their emissions reduction potential is missing. Here, we address this gap for interventions aimed at changing individual households’ use of existing equipment, such as monetary incentives or feedback. We have performed a machine learning-assisted systematic review and meta-analysis to comparatively assess the effectiveness of these interventions in reducing energy demand in residential buildings. We extracted 360 individual effect sizes from 122 studies representing trials in 25 countries. Our meta-regression confirms that both monetary and non-monetary interventions reduce the energy consumption of households, but monetary incentives, of the sizes reported in the literature, tend to show on average a more pronounced effect. Deploying the right combinations of interventions increases the overall effectiveness. We have estimated a global carbon emissions reduction potential of 0.35 GtCO2 yr−1, although deploying the most effective packages of interventions could result in greater reduction. While modest, this potential should be viewed in conjunction with the need for de-risking mitigation pathways with energy-demand reductions.

The Lancet Planetary Health, 2021

Background The global literature on the links between climate change and human health is large, i... more Background The global literature on the links between climate change and human health is large, increasing exponentially, and it is no longer feasible to collate and synthesise using traditional systematic evidence mapping approaches. We aimed to use machine learning methods to systematically synthesise an evidence base on climate change and human health. Methods We used supervised machine learning and other natural language processing methods (topic modelling and geoparsing) to systematically identify and map the scientific literature on climate change and health published between Jan 1, 2013, and April 9, 2020. Only literature indexed in English were included. We searched Web of Science Core Collection, Scopus, and PubMed using title, abstract, and keywords only. We searched for papers including both a health component and an explicit mention of either climate change, climate variability, or climate change-relevant weather phenomena. We classified relevant publications according to the fields of climate research, climate drivers, health impact, date, and geography. We used supervised and unsupervised machine learning to identify and classify relevant articles in the field of climate and health, with outputs including evidence heat maps, geographical maps, and narrative synthesis of trends in climate health-related publications. We included empirical literature of any study design that reported on health pathways associated with climate impacts, mitigation, or adaptation. Findings We predict that there are 15 963 studies in the field of climate and health published between 2013 and 2019. Climate health literature is dominated by impact studies, with mitigation and adaptation responses and their cobenefits and co-risks remaining niche topics. Air quality and heat stress are the most frequently studied exposures, with all-cause mortality and infectious disease incidence being the most frequently studied health outcomes. Seasonality, extreme weather events, heat, and weather variability are the most frequently studied climate-related hazards. We found major gaps in evidence on climate health research for mental health, undernutrition, and maternal and child health. Geographically, the evidence base is dominated by studies from high-income countries and China, with scant evidence from low-income counties, which often suffer most from the health consequences of climate change. Interpretation Our findings show the importance and feasibility of using automated machine learning to comprehensively map the science on climate change and human health in the age of big literature. These can provide key inputs into global climate and health assessments. The scant evidence on climate change response options is concerning and could significantly hamper the design of evidence-based pathways to reduce the effects on health of climate change. In the post-2015 Paris Agreement era of climate solutions, we believe much more attention should be given to climate adaptation and mitigation options and their effects on human health.

Ecological Solutions and Evidence, 2021

1. Research weaving is an approach that combines systematic mapping methods with bibliometric and... more 1. Research weaving is an approach that combines systematic mapping methods with bibliometric and scientometric analyses, shedding light on how research in a systematic map is connected or disconnected. Given its novelty, few examples exist that demonstrate methods for research weaving or highlight its value in the evidence synthesis context. 2. Here, we seek to fill this gap by applying a research weaving analysis to a previously published systematic map on the roles of buffer strips in agricultural fields. This work identified a lack of studies addressing multifunctional roles of buffer strips and a remarkable array of terminology used by researchers to describe buffer strips. Using network visualization, such as co-authorship and bibliographic coupling networks, as well as content and text analyses, we aim to build on these findings addressing questions related to collaboration, disciplinary contributions and citation and term usage patterns. 3. As a result of this work, we aim to provide workflows, tools, and recommendations for the application of research weaving across a wide range of evidence maps in any domain. We will discuss the unique challenges of conducting bibliometric analysis in an evidence synthesis context and will propose solutions to address these challenges.

Conservation Physiology, 2021

Conservation physiology represents a recently emerging arm of conservation science that applies p... more Conservation physiology represents a recently emerging arm of conservation science that applies physiological tools and techniques to understand and solve conservation issues. While a multi-disciplinary toolbox can only help to address the global biodiversity crisis, any field can face challenges while becoming established, particularly highly applied disciplines that require multi-stakeholder involvement. Gaining first-hand knowledge of the challenges that conservation physiologists are facing can help characterize the current state of the field and build a better foundation for determining how it can grow. Through an online survey of 468 scientists working at the intersection of physiology and conservation, we aimed to identify characteristics of those engaging in conservation physiology research (e.g. demographics, primary taxa of study), gauge conservation physiology’s role in contributing to on-the-ground conservation action, identify the perceived barriers to achieving success and determine how difficult any identified barriers are to overcome. Despite all participants having experience combining physiology and conservation, only one-third considered themselves to be ‘conservation physiologists’. Moreover, there was a general perception that conservation physiology does not yet regularly lead to tangible conservation success. Respondents identified the recent conceptualization of the field and the broader issue of adequately translating science into management action as the primary reasons for these deficits. Other significant barriers that respondents have faced when integrating physiology and conservation science included a lack of funding, logistical constraints (e.g. sample sizes, obtaining permits) and a lack of physiological baseline data (i.e. reference ranges of a physiological metric’s ‘normal’ or pre-environmental change levels). We identified 12 actions based on suggestions of survey participants that we anticipate will help deconstruct the barriers and continue to develop a narrative of physiology that is relevant to conservation science, policy and practice.

Systematic Reviews, 2021

Background: Investigations of transparency, reproducibility and replicability in science have bee... more Background: Investigations of transparency, reproducibility and replicability in science have been directed largely at individual studies. It is just as critical to explore these issues in syntheses of studies, such as systematic reviews, given their influence on decision-making and future research. We aim to explore various aspects relating to the transparency, reproducibility and replicability of several components of systematic reviews with meta-analysis of the effects of health, social, behavioural and educational interventions. Methods: The REPRISE (REProducibility and Replicability In Syntheses of Evidence) project consists of four studies. We will evaluate the completeness of reporting and sharing of review data, analytic code and other materials in a random sample of 300 systematic reviews of interventions published in 2020 (Study 1). We will survey authors of systematic reviews to explore their views on sharing review data, analytic code and other materials and their understanding of and opinions about replication of systematic reviews (Study 2). We will then evaluate the extent of variation in results when we (a) independently reproduce meta-analyses using the same computational steps and analytic code (if available) as used in the original review (Study 3), and (b) crowdsource teams of systematic reviewers to independently replicate a subset of methods (searches for studies, selection of studies for inclusion, collection of outcome data, and synthesis of results) in a sample of the original reviews; 30 reviews will be replicated by 1 team each and 2 reviews will be replicated by 15 teams (Study 4). Discussion: The REPRISE project takes a systematic approach to determine how reliable systematic reviews of interventions are. We anticipate that results of the REPRISE project will inform strategies to improve the conduct and reporting of future systematic reviews.

[

F1000 Research, 2021

Background: While the PRISMA flow diagram is widely used for reporting standard systematic review... more Background: While the PRISMA flow diagram is widely used for reporting standard systematic reviews (SRs), it was not designed for capturing the results of continual searches for studies in living systematic reviews (LSRs). The objectives of this study are (1) to assess how published LSRs report on the flow of studies through the different phases of the review for the different updates; (2) to propose an approach to reporting on that flow. Methods: For objective 1, we identified all LSRs published up to July 2020. We abstracted information regarding their general characteristics and how they reported on search results. For objective 2, we based our proposal for tailored PRISMA approaches on the findings from objective 1, as well as on our experience with conducting Cochrane LSRs. Results: We identified 108 living publications relating to 32 LSRs. Of the 108 publications, 7% were protocols, 24% were base versions (i.e., the first version), 62% were partial updates (i.e., does not include all typical sections of an SR), and 7% were full updates (i.e., includes all typical sections of an SR). We identified six ways to reporting the study flow: base separately, each update separately (38%);

Environmental Evidence, 2021

One of the most important steps in the process of conducting a systematic review or map is data e... more One of the most important steps in the process of conducting a systematic review or map is data extraction and the production of a database of coding, metadata and study data. There are many ways to structure these data, but to date, no guidelines or standards have been produced for the evidence synthesis community to support their production. Furthermore, there is little adoption of easily machine-readable, readily reusable and adaptable databases: these databases would be easier to translate into different formats by review authors, for example for tabulation, visualisa-tion and analysis, and also by readers of the review/map. As a result, it is common for systematic review and map authors to produce bespoke, complex data structures that, although typically provided digitally, require considerable efforts to understand, verify and reuse. Here, we report on an analysis of systematic reviews and maps published by the Collaboration for Environmental Evidence, and discuss major issues that hamper machine readability and data reuse or verification. We highlight different justifications for the alternative data formats found: condensed databases; long databases; and wide databases. We describe these challenges in the context of data science principles that can support curation and publication of machine-readable, Open Data. We then go on to make recommendations to review and map authors on how to plan and structure their data, and we provide a suite of novel R-based functions to support efficient and reliable translation of databases between formats that are useful for presentation (condensed, human readable tables), filtering and visualisation (wide databases), and analysis (long databases). We hope that our recommendations for adoption of standard practices in database formatting, and the tools necessary to rapidly move between formats will provide a step-change in transparency and replicability of Open Data in evidence synthesis.

Ecological Solutions and Evidence, 2021

1. The 'anthropause' , a period of unusually reduced human activity and mobility due to COVID-19 ... more 1. The 'anthropause' , a period of unusually reduced human activity and mobility due to COVID-19 restrictions, has serendipitously opened up unique opportunities for research on how human activities impact the environment. 2. In the field of health, COVID-19 research has led to concerns about the quality of research papers and the underlying research and publication processes due to accelerated peer review and publication schedules, increases in pre-prints and retractions. 3. In the field of environmental science, framing the pandemic and associated global lockdowns as an unplanned global human confinement experiment with urgency should raise the same concerns about the rigorousness and integrity of the scientific process. Furthermore, the recognition of an 'infodemic' , an unprecedented explosion of research, risks research waste and duplication of effort, although how information is used is as important as the quality of evidence. This highlights the need for an evidence base that is easy to find and use-that is discoverable, curated, synthesizable, synthesized. 4. We put forward a list of 10 key principles to support the establishment of a reproducible , replicable, robust, rigorous, timely and synthesizable COVID-19 environmental evidence base that avoids research waste and is resilient to the pressures to publish urgently. These principles focus on engaging relevant actors (e.g. local communities , rightsholders) in research design and production, statistical power, collaborations, evidence synthesis, research registries and protocols, open science and transparency, data hygiene (cleanliness) and integrity, peer review transparency, standardized keywords and controlled vocabularies.

Environmental Evidence, 2021

Background: Coral reefs are rapidly changing in response to local and global stressors. Research ... more Background: Coral reefs are rapidly changing in response to local and global stressors. Research to better understand and inform the management of these stressors is burgeoning. However, in situ studies of coral reef ecology are constrained by complex logistics and limited resources. Many reef studies are also hampered by the scale-dependent nature of ecological patterns, and inferences made on causal relationships within coral reef systems are limited by the scales of observation. This is because most socio-ecological studies are conducted at scales relevant to the phenomenon of interest. However, management often occurs across a significantly broader, often geopolitical, range of scales. While there is a critical need for incisive coral reef management actions at relevant spatial and temporal scales, it remains unclear to what extent the scales of empirical study overlap with the scales at which management inferences and recommendations are made. This systematic map protocol will evaluate this potential scale mismatch with the goal of raising awareness about the significance of effectively addressing and reporting the scales at which researchers collect data and make assumptions. Methods: We will use the Collaboration for Environmental Evidence (CEE) systematic mapping guidelines to identify relevant studies using a framework-based synthesis to summarise the spatial and temporal scales of coral reef fish ecology research and the scales at which management inferences or recommendations are made. Using tested pre-defined terms, we will search for relevant published academic and grey literature, including bibliographic databases, web-based search engines, and organisational websites. Inclusion criteria for the evidence map are empirical studies that focus on coral reef fish ecological organisation and processes, those informing management interventions and policy decisions, and management documents that cite coral reef research for management decision-making. Study results will be displayed graphically using data matrices and heat maps. This is the first attempt to systematically assess and compare the scales of socio-ecological research conducted on coral reef systems with their management.

Campbell Systematic Reviews, 2021

Environmental Evidence, 2020

Background: Insects play a central role in the functioning of terrestrial and freshwater ecosyste... more Background: Insects play a central role in the functioning of terrestrial and freshwater ecosystems and contribute to a multitude of ecosystem services in managed and unmanaged systems Even local declines of insect abundance and richness can have enormous ecological and economic consequences. Evidence-informed conservation actions are essential to prevent potential cascading consequences of insect declines, and to help declining populations recover. Policy-makers rely on syntheses of primary research, such as reviews and meta-analyses, when making decisions about which conservation actions to implement. These evidence reviews vary in their quality, and do not necessarily synthesise the full range of scientific evidence found in the primary literature, which hampers decision-making. This evidence review map will identify, catalogue, and describe evidence reviews that investigate actions and policies to conserve insect biodiversity. This will help policy makers identify relevant reviews and researchers to identify synthesis gaps. It will also generate a list of conservation actions which can feed into future synthesis projects. We will produce an interactive database of evidence reviews, acting as a bibliography for policy-makers, curate a list of insect conservation actions that have been reviewed, and identify synthesis gaps for conservation actions that have not been reviewed. Methods: We will search for evidence reviews across seven large, generic bibliographic databases, a database of environmental reviews, and five grey literature resources using a search string consisting of an insect substring, a biodiversity or population response substring, an evidence synthesis substring, and a conservation substring. The results will be deduplicated and then screened at title and abstract (concurrently) and full text levels against prede-fined inclusion criteria. We will initially perform consistency checking on a subset of records at each level to ensure the inclusion criteria are sufficiently clear and understood by multiple reviewers. We will extract a suite of descriptive meta-data from relevant reviews, including a description of the action and information on each review's focal taxa, biomes, and locations. If resources allow, we will apply the CEESAT critical appraisal tool for evidence reviews to assess validity of individual records and the evidence base as a whole. We will summarise our findings in an interactive database of reviews and other visualisations, including evidence atlases and heat maps (cross tabulations of the volume of evidence across two categorical variables). The findings of the evidence review map will support the identification of synthesis gaps and clusters that may warrant further attention through secondary research.

Environmental Evidence, 2020

Evidence synthesis is a vital part of evidence-informed decision-making, but high growth in the v... more Evidence synthesis is a vital part of evidence-informed decision-making, but high growth in the volume of research evidence over recent decades has made efficient evidence synthesis increasingly challenging. As the appreciation and need for timely and rigorous evidence synthesis continue to grow, so too will the need for tools and frameworks to conduct reviews of expanding evidence bases in an efficient and time-sensitive manner. Efforts to future-proof evidence synthesis through the development of new evidence synthesis technology (ESTech) have so far been isolated across interested individuals or groups, with no concerted effort to collaborate or build communities of practice in technology production. We established the evidence synthesis Hackathon to stimulate collaboration and the production of Free and Open Source Software and frameworks to support evidence synthesis. Here, we introduce a special series of papers on ESTech, and invite the readers of environmental evidence to submit manuscripts introducing and validating novel tools and frameworks. We hope this collection will help to consolidate ESTech development efforts and we encourage readers to join the ESTech revolution. In order to future-proof evidence synthesis against the evidence avalanche, we must support community enthusiasm for ESTech, reduce redundancy in tool design, collaborate and share capacity in tool production, and reduce inequalities in software accessibility.

Background: Mining activities, including prospecting, exploration, construction, operation, maint... more Background: Mining activities, including prospecting, exploration, construction, operation, maintenance, expansion, abandonment, decommissioning and repurposing of a mine can impact social and environmental systems in a range of positive and negative, and direct and indirect ways. Mining can yield a range of benefits to societies, but it may also cause conflict, not least in relation to above-ground and sub-surface land use. Similarly, mining can alter environments, but remediation and mitigation can restore systems. Boreal and Arctic regions are sensitive to impacts from development, both on social and environmental systems. Native ecosystems and aboriginal human communities are typically affected by multiple stressors, including climate change and pollution, for example. Methods: We will search a suite of bibliographic databases, online search engines and organisational websites for relevant research literature using a tested search strategy. We will also make a call for evidence ...

Synthesis of evidence from the totality of relevant research is essential to inform and improve p... more Synthesis of evidence from the totality of relevant research is essential to inform and improve prevention efforts and policy. Given the large and usually heterogeneous evidence available,reaching a thorough understanding of what works, for whom, and in what contexts, can only be achieved through a systematic and comprehensive synthesis of evidence. Many barriers impede comprehensive evidence synthesis, which leads to uncertainty about the generalizability of intervention effectiveness, including: inaccurate terminology titles/abstracts/keywords (hampering literature search efforts); ambiguous reporting of study methods (resulting ininaccurate assessments of study rigor); and poorly reported participant characteristics, outcomes, and key variables (obstructing the calculation of an overall effect or the examination of effect modifiers). To address these issues and improve the reach of primary studies through theirinclusion in evidence syntheses, we provide a set of practical guideli...

Journal of Clinical Epidemiology

Environment International, 2016

Systematic reviews and maps should be based on the best available evidence, and reviewers should ... more Systematic reviews and maps should be based on the best available evidence, and reviewers should make all reasonable efforts to source and include potentially relevant studies. However, reviewers may not be able to consider all existing evidence, since some data and studies may not be publicly available. Including non-public studies in reviews provides a valuable opportunity to increase systematic review/map comprehensiveness, potentially mitigating negative impacts of publication bias. Studies may be non-public for many reasons: some may still be in the process of being published (publication can take a long time); some may not be published due to author/publisher restrictions; publication bias may make it difficult to publish non-significant or negative results. Here, we consider what forms these non-public studies may take and the implications of including them in systematic reviews and maps. Reviewers should carefully consider the advantages and disadvantages of including non-public studies, weighing risks of bias against benefits of increased comprehensiveness. As with all systematic reviews and maps, reviewers must be transparent about methods used to obtain data and avoid risks of bias in their synthesis. We make tentative suggestions for reviewers in situations where non-public data may be present in an evidence base.

The North American signal crayfish (Pacifastacus leniusculus) is native to the Pacific North West... more The North American signal crayfish (Pacifastacus leniusculus) is native to the Pacific North West, with introduced populations extending to southern California. The species has also been widely introduced across Europe, with multiple detrimental impacts on native biota and abiotic environments. Understanding the differences between native and invasive populations of invading species may provide an insight into why some species are successful invaders, and may also aid in predicting future invaders and their impacts. Previous work shows that native British crayfish are morphologically adapted to local environments. Here we present evidence that invasive populations of North American signal crayfish in the UK show the same morphological adaptations; carapaces are narrower in running water and wider in still water, and areolae show the opposite correlation. This relationship may reflect differences in dissolved oxygen concentrations between lentic and lotic environments. No such relati...

Nature Climate Change, 2021

Assessing global progress on human adaptation to climate change is an urgent priority. Although t... more Assessing global progress on human adaptation to climate change is an urgent priority. Although the literature on adaptation to climate change is rapidly expanding, little is known about the actual extent of implementation. We systematically screened >48,000 articles using machine learning methods and a global network of 126 researchers. Our synthesis of the resulting 1,682 articles presents a systematic and comprehensive global stocktake of implemented human adaptation to climate change. Documented adaptations were largely fragmented, local and incremental, with limited evidence of transformational adaptation and negligible evidence of risk reduction outcomes. We identify eight priorities for global adaptation research: assess the effectiveness of adaptation responses, enhance the understanding of limits to adaptation, enable individuals and civil society to adapt, include missing places, scholars and scholarship, understand private sector responses, improve methods for synthesizing different forms of evidence, assess the adaptation at different temperature thresholds, and improve the inclusion of timescale and the dynamics of responses.

Nature Energy, 2021

Despite the importance of evaluating all mitigation options to inform policy decisions addressing... more Despite the importance of evaluating all mitigation options to inform policy decisions addressing climate change, a comprehensive analysis of household-scale interventions and their emissions reduction potential is missing. Here, we address this gap for interventions aimed at changing individual households’ use of existing equipment, such as monetary incentives or feedback. We have performed a machine learning-assisted systematic review and meta-analysis to comparatively assess the effectiveness of these interventions in reducing energy demand in residential buildings. We extracted 360 individual effect sizes from 122 studies representing trials in 25 countries. Our meta-regression confirms that both monetary and non-monetary interventions reduce the energy consumption of households, but monetary incentives, of the sizes reported in the literature, tend to show on average a more pronounced effect. Deploying the right combinations of interventions increases the overall effectiveness. We have estimated a global carbon emissions reduction potential of 0.35 GtCO2 yr−1, although deploying the most effective packages of interventions could result in greater reduction. While modest, this potential should be viewed in conjunction with the need for de-risking mitigation pathways with energy-demand reductions.

The Lancet Planetary Health, 2021

Background The global literature on the links between climate change and human health is large, i... more Background The global literature on the links between climate change and human health is large, increasing exponentially, and it is no longer feasible to collate and synthesise using traditional systematic evidence mapping approaches. We aimed to use machine learning methods to systematically synthesise an evidence base on climate change and human health. Methods We used supervised machine learning and other natural language processing methods (topic modelling and geoparsing) to systematically identify and map the scientific literature on climate change and health published between Jan 1, 2013, and April 9, 2020. Only literature indexed in English were included. We searched Web of Science Core Collection, Scopus, and PubMed using title, abstract, and keywords only. We searched for papers including both a health component and an explicit mention of either climate change, climate variability, or climate change-relevant weather phenomena. We classified relevant publications according to the fields of climate research, climate drivers, health impact, date, and geography. We used supervised and unsupervised machine learning to identify and classify relevant articles in the field of climate and health, with outputs including evidence heat maps, geographical maps, and narrative synthesis of trends in climate health-related publications. We included empirical literature of any study design that reported on health pathways associated with climate impacts, mitigation, or adaptation. Findings We predict that there are 15 963 studies in the field of climate and health published between 2013 and 2019. Climate health literature is dominated by impact studies, with mitigation and adaptation responses and their cobenefits and co-risks remaining niche topics. Air quality and heat stress are the most frequently studied exposures, with all-cause mortality and infectious disease incidence being the most frequently studied health outcomes. Seasonality, extreme weather events, heat, and weather variability are the most frequently studied climate-related hazards. We found major gaps in evidence on climate health research for mental health, undernutrition, and maternal and child health. Geographically, the evidence base is dominated by studies from high-income countries and China, with scant evidence from low-income counties, which often suffer most from the health consequences of climate change. Interpretation Our findings show the importance and feasibility of using automated machine learning to comprehensively map the science on climate change and human health in the age of big literature. These can provide key inputs into global climate and health assessments. The scant evidence on climate change response options is concerning and could significantly hamper the design of evidence-based pathways to reduce the effects on health of climate change. In the post-2015 Paris Agreement era of climate solutions, we believe much more attention should be given to climate adaptation and mitigation options and their effects on human health.

Ecological Solutions and Evidence, 2021

1. Research weaving is an approach that combines systematic mapping methods with bibliometric and... more 1. Research weaving is an approach that combines systematic mapping methods with bibliometric and scientometric analyses, shedding light on how research in a systematic map is connected or disconnected. Given its novelty, few examples exist that demonstrate methods for research weaving or highlight its value in the evidence synthesis context. 2. Here, we seek to fill this gap by applying a research weaving analysis to a previously published systematic map on the roles of buffer strips in agricultural fields. This work identified a lack of studies addressing multifunctional roles of buffer strips and a remarkable array of terminology used by researchers to describe buffer strips. Using network visualization, such as co-authorship and bibliographic coupling networks, as well as content and text analyses, we aim to build on these findings addressing questions related to collaboration, disciplinary contributions and citation and term usage patterns. 3. As a result of this work, we aim to provide workflows, tools, and recommendations for the application of research weaving across a wide range of evidence maps in any domain. We will discuss the unique challenges of conducting bibliometric analysis in an evidence synthesis context and will propose solutions to address these challenges.

Conservation Physiology, 2021

Conservation physiology represents a recently emerging arm of conservation science that applies p... more Conservation physiology represents a recently emerging arm of conservation science that applies physiological tools and techniques to understand and solve conservation issues. While a multi-disciplinary toolbox can only help to address the global biodiversity crisis, any field can face challenges while becoming established, particularly highly applied disciplines that require multi-stakeholder involvement. Gaining first-hand knowledge of the challenges that conservation physiologists are facing can help characterize the current state of the field and build a better foundation for determining how it can grow. Through an online survey of 468 scientists working at the intersection of physiology and conservation, we aimed to identify characteristics of those engaging in conservation physiology research (e.g. demographics, primary taxa of study), gauge conservation physiology’s role in contributing to on-the-ground conservation action, identify the perceived barriers to achieving success and determine how difficult any identified barriers are to overcome. Despite all participants having experience combining physiology and conservation, only one-third considered themselves to be ‘conservation physiologists’. Moreover, there was a general perception that conservation physiology does not yet regularly lead to tangible conservation success. Respondents identified the recent conceptualization of the field and the broader issue of adequately translating science into management action as the primary reasons for these deficits. Other significant barriers that respondents have faced when integrating physiology and conservation science included a lack of funding, logistical constraints (e.g. sample sizes, obtaining permits) and a lack of physiological baseline data (i.e. reference ranges of a physiological metric’s ‘normal’ or pre-environmental change levels). We identified 12 actions based on suggestions of survey participants that we anticipate will help deconstruct the barriers and continue to develop a narrative of physiology that is relevant to conservation science, policy and practice.

Systematic Reviews, 2021

Background: Investigations of transparency, reproducibility and replicability in science have bee... more Background: Investigations of transparency, reproducibility and replicability in science have been directed largely at individual studies. It is just as critical to explore these issues in syntheses of studies, such as systematic reviews, given their influence on decision-making and future research. We aim to explore various aspects relating to the transparency, reproducibility and replicability of several components of systematic reviews with meta-analysis of the effects of health, social, behavioural and educational interventions. Methods: The REPRISE (REProducibility and Replicability In Syntheses of Evidence) project consists of four studies. We will evaluate the completeness of reporting and sharing of review data, analytic code and other materials in a random sample of 300 systematic reviews of interventions published in 2020 (Study 1). We will survey authors of systematic reviews to explore their views on sharing review data, analytic code and other materials and their understanding of and opinions about replication of systematic reviews (Study 2). We will then evaluate the extent of variation in results when we (a) independently reproduce meta-analyses using the same computational steps and analytic code (if available) as used in the original review (Study 3), and (b) crowdsource teams of systematic reviewers to independently replicate a subset of methods (searches for studies, selection of studies for inclusion, collection of outcome data, and synthesis of results) in a sample of the original reviews; 30 reviews will be replicated by 1 team each and 2 reviews will be replicated by 15 teams (Study 4). Discussion: The REPRISE project takes a systematic approach to determine how reliable systematic reviews of interventions are. We anticipate that results of the REPRISE project will inform strategies to improve the conduct and reporting of future systematic reviews.

[

F1000 Research, 2021

Background: While the PRISMA flow diagram is widely used for reporting standard systematic review... more Background: While the PRISMA flow diagram is widely used for reporting standard systematic reviews (SRs), it was not designed for capturing the results of continual searches for studies in living systematic reviews (LSRs). The objectives of this study are (1) to assess how published LSRs report on the flow of studies through the different phases of the review for the different updates; (2) to propose an approach to reporting on that flow. Methods: For objective 1, we identified all LSRs published up to July 2020. We abstracted information regarding their general characteristics and how they reported on search results. For objective 2, we based our proposal for tailored PRISMA approaches on the findings from objective 1, as well as on our experience with conducting Cochrane LSRs. Results: We identified 108 living publications relating to 32 LSRs. Of the 108 publications, 7% were protocols, 24% were base versions (i.e., the first version), 62% were partial updates (i.e., does not include all typical sections of an SR), and 7% were full updates (i.e., includes all typical sections of an SR). We identified six ways to reporting the study flow: base separately, each update separately (38%);

Environmental Evidence, 2021

One of the most important steps in the process of conducting a systematic review or map is data e... more One of the most important steps in the process of conducting a systematic review or map is data extraction and the production of a database of coding, metadata and study data. There are many ways to structure these data, but to date, no guidelines or standards have been produced for the evidence synthesis community to support their production. Furthermore, there is little adoption of easily machine-readable, readily reusable and adaptable databases: these databases would be easier to translate into different formats by review authors, for example for tabulation, visualisa-tion and analysis, and also by readers of the review/map. As a result, it is common for systematic review and map authors to produce bespoke, complex data structures that, although typically provided digitally, require considerable efforts to understand, verify and reuse. Here, we report on an analysis of systematic reviews and maps published by the Collaboration for Environmental Evidence, and discuss major issues that hamper machine readability and data reuse or verification. We highlight different justifications for the alternative data formats found: condensed databases; long databases; and wide databases. We describe these challenges in the context of data science principles that can support curation and publication of machine-readable, Open Data. We then go on to make recommendations to review and map authors on how to plan and structure their data, and we provide a suite of novel R-based functions to support efficient and reliable translation of databases between formats that are useful for presentation (condensed, human readable tables), filtering and visualisation (wide databases), and analysis (long databases). We hope that our recommendations for adoption of standard practices in database formatting, and the tools necessary to rapidly move between formats will provide a step-change in transparency and replicability of Open Data in evidence synthesis.

Ecological Solutions and Evidence, 2021

1. The 'anthropause' , a period of unusually reduced human activity and mobility due to COVID-19 ... more 1. The 'anthropause' , a period of unusually reduced human activity and mobility due to COVID-19 restrictions, has serendipitously opened up unique opportunities for research on how human activities impact the environment. 2. In the field of health, COVID-19 research has led to concerns about the quality of research papers and the underlying research and publication processes due to accelerated peer review and publication schedules, increases in pre-prints and retractions. 3. In the field of environmental science, framing the pandemic and associated global lockdowns as an unplanned global human confinement experiment with urgency should raise the same concerns about the rigorousness and integrity of the scientific process. Furthermore, the recognition of an 'infodemic' , an unprecedented explosion of research, risks research waste and duplication of effort, although how information is used is as important as the quality of evidence. This highlights the need for an evidence base that is easy to find and use-that is discoverable, curated, synthesizable, synthesized. 4. We put forward a list of 10 key principles to support the establishment of a reproducible , replicable, robust, rigorous, timely and synthesizable COVID-19 environmental evidence base that avoids research waste and is resilient to the pressures to publish urgently. These principles focus on engaging relevant actors (e.g. local communities , rightsholders) in research design and production, statistical power, collaborations, evidence synthesis, research registries and protocols, open science and transparency, data hygiene (cleanliness) and integrity, peer review transparency, standardized keywords and controlled vocabularies.

Environmental Evidence, 2021

Background: Coral reefs are rapidly changing in response to local and global stressors. Research ... more Background: Coral reefs are rapidly changing in response to local and global stressors. Research to better understand and inform the management of these stressors is burgeoning. However, in situ studies of coral reef ecology are constrained by complex logistics and limited resources. Many reef studies are also hampered by the scale-dependent nature of ecological patterns, and inferences made on causal relationships within coral reef systems are limited by the scales of observation. This is because most socio-ecological studies are conducted at scales relevant to the phenomenon of interest. However, management often occurs across a significantly broader, often geopolitical, range of scales. While there is a critical need for incisive coral reef management actions at relevant spatial and temporal scales, it remains unclear to what extent the scales of empirical study overlap with the scales at which management inferences and recommendations are made. This systematic map protocol will evaluate this potential scale mismatch with the goal of raising awareness about the significance of effectively addressing and reporting the scales at which researchers collect data and make assumptions. Methods: We will use the Collaboration for Environmental Evidence (CEE) systematic mapping guidelines to identify relevant studies using a framework-based synthesis to summarise the spatial and temporal scales of coral reef fish ecology research and the scales at which management inferences or recommendations are made. Using tested pre-defined terms, we will search for relevant published academic and grey literature, including bibliographic databases, web-based search engines, and organisational websites. Inclusion criteria for the evidence map are empirical studies that focus on coral reef fish ecological organisation and processes, those informing management interventions and policy decisions, and management documents that cite coral reef research for management decision-making. Study results will be displayed graphically using data matrices and heat maps. This is the first attempt to systematically assess and compare the scales of socio-ecological research conducted on coral reef systems with their management.

Campbell Systematic Reviews, 2021

Environmental Evidence, 2020

Background: Insects play a central role in the functioning of terrestrial and freshwater ecosyste... more Background: Insects play a central role in the functioning of terrestrial and freshwater ecosystems and contribute to a multitude of ecosystem services in managed and unmanaged systems Even local declines of insect abundance and richness can have enormous ecological and economic consequences. Evidence-informed conservation actions are essential to prevent potential cascading consequences of insect declines, and to help declining populations recover. Policy-makers rely on syntheses of primary research, such as reviews and meta-analyses, when making decisions about which conservation actions to implement. These evidence reviews vary in their quality, and do not necessarily synthesise the full range of scientific evidence found in the primary literature, which hampers decision-making. This evidence review map will identify, catalogue, and describe evidence reviews that investigate actions and policies to conserve insect biodiversity. This will help policy makers identify relevant reviews and researchers to identify synthesis gaps. It will also generate a list of conservation actions which can feed into future synthesis projects. We will produce an interactive database of evidence reviews, acting as a bibliography for policy-makers, curate a list of insect conservation actions that have been reviewed, and identify synthesis gaps for conservation actions that have not been reviewed. Methods: We will search for evidence reviews across seven large, generic bibliographic databases, a database of environmental reviews, and five grey literature resources using a search string consisting of an insect substring, a biodiversity or population response substring, an evidence synthesis substring, and a conservation substring. The results will be deduplicated and then screened at title and abstract (concurrently) and full text levels against prede-fined inclusion criteria. We will initially perform consistency checking on a subset of records at each level to ensure the inclusion criteria are sufficiently clear and understood by multiple reviewers. We will extract a suite of descriptive meta-data from relevant reviews, including a description of the action and information on each review's focal taxa, biomes, and locations. If resources allow, we will apply the CEESAT critical appraisal tool for evidence reviews to assess validity of individual records and the evidence base as a whole. We will summarise our findings in an interactive database of reviews and other visualisations, including evidence atlases and heat maps (cross tabulations of the volume of evidence across two categorical variables). The findings of the evidence review map will support the identification of synthesis gaps and clusters that may warrant further attention through secondary research.

Environmental Evidence, 2020

Evidence synthesis is a vital part of evidence-informed decision-making, but high growth in the v... more Evidence synthesis is a vital part of evidence-informed decision-making, but high growth in the volume of research evidence over recent decades has made efficient evidence synthesis increasingly challenging. As the appreciation and need for timely and rigorous evidence synthesis continue to grow, so too will the need for tools and frameworks to conduct reviews of expanding evidence bases in an efficient and time-sensitive manner. Efforts to future-proof evidence synthesis through the development of new evidence synthesis technology (ESTech) have so far been isolated across interested individuals or groups, with no concerted effort to collaborate or build communities of practice in technology production. We established the evidence synthesis Hackathon to stimulate collaboration and the production of Free and Open Source Software and frameworks to support evidence synthesis. Here, we introduce a special series of papers on ESTech, and invite the readers of environmental evidence to submit manuscripts introducing and validating novel tools and frameworks. We hope this collection will help to consolidate ESTech development efforts and we encourage readers to join the ESTech revolution. In order to future-proof evidence synthesis against the evidence avalanche, we must support community enthusiasm for ESTech, reduce redundancy in tool design, collaborate and share capacity in tool production, and reduce inequalities in software accessibility.

This report summarises a workshop held in Stockholm at Stockholm Environment Institute (SEI), Glo... more This report summarises a workshop held in Stockholm at Stockholm Environment Institute (SEI), Global Water Partnership (GWP) and Stockholm International Water Institute (SIWI) from the 23rd to the 25th April 2018. The event brought together 25 programmers and coders from across the globe in an attempt to collectively solve some of the biggest issues facing evidence synthesis using technology. The event was generously funded by Mistra EviEM (www.eviem.se/en) and the Fenner School’s Environment & Society Synthesis Program (Australian National University). Further details of the Evidence Synthesis Hackathon can be found on the website: www.evidencesynthesishackathon.com.