Environmental Data Initiative logo
EDI Staging Environment

Data Package Metadata   View Summary

California Wildfire Resilience Core Metrics Rating Process and Results

General Information
Data Package:
Local Identifier:edi.1612.2
Title:California Wildfire Resilience Core Metrics Rating Process and Results
Alternate Identifier:DOI PLACE HOLDER
Abstract:

The California Wildfire & Forest Resilience Task Force (Task Force) is producing a toolkit that can support organizations to prioritize, plan and implement actions to lessen wildfire risk to communities and improve broader statewide ecosystem resilience. As part of this effort, a core set of metrics needed to be identified to report resilience progress. The Task Force's Science Advisory Panel engaged in a rapid Delphi process, collecting expert opinion via surveys to provide content knowledge and science support for this process. This data archive provides content 1) for transparency, to share as much of our workflow as is feasible; 2) for others to pull from as an example if they want to do a similar process; 3) to provide all the detailed results on metrics if a reader wants to look into the ratings for and definition of a particular metric. The archive supports a report and paper summarizing and describing our Delphi method and the results regarding the specific metrics considered by our experts.

Publication Date:2024-04-09
For more information:
Visit: DOI PLACE HOLDER

Time Period
Begin:
2023-10-16
End:
2023-12-03

People and Organizations
Contact:Eitzel, M.V. (University of California, Berkeley) [  email ]
Creator:Eitzel, M. V. (University of California, Berkeley)
Creator:Battles, John (University of California, Berkeley)
Creator:Smith, Jennifer (U.S. Department of Agriculture California Climate Hub)
Creator:Ostoja, Steve (U.S. Department of Agriculture California Climate Hub)

Data Entities
Data Table Name:
SummarizedMetricRanks_ForEDI_2024Mar31
Description:
Table of candidate data layers to be core reporting metrics for fire, ecological, and social resilience in California. Each row is a metric, with a topic assigned by Task Force Science Advisory Panel Staff in order to group the metrics for convenience in viewing. We have indicated if it is an existing metric in the Regional Resource Kits, or whether it is a novel, proposed metric. Survey respondents gave ratings for each metric for three resilience goals (reducing Wildfire Risk, improving Ecological Integrity, supporting Social, Economic, and Cultural Wellbeing), and for three attributes (how Feasible it was to remeasure, how Understandable it was to a broader audience, and how Sensitive it was to the process of interest). We averaged these values to produce the rating in the table. Respondents also indicated whether they felt the metric was relevant statewide throughout California, and/or for any of the specific regions (Sierra Nevada - SN, Central California - CC, Northern California - NC, Southern California - SC) - we added up the number of respondents who thought the metric was relevant for each region and statewide and that is the value in the table. We also calculated a composite score for the goals and attributes, where we asked the respondents to rate each goal in the abstract and averaged them across respondents, then used those weightings to combine the three goals (and similarly for the attributes). We also list the number of ratings for each goal for each metric (NumRatings).
Other Name:
Core_Metrics_-_Phase_1_16Oct2023
Description:
Qualtrics proprietary format containing the Round 1 survey questions, design, and logical flow. Compatible with Qualtrics format as of March 16, 2024.
Other Name:
CoreMetrics-Phase2_12Nov2023
Description:
Qualtrics proprietary format containing the Round 2 survey questions, design, and logical flow. Compatible with Qualtrics format as of March 16, 2024.
Other Name:
merged_MetricDictionaries
Description:
Dictionaries describing the methods behind and provenance of data layers in the Regional Resource Kits. The four regions (Sierra, Central California, Northern California, and Southern California) each have their own metric dictionaries available on the Task Force website as of March 2024; this pdf contains all four and can be searched simultaneously. Note that many metrics are included in more than one region, and therefore their definitions will be repeated in this document. This pdf is intended for those who want details on any of the already-existing metrics considered in our core metrics surveys. For descriptions of proposed but not yet existing metrics that were considered in Round 2, see the "Metrics Cheat Sheet" document.
Other Name:
Core_Metrics_-_Phase_1_16Oct2023_SurveyQuestions
Description:
Text for Round 1 of our expert opinion surveys on core metrics. Includes language for questions and list of metrics considered.
Other Name:
CoreMetrics-Phase2_12Nov2023_SurveyQuestions
Description:
Text for Round 2 of our expert opinion surveys on core metrics. Includes language for questions and list of metrics considered.
Other Name:
Metrics_CheatSheet
Description:
Metric cheat sheet with brief definitions of existing metrics and descriptions of proposed metrics, used in round 2 of the expert opinion surveys.
Other Name:
IndividualMetricsPhase1Report
Description:
Document showing the detailed results for each metric from the Round 1 surveys collecting expert opinion on whether existing Regional Resource Kit metrics would be good candidates for core reporting metrics.
Other Name:
AllPhase2MetricsDetails
Description:
Document showing the detailed results for each metric from the Round 2 surveys collecting expert opinion on whether existing Regional Resource Kit metrics and/or proposed novel metrics would be good candidates for core reporting metrics.
Detailed Metadata

Data Entities

Data Table

Data:https://pasta-s.lternet.edu/package/data/eml/edi/1612/2/7913e47152f5c82eb4b7b5d44b066b62
Name:SummarizedMetricRanks_ForEDI_2024Mar31
Description:Table of candidate data layers to be core reporting metrics for fire, ecological, and social resilience in California. Each row is a metric, with a topic assigned by Task Force Science Advisory Panel Staff in order to group the metrics for convenience in viewing. We have indicated if it is an existing metric in the Regional Resource Kits, or whether it is a novel, proposed metric. Survey respondents gave ratings for each metric for three resilience goals (reducing Wildfire Risk, improving Ecological Integrity, supporting Social, Economic, and Cultural Wellbeing), and for three attributes (how Feasible it was to remeasure, how Understandable it was to a broader audience, and how Sensitive it was to the process of interest). We averaged these values to produce the rating in the table. Respondents also indicated whether they felt the metric was relevant statewide throughout California, and/or for any of the specific regions (Sierra Nevada - SN, Central California - CC, Northern California - NC, Southern California - SC) - we added up the number of respondents who thought the metric was relevant for each region and statewide and that is the value in the table. We also calculated a composite score for the goals and attributes, where we asked the respondents to rate each goal in the abstract and averaged them across respondents, then used those weightings to combine the three goals (and similarly for the attributes). We also list the number of ratings for each goal for each metric (NumRatings).
Number of Records:115
Number of Columns:19

Table Structure
Object Name:SummarizedMetricRanks_ForEDI_2024Mar31.csv
Size:21018 byte
Authentication:264bb3cf0e4b2e971f4a774466a8c435 Calculated By MD5
Text Format:
Number of Header Lines:1
Record Delimiter:\n
Orientation:column
Simple Delimited:
Field Delimiter:,
Quote Character:"

Table Column Descriptions
 MetricNameTopicNovelMetricWildfireRiskEcologicalIntegrityWellbeingFeasibilityUnderstandabilitySensitivityNumRatings_WRNumRatings_EINumRatings_WBStatewideSNCCSCNCCompositeScoreCompositeAttributes
Column Name:MetricName  
Topic  
NovelMetric  
WildfireRisk  
EcologicalIntegrity  
Wellbeing  
Feasibility  
Understandability  
Sensitivity  
NumRatings_WR  
NumRatings_EI  
NumRatings_WB  
Statewide  
SN  
CC  
SC  
NC  
CompositeScore  
CompositeAttributes  
Definition:Name of metric as given in survey. Importing into R has caused spaces and other characters to be converted to periods.The topic assigned by Task Force Science Advisory Panel Staff in order to group the metrics for convenience in viewingIs the metric a novel, proposed metric ("Yes") or is it already an existing metric in the Regional Resource Kits ("No").Average rating from respondents as to how well this metric could track reduction in wildfire riskAverage rating from respondents as to how well this metric could track improvements in ecological integrityAverage rating from respondents as to how well this metric could track support of cultural, social, and economic wellbeingAverage rating from respondents as to how well this metric could be repeatible measuredAverage rating from respondents as to how well this metric could be understood by wide audiencesAverage rating from respondents as to how well this metric responds to the process of interestNumber of ratings for Wildfire Risk goalNumber of ratings for Ecological Integrity goalNumber of ratings for Wellbeing goalNumber of survey respondents who felt the metric was applicable statewide throughout CaliforniaNumber of survey respondents who felt the metric was applicable to the Sierra Nevada regionNumber of survey respondents who felt the metric was applicable to the Central California regionNumber of survey respondents who felt the metric was applicable to the Southern California regionNumber of survey respondents who felt the metric was applicable to the Northern California regionRespondents rated each resilience goal (wildfire risk, ecological integrity, wellbeing) in general from 1-5, and we averaged their ratings, then used those scores to combine the scores for the three goals for a given metric in a weighted sum.Respondents rated each metric attribute (feasibility, understandability, sensitivity) in general from 1-5, and we averaged their ratings, then used those scores to combine the three attribute scores for a given metric in a weighted sum.
Storage Type:string  
string  
string  
float  
float  
float  
float  
float  
float  
float  
float  
float  
float  
float  
float  
float  
float  
float  
float  
Measurement Type:nominalnominalnominalratioratioratioratioratioratioratioratioratioratioratioratioratioratioratioratio
Measurement Values Domain:
Definitiontext
Definitiontext
Allowed Values and Definitions
Enumerated Domain 
Code Definition
CodeNo
DefinitionThe metric is not novel and is already an existing metric in the Regional Resource Kits ("No").
Source
Code Definition
CodeYes
DefinitionThe metric is a novel, proposed metric ("Yes") not already in the Regional Resource Kits
Source
UnitUnitless
Typereal
Min1 
Max5 
UnitUnitless
Typereal
Min1 
Max5 
UnitUnitless
Typereal
Min1 
Max5 
UnitUnitless
Typereal
Min1 
Max5 
UnitUnitless
Typereal
Min1 
Max5 
UnitUnitless
Typereal
Min1 
Max5 
UnitUnitless
Typeinteger
UnitUnitless
Typeinteger
UnitUnitless
Typeinteger
UnitUnitless
Typeinteger
UnitUnitless
Typeinteger
UnitUnitless
Typeinteger
UnitUnitless
Typeinteger
UnitUnitless
Typeinteger
UnitComposite Score
Typereal
UnitComposite Score
Typereal
Missing Value Code:                                      
Accuracy Report:                                      
Accuracy Assessment:                                      
Coverage:                                      
Methods:                                      

Non-Categorized Data Resource

Name:Core_Metrics_-_Phase_1_16Oct2023
Entity Type:.qsf
Description:Qualtrics proprietary format containing the Round 1 survey questions, design, and logical flow. Compatible with Qualtrics format as of March 16, 2024.
Physical Structure Description:
Object Name:Core_Metrics_-_Phase_1_16Oct2023.qsf
Size:195868 byte
Authentication:6c9169238a5e0b8e7acb79f84991e6d8 Calculated By MD5
Externally Defined Format:
Format Name:.qsf
Data:https://pasta-s.lternet.edu/package/data/eml/edi/1612/2/e01f62cd4610535be0535f213ad0dd9f

Non-Categorized Data Resource

Name:CoreMetrics-Phase2_12Nov2023
Entity Type:.qsf
Description:Qualtrics proprietary format containing the Round 2 survey questions, design, and logical flow. Compatible with Qualtrics format as of March 16, 2024.
Physical Structure Description:
Object Name:CoreMetrics-Phase2_12Nov2023.qsf
Size:768108 byte
Authentication:939258d3c591e0cf457933e1c424a93a Calculated By MD5
Externally Defined Format:
Format Name:.qsf
Data:https://pasta-s.lternet.edu/package/data/eml/edi/1612/2/8381d69c8d3fb78eaacec45c13f9cdf1

Non-Categorized Data Resource

Name:merged_MetricDictionaries
Entity Type:application/pdf
Description:Dictionaries describing the methods behind and provenance of data layers in the Regional Resource Kits. The four regions (Sierra, Central California, Northern California, and Southern California) each have their own metric dictionaries available on the Task Force website as of March 2024; this pdf contains all four and can be searched simultaneously. Note that many metrics are included in more than one region, and therefore their definitions will be repeated in this document. This pdf is intended for those who want details on any of the already-existing metrics considered in our core metrics surveys. For descriptions of proposed but not yet existing metrics that were considered in Round 2, see the "Metrics Cheat Sheet" document.
Physical Structure Description:
Object Name:All-Metric-Dictionaries.pdf
Size:4284309 byte
Authentication:89dbc08cbafe65201fb34853cd67f9c5 Calculated By MD5
Externally Defined Format:
Format Name:application/pdf
Data:https://pasta-s.lternet.edu/package/data/eml/edi/1612/2/03f3f571e0a1d076f7484ba9691738d9

Non-Categorized Data Resource

Name:Core_Metrics_-_Phase_1_16Oct2023_SurveyQuestions
Entity Type:application/pdf
Description:Text for Round 1 of our expert opinion surveys on core metrics. Includes language for questions and list of metrics considered.
Physical Structure Description:
Object Name:Core_Metrics_-_Phase_1_16Oct2023_SurveyQuestions.pdf
Size:56485 byte
Authentication:fa62a242b911869d28bf48cfc44320f8 Calculated By MD5
Externally Defined Format:
Format Name:application/pdf
Data:https://pasta-s.lternet.edu/package/data/eml/edi/1612/2/0321977843363bca6cd0b51760f569e2

Non-Categorized Data Resource

Name:CoreMetrics-Phase2_12Nov2023_SurveyQuestions
Entity Type:application/pdf
Description:Text for Round 2 of our expert opinion surveys on core metrics. Includes language for questions and list of metrics considered.
Physical Structure Description:
Object Name:CoreMetrics-Phase2_12Nov2023_SurveyQuestions.pdf
Size:87060 byte
Authentication:1c27b05e011e4275b1fa1d804ebb1739 Calculated By MD5
Externally Defined Format:
Format Name:application/pdf
Data:https://pasta-s.lternet.edu/package/data/eml/edi/1612/2/f68af61ec9c32ee6f712968b14f225e2

Non-Categorized Data Resource

Name:Metrics_CheatSheet
Entity Type:application/pdf
Description:Metric cheat sheet with brief definitions of existing metrics and descriptions of proposed metrics, used in round 2 of the expert opinion surveys.
Physical Structure Description:
Object Name:Metrics_CheatSheet.pdf
Size:169438 byte
Authentication:4e511defedb18016471d0906ea0c178b Calculated By MD5
Externally Defined Format:
Format Name:application/pdf
Data:https://pasta-s.lternet.edu/package/data/eml/edi/1612/2/3f78ff2818306c6349b8b6a565d0716f

Non-Categorized Data Resource

Name:IndividualMetricsPhase1Report
Entity Type:application/pdf
Description:Document showing the detailed results for each metric from the Round 1 surveys collecting expert opinion on whether existing Regional Resource Kit metrics would be good candidates for core reporting metrics.
Physical Structure Description:
Object Name:IndividualMetricsPhase1Report.pdf
Size:215170 byte
Authentication:447bc5fccaac9a896d852e0ebac1a242 Calculated By MD5
Externally Defined Format:
Format Name:application/pdf
Data:https://pasta-s.lternet.edu/package/data/eml/edi/1612/2/76e5416416ca6553ef12d065a5683f35

Non-Categorized Data Resource

Name:AllPhase2MetricsDetails
Entity Type:application/pdf
Description:Document showing the detailed results for each metric from the Round 2 surveys collecting expert opinion on whether existing Regional Resource Kit metrics and/or proposed novel metrics would be good candidates for core reporting metrics.
Physical Structure Description:
Object Name:AllPhase2MetricsDetails.pdf
Size:370638 byte
Authentication:bd13b23bcba1a3718ee88d28a9c913f1 Calculated By MD5
Externally Defined Format:
Format Name:application/pdf
Data:https://pasta-s.lternet.edu/package/data/eml/edi/1612/2/5fd64d6357d2f2467f373c4232b4f6f1

Data Package Usage Rights

This information is released under the Creative Commons license - Attribution - CC BY (https://creativecommons.org/licenses/by/4.0/). The consumer of these data ("Data User" herein) is required to cite it appropriately in any publication that results from its use. The Data User should realize that these data may be actively used by others for ongoing research and that coordination may be necessary to prevent duplicate publication. The Data User is urged to contact the authors of these data if any questions about methodology or results occur. Where appropriate, the Data User is encouraged to consider collaboration or co-authorship with the authors. The Data User should realize that misinterpretation of data may occur if used out of context of the original study. While substantial efforts are made to ensure the accuracy of data and associated documentation, complete accuracy of data sets cannot be guaranteed. All data are made available "as is." The Data User should be aware, however, that data are updated periodically and it is the responsibility of the Data User to check for new versions of the data. The data authors and the repository where these data were obtained shall not be liable for damages resulting from any use or misinterpretation of the data. Thank you.

Keywords

By Thesaurus:
(No thesaurus)Forest, Shrubland, Expert opinion, Core reporting metrics, Ecological integrity, Resilient social-ecological systems, California, Delphi process

Methods and Protocols

These methods, instrumentation and/or protocols apply to all data in this dataset:

Methods and protocols used in the collection of this data package
Description:

At the request of the California Forest and Wildfire Resilience Task Force leadership, the Science Advisory Panel (SAP) developed an approach to provide timely science support for their effort to identify core metrics. We used a structured process of science review characterized by sequential rounds of surveys with questions defined by policy-led information needs. This package includes the survey language, Qualtrics files, and detailed descriptions of candidate metrics. A related report describes the process of implementing the surveys in more detail.

Round 1 of the surveys asked respondents to discuss criteria for core metrics, eliminate metrics from the existing Regional Resource Kits (RRKs) that were clearly inappropriate and/or inadequate based on those criteria, and recommend additional metrics that may have been overlooked. The Round 1 survey revealed that selecting a good set of core metrics depended on their intended use (for example, for planning versus reporting), audience (for example policy makers, scientists, or the public), and resilience goals (for example, immediate wildfire risk reduction versus long-term ecological health). This package shares the metrics suggested from Round 1 and detailed responses regarding each existing metric from the RRKs.

Round 2 of the surveys grouped metrics by topic and asked respondents to rate each metric on how well it measured each of three broad, overlapping resilience goals identified by the Task Force: Reducing wildfire risk, improving ecological integrity, and supporting social and cultural wellbeing. Ratings ranged from 1 (lowest) to 5 (highest). Respondents also evaluated three attributes for each metric: how realistic it was to remeasure (Feasibility), how easy it was to explain to wide audiences (Understandability), and how well it represented the process of interest (Sensitivity). In addition, respondents identified relevant region(s) of California (Sierra Nevada, Southern California, Central Coast, and Northern California) for each metric. In this package, we share a pdf with the detailed responses on each metric considered in Round 2.

We averaged the ratings for each metric among the different respondents' ratings, and report these in the included data table. We also include the number of ratings for each metric, the number of respondents who felt the metric was relevant to each region, and included a composite score which used the average respondent rating for the overall goals (e.g. on average respondents rated Wildfire Risk in general as 4.5 out of 5 but Ecological Integrity in general 3.5 out of 5) to weight the average ratings for those three goals for an individual metric.

People and Organizations

Publishers:
Organization:Environmental Data Initiative
Email Address:
info@edirepository.org
Web Address:
https://edirepository.org
Id:https://ror.org/0330j0z60
Creators:
Individual: M. V. Eitzel
Organization:University of California, Berkeley
Email Address:
mveitzel@berkeley.edu
Id:https://orcid.org/0000-0002-3877-1551
Individual: John Battles
Organization:University of California, Berkeley
Email Address:
jbattles@berkeley.edu
Individual: Jennifer Smith
Organization:U.S. Department of Agriculture California Climate Hub
Email Address:
jatsmith@ucdavis.edu
Individual: Steve Ostoja
Organization:U.S. Department of Agriculture California Climate Hub
Email Address:
smostoja@ucdavis.edu
Contacts:
Individual: M.V. Eitzel
Organization:University of California, Berkeley
Email Address:
mveitzel@berkeley.edu
Id:https://orcid.org/0000-0002-3877-1551

Temporal, Geographic and Taxonomic Coverage

Temporal, Geographic and/or Taxonomic information that applies to all data in this dataset:

Time Period
Begin:
2023-10-16
End:
2023-12-03
Geographic Region:
Description:The archive includes descriptions of metrics that have been calculated throughout the political boundaries of the state of California, United States of America. Bounding coordinates include areas not inside California.
Bounding Coordinates:
Northern:  42.0Southern:  32.55
Western:  -124.406167Eastern:  -114.52

Project

Parent Project Information:

Title:California Wildfire Resilience Core Metrics Rating Process
Personnel:
Individual: M.V. Eitzel
Organization:University of California, Berkeley
Email Address:
mveitzel@berkeley.edu
Id:https://orcid.org/0000-0002-3877-1551
Role:Project Leader

Maintenance

Maintenance:
Description:

There is no ongoing data collection nor do we expect changes to the dataset.

Frequency:
Other Metadata

Additional Metadata

additionalMetadata
        |___text '\n      '
        |___element 'metadata'
        |     |___text '\n         '
        |     |___element 'unitList'
        |     |     |___text '\n            '
        |     |     |___element 'unit'
        |     |     |     |  \___attribute 'id' = 'Unitless'
        |     |     |     |  \___attribute 'name' = 'Unitless'
        |     |     |     |___text '\n               '
        |     |     |     |___element 'description'
        |     |     |     |     |___text 'Count of survey responses'
        |     |     |     |___text '\n            '
        |     |     |___text '\n            '
        |     |     |___element 'unit'
        |     |     |     |  \___attribute 'id' = 'Composite Score'
        |     |     |     |  \___attribute 'name' = 'Composite Score'
        |     |     |     |___text '\n               '
        |     |     |     |___element 'description'
        |     |     |     |     |___text 'three average ratings from 1-5, each multiplied by an average rating from 1-5.'
        |     |     |     |___text '\n            '
        |     |     |___text '\n         '
        |     |___text '\n      '
        |___text '\n   '

Additional Metadata

additionalMetadata
        |___text '\n      '
        |___element 'metadata'
        |     |___text '\n         '
        |     |___element 'emlEditor'
        |     |        \___attribute 'app' = 'ezEML'
        |     |        \___attribute 'release' = '2024.03.27'
        |     |___text '\n      '
        |___text '\n   '

Copyright 2024 Environmental Data Initiative. This material is based upon work supported by the National Science Foundation under grants #2223103 and #2223104. Any opinions, findings, conclusions, or recommendations expressed in the material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Please contact us with questions, comments, or for technical assistance regarding this web site or the Environmental Data Initiative. Please read our privacy policy to know what information we collect about you and to understand your privacy rights.

EDI is a collaboration between the University of New Mexico and the University of Wisconsin – Madison, Center for Limnology:

UNM logo UW-M logo