Data Package Metadata View Summary

Fish abundance in the San Francisco Estuary (1959-2024), an integration of 10 monitoring surveys.

General Information

Data Package:

Local Identifier:

edi.1118.2

Title:

Fish abundance in the San Francisco Estuary (1959-2024), an integration of 10 monitoring surveys.

Alternate Identifier:

DOI PLACE HOLDER

Abstract:

The San Francisco Estuary (SFE) is simultaneously a central hub of water delivery in California and home to commercially important and endangered fishes, such as Chinook Salmon, Green Sturgeon, and Delta and Longfin Smelt. Extensive ecological monitoring has been conducted for over 50 years, mainly under the auspices of the Interagency Ecological Program for the San Francisco Estuary (https://iep.ca.gov/). We integrated fish catch and length data from 10 long-term monitoring surveys in the SFE. The integrated database contains survey-level data such as environmental variables and sampling effort in addition to the fish-level species, lengths, and counts. Zero catches have been filled in for any species not caught in a sample. The geographic scope includes San Francisco Bay through the upper estuary, and the timeseries spans 1959 to 2024. Sampling methods, gear, fish length metric, and other factors differ among the component surveys. Sampling designs (locations and temporal frequency) have also changed over time. Thus, it is highly recommended to inspect the documentation of the component surveys for more information on their methods.

Publication Date:

2024-08-01

For more information:
Visit:	DOI PLACE HOLDER

Time Period

Begin:

1959-06-13

End:

2024-07-28

People and Organizations
Contact:	Bashevkin, Samuel M (State Water Board) [ email ]
Creator:	Bashevkin, Samuel M (State Water Board)
Creator:	Nguyen, Trinh X (California Department of Fish and Wildlife and the Interagency Ecological Program)
Creator:	Tham, Enoch J (CDFW Water Branch)
Creator:	Gaeta, Jereme W (California Department of Fish and Wildlife and the Interagency Ecological Program)
Creator:	Mitchell, Lara (U.S. Fish and Wildlife Service)
Creator:	Khanna, Shruti (California Department of Fish and Wildlife and the Interagency Ecological Program)

Data Entities
Data Table Name:	Sample-level table
Description:	Sample-level environmental and effort data. Can be joined to the fish table with the SampleID column.
Data Table Name:	Fish-level data
Description:	Fish-level length and abundance data. Can be joined to the survey table with the SampleID column.
Data Table Name:	Length conversion equations
Description:	Length conversion equations for 20 fishes, of the form fork or total length = intercept + slope * standard length.
Other Name:	Compressed fish and survey data.
Description:	The fish and survey tables compressed into an rda file (compressed file format for the R programming language). These data are exactly identical to their csv analogs. They are provided to reduce download time for R-users. This file can be read into R using the "load" function.

Detailed Metadata

Data Entities

Data Table


Data:	https://pasta-s.lternet.edu/package/data/eml/edi/1118/2/79240c490fe74543da6b86a1c7c751b9
Name:	Sample-level table
Description:	Sample-level environmental and effort data. Can be joined to the fish table with the SampleID column.
Number of Records:	723900
Number of Columns:	25

Table Structure

Object Name:

survey.csv

Size:

160784114 bytes

Authentication:

9655493ce310f0704cf673591256645c Calculated By MD5

Text Format:

Number of Header Lines:

Record Delimiter:

\r\n

Orientation:

column

Simple Delimited:

Field Delimiter:	,
Quote Character:	"

Table Column Descriptions

Column Name:

Source

Station

Latitude

Longitude

Date

Datetime

Survey

Depth

SampleID

Method

Tide

Sal_surf

Sal_bot

Temp_surf

TurbidityNTU

TurbidityFNU

Secchi

Secchi_estimated

Tow_duration

Tow_area

Tow_volume

Cable_length

Tow_direction

Notes_tow

Notes_flowmeter

Definition:

Name of the source survey.

Unique name of each sampling location

Latitude of the sampling location

Longitude of the sampling location

Date sample was collected

Date and time sample was collected

Cumulative survey number in the source dataset

Bottom depth. Sometimes this is recorded at the time of the tow, and sometimes it's an average depth for the sampling location.

Unique identifier for each sample (i.e. tow). This is the key to join the fish and sample tables together.

Gear type

Tidal stage

Surface salinity

Bottom salinity

Surface temperature

Surface water turbidity in NTU

Surface water turbidity in FNU

Secchi depth

Was Secchi depth estimated? This is only used for FMWT data.

Duration of tow. For Salvage this is the duration of sampling.

Area sampled by tow

Volume sampled by tow. For Salvage, this value represents the daily export volume for the sampled facility.

Length of cable released when net deployed

Direction of tow, this has not been standardized among surveys.

Notes recorded about the tow

Notes recorded about the flowmeter used to calculate tow_area or tow_volume

Storage Type:

string

float

date

float

string

float

string

float

string

Measurement Type:

nominal

ratio

dateTime

ratio

nominal

ratio

nominal

ratio

nominal

Measurement Values Domain:

Allowed Values and Definitions

Enumerated Domain

Code Definition

Code	20mm
Definition	CDFW 20mm Survey
Source

Code Definition

Code	Bay Study
Definition	CDFW Bay Study
Source

Code Definition

Code	DJFMP
Definition	USFWS Delta Juvenile Fish Monitoring Program
Source

Code Definition

Code	EDSM
Definition	USFWS Enhanced Delta Smelt Monitoring
Source

Code Definition

Code	FMWT
Definition	CDFW Fall Midwater Trawl
Source

Code Definition

Code	SKT
Definition	CDFW Spring Kodiak Trawl
Source

Code Definition

Code	SLS
Definition	CDFW Smelt Larva Survey
Source

Code Definition

Code	STN
Definition	CDFW Summer Townet
Source

Code Definition

Code	Suisun
Definition	UC Davis Suisun Marsh Fish Study
Source

Code Definition

Code	Salvage
Definition	California Department of Fish and Wildlife Fish Salvage database
Source

Definition

Unique name of each sampling location

Unit	degree
Type	real
Min	37.50185
Max	39.21999207

Unit	degree
Type	real
Min	-122.4984841
Max	-121.00969

Format	YYYY-MM-DD
Precision

Format	YYYY-MM-DD hh:mm:ss
Precision

Unit	dimensionless
Type	natural
Min	1
Max	61

Unit	meter
Type	real
Min	0
Max	95.7072

Definition

Unique identifier for each sample (i.e. tow). This is the key to join the fish and sample tables together.

Allowed Values and Definitions

Enumerated Domain

Code Definition

Code	20mm net
Definition	20mm net
Source

Code Definition

Code	Beach seine
Definition	Beach seine
Source

Code Definition

Code	Kodiak trawl
Definition	Kodiak trawl
Source

Code Definition

Code	Midwater trawl
Definition	Midwater trawl
Source

Code Definition

Code	Oblique tow
Definition	Oblique tow
Source

Code Definition

Code	Otter trawl
Definition	Otter trawl
Source

Code Definition

Code	STN net
Definition	Summer townet
Source

Code Definition

Code	Normal count
Definition	Salvage sampling period code 0000
Source

Code Definition

Code	Second flush
Definition	Salvage sampling period code 9999
Source

Allowed Values and Definitions

Enumerated Domain

Code Definition

Code	Ebb
Definition	Ebb tide
Source

Code Definition

Code	Flood
Definition	Flood tide
Source

Code Definition

Code	High Slack
Definition	High slack tide
Source

Code Definition

Code	Low Slack
Definition	Low slack tide
Source

Unit	dimensionless
Type	real
Min	0
Max	37.4763743012389

Unit	dimensionless
Type	real
Min	0.031814021671517
Max	31.4059437704999

Unit	celsius
Type	real
Min	0
Max	32.2222222222222

Unit	nephelometricTurbidityUnit
Type	real
Min	0
Max	1000

Unit	formazinNephelometricUnit
Type	whole
Min	0
Max	174

Unit	centimeter
Type	real
Min	0
Max	692.999982833862

Allowed Values and Definitions

Enumerated Domain

Code Definition

Code	TRUE
Definition	The Secchi value is estimated
Source

Code Definition

Code	FALSE
Definition	The Secchi value is not estimated
Source

Unit	minute
Type	real
Min	1
Max	26

Unit	squareMeter
Type	real
Min	63.3383985842764
Max	5217.33333333333

Unit	meterCubed
Type	real
Min	0
Max	25968454.44

Unit	meter
Type	real
Min	0
Max	68.58

Definition

Direction of tow, this has not been standardized among surveys.

Definition

Notes recorded about the tow

Definition

Notes recorded about the flowmeter used to calculate tow_area or tow_volume

Missing Value Code:

Code	NA
Expl	No coordinates available

Code	NA
Expl	No coordinates available

Code	NA
Expl	Not recorded

Code	NA
Expl	The source dataset does not record survey numbers

Code	NA
Expl	Not recorded

Code	NA
Expl	Not recorded

Code	NA
Expl	Not recorded

Code	NA
Expl	Not recorded

Code	NA
Expl	Not recorded

Code	NA
Expl	Not recorded

Code	NA
Expl	Not recorded

Code	NA
Expl	Not recorded

Code	NA
Expl	Not recorded (only used for FMWT data)

Code	NA
Expl	Not recorded

Code	NA
Expl	Not recorded or this effort metric not used for this gear type

Code	NA
Expl	Not recorded or this effort metric not used for this gear type

Code	NA
Expl	Not recorded

Code	NA
Expl	Not recorded

Code	NA
Expl	Not recorded

Code	NA
Expl	Not recorded

Accuracy Report:

Accuracy Assessment:

Coverage:

Methods:

Data Table


Data:	https://pasta-s.lternet.edu/package/data/eml/edi/1118/2/5429d3e82b1671e7454c7b5d7a15c6ef
Name:	Fish-level data
Description:	Fish-level length and abundance data. Can be joined to the survey table with the SampleID column.
Number of Records:	77641452
Number of Columns:	5

Table Structure

Object Name:

fish.csv

Size:

5074160193 bytes

Authentication:

1e2147b076b71a56dbf453019e0c0f58 Calculated By MD5

Text Format:

Number of Header Lines:

Record Delimiter:

\r\n

Orientation:

column

Simple Delimited:

Field Delimiter:	,
Quote Character:	"

Table Column Descriptions

Column Name:

SampleID

Taxa

Length

Count

Notes_catch

Definition:

Unique identifier for each sample (i.e. tow). This is the key to join the fish and sample tables together.

Scientific name for the sampled taxa

Fish length in fork length or total length for species with no fork for all studies except Suisun study, for which lengths are recorded as standard length. See the Length_conversion table for equations to convert standard length to fork/total length.

Adjusted length frequency (see methods) of fish for each sample, taxa, and measured length

Notes recorded about the fish catch

Storage Type:

string

float

string

Measurement Type:

nominal

ratio

nominal

Measurement Values Domain:

Definition

Unique identifier for each sample (i.e. tow). This is the key to join the fish and sample tables together.

Definition

Scientific name for the sampled taxa

Unit	millimeter
Type	real
Min	1
Max	1945

Unit	number
Type	real
Min	0
Max	7054248

Definition

Notes recorded about the fish catch

Missing Value Code:

Code	NA
Expl	The length is unknown

Accuracy Report:

Accuracy Assessment:

Coverage:

Methods:

Data Table


Data:	https://pasta-s.lternet.edu/package/data/eml/edi/1118/2/2933237df1902243b4307f082bdc7d18
Name:	Length conversion equations
Description:	Length conversion equations for 20 fishes, of the form fork or total length = intercept + slope * standard length.
Number of Records:	20
Number of Columns:	3

Table Structure

Object Name:

Length_conversions.csv

Size:

745 bytes

Authentication:

ce31df715ae888e4610565139b57add5 Calculated By MD5

Text Format:

Number of Header Lines:

Record Delimiter:

\r\n

Orientation:

column

Simple Delimited:

Field Delimiter:	,
Quote Character:	"

Table Column Descriptions

Column Name:

Species

Intercept

Slope

Definition:

Species name

Intercept of the equation: fork or total length = intercept + slope * standard length

Storage Type:

string

float

Measurement Type:

nominal

ratio

Measurement Values Domain:

Definition

Species name

Unit	millimeter
Type	real
Min	-1.166
Max	3.362

Unit	dimensionless
Type	real
Min	0.926
Max	1.24

Missing Value Code:

Accuracy Report:

Accuracy Assessment:

Coverage:

Methods:

Non-Categorized Data Resource

Name:

Compressed fish and survey data.

Entity Type:

unknown

Description:

The fish and survey tables compressed into an rda file (compressed file format for the R programming language). These data are exactly identical to their csv analogs. They are provided to reduce download time for R-users. This file can be read into R using the "load" function.

Physical Structure Description:

Object Name:

fishsurvey_compressed.rda

Size:

31270252 bytes

Authentication:

14b03f9bd9e721619f3bb41cd93ae173 Calculated By MD5

Externally Defined Format:

Format Name:

Unknown

Data:

https://pasta-s.lternet.edu/package/data/eml/edi/1118/2/926f4aa8484f185b69bc1827fa67d40c

Data Package Usage Rights

This information is released under the Creative Commons license - Attribution - CC BY (https://creativecommons.org/licenses/by/4.0/). The consumer of these data ("Data User" herein) is required to cite it appropriately in any publication that results from its use. The Data User should realize that these data may be actively used by others for ongoing research and that coordination may be necessary to prevent duplicate publication. The Data User is urged to contact the authors of these data if any questions about methodology or results occur. Where appropriate, the Data User is encouraged to consider collaboration or co-authorship with the authors. The Data User should realize that misinterpretation of data may occur if used out of context of the original study. While substantial efforts are made to ensure the accuracy of data and associated documentation, complete accuracy of data sets cannot be guaranteed. All data are made available "as is." The Data User should be aware, however, that data are updated periodically and it is the responsibility of the Data User to check for new versions of the data. The data authors and the repository where these data were obtained shall not be liable for damages resulting from any use or misinterpretation of the data. Thank you.

Keywords

By Thesaurus:
https://vocab.lternet.edu/	estuaries, fishes, species abundance, community composition, long term monitoring, endangered species, invasive species

Methods and Protocols

These methods, instrumentation and/or protocols apply to all data in this dataset:

Methods and protocols used in the collection of this data package

Description:

1. Data integration

Data integration was performed in the statistical programming language R (R Core Team 2024) using the ‘tidyverse’ family of packages (Wickham et al. 2019) and packaged into the R package LTMRdata v2.1.0 (Bashevkin et al. 2024). The latest version of the R package is available at https://github.com/sbashevkin/LTMRdata.

Source datasets were obtained either in flat format from data publications or directly from the Access databases. When pulled from Access databases, the individual relational data tables were joined together following the relationships diagram in their respective Access databases.

To ensure consistency across datasets, columns and categorical data (e.g., species names) were renamed and units converted. All time data were converted to Pacific Time (PST/PDT). In the Salvage dataset, daily sampling occurs throughout the 24-hour period and due to the manual recording of sampling times, there are instances where time was recorded as 2 AM on the Spring daylight savings time day, a nonexistent time since 2 AM is skipped on these days. These entries were recoded to 3 AM, but that time may be incorrect and off by one hour. This fix also does not resolve additional erroneous time entries that may have been recorded before operators fully adjusted for daylight savings.

Data from the fish sampling gear(s) or sampling times (i.e., normal counts or secondary flushes in the Salvage database) of each survey were retained. Data that did not include a fish count and only a water quality sample were removed. In the Salvage dataset, these entries were identified using the unstandardized comments along with a zero-catch count.

To allow users to standardize catch across samples, we calculated metrics of effort for each sampling program and method. Sampling effort was quantified following the methods of the component surveys, as either tow area (for the otter trawls) or tow volume (for all other gear types). The tow volume values for Salvage represent the daily export volume for the sampled facility. Thus, although multiple salvage samples are collected each day, the tow volume value represents the daily effort. Some tow volumes values are 0 for Salvage and these represent days when no pumping was conducted but valid salvage counts occurred. For the Enhanced Delta Smelt Monitoring Program (EDSM) and Delta Juvenile Fish Monitoring Program (DJFMP) datasets, tow volume was set to NA when debris was detected in the flowmeter, and samples were excluded with gear condition codes 3 (poor sampling), 4 (no sample attempted), or 9 (fish captured outside of live box or cod end and could not be assigned to a specific tow).

Catch count in the Salvage dataset was calculated as salvage:

S = N (Tp/Tc),

where S = Salvage, N = the observed count of each species, Tp = the total time pumped (minutes), and Tc = the total time sampled (minutes; CDFW 2018).

Generally, surveys measure lengths for only a subset of the total catch of a given species from each sample in a process called subsampling. To estimate the frequency of each recorded length across all individuals caught in a sample, we calculated the adjusted length frequencies for each species in each sample following the methods of FMWT (James White, personal communication):

Al= C(𝐹l/M),

where Al = adjusted frequency of each recorded length, C = total catch, 𝐹l = measured frequency of each recorded length, and M = total number of fish measured. For instances when subsampling did not occur (e.g., species listed on the Endangered Species Act or the California Endangered Species Act, or a low catch that did not require subsampling), the calculated adjusted length frequency is equivalent to the recorded length frequency. Some surveys occasionally subsampled for measurement separately on different groups within a species (generally based on size classes, age, or Salmon races). In these cases, we calculated the adjusted frequencies separately for each group. In the Suisun Study database, these groups were identified with unstandardized comments describing the size or age class. When possible, these comments were converted into a fish size range that could then be assigned a group. However, not all comments could be translated into a fish size range, leaving some fish sizes unknown in the Suisun Study data. Fish sizes are occasionally unknown in many of the surveys when no individuals of a species were measured in a sample.

The datasets were then combined by simply binding them together by their now-standardized column names. The last step in data integration was to fill in zeros. The surveys did not commonly record when a species was not present in a trawl, but these absences are critically important for statistical analysis. For any samples in which a species was not recorded, we added a record of that species with a count of 0. Note that this resulted in 0s being added even for years in which a particular taxon was not sampled for (e.g., some surveys only started counting jellyfish in more recent years, but this will fill in 0s even before they were counted). Thus, users should consult the documentation of the component datasets for taxa that have not always been counted. However, zeros were filled separately for each survey, so zeros were only filled for taxa that had been recorded historically by that particular survey. This should ensure that zeros are only filled based on the taxa list (and historical observations) of each individual survey.

2. Fish lengths

While 9 of the surveys recorded fish lengths as fork length (or total length if no fork), the Suisun Study recorded standard length. We thus compiled conversion equations for 20 species to convert Suisun Study lengths (for those 20 species) to fork length (or total length if no fork). The equations are of the form:

FL = a + b × SL

where FL = fork length (or total length for species with no fork) in mm, a = intercept, b = slope, and SL = standard length in mm. The Splittail equation was calculated via a median quantile regression, while the other equations were obtained from the California Department of Fish and Wildlife Length-Weight Study (unpublished).

While the equations are provided, lengths are not converted in this dataset, so Suisun Study lengths are not comparable to the other surveys unless converted. Sampling programs also often discard (not count nor measure) fish below a size cutoff, but those cutoffs differ among surveys and over time. The integrated dataset does not account for these different cutoffs, so users should consult the documentation of their dataset of interest for these details.

3. Final format

Before publication, the joined dataset was split into separate tables for the fish-level data (taxa, length, count, etc.) and the sample-level data (environmental variables, sampling effort, etc.). The fish length conversion equations are published in a separate table.

The dataset can be downloaded and accessed via the R package deltafish (https://github.com/Delta-Stewardship-Council/deltafish). The package downloads the data from this EDI publication, processes it into a database, and provides tools to access that database within R.

4. Literature cited

Bashevkin, S. M., T. X. Nguyen, E. J. Tham, J. W. Gaeta, L. Mitchell, and S. Khana. 2024. LTMRdata: An R package to integrate data from 9 fish surveys in the San Francisco Estuary. v2.1.0. Zenodo. doi:10.5281/zenodo.6048977

[CDFW] California Department of Fish and Wildlife (2018). Chinook salmon loss estimation for Skinner Delta Fish Protective Facility and Tracy Fish Collection Facility. California Department of Fish and Wildlife.

R Core Team. 2024. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.

Wickham, H., M. Averick, J. Bryan, and others. 2019. Welcome to the Tidyverse. Journal of Open Source Software 4: 1686. doi:10.21105/joss.01686

Description:

This method step describes provenance-based metadata as specified in the LTER EML Best Practices.