iBet uBet web content aggregator. Adding the entire web to your favor.
iBet uBet web content aggregator. Adding the entire web to your favor.



Link to original content: https://pubmed.ncbi.nlm.nih.gov/26451728
Can Observation Skills of Citizen Scientists Be Estimated Using Species Accumulation Curves? - PubMed Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2015 Oct 9;10(10):e0139600.
doi: 10.1371/journal.pone.0139600. eCollection 2015.

Can Observation Skills of Citizen Scientists Be Estimated Using Species Accumulation Curves?

Steve Kelling  1 , Alison Johnston  2 , Wesley M Hochachka  1 , Marshall Iliff  1 , Daniel Fink  1 , Jeff Gerbracht  1 , Carl Lagoze  3 , Frank A La Sorte  1 , Travis Moore  4 , Andrea Wiggins  5 , Weng-Keen Wong  4 , Chris Wood  1 , Jun Yu  4
Affiliations

Can Observation Skills of Citizen Scientists Be Estimated Using Species Accumulation Curves?

Steve Kelling et al. PLoS One. .

Abstract

Volunteers are increasingly being recruited into citizen science projects to collect observations for scientific studies. An additional goal of these projects is to engage and educate these volunteers. Thus, there are few barriers to participation resulting in volunteer observers with varying ability to complete the project's tasks. To improve the quality of a citizen science project's outcomes it would be useful to account for inter-observer variation, and to assess the rarely tested presumption that participating in a citizen science projects results in volunteers becoming better observers. Here we present a method for indexing observer variability based on the data routinely submitted by observers participating in the citizen science project eBird, a broad-scale monitoring project in which observers collect and submit lists of the bird species observed while birding. Our method for indexing observer variability uses species accumulation curves, lines that describe how the total number of species reported increase with increasing time spent in collecting observations. We find that differences in species accumulation curves among observers equates to higher rates of species accumulation, particularly for harder-to-identify species, and reveals increased species accumulation rates with continued participation. We suggest that these properties of our analysis provide a measure of observer skill, and that the potential to derive post-hoc data-derived measurements of participant ability should be more widely explored by analysts of data from citizen science projects. We see the potential for inferential results from analyses of citizen science data to be improved by accounting for observer skill.

PubMed Disclaimer

Conflict of interest statement

Competing Interests: The authors have declared that no competing interests exist.

Figures

Fig 1
Fig 1. The Bird Conservation Regions (BCRs) of North America.
BCRs are ecologically distinct regions with similar bird communities. BCRs were used to cluster checklists into groups with similar likelihoods of species encounter. Data from six BCRs were used in this study: BCR 9- Great Basin, BCR 23- Prairie Hardwood Transition, BCR 30- New England/Mid-Atlantic Coast, BCR 31- Peninsular Florida, BCR 32- Coastal California, and BCR 37- Gulf Coastal Prairie. These BCRs were selected to include a range of eBird participation (Table 1) and varying degrees of diversity and patterns of bird occurrence. Map provided by the North American Bird Conservation Initiative (www.nabci-us.org).
Fig 2
Fig 2. Species accumulation curves for all individual observers in a BCR.
Each line represents the species accumulation curve derived from the mixed model fit to data from that BCR, for every individual observer in that BCR, and calculated with the standardized covariates of Sep 1st, 7am start time, travelling 1km and average percentage land cover. The fitted line for each observer is plotted to the maximum count period duration in the data from that observer. Species accumulation curves that decrease for some observers may indicate different biases in attention to birding. For example checklists under 1 hour may be more concentrated birding, whereas checklists over 1 hour may combine birding with another activity such as hiking or fishing.
Fig 3
Fig 3. Representative Individual Species Accumulation Curves and Indices.
Actual counts of species reported and observer-specific SACs for one individual observer classified as within the lowest quartile a), middle quartiles b), and highest quartile c). As the duration an observer spends collecting data for each checklist increases, the number of species observed increases. However as the duration lengthens, the rate of species accumulation decreases. The black dot on each curve is the SAC Index—the estimated number of species that individual would see during one hour of birding. Individuals were selected from the group who submitted at least 300 checklists between 2002 and 2012.
Fig 4
Fig 4. Distribution of individual SAC Indices.
The expected number of species observed in 1 hour for all observers in a BCR. Individual data submission scores are ranked from lowest to highest and the light gray region represents the lower quartile of observers, and the dark gray region the upper quartile of observers.
Fig 5
Fig 5. Comparison of detection rates of individual species of birds for observers in the lower quartile and the upper quartiles of SAC indices.
Detection rates are the proportions of checklists that record a given species and lines represent 95% bootstrap confidence intervals. The red line is a model fitted to the logit detection rates. The gray line indicates the line of equality, where detection rates for the two groups are equal; the highest quartile had statistically significant detection rates for the majority of species.
Fig 6
Fig 6. Bird species detected at the most similar and dis-similar rates by observers in the lowest quartile and highest quartile of SAC index values.
Barplots from BCR 23 of the 20 species for which detection rates are proportionally most similar (left-hand panels) and the 20 species for which detection rates are proportionally most different (right-hand panels). Detection rate is the proportion of checklists that record a given species and error bars represent 95% bootstrap confidence intervals. The 20 species for which the two groups have proportionally most similar detection rates are generally species that are fairly easy to identify by sight. The 20 species that the two groups have proportionally most different detection rates are generally species that are difficult to identify, easier to identify by sound, or often be seen as a high-flying silhouette without many distinguishing features.
Fig 7
Fig 7. Bird species detected at the most similar and dis-similar rates by observers in the lowest quartile and highest quartile of SAC index values.
Barplots from BCR 31 of the 20 species for which detection rates are proportionally most similar (left-hand panels) and the 20 species for which detection rates are proportionally most different (right-hand panels). Detection rate is the proportion of checklists that record a given species and error bars represent 95% bootstrap confidence intervals. The 20 species for which the two groups have proportionally most similar detection rates are generally species that are fairly easy to identify by sight. The 20 species that the two groups have proportionally most different detection rates are generally species that are difficult to identify, easier to identify by sound, or often be seen as a high-flying silhouette without many distinguishing features.
Fig 8
Fig 8. The change in SACs as a function of the cumulative participation in eBird.
We estimated average changes in shapes of species accumulation curves with increasing number of checklists submitted to eBird from our BCR-specific models of species accumulation curves, to visualize whether observers report more species after they have submitted more eBird checklists. Note that while increased participation leads to a higher rate of accumulation of species, this effect is highest for beginning participants and slows with increased participation.
Fig 9
Fig 9. Species distribution model accuracy with and without data submission scores (AUC a and Kappa b.).
There are 20 species models from each of the 6 BCRs: 10 for species, which are hard to identify, and 10 for species that are easier to identify, as defined by differences between the highest and lowest quartiles of data submission scores (Figs 6 and 7). In most cases the inclusion individual observer data submission scores improved model accuracy.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Bonney R, Cooper C, Dickinson JL, Kelling S, Phillips T, Rosenberg K, et al. (2009) Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy. BioScience 59: 977–984.
    1. Wood C, Sullivan B, Iliff M, Fink D, Kelling S (2011) eBird: Engaging Birders in Science and Conservation. PLOS Biol 9: e1001220 10.1371/journal.pbio.1001220 - DOI - PMC - PubMed
    1. Dickinson JL, Zuckerberg B, Bonter DN (2010) Citizen science as an ecological research tool: challenges and benefits. Annual Review of Ecology, Evolution, and Systematics 41: 149–172.
    1. Hochachka WM, Fink D, Hutchinson R, Sheldon D, Wong W-K, Kelling S (2012) Data-intensive science applied to broad-scale citizen science. Trends in Ecology & Evolution 27(2):130–137. - PubMed
    1. Lindenmayer DB, Likens GE (2010) The science and application of ecological monitoring. Biological Conservation 143: 1317–1328.

Publication types

Grants and funding

This work was supported by the Leon Levy Foundation (http://leonlevyfoundation.org), Seaver Institute, Wolf Creek Foundation, and the National Science Foundation (IIS-1238371 and IIS-1209714). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.