| Transect | Species count (richness) | Species |
|---|---|---|
| 3 | 2 | Halodule, Thalassia |
| 6 | 2 | DR: Chondria, Thalassia |
| 7 | 5 | DR: Acanthophora, DR: Chondria, Halodule, Syringodium, Thalassia |
| 8 | 3 | DR: Acanthophora, Halodule, Thalassia |
| 9 | 4 | DR: Chondria, DR: Hypnea, Syringodium, Thalassia |
| 10 | 1 | Halodule |
How Report Card Scores Are Calculated
Overview
Seagrass transects are monitored in Tampa Bay each year by multiple resource management agencies. To support these efforts, the Tampa Bay Estuary Program coordinates an annual intercalibration training prior to the field season. This intercalibration effort ensures that all participating groups are using consistent methods and interpretations. Seagrasses are independently assessed by each group at the same locations and the results are compared using report cards. The intent of these report cards is to highlight areas where each group is doing well and where efforts could be improved to ensure consistency. This document describes how the report cards are prepared and what the scores mean.
Because there is no single external ground truth, scores are based on how consistently groups agree with each other. A group that reports values close to the cross-group average for the year earns a high score and one that deviates substantially earns a lower score.
Scores are calculated for three field measurements for each seagrass species:
- Abundance: Braun-Blanquet (BB) cover category (0 = No cover, 0.1 = Solitary, 0.5 = Few, 1 = <5%, 2 = 5-25%, 3 = 25-50%, 4 = 51-75%, 5 = 76-100%)
- Blade Length: average blade length in cm
- Short Shoot Density: shoots per m²
Whether each group correctly identified which species were present is also factored into the abundance score. These three metric scores are averaged into an overall total score, which is then converted to a letter grade.
Note that “transect” is used herein to describe the intercalibration sites during training, which are quadrats at fixed locations. For the actual transect sampling, sites are at set meter marks along a transect line.
The scoring workflow, step by step
The full scoring workflow, from raw field data to a final letter grade, is:
- Identify the “true” (consensus) species at each transect.
- Calculate the “true” value for each metric and species at each transect, as the cross-group average.
- Calculate the cross-group variation in reported values for each species/transect/metric, for the transect-level weight.
- Calculate each group’s difference (reported minus true) for each species/transect/metric.
- Apply the transect-level weight from step 3 to each transect’s difference from step 4, discounting differences at transects where groups disagreed a lot.
- Combine the weighted per-transect differences from step 5 across transects into one difference value per species/metric, using a true weighted average (weighted by the same transect-level weight from step 3, not a plain average), so a transect with a larger weight counts for more.
- Roll up species into a raw metric score as the plain average of those per-species differences.
- Establish the typical spread of raw metric scores between groups within each year, using a fixed baseline period of 5 years (2020-2024). This is the baseline that every year (including years in the baseline itself) is compared against to judge whether a focal year was unusually tight or loose.
- Calibrate the score floor for that year and metric based on how tight or loose the cohort was that year relative to the typical spread from step 8, then rescale the raw score to a 0–100 numeric score.
- Get the overall Total score as the plain average of the three metrics’ numeric scores, then convert every numeric score, per-metric and Total, to a letter grade using fixed thresholds.
The steps are connected as shown below in the diagram.
The sections below work through each of these ten steps in detail. A full worked example applied to one group follows the detailed descriptions.
Step 1: The Consensus Species List
Not every species recorded across all groups counts toward scoring. A species at a given transect (site/quadrat) is considered present if at least two distinct groups reported it with non-zero abundance. This removes likely misidentifications while still being inclusive, i.e., a species does not need to be unanimous, just corroborated. False positives (found by the current group but not found by any other groups) or false negatives (found by at least two other groups but not the current group) are considered in the abundance metric calculations. Species identifications include seagrass and macroalgae. Both are scored on abundance, but only seagrass is used to assess blade length and short shoot density.
The consensus list is evaluated per transect, so a species may be on the list at one transect but not another.
Step 2: True Values
For each consensus species at each transect, true values are calculated as the cross-group average:
- Abundance: group Braun-Blanquet values (0, 0.1, 0.5, 1, 2, 3, 4, 5) are averaged directly across groups, then snapped to the nearest valid BB value (no coverage, solitary, few, <5%, 5-25%, 26-50%, 51-75%, 76-100%).
- Blade Length and Short Shoot Density: simple means across groups.
Only consensus species enter this calculation. This prevents a misidentified species (recorded by only one group) from distorting the true values for everyone else.
| Transect | Species | Abundance (0, 0.5, 1, 2, 3, 4, 5) | Blade Length (cm) | Short Shoot Density (per m²) |
|---|---|---|---|---|
| 8 | DR: Acanthophora | 1 | NA | NA |
| 8 | Halodule | 3 | 21.3 | 5.5 |
| 8 | Thalassia | 2 | 28.5 | 1.9 |
Steps 3-7: Group Differences and Metric Scores
For each metric (abundance, blade length, short shoot density), a group’s reported values are compared to the true values, species by species and transect by transect, then rolled into a single score per group per metric. Species misidentifications (false positive or false negative) are evaluated first before rollup, since they change what counts as a difference from the true values.
Species Identification Penalties
Comparing a group’s reported values to the true values across all consensus species and transects reveals two types of species identification errors:
False negatives (missed species): a species is present but the group did not record it. The group’s abundance for that species is treated as “no coverage” (BB value 0) when computing the difference. The penalty scales with the true abundance of the missed species, such that missing a rare species is a small error, while missing a dominant one is a large error.
False positives: the group recorded a species that is not on the consensus list (i.e., no other group confirmed it). The true abundance is treated as “no coverage” (BB value 0), but unlike a missed species, the penalty is a fixed one category difference from zero. What category a group happens to report for a species nobody else saw is not evidence about whether the species was really there, so the penalty does not scale with the reported abundance.
These penalties only affect the abundance score. Blade length and short shoot density cannot be meaningfully penalized for species that were not found.
For abundance, the difference is computed as a gap in ordinal category positions rather than in raw Braun-Blanquet numeric values. The eight BB categories are treated as equally spaced steps (1 = no coverage through 8 = 76–100%), so a difference of 1 means the group was one category apart from the true value, regardless of the numeric gap between those categories on the BB scale. This implicitly assumes that there is similar difficulty in distinguishing between lower abundances (e.g., solitary vs few) and higher abundances (e.g., 51-75% vs 76-100%).
| Transect | Species | Abundance reported | Abundance true | Blade Length reported | Blade Length true | Short Shoot Density reported | Short Shoot Density true |
|---|---|---|---|---|---|---|---|
| 8 | DR: Acanthophora | — | <5% | NA | NA | NA | NA |
| 8 | DR: Laurencia | few | — | NA | NA | NA | NA |
| 8 | Halodule | 5-25% | 25-50% | 15.2 | 21.3 | 4.7 | 5.5 |
| 8 | Thalassia | 5-25% | 5-25% | 21.8 | 28.5 | 1.3 | 1.9 |
Per-transect difference and weight
Group scores are based entirely on how similar their estimates compare to the across-group average (the “true” estimate). The within group difference is also weighted based on how variable the estimates were between groups. At each transect, a difference is discounted when the different groups had highly variable estimates. That disagreement reflects how hard the species/metric was to measure at that specific site.
At transect \(t\), the difference for species \(s\) is
\[ d_{s,t} = \text{reported} - \text{true} \]
for abundance (an ordinal category-position difference, see Species Identification Penalties above) and blade length (a raw difference in cm), or the symmetric percent difference
\[ d_{s,t} = \frac{\text{reported} - \text{true}}{(\text{reported}+\text{true})/2} \]
for short shoot density (see Absolute vs. percent difference below). The one exception is a false positive transect for abundance (see Species Identification Penalties above), where \(d_{s,t}\) is fixed at 1 rather than computed from the reported category, since what category was reported is not evidence about whether the species was really there.
A species weight at a given transect is
\[ w_{s,t} = \frac{1}{1+\sigma_{s,t}} \]
where \(\sigma_{s,t}\) is the standard deviation, across all groups, of their individually reported values at that transect for that species:
\[ \sigma_{s,t} = \text{SD}\!\left(x_{s,t}\right) \]
with \(x_{s,t}\) denoting all reported values across the groups for species \(s\) at transect \(t\), and the standard deviation taken across all reporting groups, not just the group being scored. It measures how much trouble groups had agreeing with each other right there. A transect where fewer than two groups reported a value has an undefined \(\sigma_{s,t}\) and gets full weight (\(w_{s,t}=1\)). This is independent of how much the true value itself might vary from one transect to another. That spatial variability plays no role anywhere in scoring.
Rolling up to a species value
The species-level difference is a true weighted average, across transects, of the per-transect differences:
\[ \bar{d}_s = \frac{\sum_{t=1}^{T} w_{s,t}\,d_{s,t}}{\sum_{t=1}^{T} w_{s,t}} \]
A transect’s influence on \(\bar{d}_s\) scales with its weight \(w_{s,t}\), so a transect discounted for poor cross-group agreement is proportionally down-weighted relative to the other transects, rather than simply contributing a smaller value while every transect still counts equally toward the denominator.
When weighting matters
The weight \(w_{s,t}\) makes a difference count for less when groups had a lot of trouble agreeing with each other at that transect. Whether that raises or lowers \(\bar{d}_s\) compared to a plain (unweighted) average of the same transects depends on whether the group’s large differences happened to fall on the hard transects (high cross-group disagreement) or the easy ones (low disagreement). The two scenarios below use one hypothetical species reported at two transects to illustrate both cases.
Scenario 1: a large difference on a hard transect is discounted, lowering the score
A group is close to the true value at an easy transect (all groups agreed closely there, \(\sigma_{s,t}=0\)) but far off at a hard transect (groups disagreed a lot, \(\sigma_{s,t}=3\)).
| σs,t | |ds,t| | ws,t / average | |
|---|---|---|---|
| Transect A (easy, σ = 0) | 0 | 0.5 | 1.00 |
| Transect B (hard, σ = 3) | 3 | 4.0 | 0.25 |
| Weighted average | 1.20 | ||
| Unweighted average | 2.25 |
Without weighting, the two transects average to 2.25. With weighting, Transect B’s large difference is discounted because groups naturally struggled to agree there, yielding a weighted average of 1.2, a substantially better score. The group missed a hard target, and the scoring system appropriately reduces the penalty for that miss.
Scenario 2: a large difference on an easy transect is not discounted, raising the score
Now the same group is far off at the easy transect but close at the hard one.
| σs,t | |ds,t| | ws,t / average | |
|---|---|---|---|
| Transect A (easy, σ = 0) | 0 | 3.0 | 1.00 |
| Transect B (hard, σ = 3) | 3 | 0.5 | 0.25 |
| Weighted average | 2.50 | ||
| Unweighted average | 1.75 |
Without weighting, the two transects average to 1.75. With weighting, the small difference at the hard transect counts for less, so it no longer offsets the large miss at the easy transect, yielding a weighted average of 2.5. This is the appropriate outcome, since missing badly on a transect where all groups should agree is a genuine measurement error and should not be diluted by being close on a transect that is naturally harder to pin down.
The weight \(w_{s,t}\) does not systematically produce better or worse scores than a simple average. Whether it is higher or lower depends on the data. What it always does is give more say to transects where cross-group agreement was good, and less relative say to transects where even the group of observers as a whole struggled to agree, regardless of whether the scored group’s own report happened to be close or far off there.
Combining species into a metric score
The metric score is a plain average, across species, of each species’ absolute difference:
\[ \text{metric score}_{\text{raw}} = \frac{1}{S}\sum_s |\bar{d}_s| \]
where \(S\) is the number of species with a defined \(\bar{d}_s\) that year. No further weighting is applied here.
| Species | Reported (avg) | True (avg) | Mean difference (d̅s) |
|---|---|---|---|
| DR: Acanthophora | solitary | <5% | -1.8 |
| DR: Chondria | no coverage | few | -1.7 |
| DR: Hypnea | no coverage | few | -2.0 |
| DR: Laurencia | <5% | no coverage | 1.0 |
| Halodule | 5-25% | 25-50% | -1.2 |
| Syringodium | <5% | <5% | -0.5 |
| Thalassia | <5% | 5-25% | -0.6 |
To see how these combine into a raw metric score, consider the abundance values for group C in Table 6 above. Taking the absolute values of the mean differences, the 7 species have \(|\bar{d}_s|\) of 1.8, 1.7, 2, 1, 1.2, 0.5, 0.6. The raw abundance score is their plain average:
\[\frac{1}{S}\sum_s|\bar{d}_s| = \frac{1.8 + 1.7 + 2 + 1 + 1.2 + 0.5 + 0.6}{7} = 1.257\]
matching the abundance value shown in the “Raw score” row of Table 11 in the full worked example below. This raw score is then scaled to a numeric score on a 0–100 scale using the range of scores across groups, which is then converted to a letter grade (see next steps).
Absolute vs. percent difference
\(d_{s,t}\) is not defined the same way for every metric:
- Abundance uses the raw difference in ordinal Braun-Blanquet category positions (as described in Species Identification Penalties above). These categories are already unitless and applied the same way to every species, so there is no reason to convert them to a percent basis.
- Blade Length uses the raw difference in cm.
- Short Shoot Density instead uses the symmetric percent difference (CV). This metric is measured in physical units (shoots/m²) where the typical magnitude varies a great deal by species (e.g., Halodule is characteristically much denser than Thalassia). The percent difference puts every species and every year on the same relative scale. The reasoning for doing so is justified below.
The weight \(\sigma_{s,t}\) follows the same basis as \(d_{s,t}\) for each metric:
- Abundance and Blade Length use \(\sigma_{s,t}\) directly, in raw units (category positions, cm).
- Short Shoot Density instead uses \(\sigma_{s,t}\)’s coefficient of variation, dividing it by the true value at that transect, for the same reason \(d_{s,t}\) does.
When \(\sigma_{s,t}\) cannot be computed (fewer than two groups reporting at that transect), the weight defaults to its neutral value, full weight (\(w_{s,t}=1\)), for every metric.
For short shoot density, percent normalization is appropriate when the typical size of a measurement disagreement itself scales with the magnitude of the true value. If disagreement is instead roughly constant regardless of magnitude, dividing by the true value doesn’t remove a scale effect, it creates one. This is why abundance and blade length scores are based on absolute differences.
The figure below demonstrates these differences by showing every species/year combination across the full training record for abundance (left), blade length (middle), and short shoot density (right). The x-axis shows the “true” values for the year/species combination and the y-axis shows the mean absolute disagreement between groups.
For short shoot density, disagreement increases with higher density (r = 0.97). Percent difference removes that effect, so a year or species with a naturally denser bed isn’t penalized more just for being dense.
The figure below confirms that the effect is removed for short shoot density. The y-axis is now shown as the percent (CV) difference actually used for scoring, instead of the raw shoots/m² difference shown above.
Steps 8-9: Score Calibration
Raw metric scores for each of the three metrics are converted to a 0–100 scale, with a minimum value of 50 that can be increased so that groups can receive better scores when they are more similar to each other. It prevents any one group from receiving a very low score due to the rescaling even if all groups performed similarly in a given year.
How calibration works
For each year and metric, the within-year standard deviation of group differences is calculated to assess the spread between groups. Each year/metric spread is then compared to a “typical” spread to determine how similar or different the groups were for that year. The typical spread is determined from a fixed baseline period calculated using the first 5 years of training data (2020-2024).
The score floor for a given year and metric is:
\[ \text{floor}_{\text{yr, met}} = \max\!\left(50,\ 50 + \left(1 - \frac{\text{SD}_{\text{yr, met}}}{\overline{\text{SD}_{\text{met}}}}\right) \times 50\right) \]
where \(\text{SD}_{\text{yr, met}}\) is the within-year spread for that metric and year, \(\overline{\text{SD}_{\text{met}}}\) is the baseline mean standard deviation for the metric described above, and 50 is a scaling constant that sets the maximum possible floor lift (grade-points). The scaling constant is a subjective choice and, in this case, using a value of 50 means that the floor can be raised a total of 50 points for years when groups score similarly. That is, the maximum lift occurs when all groups agree perfectly setting the ratio to 0, giving a floor of \(50 + 50 = 100\), which corresponds to a minimum grade of A. Years when the standard deviation exceeds the baseline value are capped at 50 so that no extra penalty is applied.
The figure below shows every group’s raw (before rescale) metric score, one point per group for every training year, to demonstrate the general spread of scores from which the standard deviations are calculated.
The next figure shows those distributions as the standard deviation for the year, compared to the mean standard deviation for the baseline period (dashed line). Years when the ratio of the standard deviation to the baseline mean standard deviation are less than 1 have a raised minimum score (i.e., groups had more similar scores), whereas years when the ratio is at least 1 receive the default floor of 50.
The table below shows the final score floors for each year and metric. These are the same values in the labels above the barplots in the previous figure.
| Year | Abundance | Blade Length | Short Shoot Density |
|---|---|---|---|
| 2020 | 50 | 81 | 69 |
| 2021 | 69 | 50 | 58 |
| 2022 | 50 | 52 | 50 |
| 2023 | 60 | 50 | 50 |
| 2024 | 52 | 50 | 56 |
| 2025 | 60 | 55 | 75 |
| 2026 | 60 | 91 | 59 |
Step 10: Letter Grades
After calibration, each group’s numeric score for each metric falls on a conventional 0–100 scale, with the lowest possible score in a given year defined by the calibrated score floor. The 0–100 label is nominal only to correspond to a recognizable scoring scheme. In practice a score can never fall below that year’s floor. These are mapped to letter grades using fixed thresholds:
| Grade | Score range |
|---|---|
| A | 95 – 100 |
| A- | 90 – 94 |
| B+ | 85 – 89 |
| B | 80 – 84 |
| B- | 75 – 79 |
| C+ | 70 – 74 |
| C | 65 – 69 |
| C- | 60 – 64 |
| D+ | 55 – 59 |
| D | below 55 |
The Total score is the unweighted average of the abundance, blade length, and short shoot density numeric scores, then converted to a letter grade using the same thresholds.
Worked Example
The example below applies the general workflow from above to group C in 2025, starting from the raw per-transect data.
Raw differences
The table below shows the raw data used for scoring. Each row is one transect (site) and species (macroalgae or seagrass), with the group’s reported value, the cross-group true value, and the weighted difference between them for each metric (a percent difference for short shoot density, a cm difference for blade length, a category-position difference for abundance). The weighted difference column is already discounted by the transect-level weight \(w_{s,t}\) and not simply the difference between reported and true (unless the weight is 1). Likewise, the “Average” row below each species is a true weighted average of the transect-level values above it (the sum of the weighted differences divided by the weights). An example for one species using the weights follows this table.
| Transect | Abundance | Blade Length | Short Shoot Density | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Reported | True | Weighted Diff | Reported | True | Weighted Diff | Reported | True | Weighted Diff | |
| DR: Acanthophora | |||||||||
| 7 | 4 | 4 | 0 | — | — | — | — | — | — |
| 8 | — | 4 | — | — | — | — | — | — | — |
| Average (ds) | — | — | -1.8 | — | — | — | — | — | — |
| DR: Chondria | |||||||||
| 6 | — | 2 | — | — | — | — | — | — | — |
| 7 | — | 3 | — | — | — | — | — | — | — |
| 9 | — | 3 | — | — | — | — | — | — | — |
| Average (ds) | — | — | -1.7 | — | — | — | — | — | — |
| DR: Hypnea | |||||||||
| 9 | — | 3 | — | — | — | — | — | — | — |
| Average (ds) | — | — | -2 | — | — | — | — | — | — |
| DR: Laurencia | |||||||||
| 8 | 3 | — | — | — | — | — | — | — | — |
| 9 | 4 | — | — | — | — | — | — | — | — |
| Average (ds) | — | — | 1 | — | — | — | — | — | — |
| Halodule | |||||||||
| 3 | 5 | 7 | -1.17 | 20.6 | 24.6 | -0.88 | 12 | 9 | 23% |
| 7 | 5 | 6 | -0.58 | 20.8 | 5.1 | 1.41 | — | — | — |
| 8 | 5 | 6 | -0.67 | 15.2 | 21.3 | -1.08 | 4.7 | 5.5 | -12% |
| 10 | 5 | 6 | -0.59 | 5.6 | 1.7 | 0.85 | — | — | — |
| Average (ds) | — | — | -1.2 | — | — | 0.4 | — | — | 7% |
| Syringodium | |||||||||
| 7 | 4 | 5 | -0.58 | 23.8 | 7 | 1.09 | — | — | — |
| 9 | 3 | 3 | 0 | 18.4 | 20.9 | -0.36 | 0.3 | 0.7 | -53% |
| Average (ds) | — | — | -0.5 | — | — | 3.5 | — | — | -80% |
| Thalassia | |||||||||
| 3 | 2 | 3 | -0.53 | 6.2 | 23 | -1.5 | 0.7 | 1 | -24% |
| 6 | 5 | 5 | 0 | 17.4 | 4.9 | 1.13 | 0 | 0.2 | -53% |
| 7 | 5 | 6 | -0.59 | 33.8 | 9.1 | 1.27 | — | — | — |
| 8 | 5 | 5 | 0 | 21.8 | 28.5 | -0.85 | 1.3 | 1.9 | -26% |
| 9 | 5 | 6 | -0.58 | 22.6 | 24.4 | -0.4 | 3.3 | 3.3 | 0% |
| Average (ds) | — | — | -0.6 | — | — | -0.6 | — | — | -40% |
To demonstrate how the weights are factored into the weighted differences and the species averages, the table below shows the full example for group C’s short shoot density for Thalassia:
| Transect | Reported | True | Percent difference | Cross-group CV (σs,t) | Weight (ws,t) | Weighted % difference |
|---|---|---|---|---|---|---|
| 3 | 0.7 | 1.0 | -35% | 0.45 | 0.69 | -24% |
| 6 | 0.0 | 0.2 | -200% | 2.78 | 0.26 | -53% |
| 8 | 1.3 | 1.9 | -37% | 0.43 | 0.70 | -26% |
| 9 | 3.3 | 3.3 | 0% | 0.10 | 0.91 | 0% |
Averaging the raw “Percent difference” column above gives -68%, which is not what is used for the species average metric difference (\(\bar{d}_s\)). The average difference is based on the weighted differences and the weights, estimated as a true weighted average. This is done by summing the weighted differences (-1.03) and then dividing by the weights (2.56), yielding -40%, matching the Average row for Thalassia’s short shoot density in Table 9 above.
Scores
For each metric, the mean absolute difference per species (\(|\bar{d}_s|\)) is listed in a single comma-separated cell. The “Raw score” row below it shows the plain average of those values, the calibrated score floor, the rescaled numeric score, and the letter grade. The raw score is a plain average since the differences per species are already weighted.
| Species |d̅s| | Raw score (average) |
Score floor | Numeric score | Letter grade |
|---|---|---|---|---|
| Abundance | ||||
DR: Acanthophora (1.8), DR: Chondria (1.7), DR: Hypnea (2), DR: Laurencia (1), Halodule (1.2), Syringodium (0.5), Thalassia (0.6) |
— | — | — | — |
Raw score (average) |
1.257 | 60 | 66.6 | C |
| Blade Length | ||||
Halodule (0.4), Syringodium (3.5), Thalassia (0.6) |
— | — | — | — |
Raw score (average) |
1.500 | 55 | 95.5 | A |
| Short Shoot Density | ||||
Halodule (0.07), Syringodium (0.8), Thalassia (0.4) |
— | — | — | — |
Raw score (average) |
0.425 | 75 | 77.6 | B- |
| Total | ||||
Average of three metric scores |
— | — | 79.9 | B- |
| The Raw score row is the plain average of |d̅s| across species. |d̅s| is a percent difference for short shoot density, a cm difference for blade length, and a category-position difference for abundance, already discounted at the transect level for cross-group disagreement (Steps 3-7). | ||||
The figure below shows how raw scores map to numeric scores across all groups for each metric. The gray line is the linear rescaling anchored at 100 for the best group and at the score floor for the worst. Group C is highlighted in blue.
Finally, the table below shows the group’s scores with and without score calibration. Because groups performed more similarly for all metrics (short shoot density in particular) compared to the baseline period, the floor raises all scores.
| Metric | Score without calibration | Score with calibration | Ratio (within-year spread / baseline mean) | Score floor |
|---|---|---|---|---|
| Abundance | 57.5 | 66.6 | 0.79 | 60 |
| Blade Length | 95.0 | 95.5 | 0.90 | 55 |
| Short Shoot Density | 54.8 | 77.6 | 0.50 | 75 |
| Average | 69.1 | 79.9 | — | — |