Evaluating a potential disturbance

In many cases biological surveys to inform the evaluation process should be conducted on multiple occasions and comparison will be made between these data to ascertain trends and changes. However, data are only comparable when the same protocol has been used.

Baseline data

Most fauna impact evaluations compare aspects of the fauna assemblage or the abundance and distribution of a particular species before and after an event. However, many proponents of a potential developed fail to remember that substantial fauna survey data collected post an event will never make up for an inadequate pre-event data set. It is therefore very important that the assessment of potential impacts is planned well in advance, and if necessary, peer reviewed before baseline data are collected. This may require a pilot study (or use of previously collected data) to develop an appreciation of the fauna assemblage (i.e. detectability) and to determine spatial and temporal variability to estimate variance that will be used in a power test to determine the survey effort.

Adequacy of sampling and power analysis

Typically, a statistical power analysis is undertaken prior to a fauna survey to determine how much data should be collected to determine impact effects. In most cases, the intent is to test the null hypothesis that the proposed development has no impact upon the fauna assemblage in the adjacent area. Confidence in the capacity to detect a difference when it is real is assessed using a statistical power test. However, rarely is a power test undertaken when planning a fauna monitoring / evaluation program. If the intent is to determine impacts, then it makes little sense not to collect sufficient data to demonstrate an effect, if it is real.

In most situations the objective for a development proponent will be to have no impacts on the fauna in adjacent areas. In testing or evaluating impacts, there will be a ‘negative’ null hypothesis, because, if it is true, then there would be no benefit in applying a treatment (Fairweather 1991).

If it is falsely concluded that there is an impact when that is not true, then this is a Type I error (e.g. a false alarm). In contrast, if the investigation concludes there is no impact, when there really is an impact, this is a Type II error (Plate 1). This situation often leads to a false sense of security and complacency. Therefore, to be sure to detect an impact when it exists, we wish to minimise Type II errors. The probability of a Type II error is determined by the value of ß, and the statistical power of the test is 1-ß (Fairweather 1991). When a monitoring program has low power, it has a high Type II error rate (Burgman and Lindenmayer 1998) and ß is directly proportional to the variability in the data set. Variance is generally reduced by increasing the sample size, so to reduce the probability of a Type II error the sample size is generally increased (Fairweather 1991).

Plate 1. Type I and II errors in assessing impacts

Power tests are used to calculate the sample size needed to detect an environmental change, provided that the magnitude of that change can be defined (e.g. 10 or 20%) and there is an estimate of variance for the data. Evaluations resulting in small sample sizes run the risk of compromising power. In this context, it is relatively easy to obtain a non-significant impact, by simply ensuring the sampling intensity is kept low. Ignoring the power of test results can result in:

• little confidence in a non-significant impact finding;
• a failure to detect the cause of impacts;
• a lack of understanding of ecological relationships; and
• complacency (Fairweather 1991).

A power test is used to:

• select the most sensitive tests for the available data;
• decide on an appropriate sample size before the data collection commences; and
• to evaluate any non-significant results (i.e. post hoc power test) (Fairweather, 1991).

Power and the precautionary principle

The Environmental Protection Act requires that the EPA apply the precautionary principle in its assessments, and this would include any assessment of unforeseen or unplanned impacts of a development on the adjacent fauna. Kriebel et al. (2001) suggested that the burden of proof that a development is not having an unforeseen or unplanned impact lies with the proponent of the activity. It is therefore the proponent’s responsibility to demonstrate that the survey effort and analysis have been adequate to demonstrate that there was no impact when that was the conclusion. This can either be done by using a pre-survey power test or a post hoc power test (Fairweather 1991).

Bio-indicators

Evaluation programs should be statistically rigorous, based on sound biological principles, effective in communicating changes to proponents, stakeholders, government assessors and the general public, and providing sufficient data to interpret the changes. There is a diverse range of bio-indicators that are used to achieve this purpose. Bio-indicators are species or taxonomic groups of species whose parameters can be used as a measure of ecosystem condition or change in condition, or in the context of monitoring impacts on fauna, changes in the fauna assemblage. Bio-indicators range in complexity from single species indicators to measures of complex ecological systems.

In some circumstances, the evaluation will focus on how a development has impacted on a particular species. For example, a development in the mid-west of Western Australia might be in an area that contains a breeding population of Malleefowl (i.e. bio-indicator for this project), and a condition attached to the approval for the development requires the proponent to demonstrate, on an ongoing basis, that the development does not affect the viability of that population. There are two possible approaches in this situation:

• establish a criterion based on the minimum population for Malleefowl in a defined area (e.g. 25 birds in 200ha); or
• compare changes in the population in the impact area with those in an adjacent control site.

Both approaches are reasonable but have problems. For example, if the population dips below the criterion (e.g. 25 birds in 200ha), it may be for reasons not related to the disturbance (i.e. no cause and effect has been demonstrated), such as, a regional drought that has suppressed breeding activity or an increase in fox and wild dog numbers has impacted the population. The comparative approach provides more information (e.g. effects of drought, predator activity) but is likely to be more expensive. Unknown factors (e.g. fox predation) might affect one or both populations which would reduce the capacity to interpret the comparative data.

In other situations, the evaluation will focus on measuring impacts on the fauna assemblage. Lamb et al. (2009) suggested there were four types of indices for summarising changes in the intactness of biodiversity, or the degree to which biodiversity deviates from an analogue condition:

• traditional diversity indices;
• species intactness indices based on occurrence;
• species intactness based on abundance; and
• multivariate community indices (e.g. rank abundance curves, ordination, taxonomic diversity).

Traditional indices such as Shannon-Weiner, Simpson and species richness have limitations because of a loss of species identity (e.g. a native species can be replaced by an exotic and the score remains unchanged), but they are useful in indicating an overall change in vertebrate biodiversity (e.g. a reduction in the number of species). However, evaluations based on changes in species richness can be seriously flawed, as they take no account of relative abundance or the inclusion of rare and transient species in either the before or after datasets and can be sensitive to sample size or sampling error. For example, an abundant species may suffer a significant decline in numbers but this would not be not detected, and a rarely caught species (e.g. difficult to catch, vagrant) is given the same value as an abundant species. Variation based on relative abundance, when considered with species richness, provides more useful information than either species richness or overall abundance.

Rehabilitation and Degradation Index

Thompson et al. (2008) reported on an index that provides a quantitative measure of the extent to which the reptile assemblage in a rehabilitated or degraded area resembles that in an analogue site (e.g. control site). It utilises a combination of diversity, taxonomic and ecological parameters and each of these parameters is further sub-divided and an overall weighted score out of 100 can be calculated for a disturbed area. This index can also be used to quantify the impact of mining, grazing, feral predators or noise and dust on functional terrestrial ecosystems in adjacent areas. Birds and small mammals can be included in the index if sufficient ecological data are available for these taxa.

The Rehabilitation and Degradation Index (RDI; Thompson et al. 2008) requires that multiple sites in the impact and control areas are surveyed. Data from multiple sites in the same fauna habitat type are generally combined to provide a single score ranging from 0-100.

Images – top: Sturt’s Desert Pea (Swainsona formosa; taken near the Flinders Ranges, SA); bottom: Type I and II errors (https://effectsizefaq.com/category/type-ii-error/)

References:

Burgman, M. A. and D. B. Lindenmayer. 1998. Conservation Biology for the Australian Environment. Surrey Beatty, Chipping Norton.

Fairweather, P. G. 1991. Statistical power and design requirements for environmental monitoring. Australian Journal of Marine and Freshwater Research 42:555-567.

Kriebel, D., J. Ticker, P. Epstein, J. Lemons, R. Levins, E. L. Loechler, M. Quinn, R. Rudel, T. Schettler, and M. Stoto. 2001. The precautionary principle in environmental science. Environmental Health Perspectives 109:871-876.

Lamb, E. G., E. Bayne, G. Holloway, J. Schieck, S. Boutin, J. Herbers, and D. L. Haughland. 2009. Indices for monitoring biodiversity change: Are some more effective than others? Ecological Indicators 9:432-444.

Thompson, S. A., G. G. Thompson, and P. C. Withers. 2008. Rehabilitation index for evaluating restoration of terrestrial ecosystems using the reptile assemblage as the bio-indicator. Ecological Indicators 8:530-549.