Establishing a monitoring system

Our Resource Library contains manuals, forms and other documents with a lot of detailed, expert information and tips from practitioners in the field and from scientific bodies. These include guidelines and advice tailored to monitoring carbon and biodiversity for REDD+ (e.g. the Sourcebook of biodiversity monitoring for REDD+  by Latham et al., 2014). We recommend exploring those resources for advice that meets your particular needs.

However, to provide a starting point, we summarise below some key considerations to keep in mind when developing your system.

Involving the right people

Before designing a monitoring methodology, it’s important to consider who should be involved in the design process. The framework suggested by Danielsen et al. (2009) describes the spectrum of options for projects led by community members, external scientists, or a combination of actors.

screen shot

As with other aspects of project design and implementation, the level of involvement of different actors in designing the methodology can affect the legitimacy, relevance and success of the project. Our case studies showcase a variety of options, but mostly sit somewhere in the middle of this spectrum as collaborations between communities and external actors.

The Forest COMPASS team learned from our collaborative projects in Guyana and Brazil that the multi-stakeholder approach can be time-consuming and complex, but it generated valuable levels of satisfaction and support. At the same time, it was important for these multi-stakeholder groups to be collectively realistic about what could be achieved, given limits on time, capacity and resources.

Generating relevant and useable data with a feasible methodology

The data produced by community-based monitoring will need to be not only useful, but also useable. A common problem in monitoring (both by scientists and other groups) is that valuable data can sometimes remain unused, or inaccessible to some key stakeholders. This could happen, for example, because the data are stored in a particular location or format and controlled by a particular person or group, or because the methodology used to collect them wasn’t consistent with other methodologies (so they couldn’t be easily analysed along with other data). Imagining early on how the data will be used (before designing the methodology or starting data collection), and thinking about how to enable that, can save time and help ensure the results lead to actions.

If the indicators being monitored are intended only for communities, it might be hard to convince government officials or companies to respond to them. On the other hand, if the project answers questions that are of interest to national or international decision-makers, but of little relevance to the community, local people might be reluctant to carry out the task, or to continue with monitoring after external donors and facilitators have withdrawn. This is especially true if the method used takes them away from their usual daily activities.

Therefore, as a critical early step, methodologies and indicators need to be chosen which are suitable, feasible and compelling to, and thoroughly understood by, key participants and audiences. Without these, data collection and analysis can be unfocused and lead to unnecessary expenditure of time and money, and frustration among participants. Our article on cost-effectiveness provides useful advice on achieving this by balancing certain trade-offs.

In each of the case studies on the Forest Compass website, we have presented the methods, themes and indicators that were used, which may provide useful ideas (see also our full table of themes, developed for the case studies). Further advice on selecting themes and indictors for community-based monitoring can be found in the Resource Library.

Making it reliable

To make sure you and others can trust the data generated through monitoring, it will need to have a certain level of accuracy and reliability. You will also need to avoid bias. Below is a list of some of the relevant criteria to consider when designing the methodology. For further information and detail, and to understand better how these issues relate to the type of issue you plan to monitor, please see the handbooks and other resources in the Resource Library.

Sample size: Are samples large enough that trends can be detected, but not so large that the project is unfeasible?

Data accuracy and precision: Accuracy is defined as being the closeness of a measure to its true value, free from systematic and observational error. Data precision is the range of possible values between which a particular observation may lie, the more precise the measurement, the more repeatable the measurement will be. Greater accuracy can be achieved by proper planning of a survey and a clearly defined survey question. Precision can be achieved by good planning, and often a greater investment of time or resources.

Table 2 reproduced from Keller 2015

Table 2 reproduced from Keller 2015

Research shows that community-led monitoring tends to be less accurate and precise than scientist-led monitoring. That doesn’t mean it’s less useful, just that there are different considerations that need to be taken into account. It may be more appropriate to invest in ensuring a survey is accurate, rather than very precise, for community-based forest monitoring.

Bias: Perhaps the most important consideration is to try to avoid bias, which can make a survey redundant or misleading for decision making. Bias is defined as any systematic error in measurement. It can result from accidental importance being given to a certain part of the survey population. For example, if human disturbance within a forest is being monitored, and the survey concentrates on areas alongside a road (because this gives easier access to the surveyors), then there will be bias in the data, because roadside forest will systematically have a higher occurrence of human disturbance than areas of forest far away from the road. Bias may also affect social surveys. For example, if a disproportionate number of household interviews are carried out along a road, the results will not be representative of remote households. Avoid bias by considering the survey aim and designing the survey so that it does not preference certain people or areas. Solutions for avoiding bias will depend on the context and what is being monitored - see the documents in the Resources section, or dedicated statistics manuals, for further advice.

Frequency/Timeframe: Often the objective of a survey will be to monitor change in an indicator, not just take a static assessment of an indicator. As a basic rule at least three data points are needed to detect a trend. Greater frequency or project duration may be very useful, but also tend to require more time and/or money. How frequently an indicator is monitored should correspond to the survey question. For example, to detect a trend in income you may only need to survey every five years, but a change in daily temperature will require daily monitoring.

Geolocation: Will it be useful, or necessary, to capture location data (e.g. using GPS – see digital technology section)? This may be essential for monitoring change in particular sites, and for validating data. Spatial data were seen as essential for ensuring the trustworthiness of data in the Moabi REDD+ safeguard monitoring project in the Democratic Republic of Congo.

Visualisation: Could photos, maps, and other forms of visualisation be valuable for making the data comprehensible and influential? The visualisation you ultimately want to achieve may significantly influence the way in which you need to collect data. For example, maps were important for visualising land-use data in the North Rupununi project, whereas photos were important for getting media coverage of illegal incursions into indigenous lands in the RuaiSMS project in Indonesia. Mapping is often an important first step in a monitoring initiative, as it helps to reveal basic characteristics and the extent of the area to be monitored.

Roadside Stump

Verification: How will you, and others, know, and be able to prove, that your data are accurate and reliable? Internal and external verification processes may be required, depending on who is collecting the data and who you hope will use them.

Controls: In order to tease apart cause and effect in a trend, monitoring initiatives will often need controls. This is to ensure that the trend that you are measuring is not being influenced by another factor you are not measuring. For example, communities might monitor the increase in yield of maize delivered by planting the crop under a nitrogen fixing tree. However, unless maize is also planted under a non-nitrogen fixing tree (a control), the community cannot tell whether the increase in maize is caused by nitrogen-fixation or some other factor, such as moisture retention in the soil under the canopy.