On the accuracy of official Chinese crop production data: Evidence from biophysical indexes of net primary production

Multilevel Assessment of the Chinese Crop Production Data Plausibility.

Maps of HANPP_harv computed using reported provincial- and prefectural-level data are shown Fig. 1 A and B, respectively. There are differences clearly visible on the maps between the two administrative data levels. In particular, there is some evidence of potential misreporting problems at the lower administration level as prefectural-level indicator values appear much higher in many areas. These differences probably occur because, for some prefectural levels, data were usually collected by adding all subadministrative data reported by the local governments together that are more prone to manipulation, while for the provincial level, a sample survey approach is used (14, 55, 56). This finding is consistent with previously reported issues with lower-level administration in the reporting of economic statistics (14, 26, 57). More discussions about HANPP results can be found in SI Appendix.

Table 1 shows the HANPP_harv/NPP_act%, for cropland by Chinese province for the year 2010 (details about HANPP_harv/NPP_act% for other land use types can be found in SI Appendix, Table S6). The ratio for 16 provinces, including Beijing, Tianjin, Hebei, Inner Mongolia, Jilin, Shandong, Henan, and Ningxia, exceeded 100%. In this case, the values of harvested NPP for cropland (including grain and residues), based on provincial crop production data taken from the China Statistical Yearbook (10), were higher than the NPP_act obtained from the MODIS NPP products. These results suggest a significant potential problem of inflated crop statistics also at the provincial level. Consequently, the national total is potentially affected by the same problem.

Since NPP harvested by humans on cropland accounts only for part of the NPP_act, a ratio of 1 for HANPP_harv/NPP_act would not be appropriate as a threshold value above which concerns about the quality of the data might emerge. NPP_act on cropland is defined as the sum of HANPP_harv and preharvest losses due to herbivory and weeds (45, 46). A “loss expansion factor,” which typically is chosen based on the level of development and fertilizer use of a country, is used to extrapolate total NPP_act from HANPP_harv (NPP_act = HANPP_harv × loss expansion factor) (46). Krausmann and coworkers (45) used values ranging from 1.36 (least-developed countries with low fertilizer use) to 1.14 (for industrialized countries with high fertilizer use). Since previous research has shown that China has a fertilizer use intensity greater than 150 kg⋅ha⁻¹⋅y⁻¹ (58), the “technology-adjusted” loss expansion factor appropriate for China suggested by the literature on HANPP can be as low as 1.14 (46). Therefore, we propose to use 1/1.14 (∼0.88) as an upper bound above which the gap between official statistics and values from MODIS becomes increasingly concerning. Thus, an indicator value exceeding 0.88, as in the case of Anhui, Jiangxi, and Hubei, suggests a potential problem of overreporting crop production for those provinces.

The indicator allows us to classify crop production data at different administrative levels in China into three groups of potential overreporting evidence: low, for values of HANPP_harv/NPP_act lower than 0.88; medium, for values between 0.88 and 1; and high, for values greater than 1. The maps showing the spatial distribution of indicator values for 2010 and 2014 at the provincial level are displayed in Fig. 2 A and B, respectively. These maps show that, for several provinces, the potential overreporting level changed over the two periods. The values for Qinghai, Hubei, Anhui, and Gansu, and especially for Jilin, where they reached about 2, all increased over the period. In contrast, the values for Beijing and Tibet decreased. Based on the above considerations, we can also interpret an index of 1 as providing some evidence that crop statistic figures could be exaggerated by about 13%, while a value of 2 could mean that the crop statistics reported amounted to more than double the actual values (127%).

A more detailed map of the indicator calculated using Chinese data at the prefectural level for 2010 is displayed in Fig. 2C. The map, using the smaller administrative units, highlights the high spatial heterogeneity of problematic data within provinces. For several prefectures, the indicator exceeded the value of 2. This difference cannot be solely explained by the different data collection protocols and statistical methodologies of different administrative levels. In most provinces, the sum of crops production data, such as wheat and maize, at the prefectural level does not match the one at provincial-level data, which is mostly larger. In general, the statistical items in the data sheets of different administrative levels contain both omissions and inconsistent definitions that make them noncomparable. Some of the statistical items available at the provincial scale are not available in its subadministrative divisions. For example, oil crops do not always contain groundnuts and rapeseeds in every prefecture. Instead, some contain other categories such as sunflower seeds or just record oil crops as a whole. Because of these issues, the production data in China obtained by means of large-scale sample surveys at the higher level are considered more reliable than the census data reported by lower-level administrative government units (59).

Temporal Variation of the HANPP_harv and NPP_act.

We looked at Chinese official grain production statistics from 2000 to 2014 at the provincial level by computing HANPP_harv on cropland in 2000, 2005, 2010, and 2014. Cropland areas, extracted from land use maps, were overlaid with the NPP_act from MODIS to extract the estimated NPP on cropland. The two temporal series were normalized, based on HANPP_harv of cropland in 2000, to show relative contributions of each factor and the changes over time, as shown in Fig. 3. Results indicate that, during this 15-y period, the normalized indicator exceeded 1 in many provinces, such as Xinjiang, Inner Mongolia, Jilin, Tianjin, Beijing, Hebei, Shandong, and Henan. In particular, in several provinces, such as Tianjin, Hebei, Jilin, Ningxia, Shandong, and Henan, the indicator exceeded 1 for all years. Notably, the MODIS NPP tended to fluctuate slightly over time and its changes in most provinces were relatively small. Conversely, the harvested NPP for some provinces increased rapidly, in a way that is inconsistent with the “actual NPP” trend. These outliers tend to be provinces like Hebei, Jilin, Ningxia, Shandong, and Henan that are major grain producers.

The potential overreporting of crop production data in China can misinform the authorities and researchers, which could lead to an underestimation of food security issues and result in supporting wrong policies and planning actions that are highly dependent on the official reported data.

Data Uncertainties and Sensitivity Analysis.

The uncertainties about the indicator derive from the MODIS NPP product and from the parameters used in the calculation of our harvested NPP based on Chinese production statistics.

The NPP product from NASA MODIS (MOD17A3) is directly used here to obtain NPP_act. Consequently, this product’s uncertainties are part of the uncertainties of this study. According to its Product User Guide (https://lpdaac.usgs.gov/documents/212/mod17_v5_user_guide.pdf), these uncertainties include 1) low spatial resolution of the weather data used for calculating NPP that affects the quality of the final NPP product; 2) the use of biome-specific physiological parameters, which do not vary with space or time; and 3) several of the input parameters for MOD17A3 NPP data depend on the land cover data product, MOD12Q1, having an accuracy of 65 to 80% (see the Product User Guide) that will also increase the uncertainty of the calculated NPP_act.

Validation studies have found that, among all of the parameters, one of the largest sources of uncertainty in the MODIS estimates comes from the maximum light use efficiency (ε_max) value used, which is an underestimate (0.68 for cropland) (60, 61). The uncertainty in harvested NPP may come from the factors used in the HANPP_harv calculation model, such as, moisture content, harvest factor, and the carbon content for crops and crop residues.

In this work, up to this point, we used the average values of these factors for the whole of Asia following major previous studies on the geographical distribution of HANPP that included China (45, 46). However, the values of harvest factor for different regions, different crop varieties, and different planting techniques can vary substantially. Rojstaczer et al. (43) pointed out that HANPP calculations can be extremely uncertain in the absence of high-resolution values for key agricultural parameters.

To address this concern, field-measured data for ε_max and more specific harvest factors derived from the literature, from Chinese studies when available, were used to improve the reliability and assess the sensibility to input parameters of the HANPP output values. We used harvest factor values obtained in local Chinese studies (62, 63) to adjust the HANPP harvested calculations. We have also drawn from more pertinent Chinese literature to adjust the MODIS values. In particular, values for ε_max reported in different papers display a high variability (range from 0.63 to 5) (64–71). Because of the limited availability of flux towers, which are used to collect the experimental data necessary to calculate ε_max in China that tend to be concentrated in more developed areas, more suitable parameters drawn from the literature had to be used (60, 61). The values used in this study and how they compare with the values used in the MODIS products and literature are shown in Table 2. In the MOD17A3 algorithm, ε_max is a multiplier of the NPP results (see the MODIS Product User Guide). Here, we adjusted the NPP_act results multiplying by a factor, calculated on the basis of the dry weight of different crops and its residues (Materials and Methods). Specific values for Chinese harvest factors at the provincial scale are also available from the literature (62, 63).

The values of our indicator for 2010 at the provincial level obtained after adjusting calculations with updated values for the harvest factor and the maximum light use efficiency, both separately and jointly, are displayed in Table 3. After updating the harvest factor, the indicator is reduced, on average, by 23% (the obtained values vary within ±13% of the average value), but is still at a relatively high level. After adjusting the indicator for ε_max, the overreporting indicator value was lowered by 51% on average, with values varying by ±9% around the average. Depending on the ε_max value used, ranging from 0.63 to 5, as reported in the literature (64–71), the indicator of potential overreporting value decreased between 30% (±7%) and 82% (±6%). After adjusting the indicator for both, the overreporting indicator values decreased by 60% (±13%). These results confirm the sensitivity of the HANPP calculations to these critical parameters, particularly ε_max.

We perform a local and global sensitivity analysis (72) to determine which of the uncertain parameters are driving the uncertainty of the indicator and thus require particular attention to reduce the uncertainty and increase the robustness of our results. The sensitivity of HANPP_harv/NPP_act with respect to each input parameter separately (local sensitivity) can be derived analytically (for details see SI Appendix). Data at the provincial scale for the year 2010 were used as an example. Results from the local sensitivity analysis show that the output indicator values are more sensitive to changes in ε_max and in the carbon content/crop production, followed by harvest factor and the moisture content, and finally by b (ratio of belowground to aboveground NPP) (Fig. 4). We further investigate the impact of the parameter’s uncertainty by using a global variance-based sensitivity analysis method that improves upon local approaches by allowing for the evaluation of the interaction between parameters. We find that, consistent with the local approach, the parameter ε_max explains most of the variability in the index (for details see SI Appendix).

When we look at prefecture–city-level indicators, the number of cities with potential overreporting data decreased sharply after the adjustment (Fig. 5). Nevertheless, after the joint adjustment of the two parameters, a dozen prefectural cities beyond medium or high potential overreporting levels remain (Fig. 5C). This suggests that the data, obtained by summing all subadministrative data such as prefectural cities, are affected by potential overreporting and high uncertainties. These results show that to reduce uncertainty of the HANPP estimates and improve detection of possible data problems at lower administrative levels, we need higher-resolution values for key parameters such as ε_max used in the HANPP estimations. Thus, more experimental field data obtained by expanding the network of observation towers in China are required.

Despite the data uncertainties mentioned above involved in the calculation of NPP based indicators, the results of this study, as well as of previous studies (13, 14, 22, 73), still deserve attention because of their relevance and can provide the basis for further research and developments. It should be noted that some of the fruits, vegetables, and cotton that grew on cropland, which also contains large amounts of carbon, are not accounted for, according to data and calculation method availability (45, 46, 52). If counted roughly, this would generate approximately a 5 ∼ 8% increase in provincial HANPP_harv and further changes in some of the provinces, such as Xinjiang, up to 30%.

The fact that HANPP_harv/NPP_act (%) for some areas exceeded 100% or even 200% is noteworthy. Several prefectural cities still display a significant level of potential overreporting of crop production statistics, even after allowing for improved parameters. Therefore, data on crops in the statistical yearbooks in China should only be used with caution for scientific research and in decision making and planning. More efforts are required to investigate the official crop production data and the underlying problems, before developing a more comprehensive understanding on this topic.

Recommendations and Prospects for Future Research.

According to a recent comprehensive study on improving agricultural and rural statistics by FAO, the World Bank, and the United Nations Statistical Commission, the quantity and quality of agricultural statistics have undergone a serious decline over the last two decades, especially in the developing world, that lacks the capacity to produce and report even a minimum set of agricultural statistics that would be required to monitor national trends (74). The traditional statistical methods, based on census or surveys, have drawbacks that particularly affect developing and underdeveloped countries with limited resources. These drawbacks include (75) 1) the need of time-consuming and labor-intensive measurements; 2) intentional over/underreporting; 3) insufficient supervision; and 4) illiteracy, etc. In many countries, including China, India, and the United States, using remote sensing to estimate biological crop yield is being explored and is likely to become a keystone of agricultural statistics in the future (76–78).

Even if the estimates on regional crop yield, based on remote-sensed measurements from satellite platforms, are affected by large errors, their use has the potential to produce more objective estimates than those from agricultural field reports that tend to be subjective, more costly, and affected by the lack of coverage (37). Thus, the quality of future Chinese statistics on crops could be improved by using such data. We expect that remote sensing and state-of-the-art processing algorithms will play an increasing role in improving the quality of agricultural statistics.

Many are the advantages in using remote sensing observations in facilitating the improvement of agricultural statistics. Remote sensing can provide the required resolution needed for estimating the spatial variability of many parameters at different spatial scales that inform primary production models. It can provide the data necessary to develop an effective and low-cost spatial representation of the territory, based on spectral information, from which natural conditions and plant physiological indexes can be derived (75). For example, National Agricultural Statistics Service (NASS) (78–80) and Monitoring Agriculture with Remote Sensing (MARS) (81) adopted high-resolution remote-sensing images combined with ground-sampling surveys to estimate crop area, with more efficiency and reduced sampling error. Remote sensing monitoring could be used to estimate crop area at a higher level of spatial disaggregation (80). Considering the budget and time constraints in relation to ground surveys as well as the limited accessibility of some target areas, high-resolution imagery may become a good substitute for ground-truth data. The results of the ground surveys and those of the analysis of satellite images can, as an example, be combined to improve the accuracy of area estimates. The European Union (82) and many other countries (83) have improved crop area estimates by integrating remote sensing observations in the methods used. The visual interpretation of images can also guide and improve field operations (80). The census and other statistical methods used in the compilation of national agricultural statistics can be affected by human errors and other forms of interference particularly when data are upscaled from a lower to a higher administrative level (76). When a huge difference is found between the production output from official national statistics and the estimates obtained by remote sensing, data and data processing methods can be further inspected and eventually recalibrated, as shown in this work, to increase their credibility. A third party, not affected by political interference and with impeccable scientific credentials, could supervise such a workflow aimed at improving the statistical accuracy of production data.

Considerable research is still needed before remote sensing can be more widely and routinely applied to estimate crop yield. One important shortcoming for the use of satellite images to estimate crop yield is the spatial resolution of the sensor, which is not sufficient to capture the variability of crops and crop performance in smallholder fields that often are less than 0.1 ha and may be intercropped as well (75). Cloud cover problems and the need for expensive ground truthing, specialist knowledge, and expensive image-processing software might limit the current usability of remote sensing (37).

Results in this study are also affected by the uncertainties embedded in key agricultural parameters and land use maps used in our estimation procedure. A recent survey on remote sensing applications to agricultural statistics has concluded that the best estimates of crop yield are still based on correctly implemented traditional survey techniques (84). Higher-resolution remote-sensing spatial and temporal data are needed to provide more reliable estimates of agricultural productivity and its changes in future studies. With further methodological and technological advancements, higher-resolution images might be easier and less costly to obtain and process, making the support of remote-sensing data for agricultural statistics more effective (37).

NPP_act in Cropland.

The estimated actual NPP for China was obtained from MODIS data for the years 2000, 2005, 2010, and 2014. In particular, we used the MOD17A3 version of the product in our analyses (54). We employed the land use/cover data from the Resource and Environment Science Data Center of the Chinese Academy of Sciences (85), with a resolution of 100 m (the available years are 2000, 2005, 2010, and 2015), and use the land use type “cropland” as a mask to extract the MODIS NPP data.

HANPP_harv in Cropland.

The biomass harvested from cropland includes aboveground crop yield and crop residues and belowground roots. The belowground NPP on cropland is assumed to be killed during harvest and is accounted for as harvested HANPP (45, 46). The calculation method is as in Eqs. 1–4:

where $C{P}_{DMi}$ and $C{R}_{DMi}$ denote the dry matter of crop production and dry matter of crop residues for each kind of crop.$B{G}_{DMi}$ denotes the belowground NPP.

Provincial crop production is derived from the China Statistical Yearbook (10). The prefectural data are available in the provinces’ statistical yearbook or cities’ statistical yearbook. The data from the yearbooks are expressed as fresh weight. Thus, they need to be converted into dry matter through moisture content (MC) from standard tables contained in previous literature studies (SI Appendix):

where $C{P}_i$ is the production data of each crop, and$M{C}_i$ is its corresponding moisture content.

There are no data for used and unused crop residues in the reported statistics. Crop residues were extrapolated by specific harvest factors from the available data. The harvest factors refer to the ratio of the crop residues to the crop production (45) (SI Appendix):

where $C{R}_{DMi}$ is the dry matter of crop residues and $H{F}_i$ is the harvest factor of each crop.

Belowground NPP, i.e., NPP allocated to roots, tubers, etc., is calculated using the ratio of belowground to aboveground NPP taken from refs. 45 and 46, which is fixed to 0.15 for all kinds of crops:

Adjustment Method for NPP_act.

The adjustment method to obtain a more representative MODIS NPP in terms of ε_max is shown in Eq. 5:

where $NP{P}_{act}^{\prime }$ is the modified NPP_act; $NP{P}_{act}$ is the original NPP extracted from MODIS data; and $C{P}_{DM1-5}$ and $C{R}_{DM1-5}$ are dry matter of crop productions and residues for paddy rice, wheat, maize, soybean, and sugarcane, respectively. $C{P}_{DM6}$ and $C{R}_{DM6}$ are the dry matter of crop productions and residues for other crops (sum of millet, sorghum, other cereals, tubers, groundnuts, rapeseeds, sesame, sunflower seeds, and sugar beets). And $A\hspace{0.5em}\text{to}\hspace{0.5em}E$ are adjusted ε_max for paddy rice, wheat, maize, soybean, and sugarcane found in the literature (Table 2).