Agritech/China
R-squared 0.844 in the paper, 0.131 in the way you would actually use it
The same rice yield model scored R-squared 0.844 under random cross-validation and 0.131 when tested on an unseen site and season.
Using 3,204 yield records from 168 rice cultivars across 10 sites, 7 years and 67 site-year environments in southern China, researchers ran the same models through five validation schemes. Random five-fold cross-validation gave R-squared 0.844. Forward-year prediction gave 0.359, leave-one-site-out 0.290, and leave-one-site-year-out 0.131.
What happened
Search for crop yield prediction papers and you will find R-squared values above 0.8 reported routinely. Those numbers get quoted to farmers, to agricultural ministries and to investors, and they are the basis on which precision agriculture products are sold.
Almost all of them come from random cross-validation: shuffle the records, hold some out, train on the rest, test on the held-out portion.
A team at the National Engineering Research Center of Plant Space Breeding at South China Agricultural University, with colleagues at the Anyang Institute of Technology and the College of Artificial Intelligence and Low Altitude Technology, took 3,204 yield records from 168 cultivars across 10 sites, 7 years and 67 site-year environments and ran them through five different validation schemes.
Random cross-validation gave R-squared 0.844. Predicting an unseen site in an unseen year gave 0.131.
The test
Five validation schemes were applied to the same data and the same five feature sets.
Random five-fold cross-validation shuffles everything, which allows records from the same site and season to appear in both training and test data.
Leave-one-year-out holds out an entire year. Forward-year prediction trains on past years and predicts the next one, which is how a model would actually be used. Leave-one-site-out holds out a location. Leave-one-site-year-out holds out a specific site in a specific year – the hardest and most realistic test.
The five feature sets covered whole-season weather, stage-specific climate information, leakage-controlled cultivar yield BLUP, field-survey traits, and heading-stage updates. That structure lets the paper ask not just how well the model does, but which information helps at which point in the season.
The numbers
| Validation scheme | What it simulates | Macro mean R-squared |
|---|---|---|
| Random five-fold cross-validation | nothing realistic | 0.844 (best model) |
| Forward-year prediction | predicting next season | 0.359 |
| Leave-one-site-out | predicting a new location | 0.290 |
| Leave-one-site-year-out | new location, new season | 0.131 |
| RMSE, best random CV model | 340.59 kg per hectare | |
| Records / cultivars | 3,204 / 168 | |
| Sites / years / site-year environments | 10 / 7 / 67 | |
The result
Random five-fold cross-validation produced the highest apparent accuracy, with the best field-survey-enhanced model reaching R-squared 0.844 and RMSE 340.59 kilograms per hectare.
Deployment-oriented performance was lower in every case, and the drop grew with how realistic the test was: 0.359 for forward-year prediction, 0.290 for leave-one-site-out, 0.131 for leave-one-site-year-out.
The feature analysis is useful in its own right. Stage-specific climate information and leakage-controlled cultivar potential improved prediction before field-survey traits were available, while field-survey traits provided the strongest later-stage accuracy. Reduced training size lowered accuracy and increased the burden of unseen cultivars. Climate-risk environments showed risk-specific error shifts rather than uniform model failure.
The authors are explicit about what they are claiming: the study supports a deployment-oriented framework rather than making a general claim of universal out-of-sample improvement, and its main value is showing how prediction skill, feature utility and model interpretation change when validation is aligned with realistic deployment.
How this compares to what came before
This is not a new criticism in principle. Spatial and temporal leakage in geoscience and agricultural machine learning has been written about for years, and there are methodology papers arguing exactly this point.
What is new is the demonstration at this scale on real multi-year, multi-site trial data, with the same models and features held constant across five schemes. That turns a methodological argument into a measurement: a factor of six, on this dataset.
It also goes further than the usual critique by showing which information retains value under realistic validation. That is constructive rather than purely corrective, and it gives a practitioner something to do.
What it means
For anyone reading crop yield prediction claims, the validation scheme is now the first thing to check, and a random cross-validation R-squared should be read as an upper bound that bears little relation to deployment.
For anyone building these models, the finding about feature timing is the actionable part. Stage-specific climate information and cultivar potential carry useful signal early in the season, when a farmer can still act. Field-survey traits are more accurate and arrive too late to change much.
For the broader machine learning field, this is a clean, well-measured instance of a general problem. Any domain where observations cluster by place and time – epidemiology, energy forecasting, traffic, finance – has the same leakage risk, and random cross-validation will flatter the model in exactly the same way.
The honest reading of 0.131 is that predicting rice yield for a new field in a new season from this kind of data is largely unsolved.
Business ideas from this paper
Validation auditing for agricultural AI. Any precision agriculture vendor quoting an R-squared can be asked which scheme produced it. An independent audit that re-runs their model under deployment-aligned validation would be uncomfortable and valuable.
Early-season decision products. The feature analysis says stage-specific climate and cultivar potential are what you have early. A product built around what is genuinely knowable before the season ends is more honest and more useful than a yield number.
Site-year transfer research. The gap between 0.359 and 0.131 is the cost of a new location. Closing it – through transfer learning, site characterisation or local calibration – is the research problem with the clearest commercial payoff.
Benchmark datasets with deployment splits. Publishing the data with the site-year splits predefined would stop the next hundred papers from reporting inflated numbers. That is a small piece of infrastructure with a large effect.
What could be wrong with it
This is a validation study, not a field trial. No farmer changed anything because of these predictions, and the paper does not claim otherwise.
The data come from official national variety registration trials in the Southern Rice Region of China. Those are managed plots with controlled agronomy, not commercial fields, and commercial variation would likely be larger.
3,204 records across 67 site-year environments is substantial and regionally bounded. Whether the same collapse occurs in other crops and regions is untested here, though there is no reason to expect otherwise.
The raw records are administered by the national body that compiles the trial reports rather than openly published, so the analysis cannot be independently rerun.
How sure can you be?
We score this 8 out of 10.
It is a peer-reviewed paper in Frontiers in Plant Science, built on multi-year multi-site data from official national variety registration trials – which is independent, systematically collected data rather than the authors’ own field plots – with named institutions at South China Agricultural University and the Anyang Institute of Technology.
It loses points because the underlying records are not openly available and the evidence is a retrospective validation study rather than a field experiment.
The core finding is robust because it is internal: the same data, the same models, five splits. That comparison does not depend on anything outside the paper.
Do this today
If you are evaluating any yield prediction product, ask which validation scheme produced the quoted accuracy. If the answer is random cross-validation, treat the number as a ceiling and ask for the forward-year figure.
If you build predictive models on data that clusters by place and time, run a leave-one-group-out validation alongside your standard one. The difference between them is the honest measure of what your model knows.
If you are a grower being sold a yield model, ask what it scored on fields it had never seen, in a season it had never seen. That is the only number that describes what you would be buying.
Source: Deployment-oriented evaluation of stage-specific information for rice yield prediction across years, sites and site-year environments in southern China, Frontiers in Plant Science, September 2026. PMID 42774268 · doi.org ().
Just Out Tech explains new research in plain language. This article was drafted with AI assistance and checked by a human against the original source.
- Random five-fold cross-validation produced R-squared 0.844 with RMSE 340.59 kg per hectare.
- The same information scenario gave macro mean R-squared of 0.359 forward-year, 0.290 leave-one-site-out, 0.131 leave-one-site-year-out.
- Stage-specific climate and leakage-controlled cultivar potential helped early; field-survey traits helped late.
- The authors frame their contribution as showing how skill changes with validation, not as a better model.
The work uses 3,204 records from official national variety registration trials across 10 sites and 7 years, and compares five validation schemes on the same data.
Questions people ask
What is leave-one-site-year-out validation?
Hold out every record from one site in one year, train on the rest, and predict that held-out combination. It simulates predicting a new field in a new season, which is the actual deployment case.
Why does random cross-validation inflate the number?
Because records from the same site and year end up in both training and test sets. The model can learn that this particular field in this particular season yields well, which it cannot do for a season that has not happened.
Is 0.131 a useless model?
It is a weak one. R-squared of 0.131 means the model explains about 13 percent of the variation in yield for an unseen site-year, which is better than nothing and far from a decision tool.
How big was the dataset?
3,204 yield records from 168 cultivars across 10 sites, 7 years and 67 site-year environments, drawn from official national variety registration trial reports for the Southern Rice Region of China.