Beyond Saturated Scores: A Leakage-Aware and Disparity-Audited Evaluation Protocol for Crop Recommendation and Yield Estimation

Abstract

Crop recommendation and yield estimation are usually reported as two unrelated machine-learning results, and both look close to perfect. We ask how much of that skill is still there once the evaluation is made stricter. We start from an exact reproduction of a published dual-model Random Forest framework (99.55% recommendation accuracy; yield R²=0.948  ) and add LEAF-Audit, a four-stage protocol made of Leakage quantification, Evaluation under distribution shift, Attribution of group disparity and Feasibility coupling. On the same two public datasets, recommendation accuracy is saturated: Random Forest (99.52%) and Naive Bayes (99.47%) are equivalent within a ±0.5-point margin, and the data look augmented. For yield, 96.8% of random-split test rows come from district–crop–season series that the training set already contains, and a historical-mean lookup almost matches Random Forest on the log scale (  0.947 against 0.962). Under a temporal split, Random Forest (log-scale ) falls behind the historical mean (0.90). XGBoost with native categorical splits (0.92) improves on that mean by 0.15 in mean absolute error, with an interval that excludes zero under district, state and year clustering, and it also beats the last-year and rolling-mean floors under district and year clustering. The same booster with integer-coded categories, and Random Forest with one-hot or target-encoded categories, do not. Across six geographic zones the worst-to-best normalised-error ratio is 1.79 (95% interval 1.64–1.96), far above a permutation null (95th percentile 1.18). We also propose a levelling-down index. Over five seeds, adaptive difficulty-based reweighting lowers the zone and cell ratios, but 79–82% of the log-ratio reduction comes from raising error in the best-served cells; inverse-frequency reweighting does not lower the ratios. A precedent filter leaves 6.9% of records without a recommendation and mostly blocks correct, newly adopted crops. What we contribute is a reusable audit protocol, not a new learner.