Rice, wheat and maize are the main staple food crops to ensure the food security in China with diversified climate condition, cropping system and environmental and socio-economic factors across provinces. Spatial variation of technical efficiency in farmers' field is helpful to understand the potential to improve farmers' yield given the inputs level and reduce the yield gap. The study is based on a large-scale farm household survey which covered 1218 rice farmers, 3566 wheat farmers and 2111 maize farmers in the main producing areas. The results indicate that rice farmers are with very high technical efficiency level, nearly 0.9 on average, with little room to improve the efficiency of agricultural inputs. Similar results have been found in wheat and maize farmers' fields, although the technical efficiency levels are lower than that of rice farmers while still at a high level with obvious variation across regions. Farmers with higher yield level also achieve better technical efficiency in most locations. Both local environmental and socio-economic factors significantly affect farmers' technical efficiency. In the context of urbanization and economic development, improved and new agricultural technologies need to be prioritized and facilitated to improve cereal yield at farm level.