The potential of legumes to contribute to production, environmental, and nutritional objectives is well known by science and policy. Yet, low shares of grain legumes are observed in current agricultural systems, which is often attributed to high yield variability and yield gaps in legume production. These limitations can be associated with sub-optimal management practices, occurrence of pests and diseases, limiting pedoclimatic conditions, and insufficient farmers’ experience, knowledge, and attitude in growing legumes. In this study, we explore the role of these different factors on the yields of soya and fava beans and, in particular, the role of the ‘knowledge gap’ as we term the difference between yields achieved by experienced farmers and those achieved by novices. A large-scale farmer survey was conducted in nine European countries in southern (France, Spain), central (Germany, Poland), north-western (Finland, Sweden, Latvia), and north-eastern Europe (UK, the Netherlands). Using correlation analysis and stochastic frontier modelling, the collected responses are analysed to explore the effect of different yield gap factors and evaluate, for the first time, the influence of farmers’ knowledge on yield gaps.