Teff (Eragrostis tef (Zucc.) Trotter) is a major food crop in Ethiopia and Eritrea. It is well adapted to Vertisols. Yields are low (around 1000 kg ha−1) despite fertilization with urea and diammonium phosphate. The objectives of this study were to understand farmers’ perception on teff production constraints and to evaluate on-farm yield response of teff to zinc (Zn) fertilization. We conducted a farm survey and a participatory fertilization experiment in three teff-based sites (peasant associations) on Vertisols in the mid highland and lowland agroecological zones in Ethiopia. Per site 10 farmers participated in the survey and on-farm experiment. Poor soil fertility in the mid highland and moisture deficit in the lowland agroecological zones were mentioned by farmers as major teff production constraints, respectively. On-farm application of Zn fertilizer at a rate of 8 kg Zn ha−1 increased teff grain and straw yields by 14% and 15% on average, respectively, which could be economically profitable. Not all plots showed a positive response, however, indicating the necessity for enhanced insight in indicators for soil Zn bioavailability as a yield-limiting factor. Our study indicates the importance of Zn in teff production on Vertisols. We propose further research on management options to prepare for effective interventions based on the farm survey and on-farm experiment.