South Africa has only recently introduced national systemic assessments at Grades 3, 6 and 9 into policy and conducted its first national assessment (Grade 3). Prior to this, South Africa had no systemic monitoring of the education system's quality apart from the results of matriculation examinations. Therefore when South Africa participated in the Third International Mathematics and Science Study (TIMSS) in 1995, it was the first opportunity for the country to gain a national overview of its learners'performance as well as an international comparative perspective. South Africa also participated in the repeat of TIMSS in 1999 (TIMSS'99) and in both studies the performance was extremely poor compared with that of other countries. These two studies, both conducted under the auspices of the IEA (International Association for the Evaluation of Educational Achievement), provided an opportunity for South Africa to obtain a national assessment of their learners' performance in mathematics and science. Furthermore given the fact that the majority of learners are not educated in their home language, but in an alternative language, South African learners also completed a Language proficiency test in English in TIMSS'99, allowing researchers the opportunity to explore the relationship between English language and mathematics and science achievement. As background questionnaires are also administered to the students, teachers and school principals in IEA studies it is possible to explore the relationship between contextual factors on school, classroom and student-level and mathematics achievement. The research reported in this article illustrates the relevance of conducting national assessments, especially within the context of international evaluations of educational achievement. Partial Least Squares analysis was used to explore the relative contribution of these contextual factors to learners' achievement together with other background variables from the student, teacher and principal questionnaires. Thereafter multilevel analysis was employed whereby a 2-level model (school and class-level and the student-level) was analysed in order to investigate the main factors explaining achievement of South African learners in mathematics. The study revealed that a number of background variables on student and class-level were found to be significant. Amongst these the learners' proficiency in English was a strong predictor of their success in mathematics. However, home language and class size were among those that were not found to have significant effect on achievement, whilst the effect of socio-economic status (SES) had a lesser effect once certain class-level factors were taken into consideration.