This study aimed to cluster the districts in Nepal based on the patterns of age-sex structures by applying factor analysis. The population data, which is grouped by 5-year age gap, by sex and by districts, was used. The factor analysis was applied to spline smooth single-year age population by sex and district. A three factor model was best fitted to the data. These three common factors were interpreted as three different patterns based on common characteristics of age and sex distribution. The study found that 23, 17 and 5 districts correlated purely to factor 1, 2 and 3, respectively. Thirty districts were found correlated with two or more factors. In conclusion, the age-sex structure varied substantially between the different districts of Nepal in 2011. The variations were explained well by a three-factor model. The method used in this study is straightforward and applicable to the further demographic study.