This study explores the impact of using assessment items with competing theories to encourage students to practice evaluative reflection and collaborative argumentation in asynchronous discussions. Thirty undergraduate students from various departments worked in small groups and took turns collaboratively discussing the given item’s answer, reaching a consensus, and posting their consensual answer on the web. The remaining participants served as evaluators to reflect on the answer and provide comments. It was found that the students made significant progress in argumentation ability and conceptual understanding of related scientific content knowledge. In the beginning of the study, the group of students majoring in the sciences outperformed counterparts with non-science majors on the level of understanding of the assessment item’s scientific concepts. At the end of the semester, the differences diminished between the two groups both on conceptual understanding and in argumentation ability.