Evaluating AI’s Effectiveness in Content Analysis: A Linguistic Comparison of Automated and Human Text Interpretation in the Aurat March Discourse on Social Media

Abstract

This study evaluates the effectiveness of AI in analyzing the emotional, cultural, and ideological aspects of feminist discourse in Pakistan. It compares human and AI interpretations of online discussions related to the Aurat March on Facebook, Instagram, and YouTube. Using qualitative methodology supported by relational content analysis (RCA) and Technofeminism (Wajcman, 2004) as a theoretical framework, the research examines how human and the ChatGPT-5 model interpret emotional tone, sarcasm and moral framing in multilingual comments written in English, Urdu and Roman Urdu. The results shows that while the AI efficiently detects explicit emotions in English, it fails in understanding sarcasm, irony and moral expressions embedded in Urdu and Roman Urdu. AI misinterpreted phrases with cultural and religious connotations, such as “khuda ka wasta hai” or “drama karna,” while human interpreters recognized their emotional and contextual depth. This shows the limitations of AI's ability to deal with culturally specific discourse. The study supports Technofeminist arguments that technology reflects the social and gender biases of its developers. The gap in AI highlights the persistence of algorithmic bias and the dominance of Western linguistic frameworks in language models. This research contributes to studies of feminist discourse and AI ethics by proposing a hybrid model that integrates human interpretation with AI-assisted sentiment analysis. This approach combines the efficiency of AI with the sensitivity of human understanding of context. The study concludes that although AI can quickly process large data sets, it cannot yet replace human understanding when analyzing emotionally and culturally complex feminist communication. The study calls for the development of AI systems that take into account cultural, gender and context aspects, and are able to interpret various linguistic facts in digital spaces.