Developing a Corpus of Paraphrased Plagiarized Documents for Urdu Language

Abstract

Plagiarism (unacknowledged reuse of text), is a wide spread problem around the world, particularly in academia. Research has shown that it is easy to detect exact copy however; it is hard to detect plagiarism when the original text has been heavily paraphrased. The focus of this research is on paraphrased plagiarism detection.

This FYP report describes our work to investigate the problem of paraphrase plagiarism in Urdu language. The main focus of this project was exploring the problem of extrinsic plagiarism detection specially when the rewritten text has been paraphrased. Our first aim is develop Urdu corpus with instances of paraphrased plagiarised documents and non-plagiarised documents. Once the corpus was built, Natural Language Processing (NLP) algorithms were applied to detect the similarity between the original document and the suspicious document. The problem of discriminating paraphrased plagiarised documents from non-plagiarised documents is casted as a supervised classification task using existing Machine Learning (ML) techniques.

Two popular and well know techniques were explored to identify paraphrased plagiarism: (1) N-gram overlap and (2) Longest Common Subsequence (LCS). The similarity between plagiarised and non-plagiarised documents was computed using overlap similarity score for unigrams, bigrams, trigrams, four-grams and five-grams and LCS. These similarity scores were used as features for classification. Evaluation was carried out using precision, recall and F1 measure. Results showed that it is not trivial to detect plagiarism created by paraphrasing the original text.