Department of Computer Science

Permanent URI for this communityhttps://repository.cuilahore.edu.pk/handle/123456789/16

Browse

Has files: YesSubject: search.filters.subject.Machine Translation

Search Results

Now showing 1 - 2 of 2
  • No Thumbnail Available
    Item
    Data Augmentation using Machine Translation for Urdu Document Classification
    (Library Information Services, COMSATS University Islamabad, Lahore Campus, 2022) Khizer But; FA19-RCS-018; LHR TP 8355; Dr Muhammad Waqas Anwar
    A huge volume of textual data is being created due to the advancement in technology and widespread use of the internet. The exponential growth of textual data in the Urdu language has given rise to its applications in spam filtering, fake news detection, website classification, sentiment analysis, etc. However, Urdu is a low-resource language and lacks the support of automated tools for its applications. This research focuses on using data from a resource-rich language to translate it into the Urdu language to overcome the data scarcity concern. For this purpose, the BBC English news articles dataset was acquired and translated using the Google-trans-new library. Further, an Urdu dataset of fake news was enriched with 1200 articles which were collected from an online resource named Urdu Point. Urdu and machine-translated datasets were merged to come up with a diverse and large augmented dataset and augmented downsized dataset. A supervised classification method was employed with TextCNN, LSTM, simple Bi-LSTM, Bi-LSTM with attention, and SVM on the datasets. Different training experiments were applied to evaluate the effectiveness of the classification with Urdu, machine translated, and augmented datasets. Training and testing on the same dataset gave good results whereas the cross-testing from machine translated to Urdu and Urdu to Machin-translated dataset resulted in a performance decline. Models trained with an augmented dataset achieved promising results when tested on the original Urdu test dataset and results were improved as compared to results achieved with no augmentation. These promising results are suggestive of the applicability of the proposed strategy of creating datasets on large scale using machine translation to automatically classify the documents in the Urdu language. Bi-LSTM resulted in the highest F-1 score of 0.97 with the augmented dataset.
  • No Thumbnail Available
    Item
    Urdu to English Based Unsupervised Machine Translation
    (Library Information Services, COMSATS University Islamabad, Lahore Campus, 2020) Ahmed Raza; SP18-RCS-005; LHR TP 5983; Dr. Muhammad Waqas Anwar
    The process of automatically converting the text from one language to another natural language is Machine Translation. Machine Translation is a subfield of computational linguistic. There are two state-of-the-art machine translation techniques i,e Neural Machine Translation (NMT), and Statistical Machine Translation (SMT). In both techniques, a large corpus is required for the training of the translation model. Urdu counts in low resource languages due to the fewer resources available for computational work. To build a good translation system available resources are not enough. Many languages present in the world have a different structure. Like in Urdu and English, Urdu structure is based on Subject Object-Verb (SOV) and the English structure is based Subject-Verb-Object (SVO). In this study, we presented Urdu to English unsupervised translation model and the practical challenges faced during the work. We try to partially remove the need for parallel corpora and proposed a method to train a Machine Translation System in an unsupervised manner. The proposed system is aimed to provide Urdu to English translation through an unsupervised manner. For this propose, we use Artetxe Author developed a toolkit that is based on Unsupervised Neural Machine Translation (UNMT). This approach tested the models of UNMT which include denoising and on-the-fly back-translation. From denoising model obtain the BLEU score 4.14 and 5.11 for two language pairs UR-EN and EN-UR. From back translation obtain the BLEU score of 5.21 and 6.28 which are better than from the previous score. Back-translation results difference from denoising technique gains +1.07 and +1.17 for two language pairs Urdu to English and English to Urdu. We also faced many challenges during work and effects on pre-processing techniques. Our approach shows promising results in translation of Urdu text into English which is mostly neglected due to its complexities