Week from 05/23 to 05/29

Start of the week

This week I was assigned to develop a Language Detector. 

Why?

As the approach for extending the NSpM framework to support multiple languages; first, we must identify the source language. For that, we must develop a language detector, which can detect languages of small sentences; since the questions are not long texts as documents.

How?

Creating the dataset

The datasets were created using the QLAD-7 datasets which are attained using IRbench here. Since the dataset contained additional information, I extracted the sentences and the languages and created a dataset. 

Testing existing tools

First, I tried out the Apache OpenNLP language detector tool, which claimed it is able to detect 103 languages. But for that, they have required to have at least 2 or more sentences from the same language. Therefore I decided to evaluate QLAD-7 dataset for the languages English, Spanish, German, Italian, French, Dutch, Romanian. The accuracy score was 0.8593. As they mentioned I assume that the low score is a result of shorter text in the QLAD-7 dataset.

Building a Language detector on my own

Since It was a classification problem, I decided to build a simple Naive Bayes classifier. Trained the dataset on QLAD-7 train dataset and tested it in QLAD-8 test dataset. The model accuracy for the test dataset was 0.9883. The progress is in this github repo here. Even the training dataset consisted of sentences of  English, Spanish, German, Italian, French, Dutch, Romanian languages; the test dataset only contained 4 languages. The confusion matrix created using the model predictions.



End of the week

At the end of the week, I discussed my results with my mentors. They advise to research more about language detection and evaluate more existing products.

Comments

Post a Comment

Popular posts from this blog

DBpedia Neural Multilingual QA - GSoC project summery

 GSoC blog This blog includes a detailed description of the work I carried out during the GSoC period. The above mentioned blog includes the tasks carried out on a weekly basis. Brief Description of my work Implement airML using KBox and pip.  This task was to create and distribute KBox  into a pip package called airML, which will allow users to share and dereference ML models Share the Monument dataset with airML. Train the monument dataset and place it in a public repository and dereference it with airML. Create the language detector dataset. Iterate over all questions in IRbench , creating the dataset. Create a language detector model and train it. After creating the model, I evaluated other existing language models and wrote a research paper with my mentor. Experimentation in Machine Translation methods. Creating datasets. Annotating Datasets. Evaluating different methods. The full description of Tasks carried out and the obtained results are included in this bl...
Read more

Week from 07/12 to 07/18

 Since we are writing a research paper on "Assessing Language Identification Methods and Frameworks over Linked Data", I was tasked to do some work regarding the paper this week. These are the work done in this week, Created a readme file including the results obtained so far the research. The complete results can be found in this Github repo .   Wrote the evaluation part for the research paper including how the experimentation is done. Refactored the code and put a PR to the main repo.
Read more

Week from 07/19 to 07/25

 This week we started to focus on the Machine Translation task. First I tried to understand how the "Tensorflow Neural Machine Translation" model is implemented. And then trained the model on QALD datasets. How the datasets are created? From the QALD datasets from QALD-3 to QALD-7, I created a dataset which consists of language pairs such as English-Spanish, English-Deutsch, etc. These pairs are created for all languages Deutsch, Spanish, French Brazilian Portuguese, Portuguese, Italian, Dutch, Hindi, Romanian, Persian, and Russian. How the evaluation is done? Using the " Tensorflow Neural Machine translation with attention model ", trained the datasets created as said above and got the following results, language accuracy % spanish 60.6299 german 65.8595 french 63.3587 russian 14.6666 italian 31.6301 portugese 3.33333 pt_BR 4.54545 hindi 37.3333 dutch 61.9422 persian 8.14479 romanian 52.3026 Observations It is observed that the results are very poor. This can be du...
Read more