Week from 05/30 to 06/06

This week I started evaluating existing language detection tools like openNLP, Tika and langdetect. These tools are evaluated on qald test datasets.

Why? 

As the approach for extending the NSpM framework to support multiple languages; first, we must identify the source language. For that, we developed a language detector past week, which can detect languages of small sentences. To compare how our model and existing language models perform on qald datasets and choose a better tool for the use of our project.

How?

First I extracted the qald test and training datasets using IRbench and then created csv files using them. The qald training datasets consisted of these languages,

QALD 3 : English, Deutsch, French, Spanish, Italian and Dutch.
QALD 4 : English, Deutsch, French, Spanish,  Italian, Dutch and Romanian.
QALD 5 : English, Deutsch, French, Spanish, Italian, Dutch and Romanian.
QALD 6 : English, Deutsch, French, Spanish, Italian, Dutch, Romanian and Persian.
QALD 7: English, Deutsch, French, Spanish, Brazilian Portuguese, Italian, Dutch, Hindi, Romanian and Persian.
QALD 8:English, Deutsch, French, Spanish, Italian, Dutch, Hindi, Romanian and Persian
QALD 9: English, Deutsch, French, Spanish, Brazilian Portuguese, Italian, Dutch, Hindi, Romanian, Persian, Portuguese and Russian.

qald test datasets consisted of these languages, 

QALD 3 : English, Deutsch, French, Spanish, Italian and Dutch.
QALD 4 : English, Deutsch, French, Spanish,  Italian, Dutch and Romanian.
QALD 5 : English, Deutsch, French, Spanish, Italian, Dutch and Romanian.
QALD 6 : English, Deutsch, French, Spanish, Italian, Dutch, Romanian and Persian.
QALD 7: English, Deutsch, French, Spanish, Brazilian Portuguese, Italian, Dutch, Hindi, Romanian and Persian.
QALD 8 : English
QALD 9 : English, Deutsch, French, Spanish, Brazilian Portuguese, Italian, Dutch, Hindi, Romanian, Persian, Portuguese and Russian.

This is how the openNLP, Tika and langdetect performed on qald 7 test dataset,

LangTag(s)  : 0.98
openNLP      : 0.90
langdetect   : 0.95
Tika              : 0.91

LangTag(s) is the model we trained on the qald-7-training dataset.

Problems encountered?
  • Uncompleted test datasets.(QALD 8 dataset consisted only English)

End of the week

After discussing with the mentor and the co-mentors we decided on writing a research paper on LangTag.

Comments

Post a Comment

Popular posts from this blog

DBpedia Neural Multilingual QA - GSoC project summery

 GSoC blog This blog includes a detailed description of the work I carried out during the GSoC period. The above mentioned blog includes the tasks carried out on a weekly basis. Brief Description of my work Implement airML using KBox and pip.  This task was to create and distribute KBox  into a pip package called airML, which will allow users to share and dereference ML models Share the Monument dataset with airML. Train the monument dataset and place it in a public repository and dereference it with airML. Create the language detector dataset. Iterate over all questions in IRbench , creating the dataset. Create a language detector model and train it. After creating the model, I evaluated other existing language models and wrote a research paper with my mentor. Experimentation in Machine Translation methods. Creating datasets. Annotating Datasets. Evaluating different methods. The full description of Tasks carried out and the obtained results are included in this blog . The results ob
Read more

Week from 07/12 to 07/18

 Since we are writing a research paper on "Assessing Language Identification Methods and Frameworks over Linked Data", I was tasked to do some work regarding the paper this week. These are the work done in this week, Created a readme file including the results obtained so far the research. The complete results can be found in this Github repo .   Wrote the evaluation part for the research paper including how the experimentation is done. Refactored the code and put a PR to the main repo.
Read more

Machine Translation Task

 This experimentation was done as follows, Dataset Creation using QALD datasets. Train and evaluate the initial dataset on Tensorflow Neural Machine translation with attention  model . Using  DBpedia Spotlight  to annotate the entities and create templates. Manually evaluate the entity recognition process. Train and evaluate the correctly annotated text on Tensorflow Neural Machine translation with attention  model . 1. Dataset Creation using QALD datasets. From the QALD datasets from QALD-3 to QALD-7, I created a dataset which consists of language pairs such as English-Spanish, English-Deutsch, etc. These pairs are created for all languages Deutsch, Spanish, French Brazilian Portuguese, Portuguese, Italian, Dutch, Hindi, Romanian, Persian, and Russian. 2.  Train and evaluate the initial dataset on Tensorflow Neural Machine translation with attention model. Using the " Tensorflow Neural Machine translation with attention model ", trained the initial datasets created as said a
Read more