DeepPavlov: Keras for Natural Language Processing helps answer questions about COVID-2019

In an area of ​​deep learning such as image processing, the Keras library plays a key role, drastically simplifying transfer learning and using pre-trained models. In the field of natural language processing (NLP), to solve fairly complex problems, such as answering questions or classifying intentions, you have to combine a series of models. In this article, we will show how the DeepPavlov library simplifies the construction of model chains for NLP. Based on DeepPavlov and using Azure ML, we will build a question-answer neural network trained on the COVID-19 data set.

, , , BERT. BERT, , , , .

BERT , , BERT . , , , , — , — , TF-IDF ( ) .

, . , :

• BERT,
• ,

.

DeepPavlov

DeepPavlov . :

• ;
• , config-;
• , ;
• Python SDK .

NLP . REST API Microsoft Bot Framework. , DeepPavlov , Keras .

DeepPavlov - demo.deeppavlov.ai.

BERT DeepPavlov

BERT. DeepPavlov , , Twitter. chainer , :

• simple_vocab (y), , (y_ids);
• transformers_bert_preprocessor x BERT;
• transformers_bert_embedder BERT-
• one_hotter y_ids one-hot encoding, ;
• keras_classification_model — , CNN ;
• proba2labels — , .

:

• dataset_reader — ;
• train — ;
• .

, :

python -m deeppavlov install sentiment_twitter_bert_emb.json
python -m deeppavlov download sentiment_twitter_bert_emb.json
python -m deeppavlov train sentiment_twitter_bert_emb.json

install (, Keras, transformers ..), download , .

, :

python -m deeppavlov interact sentiment_twitter_bert_emb.json

Python SDK:

model = build_model(configs.classifiers.sentiment_twitter_bert_emb)
result = model(["This is input tweet that I want to analyze"])

: ODQA

, BERT, , , ODQA (Open Domain Question Answering). ODQA , Wikipedia. , , . BERT .

, ODQA :

• , .. ;
• BERT-, — .

ODQA DeepPavlov , R-NET, BERT. , , ODQA BERT. "" COVID-19 OpenResearch Dataset, 52 000 COVID-19. , .

Azure ML

Azure Machine Learning, Notebooks. AzureML — Dataset. COVID-19 Semantic Scholar. , JSON-.

Azure ML Dataset. Azure ML Portal, Datasets from web files. file, . tabular, . URL, .

, , . , notebook compute, . ODQA , Azure ML NC12 112 . .

:

from azureml.core import Workspace, Dataset
workspace = Workspace.from_config()
dataset = Dataset.get_by_name(workspace, name='COVID-NC')

.tar.gz. , UNIX:

mnt_ctx = dataset.mount('data')
mnt_ctx.start()
!tar -xvzf ./data/noncomm_use_subset.tar.gz
mnt_ctx.stop()

. noncomm_use_subset .json, abstract body_text. , Python-:

from os.path import basename
def get_text(s):
    return ' '.join([x['text'] for x in s])

os.makedirs('text',exist_ok=True)

for fn in glob.glob('noncomm_use_subset/pdf_json/*'):
    with open(fn) as f:
        x = json.load(f)
    nfn = os.path.join('text',basename(fn).replace('.json','.txt'))
    with open(nfn,'w') as f:
        f.write(get_text(x['abstract']))
        f.write(get_text(x['body_text']))

text, . :

!rm -fr noncomm_use_subset

ODQA

, ODQA DeepPavlov. en_odqa_infer_wiki:

import sys
!{sys.executable} -m pip --quiet install deeppavlov
!{sys.executable} -m deeppavlov install en_odqa_infer_wiki
!{sys.executable} -m deeppavlov download en_odqa_infer_wiki

. , , , . !

, :

from deeppavlov import configs
from deeppavlov.core.commands.infer import build_model
odqa = build_model(configs.odqa.en_odqa_infer_wiki)
answers = odqa([ "Where did guinea pigs originate?",
                 "When did the Lynmouth floods happen?" ])

:

['Andes of South America', '1804']

, Wikipedia. , :

• What is coronavirus? — a strain of a particular virus
• What is COVID-19? — nest on roofs or in church towers
• Where did COVID-19 originate? — northern coast of Appat
• When was the last pandemic? — 1968

, … , , . — .

(ranker), . DeepPavlov. ODQA en_ranker_tfidf_wiki, data_path, , :

from deeppavlov.core.common.file import read_json
model_config = read_json(configs.doc_retrieval.en_ranker_tfidf_wiki)
model_config["dataset_reader"]["data_path"] = os.path.join(os.getcwd(),"text")
model_config["dataset_reader"]["dataset_format"] = "txt"
model_config["train"]["batch_size"] = 1000

, .

, :

doc_retrieval = train_model(model_config)
doc_retrieval(['hydroxychloroquine'])

, .

ODQA , :

# Download all the SQuAD models
squad = build_model(configs.squad.multi_squad_noans_infer, download = True)
# Do not download the ODQA models, we've just trained it
odqa = build_model(configs.odqa.en_odqa_infer_wiki, download = False)
odqa(["what is coronavirus?","is hydroxychloroquine suitable?"])

:

['an imperfect gold standard for identifying King County influenza admissions',
 'viral hepatitis']

…

BERT Q&A

DeepPavlov , Stanford Question AnsweringDataset (SQuAD): R-NET BERT. R-NET. BERT. squad_bert_infer - BERT:

!{sys.executable} -m deeppavlov install squad_bert_infer
bsquad = build_model(configs.squad.squad_bert_infer, download = True)

ODQA, :

{
   "class_name": "logit_ranker",
   "squad_model": 
    {"config_path": ".../multi_squad_noans_infer.json"},
   "in": ["chunks","questions"],
   "out": ["best_answer","best_answer_score"]
}

, multi_squad_noans_infer. ODQA, squad_model squad_bert_infer:

odqa_config = read_json(configs.odqa.en_odqa_infer_wiki)
odqa_config['chainer']['pipe'][-1]['squad_model']['config_path'] = 
                    '{CONFIGS_PATH}/squad/squad_bert_infer.json'

, :

odqa = build_model(odqa_config, download = False)
odqa(["what is coronavirus?",
      "is hydroxychloroquine suitable?",
      "which drugs should be used?"])

, :

, Azure Machine Learning NLP DeepPavlov - . DeepPavlov , , , . COVID Kaggle , , DeepPavlov Azure Machine Learning. , DeepPavlov – .

Azure ML DeepPavlov . , . . Data Science , , , !