العنوان "قراءة مقالات لك". مارس 2020. الجزء 1

مرحبا يا هابر! نواصل نشر مراجعات للمقالات العلمية من أعضاء مجتمع Open Data Science من قناة #article_essense. إذا كنت ترغب في استقبالهم قبل أي شخص آخر - انضم إلى المجتمع !

مقالات اليوم:

1. تصنيف وترتيب قابل للتمييز السريع (Google Brain، 2020)
2. MaxUp: A Simple Way to Improve Generalization of Neural Network Training (UT Austin, 2020)
3. Deep Nearest Neighbor Anomaly Detection (Jerusalem, Israel, 2020)
4. AutoML-Zero: Evolving Machine Learning Algorithms From Scratch (Google, 2020)
5. SpERT: Span-based Joint Entity and Relation Extraction with Transformer Pre-training (RheinMain University, Germany, 2019)
6. High-Resolution Daytime Translation Without Domain Labels (Samsung AI Center, Moscow, 2020)
7. Incremental Few-Shot Object Detection (UK, 2020)

1. Fast Differentiable Sorting and Ranking

: Mathieu Blondel, Olivier Teboul, Quentin Berthet, Josip Djolonga (Google Brain, 2020)

: ( belskikh)

Google Brain (O(n logn) , O(n) ) end2end . : differentiable Spearman’s rank correlation coefficient and soft least trimmed squares

(permutations). , (permutohedron) ( n- , n- , n! , ). .

:

1. Top-k classification loss function
2. Label ranking via soft Spearman’s rank correlation coefficient
3. Robust regression via soft least trimmed squares

Top-k classification loss function
CIFAR-10 CIFAR-100 (4 Conv2D with 2 max- pooling layers, ReLU activation, 2 fully connected layers with batch norm on each) .

r_Q r_E. , (nlogn vs n^2)

Label ranking via soft Spearman’s rank correlation coefficient , Robust regression via soft least trimmed squares -k , . , .. ..

, , — , .

2. MaxUp: A Simple Way to Improve Generalization of Neural Network Training

: Chengyue Gong, Tongzheng Ren, Mao Ye, Qiang Liu (UT Austin, 2020)

: ( belskikh)

, -1 85.5% 85.8% (EfficientNet B-8) .

: m , ( ), . ( , ).

, Maxup — gradient-norm regularization . Maxup , .

ResNet-50 , . CutMix+MaxUp EfficientNet B-7 85.8% -1. .

NLP — Penn Treebank WikiText-2.

Adversarial Certification ( adversarial .

MaxUp (, BERT), transfer semi-supervised learning.

3. Deep Nearest Neighbor Anomaly Detection

: Liron Bergman, Niv Cohen, Yedid Hoshen (Jerusalem, Israel, 2020)

: ( belskikh)

kNN - () SOTA anomaly detection , group anomaly detection.

kNN anomaly detection.

1. , kNN.

   
   

2. Semi-supervised Anomaly Detection:

   
   
   • , "";
   • , K kNN;
   • , / .
   
   

3. Unsupervised Anomaly Detection:

   
   
   • ;
   • , , , kNN ( 50%);
   • "", .1 (Semi-supervised Anomaly Detection).
   
   

4. Group Image Anomaly Detection:

   
   
   • , , ( , );
   • ;
   • kNN .1.
   
   

, () ( DN2 (Deep Nearest-Neighbors)).

:

• , ( );
• K 2.

group anomaly detection mean ( max concat).

: , ResNet , - angular , . " - kNN -", .

4. AutoML-Zero: Evolving Machine Learning Algorithms From Scratch

: Esteban Real, Chen Liang, David R. So, Quoc V. Le (Google, 2020)
:: GitHub project
: ( belskikh)

AutoML — , , ML — : , , , , .. — .

— . , . "typically learned by high-school level". , .

- — Setup, Predict Learn. evaluation , , .

, , , :

1. / ;
2. ;
3. .

CIFAR-10 MNIST . , , .

. :

1. , ;
2. ;
3. ( );
4. , , , .

( ) Figure 5 , , , :

1. 
   , noisy ReLU.
2. /
   LR decay.
3. ( 10 CIFAR, 2 )
   . , , - .

- , , AutoML-Zero -, , .

5. SpERT: Span-based Joint Entity and Relation Extraction with Transformer Pre-training

: Markus Eberts, Adrian Ulges (RheinMain University, Germany, 2019)
:: GitHub project
: ( Vadbeg)

, , entity recognition, relation classification.

Span-based – (entity). , . :

1. , ( entity) .
2. ( , ) .
3. pre-trained BERT ( –> ).

:

BERT. byte-pair encoded ( treehouse, a tree house). . BERT n + 1 (n – ), (n + 1)’ . , . : {we, will, rock, you} {we}, {we, will}, {will, rock, you}, ..

:

1. Span classification:

   
   
   • (, s3), . maxpooling.
   • width embeddings . – . , : - , .
   • width embedding maxpooling .
   • (n + 1)’ BERT. .
   • softmax classifier. span’ ( None)
   
   

2. Span filtering:
   . None spans , 10.

   
   

3. Relation classification:

   
   
   • , , . 1.c
   • –> (). maxpooling. -> .
   • ( ). sigmoid-layer classifier. . None.
   
   

. entity recognition relation classification ( ). . . , 100 .

( ). ( , ).

BERT. pretrained BERT ( — SciBERT).

. SOTA :

• Relation Extraction on CoNLL04;
• Relation Extraction on ADE Corpus ;
• Joint Entity and Relation Extraction on SciERC.

6. High-Resolution Daytime Translation Without Domain Labels

: Ivan Anokhin, Pavel Solovev, Denis Korzhenkov, Alexey Kharlamov, Taras Khakhulin, Alexey Silvestrov, Sergey Nikolenko, Victor Lempitsky, Gleb Sterkin (Samsung AI Center, Moscow, 2020)
:: Video :: Blog
: ( dkorzhenkov)

HiDT — image-to-image , .

: — , — .

img2img translation , :

1. CycleGAN , , ( , - ).
2. UNIT ( , — “ ”. , — ).
3. MUNIT ( , — , )
4. FUNIT . , . . . — .

?

( , — , ). HiDT . , , , ( , ).

, , — “ ”, . — , ( — projection discriminator).

, , . hi-res , lo-res multiframe image restoration.

, . artistic style transfer.

style transfer: — , o — .

7. Incremental Few-Shot Object Detection

: Juan-Manuel Perez-Rua, Xiatian Zhu, Timothy Hospedales, Tao Xiang (UK, 2020)

: ( belskikh)

CentreNet Incremental Few-Shot Detection (iFSD) ( ). few-shot -, + .

anchor-box free CentreNet ( Objects as Points), , Hourglass () , . . 3 .

, , ( , . Few Shot Vid-to-Vid), .

, / :

1. CentreNet .
2. , , .
3. - , — .
4. , . .. ground truth , 1, ground truth . L1 ground truth.
5. .

-, 1, 5 10 . + - .

.

7 2- .