2019 DS/ML digest 16

2019 DS/ML digest 16

Posted by snakers41 on October 3, 2019

Speech

NLP

  • Use GPT-2 for steganography !
  • Russian sentiment analysis library based on FastText and some small corpus;
  • 10x smaller transformer. Looks amazingly similar in essence to classic LSTM-like models … it shares weights between layers. Really looking forward to trying this as drop-in replacement to some of our RNN units;

Coding

  • What is new in python 3.8
  • VS Code becomes better;

ML

CV

  • Facebook develops fashion++ for fashion look improvements
  • Very cool real life application of semseg in trains
  • Very cool idea - a residual block composed of multiple convolutional feature streams
  • Deconstruction of the convolution:
    • Shift layer
    • Active Shift Layer
    • TLDR - same as depthwise separable convolutions, but uses (learnable) shift operator instead of first convolution
    • Downside - there is no mainsteam / proper PyTorch implementation that does not require custom CUDA kernels
  • Using narration in instructional videos for cross-modal training

Datasets

  • FaceForensics - a deep fake detection dataset. The dataset itself is behind some academic registration wall, but they do not check what you type in;
  • Image Harmonization dataset
  • GitHub code search challenge - very cool - but train / test relation is kind of murky;
  • Google releases parahprase dataset - PAWG