2019 DS/ML digest 06

2019 DS/ML digest 06

Posted by snakers41 on March 18, 2019

NLP

  • This kind of went below my radar and I am 2 years late, but we have 4 main alertnatives in NLP:
    • LSTM / GRU;
    • TCN / Trellis;
    • Transformer;
    • And QRNN - interesting thread:
      The QRNN provides similar accuracy to the LSTM but can be betwen 2 and 17 times faster than the highly optimized NVIDIA cuDNN LSTM implementation depending on the use case.
      And yeah, looks like they need cupy for some of their layers … meh - turns out it is not a plug and play replacement for LSTM;
  • Google’s new mobile STT?;

Articles / blog posts / etc

  • TF for microcontrollers;
  • RNN based handwriting recognition by Google: strokes => Bezier curves => RNN / QRNN => decoder;
  • Google’s 3D face mesh detection: our ML pipeline consists of two real-time deep neural network models that work together: A detector that operates on the full image and computes face locations, and a generic 3D mesh model that operates on those locations and predicts the approximate surface geometry via regression;
  • Google open-sources TF-Replicator for multi-GPU;
  • Hierarchical RL?;
  • RNN based lemmatization;
  • Pandas scaling;
  • Open AI is a for profit company now;
  • Facebook had a barrage of posts about their datacenters / hardware acceleration:
    • Unification of hardware acceleration slots;
    • Facebook’s ML inference;
    • Hm, this is why they needed all of this PyTorch 1.0 stuff:
  • Use pre-existing ontologies as a source of annotation;
  • Metaclasses in python;
  • Python 3.7 cool features: data class, order of dicts, importlib.resources, asyncion enhancements;
  • Intro to asyncio now even with zero boilerplate?;
  • DVC walk-through;
  • Why framework porting does not work in ML (RU);

Datasets

  • New large (500k images) fashion image dataset;
  • Looks like BigEarth is being released soon;

Papers