2020 DS/ML digest 05

2020 DS/ML digest 05

Posted by snakers41 on April 9, 2020

Speech

Towards End-to-end ASR - a prez by Google - https://drive.google.com/file/d/1Rpob1-C223L9UWTiLJ6_Dy12mTA3YyTn/view
Improving voice call quality with WaveRNN - https://ai.googleblog.com/2020/04/improving-audio-quality-in-duo-with.html
Going beyond human capabilities on many ML tasks using GAFA scale compute - https://arxiv.org/abs/1909.01736

NLP

A new review article on transformers - https://lilianweng.github.io/lil-log/2020/04/07/the-transformer-family.html - mostly very SOTA stuff, but cool ideas

ML

A practical review of mobile architectures - https://machinethink.net/blog/mobile-architectures/
Machine learning algo calibration (RU)
A Neural Weather Model for Eight-Hour Precipitation Forecasting - https://ai.googleblog.com/2020/03/a-neural-weather-model-for-eight-hour.html
Once again stumbled upon this - https://github.com/mapillary/inplace_abn - noticed this pattern used in a brand new TResNet. Looks like it matters for CV a lot


A Step Towards Protecting Patients from Medication Errors https://ai.googleblog.com/2020/04/a-step-towards-protecting-patients-from.html

  • An LSTM
  • Nearly all (93%) top-10 lists contained at least one medication that would be ordered by clinicians for the given patient within the next day.
  • Fifty-five percent of the time, the model correctly placed medications prescribed by the doctor as one of the top-10 most likely medications
  • and 75% of ordered medications were ranked in the top-25.
  • Even for ‘false negatives’ — cases where the medication ordered by doctors did not appear among the top-25 results — the model highly ranked a medication
  • in the same class 42% of the time. This performance was not explained by the model simply predicting previously prescribed medications. Even when we blinded the model to previous medication orders, it maintained high performance.


Graphcore IPUs review https://arxiv.org/pdf/1912.03413.pdf. TLDR - roughly similar to Tesla V100, but consumes much less electricity. They have Python library, plan PyTorch support, looks like they are really shipping something. Also likely that their accelerators will be single-slot.
Plain thing - mahalanobis distance - basically eucledian distance with normalization
How heap sort works https://habr.com/ru/company/edison/blog/495420/
Read Ben Evans if you do not yet https://mailchi.mp/4967d3a66e90/benedicts-newsletter-no-451085?e=b7fff6bc1c
Yeah, Google sold Boston Dynamics https://ai.googleblog.com/2020/04/exploring-nature-inspired-robot-agility.html

Papers

TResNet: High Performance GPU-Dedicated Architecture http://arxiv.org/abs/2003.13630

  • Some new tricks for optimizing mostly CV networks
  • Result - even more efficient network than EfficientNet, also more suitable for training fast on GPU
  • Ideas: SpaceToDepth stem, Anti-Alias downsampling, In-Place Activated BatchNorm, Blocks selection and SE layers

Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?

  • https://arxiv.org/abs/2003.11539v1
  • Learning a supervised or self-supervised representation on the meta-training set, followed by training a linear classifier on top of this representation
  • Using a good learned embedding model can be more effective than sophisticated meta-learning algorithms
  • Self knowledge distillation

Few-Shot Unsupervised Image-to-Image Translation https://arxiv.org/pdf/1905.01723.pdf

Sending your DNS queries
Going beyond human capabilities on many ML tasks using GAFA scale compute - https://arxiv.org/abs/1909.01736
Image matting w/o green screen https://arxiv.org/pdf/2004.00626.pdf

Datasets

Some tools were built to work with Waymo dataset - https://medium.com/snarkhub/extending-snark-hub-capabilities-to-handle-waymo-open-dataset-4dc7b7d8ab35
A bit of info on how Google does landmark recognition - https://ai.googleblog.com/2020/04/announcing-2020-image-matching.html