Speech
- Speed up your CNN speech ASR model 2x by removing batch-norm / skips and adding skips - I tried it, does not work for me;
NLP
- Google does a joint NMT system for all languages at once
- Gradient’s piece of perplexity / metrics for LMs
- http://nlp.fast.ai/classification/2019/09/10/multifit.html
- Google’s search embraces transformers
- Tiny BERT:
- The only problem is that they distill a large model into a small one
- Ofc the tiny model is faster and smaller
- But you need the large model anyway, the do not train a small model from scratch
- Fast BERT repo;
ML
- A mature view on what role ML should serve
- PyTorch is awesome and it dominates?
- Robotic platforms becoming more standard? Like RPI?
- TLDR - check exif when loading user-created data into ML models
- Named Tensors are very cool idea. Early stage
- Key problem in medical ML - stratified testing
- http://www.offconvex.org/2019/10/03/NTK/)
CV
- Mobilenet by Facebook - FbNet
- Collecting images on Yandex toloka
- Super resolution in medical ML approved by FDA?
- How FaceID works, kind of
- Cool LR hacks for deep learning
- How Pinterest search works;
Code / libs / python
- Fuzzy string matching library in python;
- https://cli.dev/
- Instead of arparse https://omegaconf.readthedocs.io/en/latest/usage.html#access-and-manipulation?
- Use attrs? https://glyph.twistedmatrix.com/2016/08/attrs.html / http://www.attrs.org/en/stable/why.html