NLP
- This kind of went below my radar and I am 2 years late, but we have 4 main alertnatives in NLP:
- LSTM / GRU;
- TCN / Trellis;
- Transformer;
- And QRNN - interesting thread:
And yeah, looks like they need cupy for some of their layers … meh - turns out it is not a plug and play replacement for LSTM;The QRNN provides similar accuracy to the LSTM but can be betwen 2 and 17 times faster than the highly optimized NVIDIA cuDNN LSTM implementation depending on the use case.
- Google’s new mobile STT?;
Articles / blog posts / etc
- TF for microcontrollers;
- RNN based handwriting recognition by Google: strokes => Bezier curves => RNN / QRNN => decoder;
- Google’s 3D face mesh detection: our ML pipeline consists of two real-time deep neural network models that work together: A detector that operates on the full image and computes face locations, and a generic 3D mesh model that operates on those locations and predicts the approximate surface geometry via regression;
- Google open-sources TF-Replicator for multi-GPU;
- Hierarchical RL?;
- RNN based lemmatization;
- Pandas scaling;
- Open AI is a for profit company now;
- Facebook had a barrage of posts about their datacenters / hardware acceleration:
- Unification of hardware acceleration slots;
- Facebook’s ML inference;
- Hm, this is why they needed all of this PyTorch 1.0 stuff:
- Use pre-existing ontologies as a source of annotation;
- Metaclasses in python;
- Python 3.7 cool features: data class, order of dicts, importlib.resources, asyncion enhancements;
- Intro to asyncio now even with zero boilerplate?;
- DVC walk-through;
- Why framework porting does not work in ML (RU);
Datasets
Papers
- Knowledge distillation works in semseg?