2021 DS/ML digest 02

2021 DS/ML digest 02

Posted by snakers41 on February 27, 2021

Speech

Lyra: A New Very Low-Bitrate Codec for Speech Compression - https://ai.googleblog.com/2021/02/lyra-new-very-low-bitrate-codec-for.html

ML / Papers

Microsoft Vision Model ResNet-50 combines web-scale data and multi-task learning to achieve state of the art - https://www.microsoft.com/en-us/research/blog/microsoft-vision-model-resnet-50-combines-web-scale-data-and-multi-task-learning-to-achieve-state-of-the-art/
Denoised smoothing: Provably defending pretrained classifiers against adversarial examples - https://www.microsoft.com/en-us/research/blog/denoised-smoothing-provably-defending-pretrained-classifiers-against-adversarial-examples/
TF 3D exists - https://ai.googleblog.com/2021/02/3d-scene-understanding-with-tensorflow.html
Why are machine learning algorithms hard to tune? - https://engraved.ghost.io/why-machine-learning-algorithms-are-hard-to-tune/
This is how we lost control of our faces - https://www.technologyreview.com/2021/02/05/1017388/ai-deep-learning-facial-recognition-data-history/
ObjectAug - https://arxiv.org/pdf/2102.00221.pdf


Searching for Fast Model Families on Datacenter Accelerators


High-Performance Large-Scale Image Recognition Without Normalization

  • http://arxiv.org/abs/2102.06171
  • Adaptive gradient clipping technique - clips gradients based on the unit-wise ratio of gradient norms to parameter norms
  • Smaller models match the test accuracy of an EffNet-B7 on ImageNet while being up to 8.7x faster to train
  • Overall recipe seems very complicated: apply the normalizer-free setup of Brock et al. (2021) to an SE-ResNeXt-D, with modified width and depth patterns, and a second spatial convolution. Second, apply AGC to every parameter except for the linear weight of the classifier layer


The Technology Behind Cinematic Photos

  • https://ai.googleblog.com/2021/02/the-technology-behind-cinematic-photos.html
  • A custom 5-camera rig as well as another dataset of Portrait photos captured on Pixel 4. Both datasets included ground-truth depth from multi-view stereo that is critical for training a model
  • Cinematic photo effect only needs the relative depths of objects in the scene, not the absolute depths
  • median filtering to improve the edges, and also infer segmentation masks of any people in the photo using a DeepLab segmentation model


Speller100: Zero-shot spelling correction at scale for 100-plus languages

  • https://www.microsoft.com/en-us/research/blog/speller100-zero-shot-spelling-correction-at-scale-for-100-plus-languages/
  • 100-plus languages
  • The foundation of Speller100 is based on the concept of language families
  • Two types of spelling errors. One is non-word error, and the other is real-word error
  • A spelling correction pretraining task to enrich standard Transformer-based models
  • Trained by corrupting text with an arbitrary noise function and learning a model to reconstruct the original text
  • Character-level mutations in order to mimic spelling errors
  • Just pre-training achieves 50% of correction recall
  • No code or released models

Datasets

Overhead MNIST - https://arxiv.org/abs/2102.04266

Code

XOR trick - https://florian.github.io/xor-trick/
Python imports - https://habr.com/ru/post/543832/
Python is 30 years old - https://www.opennet.ru/opennews/art.shtml?num=54627
Advanced Git Features You Didn’t Know You Needed - https://martinheinz.dev/blog/43
Unravelling syntactic sugar in Python - https://snarky.ca/tag/syntactic-sugar/
Bash shortcuts - https://habr.com/ru/company/lanit/blog/537596/
Compose finally having proper support of GPUs - https://www.docker.com/blog/deploy-gpu-accelerated-applications-on-amazon-ecs-with-docker-compose/?
mypyc - https://habr.com/ru/company/exness/blog/542106/
What the hell is a Bloom Filter? - https://diogodanielsoaresferreira.github.io/bloom-filter/

Tech

GOOGLE CLOSES STADIA STUDIOS - https://digitstodollars.com/2021/02/12/google-close-stadia-studios/
Smart supermarket carts - https://habr.com/ru/company/itsoft/blog/543662/
SPAC bonanza - https://digitstodollars.com/2021/02/19/spac-a-mole/
People leaving SF - https://habr.com/ru/post/542390/
A sneak peek at MetaHuman Creator: high-fidelity digital humans made easy https://www.unrealengine.com/en-US/blog/a-sneak-peek-at-metahuman-creator-high-fidelity-digital-humans-made-easy
Nvidia thinking about limiting mining - https://blogs.nvidia.com/blog/2021/02/18/geforce-cmp/
Some bitcoin opinions - https://www.bridgewater.com/research-and-insights/ray-dalio-what-i-think-of-bitcoin
AI Dungeon-maker Latitude raises $3.3M to build games with ‘infinite’ story possibilities - https://techcrunch.com/2021/02/04/latitude-seed-funding/
Wireguard vs OpenVPN? - https://habr.com/ru/company/ruvds/blog/537010/
Wireguard guide - https://linuxize.com/post/how-to-set-up-wireguard-vpn-on-ubuntu-20-04/

Blogs

This blog is just awesome - https://www.strangeloopcanon.com/
Why did I leave Google or, why did I stay so long? https://paygo.media/p/25171
The Impact of News Aggregators on Internet News Consumption: The Case of Localization - https://www.gsb.stanford.edu/faculty-research/working-papers/impact-news-aggregators-internet-news-consumption-case-localization
Catching Cyberbullies with Neural Networks - https://thegradient.pub/catching-cyberbullies-with-neural-networks/
Stakes create good software - https://cerebralab.com/Stakes create good software
Gradually, then suddenly - https://www.oreilly.com/radar/gradually-then-suddenly/