2021 DS/ML digest 03

2021 DS/ML digest 03

Posted by snakers41 on April 1, 2021

Speech

Some nice TTS tutorials

LEAF: A Learnable Frontend for Audio Classification

Google Chrome’s new Live Caption feature rolls out to transcribe speech in videos https://www.xda-developers.com/google-chrome-live-caption-feature-rolls-out-transcribe-speech-videos

ML / Papers

Last Week in AI - https://lastweekin.ai/p/107
Torch Audio v0.8 released - https://github.com/pytorch/audio/releases/tag/v0.8.0
PyTorch 1.8 released - https://t.me/snakers4/2672
Image animation - https://futurism.com/the-byte/ai-tool-deep-nostalgia-lets-you-reanimate-your-dead-relatives
Nice free background removing API - https://habr.com/ru/post/546036/
MARGIN STACKING AND THE COST OF AI - https://digitstodollars.com/2021/03/10/margin-stacking-and-the-cost-of-ai/
When to Assume Neural Networks Can Solve a Problem - https://www.skynettoday.com/editorials/what-can-nns-solve/
Sparse operation support with XNNPACK and TensorFlow Lite - https://ai.googleblog.com/2021/03/accelerating-neural-networks-on-mobile.html
TracIn — A Simple Method to Estimate Training Data Influence - https://ai.googleblog.com/2021/02/tracin-simple-method-to-estimate.html
Multimodal Neurons in Artificial Neural Networks - https://distill.pub/2021/multimodal-neurons/
Contactless Sleep Sensing in Nest Hub - https://ai.googleblog.com/2021/03/contactless-sleep-sensing-in-nest-hub.html
Factorized layers revisited: Compressing deep networks without playing the lottery -https://www.microsoft.com/en-us/research/blog/factorized-layers-revisited-compressing-deep-networks-without-playing-the-lottery/
Adobe has built super-resolution - https://blog.adobe.com/en/publish/2021/03/10/from-the-acr-team-super-resolution.html
Constructing Transformers For Longer Sequences with Sparse Attention Methods - https://ai.googleblog.com/2021/03/constructing-transformers-for-longer.html
Progress and Challenges in Long-Form Open-Domain Question Answering - https://ai.googleblog.com/2021/03/progress-and-challenges-in-long-form.html
Google Earth Engine as large scale data source - https://habr.com/ru/post/549142/

TLDR - bing has transformers in theis search now as well - https://www.microsoft.com/en-us/research/blog/the-science-behind-semantic-search-how-ai-from-bing-is-powering-azure-cognitive-search/

  • extractive summarization
  • semantic highlights of relevant words or phrases on answers
  • spell correction
  • instant answers

This pony does not exist

SEER: The start of a more powerful, flexible, and accessible era for computer vision - https://ai.facebook.com/blog/seer-the-start-of-a-more-powerful-flexible-and-accessible-era-for-computer-vision

  • Billion-parameter self-supervised computer vision model
  • Pretraining on a billion random, unlabeled and uncurated public Instagram images
  • 100% of Imagenet => 84.2% top 1 accuracy, 10% of ImageNet => 77.9%, 1% ImageNet => 60.5%
  • Real interest - comparison of the different self-supervision methods - https://github.com/facebookresearch/vissl/blob/master/MODEL_ZOO.md
  • 512 NVIDIA V100 32GB GPUs
  • During fine tuning - the network is not frozen lol

A New Lens on Understanding Generalization in Deep Learning - https://ai.googleblog.com/2021/03/a-new-lens-on-understanding.html

  • Models that train quickly on infinite data are the same models that generalize well if they are instead trained on finite data
  • Trained a GAN on CIFAR-10 => used it to generate ~6 million images (“virtually infinite” data)
  • Good models and training procedures (1) optimize quickly in the ideal world and (2) do not optimize too quickly in the real world
  • The primary effect of pre-training => turns the network into a “fast learner” for online optimization
  • Good data-augmentations (1) do not significantly harm ideal world optimization (i.e., augmented samples don’t look too “out of distribution”) or (2) inhibit real world optimization speed (so the real world takes longer to fit its train set)

Datasets

VoxPopuli - https://github.com/facebookresearch/voxpopuli:

  • 2k annotated hours of speech for several European languages
  • First datasets from corporations that is original

Spotify podcast datset - https://podcastsdataset.byspotify.com/

Code

Deep Dive into Docker Internals - Union Filesystem - https://martinheinz.dev/blog/44
Naming in programming - https://cerebralab.com/The second hardest thing in programming - Part 1
Load balancing benchmarks - https://www.loggly.com/blog/benchmarking-5-popular-load-balancers-nginx-haproxy-envoy-traefik-and-alb/
A good idea - setting up more proper PostgreSQL permissions in future for new projects - https://www.jujens.eu/posts/en/2021/Mar/10/db-user-migrations/
Ray tracing 101 - https://medium.com/swlh/ray-tracing-from-scratch-in-python-41670e6a96f9
Pattern matching in 3.10 in python - why - no speed benefit - https://habr.com/ru/company/yandex_praktikum/blog/547902/
Your Docker build needs a smoke test - https://pythonspeed.com/articles/test-your-docker-build/
The worst so-called “best practice” for Docker - https://pythonspeed.com/articles/security-updates-in-docker/

Tech

Google admits Kubernetes container tech is so complex, it’s had to roll out an Autopilot feature to do it all for you - https://www.theregister.com/2021/02/25/google_kubernetes_autopilot/
The new Google Pay repeats all the same mistakes of Google Allo - https://arstechnica.com/gadgets/2021/03/the-new-google-pay-repeats-all-the-same-mistakes-of-google-allo/
An update on Android’s audio latency - https://android-developers.googleblog.com/2021/03/an-update-on-androids-audio-latency.html
Do Amazon ads bring in more cash than AWS? - https://www.ben-evans.com/benedictevans/2021/3/14/do-amazon-ads-bring-in-more-cash-than-aws
NVME naming peculiarity in Linux - https://habr.com/ru/post/491454/
How search engines work - https://habr.com/ru/post/545634/
INTEL SELECTS OPTION C – ALL OF THE ABOVE - https://digitstodollars.com/2021/03/23/intel-selects-option-c-all-of-the-above/
Google cuts app store commissions - https://android-developers.googleblog.com/2021/03/boosting-dev-success.html
Inside Facebook Reality Labs: Wrist-based interaction for the next computing platform - https://tech.fb.com/inside-facebook-reality-labs-wrist-based-interaction-for-the-next-computing-platform

Blogs

If you’ve learned from the best, you’re doing it wrong - https://cerebralab.com/If you’ve learned from the best%2C you’re doing it wrong
‘This is bigger than just Timnit’: How Google tried to silence a critic and ignited a movement - https://www.fastcompany.com/90608471/timnit-gebru-google-ai-ethics-equitable-tech-movement
tl;dr: The time when Microsoft banned my entire country for cheating at Club Bing - https://github.com/eyal0/Chicken-story/blob/main/README.md
Skepticism re The Crypto Utopia https://www.strangeloopcanon.com/p/skepticism-re-the-crypto-utopia
Advanced Image Management in Docker Hub - https://www.docker.com/blog/advanced-image-management-in-docker-hub/
Solving the innersource discovery problem - https://github.blog/2021-03-23-solving-the-innersource-discovery-problem/
You’re Doing It Wrong: Notes on Criticism and Technology Hype - https://sts-news.medium.com/youre-doing-it-wrong-notes-on-criticism-and-technology-hype-18b08b4307e5
The dispassionate developer - https://blog.ploeh.dk/2021/03/22/the-dispassionate-developer/
Linux admin about NFC - https://aminux.wordpress.com/2021/03/31/nft-new-crypto-revolution-thing/