2018 DS/ML digest 23

2018 DS/ML digest 23

Posted by snakers41 on September 6, 2018

Buy me a coffeeBuy me a coffee

Become a PatronBecome a Patron

Market / hardware:

  • GPU purchase guide update 2018. TLDR:
    • New generation GPUs are most cost effective new;
    • But USED 1080Tis are the best option;
    • Specialized solutions (Titan, TPU, Tesla) - are not competitive;

Papers / blog posts / releases:

  • Building a semantic search engine:
    • Mostly generic advice with links to popular solutions;
    • A fast search library in python for production;
  • Google Open Images 2nd place solution - this is insanity:
    • 512 GPUs, Japanese guys using Chainer;
    • Co-occurrence loss - for bounding box proposals that are spatially close to the ground truth boxes with a subject class annotation, co-occurrence loss ignores all learning signals for classifying
      the part classes of the subject class;
    • Train models exclusively on rare classes and ensemble them with the rest of the models. We find this technique beneficial especially for the first 250 rarest classes, sorted by their occurrence count;
    • Feature Pyramid Network (FPN) with SE-ResNeXt-101 and SENet-154;
    • Extension ablation tests;
  • Crazy vid2vid papers:
    • Everybody dance now:
      • Pose estimation + pix2pix + temporal smoothing;
    • Nvidia vid2vid:
  • Real-World perception for Embodied Agents - learning in sumulation:
    • 572 full 3D scanned buildings / 211k m^2;
    • This is insanity!
  • Facebook’s unsupervised machine translation;
    • Equivalent to supervised approaches trained with nearly 100,000 reference translations;
    • Steps:
      • Learn word embeddings (vectorial representations of words) for every word in each language;
      • Learn a rotation of the word embeddings in one language to match the word embeddings in the other language, using a combination of various new and old techniques, such as adversarial training;
      • After the rotation, word translation is performed via nearest neighbor search;
      • Equipped with a language model and the word-by-word initialization, we can now build an early version of a translation system;
      • Treat these system translations (original sentence in Urdu, translation in English) as ground truth data to train an MT system in the opposite direction;


  • Finally proper guiedes and explanations about:
    • Asyncio / “callbacks” / concurrency / threading / multi-processing;
    • TLDR - for DS mostly people use multi-processing;
    • Explanation of different forms of concurrency
    • Asyncio guide
  • Yet anotherlist of Python hacks (RU) hacks

Just for lulz

Internet / tech

  • Apply buying AR companies
  • Tesla equals iPhone?
    • Making decent cars is only an entry ticket
    • Everyone will have cheap (er) batteries, Tesla only helped the shift
    • Assembly makes difference, traditional cars (and OEMs) consist of a mesh of independent components
    • Complex cars with simple software => simple cars with complex software
    • Eliminating dealers from value chain
    • Tesla needs to solve autonomy with CV
  • Li battery prices
  • Snapchat MAU 188m
  • Instagram to launch a shopping app