**Internet**

- Ben Evans about 5G
- A good piece on toxicity and hypocrisy in IT
- You will be able to export all of your photos from FaceBook to google photos
- How to check Russian passports

**Code / python**

- Viewing diffs for notebooks
- Most copied SO answer is a bit wrong and even Oracle does not respect SO license
- Useful pandas functions

**ML**

- Double descent phenomenon occurs in CNNs, ResNets, and transformers
- AI misinformation
- Suprise-surprise, FaceBook uses statistical analysis to mine its server logs
- How FaceBook mines the app problem logs
- Develop python packages in nbdev in notebooks?
- Cool idea - use cryptography as a part of your neural network with homomorphic algorithms
- Reproducibility crisis in ML and science

**Datasets**

- 500k image x-ray dataset by Google

**Papers**

What’s Hidden in a Randomly Weighted Neural Network?

- Link
- TLDR - show practically that the lottery ticket hypothesis works on ImageNet
- Randomly weighted neural networks contain subnetworks achieve impressive performance without ever training the weight values
- There is a subnetwork of Wide ResNet-50 (with random weights) that is smaller than but matches the performance ofa ResNet-34 trained on ImageNet
- Edge-popup algorithm
- Despite the hype that they do not actually “learn” weights, they essentially learn scores to include some weights with something very similar to SGD
- Also as usual there is no comparsion of the compute required vs. actual training

Fast Sparse ConvNets

- Link
- Before - use dense “regularized” blocks (separable convs, inverted bottlenecks) => higher efficiency / higher performance
- A family of efficient sparse kernels for ARM and WebAssembly
- Efficient implementations of sparse primitives => sparse versions of MobileNet v1, MobileNet v2 and EfficientNet
- MobileNets => FLOPs and parameter counts in these architectures are dominated by the 1×1 convolutions
- Fast kernels for sparse matrix-dense matrix multiplication (SpMM) specifically targeted at the acceleration of sparse neural networks
- Target the sparsity range of 70-95%
- Reduces inference times by 1.3 − 2.4×, parameter counts by over 2× and number of floating-point operations (FLOPs) by up to 3× relative to the previous generations
- They use To make the networks sparse we use the gradual magnitude pruning technique

- Link