A list of machine learning resources published between 2011 and 2021. More recent resources can be found in my repositories Awesome Generative AI and AI in 2023.
- High-Resolution Image Synthesis with Latent Diffusion Models A paper by Rombach et al. on diffusion models for image generation, later used as the model behind Stable Diffusion. #synthesis
- Evaluating Large Language Models Trained on Code. A paper by OpenAI introducing Codex, a GPT language model that translates language into code. Accompanying blog post. #nlp
- Learning Transferable Visual Models From Natural Language Supervision A paper by OpenAI on CLIP (Contrastive Language-Image Pre-Training). Accompanying blog post. #nlp #vision
- Language Models are Few-Shot Learners A paper by OpenAI on the training of GPT-3, an autoregressive language model with 175 billion parameters. #nlp
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. A paper by Mingxing Tan and Quoc V. Le on a scaling method to achieve small and fast neural networks with high classification accuracy.
- The Bitter Lesson. An essay by Richard Sutton on general-purpose methods, the leveraging of computation, and the fallacy of domain knowledge. #ai #reinforcementlearning
- Language Models are Unsupervised Multitask Learners. A paper by OpenAI's A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever introducing the GPT-2 model for word prediction. Accompanying blog post. #nlp
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. A paper by Google AI Language on a state-of-the-art language model for a wide variety of NLP tasks. Model weights available at https://github.com/google-research/bert. #nlp
- Neural Ordinary Differential Equations. A paper by R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. Duvenaud that defines neural networks as continuously evolving systems by parameterizing the derivative of their hidden states, using ODE solvers to find their final state. #dnn #ode
- Artificial Intelligence — The Revolution Hasn’t Happened Yet. An essay by Michael I. Jordan on the current state of AI. #ai
- Faceswap. Faceswap is a tool that utilizes deep learning to recognize and swap faces in pictures and videos. #deeplearning #video #fake
- A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs. A paper by Vicarious introducing the recursive cortical network, a generative model with prior structure that facilitates model building, authored by D. George, W. Lehrach, K. Kansky, M. Lázaro-Gredilla, C. Laan, B. Marthi, X. Lou, Z. Meng, Y. Liu, H. Wang, A. Lavin and D. S. Pho. #captcha #vision
- ShortScience.org Summaries of machine learning papers, provided by the community. Acompanying paper. #websites
- Foolbox v0.8.0: A Python toolbox to benchmark the robustness of machine learning models. A Python toolbox to create adversarial examples that fool neural networks. Code repo. #adversarial #deeplearning #toolboxes #python
- Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks. A paper by Pranav Rajpurkar, Awni Y. Hannun et al. that uses a 34-layer convolutional neural network which maps a sequence of ECG samples to a sequence of rhythm classes. #biomedical #cnn #deeplearning
- Attention Is All You Need. A paper by Google Brain introducing Transformer networks for sequence transduction. #nlp
- Functional neuroimaging of high-risk 6-month-old infants predicts a diagnosis of autism at 24 months of age. A paper by Robert W. Emerson et al. that uses pattern recognition techniques (SVM) on fMRI data. #autism #fmri #matlab #biomedical
- Supercharge your Computer Vision models with the TensorFlow Object Detection API Release of TensorFlow Object Detection API. #Google #tensorflow #detection
- pix2code: Generating Code from a Graphical User Interface Screenshot. A paper by Tony Beltramelli that converts a graphical user interface screenshot created by a designer into computer code. #gui #deeplearning #design
- The GAN Zoo. A list of all named GANs. Accompanying blog post. #GAN #zoo
- Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks CycleGAN paper by J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Official Torch implementation. #GAN #paper
- Distill.pub A web-based peer-reviewed journal dedicated to clear explanations of machine learning. #journals
- Learning to Discover Cross-Domain Relations with Generative Adversarial Networks. DiscoGAN paper by T. Kim, M. Cha, H. Kim, J. Lee, and J. Kim. Official Torch implementation. #GAN
- Machine Learning and Misinformation. A blog post by Paul Soulos about the destabilizing effects that disruptive technologies have when taken in a fragile social and economic climate that is slow to adapt. #GAN #psychology #beliefechos #confirmationbias
- Dermatologist-level classification of skin cancer with deep neural networks. A paper by Andre Esteva on using convolutional neural networks to classify skin lesions. #cnn #deeplearning #tensorflow #biomedical
- Deep Learning: The Unreasonable Effectiveness of Randomness. A blog post by Carlos E. Perez about meta-learning (including a machine that learns variants of the LSTM node), thoughts on models memorizing the training data, and networks with random weights. #metalearning #randomness
- Uncertainty in Deep Learning. A blog post by Yarin Gal about his PhD Thesis on Bayesian Deep Learning through dropout. #dropout #bayesian #dl
- Convolutional Variational Autoencoder, trained on MNIST. Interactive demo of convolutional variational autoencoder. #interactive #visualizations #cnn #vae
- How to Use t-SNE Effectively. An interactive exploration of the t-SNE algorithm. #visualizations #interactive #tSNE
- cleverhans v2.0.0: an adversarial machine learning library. Cleverhans is a software library that provides standardized reference implementations of adversarial example construction techniques and adversarial training. #adversarial #deeplearning #toolboxes #python
- Open Sourcing a Deep Learning Solution for Detecting NSFW Images. A Caffe DNN (ResNet 50) for detecting NSFW images, by Yahoo. #cnn
- Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. A paper by A. Nguyen, A. Dosovitskiy, J. Yosinski, T. Brox, and J. Clune. #papers #deeplearning #synthesis
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. A Google Research paper on end-to-end automated translation. Accompanying blog post #lstm
- Approaching (Almost) Any Machine Learning Problem. A post on practical ML pipelines by Abhishek Thakur. #pipelines #python
- Matching Networks for One Shot Learning. A paper by Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. #papers #oneshot
- Gradient Boosting Interactive Playground. An interactive demonstration-explanation of gradient boosting algorithm applied to classification problem. #visualizations #interactive #GB
- Awesome Deep Learning Papers. A curated list of the most cited deep learning papers since 2012. #dl #papers
- ML + AI advances 2016. An exhaustive list of ML and AI headlines of the past year. #overview #2016
- Magenta: Music and Art Generation with Machine Intelligence. A Google project to make art with TensorFlow. #art #tensorflow
- Interactive demonstrations for ML courses. A list of interactive demos: ROC curve, convnet.js (RandomForest, SVM, Neural network), Decision tree, Gradient Boosting (and decision tree for regression), Mini-course on reinforcement learning, TensorFlow NN, Image Reconstruction, t-SNE, Variational AutoEncoder, Generative adversarial networks, etc. #visualizations #interactive #list
- Movidius Fathom, a Neural Network Compute Framework embedded on a USB stick. #usb #hardware #deeplearning
- OpenAI Gym. A toolkit for developing and comparing reinforcement learning algorithms. #reinforcementlearning #openai #python
- CreativeAi. A space to share research and experiments that deal with Creativity and A.I. #art #ai
- A Neural Network Playground. Play with neural networks. Brought by the TensorFlow team. #visualizations #interactive #tensorflow
- Convolution arithmetic. A set of animations illustrating different convolutions, from the paper "A technical report on convolution arithmetic in the context of deep learning" by Vincent Dumoulin and Francesco Visin. #cnn #deeplearning
- XGBoost: A Scalable Tree Boosting System. A paper by Tianqi Chen and Carlos Guestrin. #papers #boosting
- Communication-Efficient Learning of Deep Networks from Decentralized Data A paper by researchers from Google introducing federated learning. #papers #federatedlearning
- An Interactive Node-Link Visualization of Convolutional Neural Networks. A 3D interface by Adam W. Harley. #visualizations #interactive #cnn
- Anthony Goldbloom gives you the secret to winning Kaggle competitions. A blog post reporting that XGBoost and Neural Networks are winning the majority of Kaggle competitions. #kaggle #xgboost #nn
- Human-level concept learning through probabilistic program induction. A paper by Brenden M. Lake, Ruslan Salakhutdinov, and Joshua B. Tenenbaum. #papers #one-shot #bayes
- Deep Residual Learning for Image Recognition. A paper by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun introducing the ResNet architecture. #deeplearning
- A mathematical motivation for complex-valued convolutional networks. A paper by Joan Bruna, Soumith Chintala, Yann LeCun, Serkan Piantino, Arthur Szlam, and Mark Tygert. #cnn #wavelets
- TensorFlow, an open source software library for machine intelligence by Google. #opensource #google #deeplearning #tensorflow
- Deepart.io. Generate images styled like your favorite artist #art
- Grasp-and-Lift EEG Detection Winners' Interview: 1st place, Cat & Dog. #kaggle #timeseries #eeg
- Code and documentation for the winning sollution at the Grasp-and-Lift EEG Detection challenge. #code #python #github #kaggle
- Grasp-and-Lift EEG Detection Winners' Interview: 3rd place, Team HEDJ. #kaggle #timeseries #eeg
- Spatial Transformer Networks. A paper by Max Jaderberg, Karen Simonyan, Andrew Zisserman, Koray Kavukcuoglu. And here's a demo on traffic sign recognition: The power of Spatial Transformer Networks #papers #code
- Understanding LSTM Networks. A blog post tutorial on LSTMs by Christopher Olah. #lstm #rnn
- A Neural Algorithm of Artistic Style. A paper by Leon A. Gatys, Alexander S. Ecker, Matthias Bethge. #papers #art
- GitXiv: Collaborative Open Computer Science. A space to share links to open computer science projects. Each project is presented as arXiv + Github + Links + Discussion. #github #arxiv
- Caffe Model Zoo. Caffe models for different tasks with all kinds of architectures and data. More on the GitHub Wiki #caffe #models #pretrained
- Why Deep Learning Is a Hindrance to Progress Toward True AI. Blog post on the role of time in unsupervised learning. #unsupervisedlearning
- Inceptionism: Going Deeper into Neural Networks. Blog post creating feature-enhanced images by turning recurrent neural networks upside down. #rnn
- The Holy Grail of Deep Learning: Modelling Invariances. A blog post by Ferenc Huszár. #deeplearning
- Neural Turing Machines. A paper by Alex Graves, Greg Wayne and Ivo Danihelka. #papers
- How to Evaluate Machine Learning Models: Hyperparameter Tuning. #hyperparameters
- Competing in a data science contest without reading the data. Blog post that introduces the wacky boosting algorithm. #kaggle #boosting
- Model-Based Machine Learning (Early Access): an online book. #books
- Yann LeCun, Yoshua Bengio & Geoffrey Hinton - Deep Learning. Article in nature (paywalled). #papers #deeplearning
- Siamese Neural Networks for One-Shot Image Recognition A paper by Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. #papers #deeplearning
- The Unreasonable Effectiveness of Recurrent Neural Networks. Blog post by Andrej Karpathy. #rnn
- U-Net: Convolutional Networks for Biomedical Image Segmentation. A paper by O. Ronneberger, P. Fischer, and T. Brox on U-Net image segmentation, including source code. #papers #matlab #caffe
- Early Stopping is Nonparametric Variational Inference. A Bayesian interpretation of early-stopping stochastic gradient descent. #papers
- CS224d: Deep Learning for Natural Language Processing. #courses
- Data Mining Courses. Overview of online machine learning and data mining courses. #courses
- Tutorial to configure an AWS instance to run Theano. #gpu #theano #aws
- Using convolutional neural nets to detect facial keypoints tutorial. Tutorial by Daniel Nouri on using Theano and Lasagne for detecting facial keypoints through deep convolutional regression. #deeplearning #cnn #theano #pretrained #regression
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. A paper by Sergey Ioffe and Christian Szegedy. #papers #deeplearning
- Visualizing DBSCAN Clustering. Interactive demo of DBSCAN clustering algorithm. #clustering #dbscan #interactive #visualization
- Do Deep Nets Really Need to be Deep? "Once a deep network is trained, a shallow network can learn the same function from the outputs of the deep network. The shallow network can't learn the same function directly from the data. This indicates that deep learning could be an optimization/learning trick". Via Nuit Blanche #papers #deeplearning
- Learning Deep Architectures for AI. A paper on the motivation for deep architectures, by Yoshua Bengio. #papers #ai
- "Using convolutional neural nets to detect facial keypoints tutorial." A tutorial by Daniel Nouri on how to perform regression using Theano + Lasagne + nolearn to find keypoints in pictures of face. #tutorial #theano #deeplearning
- "Deep Learning Tutorial". A tutorial on creating an AWS instance to run Theano. #kaggle #deeplearning #aws
- Do We Need Hundreds of Classifiers to Solve Real World Classification Problems? A paper by Manuel Fernández-Delgado, Eva Cernadas and Senén Barro. #paper #classification
- So You Wanna Try Deep Learning? Crash-course into deep learning including a one-file python implementation of a deep neural network. #code
- Tom M. Mitchell - The Discipline of Machine Learning. #papers
- An interview with Judea Pearl #interviews #ai
- Long short term memory. Wikipedia entry. #lstm #rnn
- Deep Learning: Past, Present and Future. Slides by Kyunghyun Cho. #tutorial #deeplearning
2014-09-08
- Accelerate Machine Learning with the cuDNN Deep Neural Network Library #deeplearning #toolboxes
- Intro to Artificial Intelligence. Course by Sebastian Thrun and Peter Norvig on Udacity. #courses #online_courses
- Using deep learning to listen for whales. Daniel Nouri's notes on the Kaggle Whale Detection Challenge. #deeplearning #kaggle
- Kernel Methods Match Deep Neural Networks On TIMIT. Via Reddit #kernelmethods #deeplearning #papers
- A Deep Learning Tutorial: From Perceptrons to Deep Networks. High-level overview of deep learning. #tutorials #deeplearning
- Stochastic Pooling for Regularization of Deep Convolutional Neural Networks. A talk by Matthew D. Zeiler. Stochastic pooling: treat activations as probabilities and pick one with the corresponding probability. At test time: weight activations with probabilities. Implemented using GPUmat with cuda-convnet #videos #deeplearning
- The History of AI. #AI #charts
- Who is the best at X? Benchmark visual recognition tests and comparison of results by state-of-the-art algorithms. #benchmarks #vision
- Deep-er Kernels. A talk by John Shawe-Taylor #videos #kernelmethods #deeplearning
- Recommending music on Spotify with deep learning. A blog post by Sander Dieleman #deeplearning
- Deep Learning. An MIT Press book by Yoshua Bengio, Ian Goodfellow and Aaron Courville. #books #pdf #deeplearning
- Signal and Image Classification - Stephane Mallat Technion lecture. Via Nuit Blanche #videos #wavelets #deeplearning
- Transforming Auto-encoders. Paper by G. E. Hinton, A. Krizhevsky & S. D. Wang, introducing the idea of "capsules". #papers #deeplearning #capsules
- The Data Scientist on a Quest to Turn Computers Into Doctors. Article on Jeremy Howard's company Enlitic #deeplearning #vision #medicine #companies #articles
- Indolent or aggressive? A computerised pathologist that can outperform its human counterparts could transform the field of cancer diagnosis. Article on Daphne Koller's Computational Pathologist. #articles #medicine #vision
- Timeseries Classification: KNN & DTW. An iPython notebook by Mark Regan. #python #dtw #knn #timeseries
- Neural Networks course by Hugo Larochelle. #courses #videos #deeplearning
- Neural Networks and Deep Learning. Online book by Michael Nielsen. #books #deeplearning
- Winning solution for the DecMeg2014 challenge. #code #matlab #github #kaggle
- Deep Learning and Convolutional Kernel Networks. #deeplearning #kernelmethods
- UFLDL Tutorial. Stanford tutorial on deep learning. My solutions on Github: Sparse Autoencoder., Sparse Autoencode Vectorized, Softmax Regression, Self-Taught Learning, Linear Decoder, Stacked Autoencoder and Convolutional Neural Network. #courses #deeplearning
- Kaggle Competition Past Solutions. #kaggle #code
- Towards End-to-End Speech Recognition with Recurrent Neural Networks. A paper by Alex Graves and Navdeep Jaitly. #rnn #lstm
- Generative Adversarial Networks. A paper by Ian J. Goodfellow et al. that proposes a framework for estimating generative models via an adversarial process. #gan
- ConvNetJS: Deep Learning in your browser. Javascript machine learning #code #deeplearning
- "Learning Hierarchies Of Invariant Features". Slides by Yann LeCun. #tutorial #deeplearning
- ImageNet Classification with Deep Convolutional Neural Networks. A paper by Alex Krizhevsky, Ilya Sutskever and Geoffrey E. Hinton. #deeplearning #cnn
- How to Grow a Mind: Statistics, Structure, and Abstraction. A paper by Joshua B. Tenenbaum, Charles Kemp, Thomas L. Griffiths, and Noah D. Goodman. #bayes #brain #science