Machine Learning Guide
About This Show
This series aims to teach you the high level fundamentals of machine learning from A to Z. I'll teach you the basic intuition, algorithms, and math. We'll discuss languages and frameworks, deep learning, and more. Audio may be an inferior medium to task; but with all our exercise, commute, and chores hours of the day, not having an audio supplementary education would be a missed opportunity. And where your other resources will provide you the machine learning trees, I’ll provide the forest. Additionally, consider me your syllabus. At the end of every episode I’ll provide the best-of-the-best resources curated from around the web for you to learn each episode’s details.
Most Recent Episode
22. Deep NLP 1
Recurrent Neural Networks (RNNs) and Word2Vec. ## Resources - Articles: Unreasonable Effectiveness of RNNs (http://karpathy.github.io/2015/05/21/rnn-effectiveness/), Deep Learning, NLP, and Representations (http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/), Understanding LSTM Networks (http://colah.github.io/posts/2015-08-Understanding-LSTMs/) - cs224n - Deep NLP (https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_TcylNr7EkRe6) `course:medium` (replaces cs224d) - TensorFlow Tutorials (https://www.tensorflow.org/tutorials/word2vec) `tutorial:medium` (start at Word2Vec + next 2 pages) - The usual DL resources (pick one): ** Deep Learning Book (http://amzn.to/2tXgCiT) (Free HTML version (http://www.deeplearningbook.org/)) `book:hard` comprehensive DL bible; highly mathematical ** Fast.ai (http://course.fast.ai/) `course:medium` practical DL for coders ** Neural Networks and Deep Learning (http://neuralnetworksanddeeplearning.com/) `book:medium` shorter online "book" ## Episode Deep NLP pros - Language complexity & nuances ** Feature engineering / learning ** Salary = degree*field, not + ** Multiple layers: pixels => lines => objects ** Multiple layers of language - Once model to rule them all; E2E models Sequence vs non-sequence - DNN = ANN = MLP = Feed Forward - RNNs for sequence (time series) RNNs - Looped hidden layers, learns nuances by combined features - Carries info through time: language model - Translation, sentiment, classification, POS, NER, ... - Seq2seq, encode/decode Word2Vec (https://www.tensorflow.org/tutorials/word2vec) - One-hot (sparse) doesn't help (plus sparse = compute) - Word embeddings ** Euclidean distance for synonyms / similar, Cosine for "projections" . king + queen - man = woman ** t-SNE (t-distributed stochastic neighbor embedding) - Vector Space Models (VSMs). Learn from context, predictive vs count-based - Predictive methods (neural probabilistic language models) - Learn model parameters which predict contexts ** Word2vec ** CBOW / Skip-Gram (cbow predicts center from context, skip-gram context from center. Small v large datasets) ** DNN, Softmax hypothesis fn, NCE loss (noise contrastive estimation) - Count-based methods / Distributional Semantics - (compute the statistics