机器学习知识交换所（MLKX）

I think the best way for practice-focused methodology is something like‘practice — learning — practice’, that means where students first come with some existing projects with problems and solutions (practice) to get familiar with traditional methods in the area and perhaps also with their methodology. After practicing with some elementary experiences, they can go into the books and study the underlying theory, which serves to guide their future advanced practice and will enhance their toolbox of solving practical problems. Studying theory also further improves their understanding on the elementary experiences, and will help them acquire advanced experiences more quickly.

Nam Vu – Top-down learning path: machine learning for software engineers|

What if I’m Not Good at Mathematics

5 Techniques To Understand Machine Learning Algorithms Without the Background in Mathematics

How do I learn machine learning?

What is the difference between Data Analytics, Data Analysis, Data Mining, Data Science, Machine Learning, and Big Data?

Learning How to Learn

Don’t Break The Chain

How to learn on your own

Kaggle knowledge competitions

Kaggle Competitions: How and where to begin?

How a Beginner Used Small Projects To Get Started in Machine Learning and Compete on Kaggle

Master Kaggle By Competing Consistently

Courses

GeneralMachine Learning Topics

A Visual Introduction to Machine Learning

A Gentle Guide to Machine Learning

Introduction to Machine Learning for Developers

Machine Learning basics for a newbie

How do you explain Machine Learning and Data Mining to non Computer Science people?

Machine Learning: Under the hood. Blog post explains the principles of machine learning in layman terms. Simple and clear

What is machine learning, and how does it work?

Deep Learning – A Non-Technical Introduction

The Machine Learning Mastery Method

Machine Learning for Programmers

Applied Machine Learning with Machine Learning Mastery

Python Machine Learning Mini-Course

Machine Learning Algorithms Mini-Course

Machine learning is fun

Machine Learning is Fun!

Part 2: Using Machine Learning to generate Super Mario Maker levels

Part 3: Deep Learning and Convolutional Neural Networks

Part 4: Modern Face Recognition with Deep Learning

Part 5: Language Translation with Deep Learning and the Magic of Sequences

Inky Machine Learning

Part 1: What is Machine Learning ?

Part 2: Supervised Learning and Unsupervised Learning

Machine learning: an in-depth, non-technical guide

Overview, goals, learning types, and algorithms

Data selection, preparation, and modeling

Model evaluation, validation, complexity, and improvement

Model performance and error analysis

Unsupervised learning, related fields, and machine learning in practice

Video Series

Machine Learning for Hackers

Fresh Machine Learning

Machine Learning Recipes with Josh Gordon

Everything You Need to know about Machine Learning in 30 Minutes or Less

A Friendly Introduction to Machine Learning

Nuts and Bolts of Applying Deep Learning – Andrew Ng

BigML Webinar

Video

Resources

mathematicalmonk’s Machine Learning tutorials

Machine learning in Python with scikit-learn

GitHub repository

Blog

My playlist – Top YouTube Videos on Machine Learning, Neural Network & Deep Learning

16 New Must Watch Tutorials, Courses on Machine Learning

DeepLearning.TV

Learning To See

MOOC

Udacity’s Intro to Machine Learning

Udacity Intro to Machine Learning Review

Udacity’s Supervised, Unsupervised & Reinforcement

Machine Learning Foundations: A Case Study Approach

Coursera’s Machine Learning

Video only

Coursera Machine Learning review

Coursera: Machine Learning Roadmap

Machine Learning Distilled

BigML training

Coursera’s Neural Networks for Machine Learning

Taught by Geoffrey Hinton, a pioneer in the field of neural networks

Machine Learning - CS - Oxford University

Creative Applications of Deep Learning with TensorFlow

Intro to Descriptive Statistics

Intro to Inferential Statistics

Resources

Learn Machine Learning in a Single Month

The Non-Technical Guide to Machine Learning & Artificial Intelligence

Machine Learning for Software Engineers on Hacker News

Machine Learning for Developers

Machine Learning Advice for Developers

Machine Learning For Complete Beginners

Getting Started with Machine Learning: For absolute beginners and fifth graders

How to Learn Machine Learning: The Self-Starter Way

Machine Learning Self-study Resources

Level-Up Your Machine Learning

Enough Machine Learning to Make Hacker News Readable Again

Video

Slide

Dive into Machine Learning

Machine Learning courses in Universities

Stanford

Machine Learning Summer Schools

Oxford

Cambridge

NLP

Kyunghyun Cho’s NLP course in NYU

Stanford Natural Language ProcessingIntro NLP course with videos. This hasno deep learning. But it is a good primer for traditional nlp.

Stanford CS 224D: Deep Learning for NLP class

Richard Socher. (2016) Class with syllabus, and slides. Videos:2015 lectures/2016 lectures

Michael Collins– one of the best NLP teachers. Check out the material on the courses he is teaching.

Intro to Natural Language Processingon Coursera by U of Michigan

Intro to Artificial Intelligencecourse on Udacity which also covers NLP

Deep Learning for Natural Language Processing (2015 classes)by Richard Socher

Deep Learning for Natural Language Processing (2016 classes)by Richard Socher. Updated to make use of Tensorflow. Note that there are some lectures missing (lecture 9, and lectures 12 onwards).

Natural Language Processing– course on Coursera that was only done in 2013. The videos are not available at the moment. Also Mike Collins is a great professor and his notes and lectures are very good.

Statistical Machine Translation– a Machine Translation course with great assignments and slides.

Natural Language Processing SFU– course byProf Anoop Sarkaron Natural Language Processing. Good notes and some good lectures on youtube about HMM.

Udacity Deep LearningDeep Learning course on Udacity (using Tensorflow) which covers a section on using deep learning for NLP tasks (covering Word2Vec, RNN’s and LSTMs).

NLTK with Python 3 for Natural Language Processingby Harrison Kinsley(sentdex). Good tutorials with NLTK code implementation.

People

Mikolovet al. 2013. Performs well on word similarity and analogy task. Expands on famous example: King – Man + Woman = Queen

Yoav Goldberg

Quoc V. Le

Source Codes

General Machine Learning Topics

Games

Halite: A.I. Coding Game

Vindinium: A.I. Programming Challenge

General Video Game AI Competition

Angry Birds AI Competition

The AI Games

Fighting Game AI Competition

CodeCup

Student StarCraft AI Tournament

AIIDE StarCraft AI Competition

CIG StarCraft AI Competition

CodinGame – AI Bot Games

NLP

Word2Vec source code

FastText Code

gensim

Memory networks are implemented inMemNN. Attempts to solve task of reason attention and memory.

Stack RNN source codeandblog post

Pre-trained word embeddings for WSJ corpusby Koc AI-Lab

HLBL language modelby Turian

Real-valued vector “embeddings”by Dhillon

Improving Word Representations Via Global Context And Multiple Word Prototypesby Huang

Dependency based word embeddings

Global Vectors for Word Representations

TwitIE: An Open-Source Information Extraction Pipeline for Microblog Text

*Node.js and Javascript– – Node.js Libaries for NLP

Twitter-text– A JavaScript implementation of Twitter’s text processing library

Knwl.js– A Natural Language Processor in JS

Retext– Extensible system for analyzing and manipulating natural language

NLP Compromise– Natural Language processing in the browser

Natural– general natural language facilities for node

Python– Python NLP Libraries

Scikit-learn: Machine learning in Python

Natural Language Toolkit (NLTK)

Pattern– A web mining module for the Python programming language. It has tools for natural language processing, machine learning, among others.

TextBlob– Providing a consistent API for diving into common natural language processing (NLP) tasks. Stands on the giant shoulders of NLTK and Pattern, and plays nicely with both.

YAlign– A sentence aligner, a friendly tool for extracting parallel sentences from comparable corpora.

jieba– Chinese Words Segmentation Utilities.

SnowNLP– A library for processing Chinese text.

KoNLPy– A Python package for Korean natural language processing.

Rosetta– Text processing tools and wrappers (e.g. Vowpal Wabbit)

BLLIP Parser– Python bindings for the BLLIP Natural Language Parser (also known as the Charniak-Johnson parser)

PyNLPl– Python Natural Language Processing Library. General purpose NLP library for Python. Also contains some specific modules for parsing common NLP formats, most notably forFoLiA, but also ARPA language models, Moses phrasetables, GIZA++ alignments.

python-ucto– Python binding to ucto (a unicode-aware rule-based tokenizer for various languages)

python-frog– Python binding to Frog, an NLP suite for Dutch. (pos tagging, lemmatisation, dependency parsing, NER)

python-zpar– Python bindings forZPar, a statistical part-of-speech-tagger, constiuency parser, and dependency parser for English.

colibri-core– Python binding to C++ library for extracting and working with with basic linguistic constructions such as n-grams and skipgrams in a quick and memory-efficient way.

spaCy– Industrial strength NLP with Python and Cython.

PyStanfordDependencies– Python interface for converting Penn Treebank trees to Stanford Dependencies.

*C++– – C++ Libraries

MIT Information Extraction Toolkit– C, C++, and Python tools for named entity recognition and relation extraction

CRF++– Open source implementation of Conditional Random Fields (CRFs) for segmenting/labeling sequential data & other Natural Language Processing tasks.

CRFsuite– CRFsuite is an implementation of Conditional Random Fields (CRFs) for labeling sequential data.

BLLIP Parser– BLLIP Natural Language Parser (also known as the Charniak-Johnson parser)

colibri-core– C++ library, command line tools, and Python binding for extracting and working with basic linguistic constructions such as n-grams and skipgrams in a quick and memory-efficient way.

ucto– Unicode-aware regular-expression based tokenizer for various languages. Tool and C++ library. Supports FoLiA format.

libfolia– C++ library for theFoLiA format

frog– Memory-based NLP suite developed for Dutch: PoS tagger, lemmatiser, dependency parser, NER, shallow parser, morphological analyzer.

MeTA–MeTA : ModErn Text Analysisis a C++ Data Sciences Toolkit that facilitates mining big text data.

Mecab (Japanese)

Mecab (Korean)

Moses

*Java– – Java NLP Libraries

ReVerbWeb-Scale Open Information Extraction

OpenRegexAn efficient and flexible token-based regular expression language and engine.

CogcompNLP– Core libraries developed in the U of Illinois’ Cognitive Computation Group.

*Scala– – Scala NLP Libraries

Saul– Library for developing NLP systems, including built in modules like SRL, POS, etc.

Clojure

Clojure-openNLP– Natural Language Processing in Clojure (opennlp)

Infections-clj– Rails-like inflection library for Clojure and ClojureScript

Ruby

Kevin Dias’sA collection of Natural Language Processing (NLP) Ruby libraries, tools and software

Service

Wit-ai– Natural Language Interface for apps and devices.

Iris– Free text search API over large public document collections.

Books

Beginner Books

Data Smart: Using Data Science to Transform Information into Insight 1st Edition

Data Science for Business: What you need to know about data mining and data analytic-thinking

Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die

Practical Books

Machine Learning for Hackers

GitHub repository(R)

GitHub repository(Python)

Python Machine Learning

GitHub repository

Programming Collective Intelligence: Building Smart Web 2.0 Applications

Machine Learning: An Algorithmic Perspective, Second Edition

GitHub repository

Resource repository

Introduction to Machine Learning with Python: A Guide for Data Scientists

GitHub repository

Data Mining: Practical Machine Learning Tools and Techniques, Third Edition

Teaching material

Slides for Chapters 1-5 (zip)

Slides for Chapters 6-8 (zip)

Machine Learning in Action

GitHub repository

Reactive Machine Learning Systems(MEAP)

GitHub repository

An Introduction to Statistical Learning

GitHub repository(R)

GitHub repository(Python)

Videos

Building Machine Learning Systems with Python

GitHub repository

Learning scikit-learn: Machine Learning in Python

GitHub repository

Probabilistic Programming & Bayesian Methods for Hackers

Probabilistic Graphical Models: Principles and Techniques

Machine Learning: Hands-On for Developers and Technical Professionals

Machine Learning Hands-On for Developers and Technical Professionals review

GitHub repository

Learning from Data

Online tutorials

Reinforcement Learning: An Introduction (2nd Edition)

GitHub repository

Machine Learning with TensorFlow(MEAP)

Papers

General Machine Learning Topics

Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning [arXiv]

Overcoming catastrophic forgetting in neural networks [arXiv]

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer[OpenReview]

A Way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs [arXiv]

Importance Sampling with Unequal Support [arXiv]

Quasi-Recurrent Neural Networks [arXiv]

Capacity and Learnability in Recurrent Neural Networks [OpenReview]

Unrolled Generative Adversarial Networks [OpenReview]

Deep Information Propagation [OpenReview]

Structured Attention Networks [OpenReview]

Incremental Sequence Learning [arXiv]

b-GAN: Unified Framework of Generative Adversarial Networks [OpenReview]

A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks [OpenReview]

Categorical Reparameterization with Gumbel-Softmax [arXiv]

Computer Vision

Image-to-Image Translation with Conditional Adversarial Networks [arXiv]

Lip Reading Sentences in the Wild [arXiv]

Deep Residual Learning for Image Recognition[arXiv]

Rethinking the Inception Architecture for Computer Vision [arXiv]

Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks [arXiv]

Deep Speech 2: End-to-End Speech Recognition in English and Mandarin [arXiv]

Reinforcement Learning

Learning to reinforcement learn [arXiv]

A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models [arXiv]

The Predictron: End-To-End Learning and Planning [OpenReview]

Third-Person Imitation Learning[OpenReview]

Generalizing Skills with Semi-Supervised Reinforcement Learning [OpenReview]

Sample Efficient Actor-Critic with Experience Replay [OpenReview]

Reinforcement Learning with Unsupervised Auxiliary Tasks[arXiv]

Neural Architecture Search with Reinforcement Learning [OpenReview]

Towards Information-Seeking Agents [OpenReview]

Multi-Agent Cooperation and the Emergence of (Natural) Language [OpenReview]

Improving Policy Gradient by Exploring Under-appreciated Rewards [OpenReview]

Stochastic Neural Networks for Hierarchical Reinforcement Learning [OpenReview]

Tuning Recurrent Neural Networks with Reinforcement Learning [OpenReview]

RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning [arXiv]

Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning [OpenReview]

Learning to Perform Physics Experiments via Deep Reinforcement Learning [OpenReview]

Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU [OpenReview]

Learning to Compose Words into Sentences with Reinforcement Learning[OpenReview]

Deep Reinforcement Learning for Accelerating the Convergence Rate [OpenReview]

#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning[arXiv]

Learning to Compose Words into Sentences with Reinforcement Learning [OpenReview]

Learning to Navigate in Complex Environments [arXiv]

Unsupervised Perceptual Rewards for Imitation Learning [OpenReview]

Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic [OpenReview]

NLP

General Topics

Strategies for Training Large Vocabulary Neural Language Models[arXiv]

Multilingual Language Processing From Bytes[arXiv]

Learning Document Embeddings by Predicting N-grams for Sentiment Classification of Long Movie Reviews[arXiv]

Target-Dependent Sentiment Classification with Long Short Term Memory[arXiv]

Reading Text in the Wild with Convolutional Neural Networks [arXiv]

Deep Reinforcement Learning with a Natural Language Action Space[arXiv]

Sequence Level Training with Recurrent Neural Networks [arXiv]

Teaching Machines to Read and Comprehend[arxiv]

Semi-supervised Sequence Learning[arXiv]

Multi-task Sequence to Sequence Learning[arXiv]

Alternative structures for character-level RNNs[arXiv]

Larger-Context Language Modeling[arXiv]

A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding[arXiv]

Towards Universal Paraphrastic Sentence Embeddings [arXiv]

BlackOut: Speeding up Recurrent Neural Network Language Models With Very Large Vocabularies [arXiv]

Sequence Level Training with Recurrent Neural Networks [arXiv]

Natural Language Understanding with Distributed Representation [arXiv]

sense2vec – A Fast and Accurate Method for Word Sense Disambiguation In Neural Word Embeddings [arXiv]

LSTM-based Deep Learning Models for non-factoid answer selection [arXiv]

Review Articles

Deep Learning for Web Search and Natural Language Processing

Probabilistic topic models

Natural language processing: an introduction

A unified architecture for natural language processing: Deep neural networks with multitask learning

A Critical Review of Recurrent Neural Networksfor Sequence Learning

Deep parsing in Watson

Online named entity recognition method for microtexts in social networking services: A case study of twitter

Word Vectors

A Primer on Neural Network Models for Natural Language ProcessingYoav Goldberg. October 2015. No new info, 75 page summary of state of the art.

A neural probabilistic language modelBengio 2003. Seminal paper on word vectors.

Efficient Estimation of Word Representations in Vector Space

Distributed Representations of Words and Phrases and their Compositionality

Linguistic Regularities in Continuous Space Word Representations

Enriching Word Vectors with Subword Information

Deep Learning, NLP, and Representations

GloVe: Global vectors for word representationPennington, Socher, Manning. 2014. Creates word vectors and relates word2vec to matrix factorizations.Evalutaion section led to controversybyYoav Goldberg

Infinite Dimensional Word Embeddings– new

Skip Thought Vectors– word representation method

Adaptive skip-gram– similar approach, with adaptive properties

Named Entity Recognition

A survey of named entity recognition and classification

Benchmarking the extraction and disambiguation of named entities on the semantic web

Knowledge base population: Successful approaches and challenges

SpeedRead: A fast named entity recognition Pipeline

Sentiment Analysis

Recursive Deep Models for Semantic Compositionality Over a Sentiment TreebankSocher et al. 2013. Introduces Recursive Neural Tensor Network and dataset: “sentiment treebank.” Includesdemo site. Uses a parse tree.

Distributed Representations of Sentences and Documents

Deep Recursive Neural Networks for Compositionality in Language

Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

Semi-supervised Sequence Learning

Bag of Tricks for Efficient Text Classification

Adversarial Training Methods for Semi-Supervised Text Classification[arXiv]

Neural Machine Translation & Dialog

Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation(abstract)

On Using Very Large Target Vocabulary for Neural Machine Translation

Sequence to Sequence Learning with Neural Networks(nips presentation). Uses seq2seq to generate translations.

Addressing the Rare Word Problem in Neural Machine Translation(abstract)

Effective Approaches to Attention-based Neural Machine Translation

Context-Dependent Word Representation for Neural Machine Translation

Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Neural Machine Translation by jointly learning to align and translateBahdanau, Cho 2014. “comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation.” Implements attention mechanism.English to French Demo

Cross-lingual Pseudo-Projected Expectation Regularization for Weakly Supervised Learning

Generating Chinese Named Entity Data from a Parallel Corpus

IXA pipeline: Efficient and Ready to Use Multilingual NLP tools

Iterative Refinement for Machine Translation [OpenReview]

A Convolutional Encoder Model for Neural Machine Translation [arXiv]

Improving Neural Language Models with a Continuous Cache [OpenReview]

Vocabulary Selection Strategies for Neural Machine Translation [OpenReview]

Towards an automatic Turing test: Learning to evaluate dialogue responses [OpenReview]

Dialogue Learning With Human-in-the-Loop [OpenReview]

Batch Policy Gradient Methods for Improving Neural Conversation Models [OpenReview]

Learning through Dialogue Interactions [OpenReview]

Dual Learning for Machine Translation[arXiv]

Unsupervised Pretraining for Sequence to Sequence Learning [arXiv]

Neural Responding Machine for Short-Text ConversationShang et al. 2015 Uses Neural Responding Machine. Trained on Weibo dataset. Achieves one round conversations with 75% appropriate responses.

A Neural Network Approach to Context-Sensitive Generation of Conversational ResponsesSordoni et al. 2015. Generates responses to tweets. UsesRecurrent Neural Network Language Model (RLM) architecture of (Mikolov et al., 2010).source code:RNNLM Toolkit

Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network ModelsSerban, Sordoni, Bengio et al. 2015. Extendshierarchical recurrent encoder-decoderneural network (HRED).

Attention with Intention for a Neural Network Conversation ModelYao et al. 2015 Architecture is three recurrent networks: an encoder, an intention network and a decoder.

A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues

A Neural Conversation ModelVinyals,Le2015. Uses LSTM RNNs to generate conversational responses. Usesseq2seq framework. Seq2Seq was originally designed for machine translation and it “translates” a single sentence, up to around 79 words, to a single sentence response, and has no memory of previous dialog exchanges. Used in GoogleSmart Reply feature for Inbox

Incorporating Copying Mechanism in Sequence-to-Sequence LearningGu et al. 2016 Proposes CopyNet, builds on seq2seq.

A Persona-Based Neural Conversation ModelLi et al. 2016 Proposes persona-based models for handling the issue of speaker consistency in neural response generation. Builds on seq2seq.

Deep Reinforcement Learning for Dialogue GenerationLi et al. 2016. Uses reinforcement learing to generate diverse responses. Trains 2 agents to chat with each other. Builds on seq2seq.

Deep learning for chatbotsArticle summary of state of the art, and challenges for chatbots.

Deep learning for chatbots. part 2Implements a retrieval based dialog agent using dual encoder lstm with TensorFlow, based on the Ubuntu dataset [paper] includessource code

UsesRecurrent Neural Network Language Model (RLM) architecture of (Mikolov et al., 2010).source code:RNNLM Toolkit

Image Captioning

Show, Attend and Tell: Neural Image Caption Generation with Visual AttentionXu et al. 2015 Creates captions by feeding image into a CNN which feeds into hidden state of an RNN that generates the caption. At each time step the RNN outputs next word and the next location to pay attention to via a probability over grid locations. Uses 2 types of attention soft and hard. Soft attention uses gradient descent and backprop and is deterministic. Hard attention selects the element with highest probability. Hard attention uses reinforcement learning, rather than backprop and is stochastic.

Open source implementation in TensorFlow

Memory and Attention Models

Memory Networks

End-To-End Memory NetworksSukhbaatar et. al 2015.

Towards AI-Complete Question Answering: A Set of Prerequisite Toy TasksWeston 2015. Classifies QA tasks like single factoid, yes/no etc. Extends memory networks.

Evaluating prerequisite qualities for learning end to end dialog systemsDodge et. al 2015. Tests Memory Networks on 4 tasks including reddit dialog task. SeeJason Weston lecture on MemNN

Neural Turing Machines

Olah and Carter blog on NTM

Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets

Reasoning, Attention and Memory RAM workshop at NIPS 2015. slides included

General NLP topics

Neural autocoder for paragraphs and documents– LSTM representation