引子

我们团队线上主力是tensorflow，我个人私下用Pytorch比较多。TF由于静态图的设计原则，一直以来以对初学者不友好出名，而Pytorch基于动态图，对Python侵入较少，新手无痛上手，经常安利给团队小伙伴。

学习二者的第一步是看官网的教程，但教程的共同特点是代码零零散散，不利于快速调试。网上看过很多教程，大多不是很满意，个人认为好的教程应该满足下面几个点：

简洁，代码量要少

机器学习的模型，代码量一般不多，在教程里面，不应该包含太多东西，比如外部参数配置、层层封装等。最好是一个main文件，一两百行代码搞定。但麻雀虽小五脏俱全，数据预处理、网络构建、train、eval等流程都要全。 举个例子，NLP中常见的Language Model是LSTM，这个Pytorch教程的核心模块main.py文件代码只有120行左右，TF半的教程也只有120行左右。看下Pytorch的代码，很简洁：

# Some part of the code was referenced from below.
# https://github.com/pytorch/examples/tree/master/word_language_model 
import torch
import torch.nn as nn
import numpy as np
from torch.nn.utils import clip_grad_norm
from data_utils import Dictionary, Corpus


# Device configuration
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

# Hyper-parameters
embed_size = 128
hidden_size = 1024
num_layers = 1
num_epochs = 5
num_samples = 1000     # number of words to be sampled
batch_size = 20
seq_length = 30
learning_rate = 0.002

# Load "Penn Treebank" dataset
corpus = Corpus()
ids = corpus.get_data('data/train.txt', batch_size)
vocab_size = len(corpus.dictionary)
num_batches = ids.size(1) // seq_length


# RNN based language model
class RNNLM(nn.Module):
    def __init__(self, vocab_size, embed_size, hidden_size, num_layers):
        super(RNNLM, self).__init__()
        self.embed = nn.Embedding(vocab_size, embed_size)
        self.lstm = nn.LSTM(embed_size, hidden_size, num_layers, batch_first=True)
        self.linear = nn.Linear(hidden_size, vocab_size)

    def forward(self, x, h):
        # Embed word ids to vectors
        x = self.embed(x)

        # Forward propagate LSTM
        out, (h, c) = self.lstm(x, h)

        # Reshape output to (batch_size*sequence_length, hidden_size)
        out = out.reshape(out.size(0)*out.size(1), out.size(2))

        # Decode hidden states of all time steps
        out = self.linear(out)
        return out, (h, c)

model = RNNLM(vocab_size, embed_size, hidden_size, num_layers).to(device)

# Loss and optimizer
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)

# Truncated backpropagation
def detach(states):
    return [state.detach() for state in states] 

# Train the model
for epoch in range(num_epochs):
    # Set initial hidden and cell states
    states = (torch.zeros(num_layers, batch_size, hidden_size).to(device),
              torch.zeros(num_layers, batch_size, hidden_size).to(device))

    for i in range(0, ids.size(1) - seq_length, seq_length):
        # Get mini-batch inputs and targets
        inputs = ids[:, i:i+seq_length].to(device)
        targets = ids[:, (i+1):(i+1)+seq_length].to(device)

        # Forward pass
        states = detach(states)
        outputs, states = model(inputs, states)
        loss = criterion(outputs, targets.reshape(-1))

        # Backward and optimize
        model.zero_grad()
        loss.backward()
        clip_grad_norm(model.parameters(), 0.5)
        optimizer.step()

        step = (i+1) // seq_length
        if step % 100 == 0:
            print ('Epoch [{}/{}], Step[{}/{}], Loss: {:.4f}, Perplexity: {:5.2f}'
                   .format(epoch+1, num_epochs, step, num_batches, loss.item(), np.exp(loss.item())))

# Test the model
with torch.no_grad():
    with open('sample.txt', 'w') as f:
        # Set intial hidden ane cell states
        state = (torch.zeros(num_layers, 1, hidden_size).to(device),
                 torch.zeros(num_layers, 1, hidden_size).to(device))

        # Select one word id randomly
        prob = torch.ones(vocab_size)
        input = torch.multinomial(prob, num_samples=1).unsqueeze(1).to(device)

        for i in range(num_samples):
            # Forward propagate RNN 
            output, state = model(input, state)

            # Sample a word id
            prob = output.exp()
            word_id = torch.multinomial(prob, num_samples=1).item()

            # Fill input with sampled word id for the next time step
            input.fill_(word_id)

            # File write
            word = corpus.dictionary.idx2word[word_id]
            word = '\n' if word == '<eos>' else word + ' '
            f.write(word)

            if (i+1) % 100 == 0:
                print('Sampled [{}/{}] words and save to {}'.format(i+1, num_samples, 'sample.txt'))

# Save the model checkpoints
torch.save(model.state_dict(), 'model.ckpt')

其他教程就喜欢做各种封装了，先是各种类：

请不要忘了，这只是个教程，TF的很多教程就更过分了，绕来绕去。 对新手来讲，最重要的是不要给我多余的信息。

覆盖全，主流模型都包括

虽然有些教程的代码也很简洁，但不全。每个人写代码都有一定的分割，熟悉了一个人的后，换一个人的写法，总要适应一下。对于NLP来讲，有些人就喜欢batch_size first。 这个Pytorch教程，主流模型都有涉及，其代码组织架构如下：

TF的代码组织架构如下：

TF的版本还包含notebook，对新手更友好。

版本要新

就不在这里吐槽TF的版本迭代导致API不兼容，还好virtualenv很好用，多部署几套环境而已。聊下Pytorch，在0.4之前，一直被人诟病的是Tensor和Variable的部分功能重合，导致代码冗余。0.4之后，Tensor和Vaiable合并，代码干净了很多。另外一个是Pytorch的gpu模式，0.4之后增加了device，又可以少写一点冗余代码。

这两个教程的stars和contributors够多，版本更新追的很快。当然，教程代码一般没有太多问题，issue作用不是特别大，不过多总比少好，多一个交流的点。

地址：

yunjey/pytorch-tutorial​ github.com
aymericdamien/TensorFlow-Examples​ github.com