前言

本文使用tensorflow训练线性回归模型，并将其与scikit-learn做比较。数据集来自Andrew Ng的网上公开课程Deep Learning

代码

#!/usr/bin/env python
# -*- coding=utf-8 -*-
# @author: 陈水平 
# @date: 2016-12-30
# @description: compare scikit-learn and tensorflow, using linear regression data from deep learning course by Andrew Ng.
# @ref: http://openclassroom.stanford.edu/MainFolder/DocumentPage.php?course=DeepLearning&doc=exercises/ex2/ex2.html

import tensorflow as tf
import numpy as np
from sklearn import linear_model

# Read x and y
x_data = np.loadtxt("ex2x.dat")
y_data = np.loadtxt("ex2y.dat")


# We use scikit-learn first to get a sense of the coefficients
reg = linear_model.LinearRegression()
reg.fit(x_data.reshape(-1, 1), y_data)

print "Coefficient of scikit-learn linear regression: k=%f, b=%f" % (reg.coef_, reg.intercept_)


# Then we apply tensorflow to achieve the similar results
# The structure of tensorflow code can be divided into two parts:

# First part: set up computation graph
W = tf.Variable(tf.random_uniform([1], -1.0, 1.0))
b = tf.Variable(tf.zeros([1]))
y = W * x_data + b

loss = tf.reduce_mean(tf.square(y - y_data)) / 2
optimizer = tf.train.GradientDescentOptimizer(0.07)  # Try 0.1 and you will see unconvergency
train = optimizer.minimize(loss)

init = tf.initialize_all_variables()

# Second part: launch the graph
sess = tf.Session()
sess.run(init)

for step in range(1500):
    sess.run(train)
    if step % 100 == 0:
        print step, sess.run(W), sess.run(b)
print "Coeeficient of tensorflow linear regression: k=%f, b=%f" % (sess.run(W), sess.run(b))

输出如下：

Coefficient of scikit-learn linear regression: k=0.063881, b=0.750163
0 [ 0.45234478] [ 0.10217379]
100 [ 0.13166969] [ 0.4169243]
200 [ 0.09332827] [ 0.58935112]
300 [ 0.07795752] [ 0.67282093]
400 [ 0.07064758] [ 0.71297228]
500 [ 0.06713474] [ 0.73227954]
600 [ 0.06544565] [ 0.74156356]
700 [ 0.06463348] [ 0.74602771]
800 [ 0.06424291] [ 0.74817437]
900 [ 0.06405514] [ 0.74920654]
1000 [ 0.06396478] [ 0.74970293]
1100 [ 0.06392141] [ 0.74994141]
1200 [ 0.06390052] [ 0.75005609]
1300 [ 0.06389045] [ 0.7501114]
1400 [ 0.0638856] [ 0.75013816]
Coeeficient of tensorflow linear regression: k=0.063883, b=0.750151

思考

对于tensorflow，梯度下降的步长alpha参数需要很仔细的设置，步子太大容易扯到蛋导致无法收敛；步子太小容易等得蛋疼。迭代次数也需要细致的尝试。