TypeError将Pandas数据框转换为Spark数据框

我有一个叫做Pandas的数据pdf框，它只是float64s的四列。这是前五行：

pdf[:5]

      x1         x2        x3          y
0   9.082060  12.837502  6.484107  10.985202
1   9.715981  14.870818  8.026042  12.815644
2  11.303901  21.286343  7.787188  15.786915
3   9.910293  20.533151  6.991775  14.775010
4  12.394907  15.401446  7.101058  13.213897

和dtypes：

pdf.dtypes

x1    float64
x2    float64
x3    float64
y     float64
dtype: object

但是当我尝试将其转换为Spark数据帧时：

sdf = sqlContext.createDataFrame(pdf)

TypeErrorTraceback (most recent call last)
<ipython-input-54-a40cb79104b5> in <module>()
      5                     ])
      6 
----> 7 sdf = sqlContext.createDataFrame(pdf)

/usr/lib/spark/python/pyspark/sql/context.py in createDataFrame(self, data, schema, samplingRatio)
    423             rdd, schema = self._createFromRDD(data, schema, samplingRatio)
    424         else:
--> 425             rdd, schema = self._createFromLocal(data, schema)
    426         jrdd = self._jvm.SerDeUtil.toJavaArray(rdd._to_java_object_rdd())
    427         jdf = self._ssql_ctx.applySchemaToPythonRDD(jrdd.rdd(), schema.json())

/usr/lib/spark/python/pyspark/sql/context.py in _createFromLocal(self, data, schema)
    339 
    340         if schema is None or isinstance(schema, (list, tuple)):
--> 341             struct = self._inferSchemaFromList(data)
    342             if isinstance(schema, (list, tuple)):
    343                 for i, name in enumerate(schema):

/usr/lib/spark/python/pyspark/sql/context.py in _inferSchemaFromList(self, data)
    239             warnings.warn("inferring schema from dict is deprecated,"
    240                           "please use pyspark.sql.Row instead")
--> 241         schema = reduce(_merge_type, map(_infer_schema, data))
    242         if _has_nulltype(schema):
    243             raise ValueError("Some of types cannot be determined after inferring")

/usr/lib/spark/python/pyspark/sql/types.py in _infer_schema(row)
    829 
    830     else:
--> 831         raise TypeError("Can not infer schema for type: %s" % type(row))
    832 
    833     fields = [StructField(k, _infer_type(v), True) for k, v in items]

TypeError: Can not infer schema for type: <type 'str'>

如果我尝试指定架构：

schema = StructType([StructField('y', DoubleType()),
                     StructField('x1', DoubleType()),
                     StructField('x2', DoubleType()),
                     StructField('x3', DoubleType())
                    ])
sdf = sqlContext.createDataFrame(pdf, schema)

然后我们得到一个略有不同的错误：

TypeErrorTraceback (most recent call last)
<ipython-input-55-a7d2b6d09ed3> in <module>()
      5                     ])
      6 
----> 7 sdf = sqlContext.createDataFrame(pdf, schema)

/usr/lib/spark/python/pyspark/sql/context.py in createDataFrame(self, data, schema, samplingRatio)
    423             rdd, schema = self._createFromRDD(data, schema, samplingRatio)
    424         else:
--> 425             rdd, schema = self._createFromLocal(data, schema)
    426         jrdd = self._jvm.SerDeUtil.toJavaArray(rdd._to_java_object_rdd())
    427         jdf = self._ssql_ctx.applySchemaToPythonRDD(jrdd.rdd(), schema.json())

/usr/lib/spark/python/pyspark/sql/context.py in _createFromLocal(self, data, schema)
    348         elif isinstance(schema, StructType):
    349             for row in data:
--> 350                 _verify_type(row, schema)
    351 
    352         else:

/usr/lib/spark/python/pyspark/sql/types.py in _verify_type(obj, dataType)
   1132     if _type is StructType:
   1133         if not isinstance(obj, (tuple, list)):
-> 1134             raise TypeError("StructType can not accept object %r in type %s" % (obj, type(obj)))
   1135     else:
   1136         # subclass of them can not be fromInternald in JVM

TypeError: StructType can not accept object 'x1' in type <type 'str'>

我缺少明显的东西吗？有没有人成功地从Pandas数据框架构建了Spark数据框架？该版本适用于Python 2.7，Spark v1.6.1和Pandas v0.18.1。