我是 Spark 机器学习的新手（2 天大）我正在 Spark Shell 中执行以下代码我试图预测一些值我看到这个错误帖子在 Stackoverflow 中可用，但我无法使用正确的解决方案修复我的代码所以再次发布这个问题为同样的道歉
输入数据：
1.00,1.00,9.00
1.00,2.00,10.00
1.00,3.00,9.00
1.00,4.00,9.00
1.00,5.00,9.00
1.00,6.00,9.45
1.00,7.00,9.45
1.00,8.00,9.45
1.00,9.00,9.45
代码：
val df = spark.read.csv("/root/Predictiondata.csv").toDF("Userid", "Date", "Intime")
import org.apache.spark.sql.types.DoubleType
val featureDf = df.select( df("Userid").cast(DoubleType).as("Userid"),df("Date").cast(DoubleType).as("Date"),df("Intime").cast(DoubleType).as("Intime")).toDF()
import org.apache.spark.mllib.linalg.Vectors
import org.apache.spark.mllib.regression.LabeledPoint
val data = featureDf.select("Userid","Date","Intime").map(r => LabeledPoint(r(0).toString.toDouble,Vectors.dense(r(1).toString.toDouble,r(2).toString.toDouble))).toDF()
import org.apache.spark.ml.regression.LinearRegression
val lr = new LinearRegression()
val lrModel = lr.fit(data)
错误：
scala> val lrModel = lr.fit(data)
java.lang.IllegalArgumentException: requirement failed: Column features must be of type org.apache.spark.ml.linalg.VectorUDT@3bfc3ba7 but was actually org.apache.spark.mllib.linalg.VectorUDT@f71b0bce.
at scala.Predef$.require(Predef.scala:224)
at org.apache.spark.ml.util.SchemaUtils$.checkColumnType(SchemaUtils.scala:42)
at org.apache.spark.ml.PredictorParams$class.validateAndTransformSchema(Predictor.scala:51)
at org.apache.spark.ml.Predictor.validateAndTransformSchema(Predictor.scala:72)
at org.apache.spark.ml.Predictor.transformSchema(Predictor.scala:122)
at org.apache.spark.ml.PipelineStage.transformSchema(Pipeline.scala:74)
at org.apache.spark.ml.Predictor.fit(Predictor.scala:90)
... 48 elided
非常感谢任何帮助或建议。
提前致谢
请将 Spark 2+ 与DataFrame API与VectorAssembler一起使用
像这样的东西（还没有测试过）：
import spark.implicits._
val data = spark.read
.option("inferSchema", true)
.csv("/root/Predictiondata.csv")
.toDF("Userid", "Date", "Intime")
val dataWithFeatures = new VectorAssembler()
.setInputCols(Array("Date", "Intime"))
.transform(data)
val dataWithLabelFeatures = dataWithFeatures
.withColumn("label", $"Userid")
val lrModel = new LinearRegression().fit(dataWithLabelFeatures)
另外，看看Pipeline
本文收集自互联网，转载请注明来源。
如有侵权，请联系[email protected] 删除。
我来说两句