Skip to content

Instantly share code, notes, and snippets.

@jlln

jlln/SparkSeqSelect.scala

Created August 4, 2016 04:29
Show Gist options
  • Save jlln/68d7ddf6d4f3bf9cce798a87310f1c73 to your computer and use it in GitHub Desktop.
Save jlln/68d7ddf6d4f3bf9cce798a87310f1c73 to your computer and use it in GitHub Desktop.
Download ZIP
Selecting spark columns using a Seq
Raw
SparkSeqSelect.scala
val retained_features:List[String] = group_mean_columns.filter(x=> !missings.contains(x._2)).map(_._1).toList :+ "LogAdjustedDemand"
//Select these columns in the training dataset
val model_training_data = training_data_all.select(retained_features.head,retained_features.tail: _*)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment