org.apache.spark.ml

Class PredictionModel<FeaturesType,M extends PredictionModel<FeaturesType,M>>

java.lang.Object

org.apache.spark.ml.PipelineStage

org.apache.spark.ml.Transformer

org.apache.spark.ml.Model<M>

org.apache.spark.ml.PredictionModel<FeaturesType,M>

All Implemented Interfaces:

java.io.Serializable, Params, Identifiable

Direct Known Subclasses:

ClassificationModel, DecisionTreeRegressionModel, GBTClassificationModel, GBTRegressionModel, MultilayerPerceptronClassificationModel, RandomForestRegressionModel, RegressionModel

public abstract class PredictionModel<FeaturesType,M extends PredictionModel<FeaturesType,M>>
extends Model<M>

:: DeveloperApi :: Abstraction for a model for prediction tasks (regression and classification).

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
PredictionModel()

Method Summary

Methods

Modifier and Type	Method and Description
Param<java.lang.String>	featuresCol() Param for features column name.
protected DataType	featuresDataType() Returns the SQL DataType corresponding to the FeaturesType type parameter.
java.lang.String	getFeaturesCol()
java.lang.String	getLabelCol()
java.lang.String	getPredictionCol()
Param<java.lang.String>	labelCol() Param for label column name.
int	numFeatures() Returns the number of features the model was trained on.
protected abstract double	predict(FeaturesType features) Predict label for the given features.
Param<java.lang.String>	predictionCol() Param for prediction column name.
M	setFeaturesCol(java.lang.String value)
M	setPredictionCol(java.lang.String value)
Dataset<Row>	transform(Dataset<?> dataset) Transforms dataset by reading from featuresCol, calling predict(), and storing the predictions as a new column predictionCol.
protected Dataset<Row>	transformImpl(Dataset<?> dataset)
StructType	transformSchema(StructType schema) :: DeveloperApi ::
StructType	validateAndTransformSchema(StructType schema, boolean fitting, DataType featuresDataType) Validates and transforms the input schema with the provided param map.

Methods inherited from class org.apache.spark.ml.Model

copy, hasParent, parent, setParent

Methods inherited from class org.apache.spark.ml.Transformer

transform, transform, transform

Methods inherited from class org.apache.spark.ml.PipelineStage

transformSchema

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.ml.param.Params

clear, copy, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn, validateParams

Methods inherited from interface org.apache.spark.ml.util.Identifiable

toString, uid

Constructor Detail

PredictionModel

public PredictionModel()

Method Detail

setFeaturesCol

public M setFeaturesCol(java.lang.String value)

setPredictionCol

public M setPredictionCol(java.lang.String value)

numFeatures

public int numFeatures()

Returns the number of features the model was trained on. If unknown, returns -1

featuresDataType

protected DataType featuresDataType()

Returns the SQL DataType corresponding to the FeaturesType type parameter.

This is used by validateAndTransformSchema(). This workaround is needed since SQL has different APIs for Scala and Java.

The default value is VectorUDT, but it may be overridden if FeaturesType is not Vector.

Returns:

(undocumented)

transformSchema

public StructType transformSchema(StructType schema)

Description copied from class: PipelineStage

:: DeveloperApi ::

Derives the output schema from the input schema.

Specified by:

transformSchema in class PipelineStage

Parameters:

schema - (undocumented)

Returns:

(undocumented)

transform

public Dataset<Row> transform(Dataset<?> dataset)

Transforms dataset by reading from featuresCol, calling predict(), and storing the predictions as a new column predictionCol.

Specified by:

transform in class Transformer

Parameters:

dataset - input dataset

Returns:

transformed dataset with predictionCol of type Double

transformImpl

protected Dataset<Row> transformImpl(Dataset<?> dataset)

predict

protected abstract double predict(FeaturesType features)

Predict label for the given features. This internal method is used to implement transform() and output predictionCol.

Parameters:

features - (undocumented)

Returns:

(undocumented)

validateAndTransformSchema

public StructType validateAndTransformSchema(StructType schema,
                                    boolean fitting,
                                    DataType featuresDataType)

Validates and transforms the input schema with the provided param map.

Parameters:

schema - input schema

fitting - whether this is in fitting

featuresDataType - SQL DataType for FeaturesType. E.g., VectorUDT for vector features.

Returns:

output schema

labelCol

public Param<java.lang.String> labelCol()

Param for label column name.

Returns:

(undocumented)

getLabelCol

public java.lang.String getLabelCol()

featuresCol

public Param<java.lang.String> featuresCol()

Param for features column name.

Returns:

(undocumented)

getFeaturesCol

public java.lang.String getFeaturesCol()

predictionCol

public Param<java.lang.String> predictionCol()

Param for prediction column name.

Returns:

(undocumented)

getPredictionCol

public java.lang.String getPredictionCol()