Predictor (Spark 3.5.5 JavaDoc) (original) (raw)

• Object

• • org.apache.spark.ml.PipelineStage
  • • org.apache.spark.ml.Estimator
      • • org.apache.spark.ml.Predictor<FeaturesType,Learner,M>

• Type Parameters:
  FeaturesType - Type of features. E.g., VectorUDT for vector features.
  Learner - Specialization of this class. If you subclass this type, use this type parameter to specify the concrete type.
  M - Specialization of PredictionModel. If you subclass this type, use this type parameter to specify the concrete type for the corresponding model.
  All Implemented Interfaces:
  java.io.Serializable, org.apache.spark.internal.Logging, Params, HasFeaturesCol, HasLabelCol, HasPredictionCol, PredictorParams, Identifiable
  Direct Known Subclasses:
  Classifier, Regressor

public abstract class Predictor<FeaturesType,Learner extends Predictor<FeaturesType,Learner,M>,M extends PredictionModel<FeaturesType,M>>
extends Estimator
implements PredictorParams
Abstraction for prediction problems (regression and classification). It accepts all NumericType labels and will automatically cast it to DoubleType in fit(). If this predictor supports weights, it accepts all NumericType weights, which will be automatically casted to DoubleType in fit().
See Also:
Serialized Form

• • Nested Class Summary

     * ### Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging  
     `org.apache.spark.internal.Logging.SparkShellLoggingFilter`  

  • Constructor Summary

  Constructors

  Constructor and Description
  Predictor()

  • Method Summary

  All Methods Instance Methods Abstract Methods Concrete Methods

  Modifier and Type	Method and Description
  abstract Learner	copy(ParamMap extra) Creates a copy of this instance with the same UID and some extra params.
  Param	featuresCol() Param for features column name.
  M	fit(Dataset<?> dataset) Fits a model to the input data.
  Param	labelCol() Param for label column name.
  Param	predictionCol() Param for prediction column name.
  Learner	setFeaturesCol(String value)
  Learner	setLabelCol(String value)
  Learner	setPredictionCol(String value)
  StructType	transformSchema(StructType schema) Check transform validity and derive the output schema from the input schema.

     * ### Methods inherited from class org.apache.spark.ml.[Estimator](../../../../org/apache/spark/ml/Estimator.html "class in org.apache.spark.ml")  
     `[fit](../../../../org/apache/spark/ml/Estimator.html#fit-org.apache.spark.sql.Dataset-org.apache.spark.ml.param.ParamMap-), [fit](../../../../org/apache/spark/ml/Estimator.html#fit-org.apache.spark.sql.Dataset-org.apache.spark.ml.param.ParamPair-org.apache.spark.ml.param.ParamPair...-), [fit](../../../../org/apache/spark/ml/Estimator.html#fit-org.apache.spark.sql.Dataset-org.apache.spark.ml.param.ParamPair-scala.collection.Seq-), [fit](../../../../org/apache/spark/ml/Estimator.html#fit-org.apache.spark.sql.Dataset-scala.collection.Seq-)`  
     * ### Methods inherited from class org.apache.spark.ml.[PipelineStage](../../../../org/apache/spark/ml/PipelineStage.html "class in org.apache.spark.ml")  
     `[params](../../../../org/apache/spark/ml/PipelineStage.html#params--)`  
     * ### Methods inherited from class Object  
     `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`  
     * ### Methods inherited from interface org.apache.spark.ml.[PredictorParams](../../../../org/apache/spark/ml/PredictorParams.html "interface in org.apache.spark.ml")  
     `[validateAndTransformSchema](../../../../org/apache/spark/ml/PredictorParams.html#validateAndTransformSchema-org.apache.spark.sql.types.StructType-boolean-org.apache.spark.sql.types.DataType-)`  
     * ### Methods inherited from interface org.apache.spark.ml.param.shared.[HasLabelCol](../../../../org/apache/spark/ml/param/shared/HasLabelCol.html "interface in org.apache.spark.ml.param.shared")  
     `[getLabelCol](../../../../org/apache/spark/ml/param/shared/HasLabelCol.html#getLabelCol--)`  
     * ### Methods inherited from interface org.apache.spark.ml.param.shared.[HasFeaturesCol](../../../../org/apache/spark/ml/param/shared/HasFeaturesCol.html "interface in org.apache.spark.ml.param.shared")  
     `[getFeaturesCol](../../../../org/apache/spark/ml/param/shared/HasFeaturesCol.html#getFeaturesCol--)`  
     * ### Methods inherited from interface org.apache.spark.ml.param.shared.[HasPredictionCol](../../../../org/apache/spark/ml/param/shared/HasPredictionCol.html "interface in org.apache.spark.ml.param.shared")  
     `[getPredictionCol](../../../../org/apache/spark/ml/param/shared/HasPredictionCol.html#getPredictionCol--)`  
     * ### Methods inherited from interface org.apache.spark.ml.param.[Params](../../../../org/apache/spark/ml/param/Params.html "interface in org.apache.spark.ml.param")  
     `[clear](../../../../org/apache/spark/ml/param/Params.html#clear-org.apache.spark.ml.param.Param-), [copyValues](../../../../org/apache/spark/ml/param/Params.html#copyValues-T-org.apache.spark.ml.param.ParamMap-), [defaultCopy](../../../../org/apache/spark/ml/param/Params.html#defaultCopy-org.apache.spark.ml.param.ParamMap-), [defaultParamMap](../../../../org/apache/spark/ml/param/Params.html#defaultParamMap--), [explainParam](../../../../org/apache/spark/ml/param/Params.html#explainParam-org.apache.spark.ml.param.Param-), [explainParams](../../../../org/apache/spark/ml/param/Params.html#explainParams--), [extractParamMap](../../../../org/apache/spark/ml/param/Params.html#extractParamMap--), [extractParamMap](../../../../org/apache/spark/ml/param/Params.html#extractParamMap-org.apache.spark.ml.param.ParamMap-), [get](../../../../org/apache/spark/ml/param/Params.html#get-org.apache.spark.ml.param.Param-), [getDefault](../../../../org/apache/spark/ml/param/Params.html#getDefault-org.apache.spark.ml.param.Param-), [getOrDefault](../../../../org/apache/spark/ml/param/Params.html#getOrDefault-org.apache.spark.ml.param.Param-), [getParam](../../../../org/apache/spark/ml/param/Params.html#getParam-java.lang.String-), [hasDefault](../../../../org/apache/spark/ml/param/Params.html#hasDefault-org.apache.spark.ml.param.Param-), [hasParam](../../../../org/apache/spark/ml/param/Params.html#hasParam-java.lang.String-), [isDefined](../../../../org/apache/spark/ml/param/Params.html#isDefined-org.apache.spark.ml.param.Param-), [isSet](../../../../org/apache/spark/ml/param/Params.html#isSet-org.apache.spark.ml.param.Param-), [onParamChange](../../../../org/apache/spark/ml/param/Params.html#onParamChange-org.apache.spark.ml.param.Param-), [paramMap](../../../../org/apache/spark/ml/param/Params.html#paramMap--), [params](../../../../org/apache/spark/ml/param/Params.html#params--), [set](../../../../org/apache/spark/ml/param/Params.html#set-org.apache.spark.ml.param.Param-T-), [set](../../../../org/apache/spark/ml/param/Params.html#set-org.apache.spark.ml.param.ParamPair-), [set](../../../../org/apache/spark/ml/param/Params.html#set-java.lang.String-java.lang.Object-), [setDefault](../../../../org/apache/spark/ml/param/Params.html#setDefault-org.apache.spark.ml.param.Param-T-), [setDefault](../../../../org/apache/spark/ml/param/Params.html#setDefault-scala.collection.Seq-), [shouldOwn](../../../../org/apache/spark/ml/param/Params.html#shouldOwn-org.apache.spark.ml.param.Param-)`  
     * ### Methods inherited from interface org.apache.spark.ml.util.[Identifiable](../../../../org/apache/spark/ml/util/Identifiable.html "interface in org.apache.spark.ml.util")  
     `[toString](../../../../org/apache/spark/ml/util/Identifiable.html#toString--), [uid](../../../../org/apache/spark/ml/util/Identifiable.html#uid--)`  
     * ### Methods inherited from interface org.apache.spark.internal.Logging  
     `$init$, initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, initLock, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning, org$apache$spark$internal$Logging$$log__$eq, org$apache$spark$internal$Logging$$log_, uninitialize`

• • Constructor Detail

     * #### Predictor  
     public Predictor()  

  • Method Detail

    * #### copy  
    public abstract [Learner](../../../../org/apache/spark/ml/Predictor.html "type parameter in Predictor") copy([ParamMap](../../../../org/apache/spark/ml/param/ParamMap.html "class in org.apache.spark.ml.param") extra)  
    Description copied from interface: `[Params](../../../../org/apache/spark/ml/param/Params.html#copy-org.apache.spark.ml.param.ParamMap-)`  
    Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See `defaultCopy()`.  
    Specified by:  
    `[copy](../../../../org/apache/spark/ml/param/Params.html#copy-org.apache.spark.ml.param.ParamMap-)` in interface `[Params](../../../../org/apache/spark/ml/param/Params.html "interface in org.apache.spark.ml.param")`  
    Specified by:  
    `[copy](../../../../org/apache/spark/ml/Estimator.html#copy-org.apache.spark.ml.param.ParamMap-)` in class `[Estimator](../../../../org/apache/spark/ml/Estimator.html "class in org.apache.spark.ml")<[M](../../../../org/apache/spark/ml/Predictor.html "type parameter in Predictor") extends [PredictionModel](../../../../org/apache/spark/ml/PredictionModel.html "class in org.apache.spark.ml")<[FeaturesType](../../../../org/apache/spark/ml/Predictor.html "type parameter in Predictor"),[M](../../../../org/apache/spark/ml/Predictor.html "type parameter in Predictor")>>`  
    Parameters:  
    `extra` \- (undocumented)  
    Returns:  
    (undocumented)  
    * #### featuresCol  
    public final [Param](../../../../org/apache/spark/ml/param/Param.html "class in org.apache.spark.ml.param")<String> featuresCol()  
    Param for features column name.  
    Specified by:  
    `[featuresCol](../../../../org/apache/spark/ml/param/shared/HasFeaturesCol.html#featuresCol--)` in interface `[HasFeaturesCol](../../../../org/apache/spark/ml/param/shared/HasFeaturesCol.html "interface in org.apache.spark.ml.param.shared")`  
    Returns:  
    (undocumented)  
    * #### fit  
    public [M](../../../../org/apache/spark/ml/Predictor.html "type parameter in Predictor") fit([Dataset](../../../../org/apache/spark/sql/Dataset.html "class in org.apache.spark.sql")<?> dataset)  
    Description copied from class: `[Estimator](../../../../org/apache/spark/ml/Estimator.html#fit-org.apache.spark.sql.Dataset-)`  
    Fits a model to the input data.  
    Specified by:  
    `[fit](../../../../org/apache/spark/ml/Estimator.html#fit-org.apache.spark.sql.Dataset-)` in class `[Estimator](../../../../org/apache/spark/ml/Estimator.html "class in org.apache.spark.ml")<[M](../../../../org/apache/spark/ml/Predictor.html "type parameter in Predictor") extends [PredictionModel](../../../../org/apache/spark/ml/PredictionModel.html "class in org.apache.spark.ml")<[FeaturesType](../../../../org/apache/spark/ml/Predictor.html "type parameter in Predictor"),[M](../../../../org/apache/spark/ml/Predictor.html "type parameter in Predictor")>>`  
    Parameters:  
    `dataset` \- (undocumented)  
    Returns:  
    (undocumented)  
    * #### labelCol  
    public final [Param](../../../../org/apache/spark/ml/param/Param.html "class in org.apache.spark.ml.param")<String> labelCol()  
    Description copied from interface: `[HasLabelCol](../../../../org/apache/spark/ml/param/shared/HasLabelCol.html#labelCol--)`  
    Param for label column name.  
    Specified by:  
    `[labelCol](../../../../org/apache/spark/ml/param/shared/HasLabelCol.html#labelCol--)` in interface `[HasLabelCol](../../../../org/apache/spark/ml/param/shared/HasLabelCol.html "interface in org.apache.spark.ml.param.shared")`  
    Returns:  
    (undocumented)  
    * #### predictionCol  
    public final [Param](../../../../org/apache/spark/ml/param/Param.html "class in org.apache.spark.ml.param")<String> predictionCol()  
    Param for prediction column name.  
    Specified by:  
    `[predictionCol](../../../../org/apache/spark/ml/param/shared/HasPredictionCol.html#predictionCol--)` in interface `[HasPredictionCol](../../../../org/apache/spark/ml/param/shared/HasPredictionCol.html "interface in org.apache.spark.ml.param.shared")`  
    Returns:  
    (undocumented)  
    * #### setFeaturesCol  
    public [Learner](../../../../org/apache/spark/ml/Predictor.html "type parameter in Predictor") setFeaturesCol(String value)  
    * #### setLabelCol  
    public [Learner](../../../../org/apache/spark/ml/Predictor.html "type parameter in Predictor") setLabelCol(String value)  
    * #### setPredictionCol  
    public [Learner](../../../../org/apache/spark/ml/Predictor.html "type parameter in Predictor") setPredictionCol(String value)  
    * #### transformSchema  
    public [StructType](../../../../org/apache/spark/sql/types/StructType.html "class in org.apache.spark.sql.types") transformSchema([StructType](../../../../org/apache/spark/sql/types/StructType.html "class in org.apache.spark.sql.types") schema)  
    Check transform validity and derive the output schema from the input schema.  
     We check validity for interactions between parameters during `transformSchema` and raise an exception if any parameter value is invalid. Parameter value checks which do not depend on other parameters are handled by `Param.validate()`.  
     Typical implementation should first conduct verification on schema change and parameter validity, including complex parameter interaction checks.  
    Specified by:  
    `[transformSchema](../../../../org/apache/spark/ml/PipelineStage.html#transformSchema-org.apache.spark.sql.types.StructType-)` in class `[PipelineStage](../../../../org/apache/spark/ml/PipelineStage.html "class in org.apache.spark.ml")`  
    Parameters:  
    `schema` \- (undocumented)  
    Returns:  
    (undocumented)