Spark 3.5.5 ScalaDoc - org.apache.spark (original) (raw)

•  case class Aggregator[K, V, C](createCombiner: (V) ⇒ C, mergeValue: (C, V) ⇒ C, mergeCombiners: (C, C) ⇒ C) extends Product with Serializable
  :: DeveloperApi :: A set of functions used to aggregate data.
•  class BarrierTaskContext extends TaskContext with Logging
  :: Experimental :: A TaskContext with extra contextual info and tooling for tasks in a barrier stage.
•  class BarrierTaskInfo extends AnyRef
  :: Experimental :: Carries all task infos of a barrier task.
•  class ComplexFutureAction[T] extends FutureAction[T]
  A FutureAction for actions that could trigger multiple Spark jobs.
•  class ContextAwareIterator[+T] extends Iterator[T]
  :: DeveloperApi :: A TaskContext aware iterator.
•  abstract class Dependency[T] extends Serializable
  :: DeveloperApi :: Base class for dependencies.
•  class ErrorClassesJsonReader extends AnyRef
  A reader to load error information from one or more JSON files.
•  case class ExceptionFailure(className: String, description: String, stackTrace: Array[StackTraceElement], fullStackTrace: String, exceptionWrapper: Option[ThrowableSerializationWrapper], accumUpdates: Seq[AccumulableInfo] = Seq.empty, accums: Seq[AccumulatorV2[_, _]] = Nil, metricPeaks: Seq[Long] = Seq.empty) extends TaskFailedReason with Product with Serializable
  :: DeveloperApi :: Task failed due to a runtime exception.
•  case class ExecutorLostFailure(execId: String, exitCausedByApp: Boolean = true, reason: Option[String]) extends TaskFailedReason with Product with Serializable
  :: DeveloperApi :: The task failed because the executor that it was running on was lost.
•  case class FetchFailed(bmAddress: BlockManagerId, shuffleId: Int, mapId: Long, mapIndex: Int, reduceId: Int, message: String) extends TaskFailedReason with Product with Serializable
  :: DeveloperApi :: Task failed to fetch shuffle data from a remote node.
•  trait FutureAction[T] extends Future[T]
  A future for the result of an action to support cancellation.
•  class HashPartitioner extends Partitioner
  A org.apache.spark.Partitioner that implements hash-based partitioning using Java's Object.hashCode.
•  class InterruptibleIterator[+T] extends Iterator[T]
  :: DeveloperApi :: An iterator that wraps around an existing iterator to provide task killing functionality.
•  sealed abstract final class JobExecutionStatus extends Enum[JobExecutionStatus]
•  trait [JobSubmitter](JobSubmitter.html "Handle via which a "run" function passed to a ComplexFutureAction can submit jobs for execution.") extends AnyRef
  Handle via which a "run" function passed to a ComplexFutureActioncan submit jobs for execution.
•  abstract class NarrowDependency[T] extends Dependency[T]
  :: DeveloperApi :: Base class for dependencies where each partition of the child RDD depends on a small number of partitions of the parent RDD.
•  class OneToOneDependency[T] extends NarrowDependency[T]
  :: DeveloperApi :: Represents a one-to-one dependency between partitions of the parent and child RDDs.
•  trait Partition extends Serializable
  An identifier for a partition in an RDD.
•  trait PartitionEvaluator[T, U] extends AnyRef
  An evaluator for computing RDD partitions.
•  trait PartitionEvaluatorFactory[T, U] extends Serializable
  A factory to create PartitionEvaluator.
•  abstract class Partitioner extends Serializable
  An object that defines how the elements in a key-value pair RDD are partitioned by key.
•  trait QueryContext extends AnyRef
•  class RangeDependency[T] extends NarrowDependency[T]
  :: DeveloperApi :: Represents a one-to-one dependency between ranges of partitions in the parent and child RDDs.
•  class RangePartitioner[K, V] extends Partitioner
  A org.apache.spark.Partitioner that partitions sortable records by range into roughly equal ranges.
•  class SerializableWritable[T <: Writable] extends Serializable
•  class ShuffleDependency[K, V, C] extends Dependency[Product2[K, V]] with Logging
  :: DeveloperApi :: Represents a dependency on the output of a shuffle stage.
•  class SimpleFutureAction[T] extends FutureAction[T]
  A FutureAction holding the result of an action that triggers a single job.
•  class SparkConf extends Cloneable with Logging with Serializable
  Configuration for a Spark application.
•  class SparkContext extends Logging
  Main entry point for Spark functionality.
•  class SparkEnv extends Logging
  :: DeveloperApi :: Holds all the runtime environment objects for a running Spark instance (either master or worker), including the serializer, RpcEnv, block manager, map output tracker, etc.
•  class SparkException extends Exception with SparkThrowable
•  trait SparkExecutorInfo extends Serializable
•  class SparkFirehoseListener extends SparkListenerInterface
•  trait SparkJobInfo extends Serializable
•  trait SparkStageInfo extends Serializable
•  class SparkStatusTracker extends AnyRef
  Low-level status reporting APIs for monitoring job and stage progress.
•  trait SparkThrowable extends AnyRef
•  case class TaskCommitDenied(jobID: Int, partitionID: Int, attemptNumber: Int) extends TaskFailedReason with Product with Serializable
  :: DeveloperApi :: Task requested the driver to commit, but was denied.
•  abstract class TaskContext extends Serializable
  Contextual information about a task which can be read or mutated during execution.
•  sealed trait TaskEndReason extends AnyRef
  :: DeveloperApi :: Various possible reasons why a task ended.
•  sealed trait TaskFailedReason extends TaskEndReason
  :: DeveloperApi :: Various possible reasons why a task failed.
•  case class TaskKilled(reason: String, accumUpdates: Seq[AccumulableInfo] = Seq.empty, accums: Seq[AccumulatorV2[_, _]] = Nil, metricPeaks: Seq[Long] = Seq.empty) extends TaskFailedReason with Product with Serializable
  :: DeveloperApi :: Task was killed intentionally and needs to be rescheduled.
•  class TaskKilledException extends RuntimeException
  :: DeveloperApi :: Exception thrown when a task is explicitly killed (i.e., task failure is expected).