Spark 4.1.0 ScalaDoc - org.apache.spark.sql (original) (raw)

•  class AnalysisException extends Exception with SparkThrowable with Serializable with WithOrigin
  Thrown when a query fails to analyze, usually because the query itself is invalid.
•  class Column extends Logging with TableValuedFunctionArgument
  A column that will be computed based on the data in a DataFrame.
•  class ColumnName extends Column
  A convenient class used for constructing schema.
•  trait CreateTableWriter[T] extends WriteConfigMethods[CreateTableWriter[T]]
  Trait to restrict calls to create and replace operations.
•  type DataFrame = Dataset[Row]
•  abstract class DataFrameNaFunctions extends AnyRef
  Functionality for working with missing data in DataFrames.
•  abstract class DataFrameReader extends AnyRef
  Interface used to load a Dataset from external storage systems (e.g.
•  abstract class DataFrameStatFunctions extends AnyRef
  Statistic functions for DataFrames.
•  abstract class DataFrameWriter[T] extends AnyRef
  Interface used to write a org.apache.spark.sql.Dataset to external storage systems (e.g.
•  abstract class DataFrameWriterV2[T] extends CreateTableWriter[T]
  Interface used to write a org.apache.spark.sql.Dataset to external storage using the v2 API.
•  abstract class Dataset[T] extends Serializable
  A Dataset is a strongly typed collection of domain-specific objects that can be transformed in parallel using functional or relational operations.
•  abstract class DatasetHolder[T] extends AnyRef
  A container for a org.apache.spark.sql.Dataset, used for implicit conversions in Scala.
•  trait Encoder[T] extends Serializable
  Used to convert a JVM object of type T to and from the internal Spark SQL representation.
•  trait EncoderImplicits extends LowPrioritySQLImplicits with Serializable
  EncoderImplicits used to implicitly generate SQL Encoders.
•  class ExperimentalMethods extends AnyRef
  :: Experimental :: Holder for experimental methods for the bravest.
•  trait ExtendedExplainGenerator extends AnyRef
  A trait for a session extension to implement that provides addition explain plan information.
•  abstract class ForeachWriter[T] extends Serializable
  The abstract class for writing custom logic to process data generated by a query.
•  abstract class KeyValueGroupedDataset[K, V] extends Serializable
  A Dataset has been logically grouped by a user specified grouping key.
•  trait LowPrioritySQLImplicits extends AnyRef
  Lower priority implicit methods for converting Scala objects intoorg.apache.spark.sql.Datasets.
•  abstract class MergeIntoWriter[T] extends AnyRef
  MergeIntoWriter provides methods to define and execute merge actions based on specified conditions.
•  class Observation extends AnyRef
  Helper class to simplify usage of Dataset.observe(String, Column, Column*):
•  abstract class RelationalGroupedDataset extends AnyRef
  A set of methods for aggregations on a DataFrame, created by groupBy,cube or rollup (and also pivot).
•  trait Row extends Serializable
  Represents one row of output from a relational operator.
•  class RowFactory extends AnyRef
  A factory class used to construct [Row](Row.html) objects.
•  abstract class RuntimeConfig extends AnyRef
  Runtime configuration interface for Spark.
•  abstract class SQLContext extends Logging with Serializable
  The entry point for working with structured data (rows and columns) in Spark 1.x.
•  abstract class SQLImplicits extends EncoderImplicits with Serializable
  A collection of implicit methods for converting common Scala objects intoorg.apache.spark.sql.Datasets.
•  sealed final class SaveMode extends Enum[SaveMode]
  SaveMode is used to specify the expected behavior of saving a DataFrame to a data source.
•  abstract class SparkSession extends Serializable with Closeable
  The entry point to programming Spark with the Dataset and DataFrame API.
•  class SparkSessionExtensions extends AnyRef
  :: Experimental :: Holder for injection points to the SparkSession.
•  trait SparkSessionExtensionsProvider extends (SparkSessionExtensions) => Unit
  Base trait for implementations used by SparkSessionExtensions
•  abstract class TableValuedFunction extends AnyRef
  Interface for invoking table-valued functions in Spark SQL.
•  class TypedColumn[-T, U] extends Column
  A Column where an Encoder has been given for the expected input and return type.
•  abstract class UDFRegistration extends AnyRef
  Functions for registering user-defined functions.
•  case class WhenMatched[T] extends Product with Serializable
  A class for defining actions to be taken when matching rows in a DataFrame during a merge operation.
•  case class WhenNotMatched[T] extends Product with Serializable
  A class for defining actions to be taken when no matching rows are found in a DataFrame during a merge operation.
•  case class WhenNotMatchedBySource[T] extends Product with Serializable
  A class for defining actions to be performed when there is no match by source during a merge operation in a MergeIntoWriter.
•  trait WriteConfigMethods[R] extends AnyRef
  Configuration methods common to create/replace operations and insert/overwrite operations.