functions (Spark 3.5.5 JavaDoc) (original) (raw)
Modifier and Type
Method and Description
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[abs](../../../../org/apache/spark/sql/functions.html#abs-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the absolute value of a numeric value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[acos](../../../../org/apache/spark/sql/functions.html#acos-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[acos](../../../../org/apache/spark/sql/functions.html#acos-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[acosh](../../../../org/apache/spark/sql/functions.html#acosh-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[acosh](../../../../org/apache/spark/sql/functions.html#acosh-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[add_months](../../../../org/apache/spark/sql/functions.html#add%5Fmonths-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") startDate,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") numMonths)
Returns the date that is numMonths after startDate.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[add_months](../../../../org/apache/spark/sql/functions.html#add%5Fmonths-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") startDate, int numMonths)
Returns the date that is numMonths after startDate.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[aes_decrypt](../../../../org/apache/spark/sql/functions.html#aes%5Fdecrypt-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key)
Returns a decrypted value of input.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[aes_decrypt](../../../../org/apache/spark/sql/functions.html#aes%5Fdecrypt-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mode)
Returns a decrypted value of input.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[aes_decrypt](../../../../org/apache/spark/sql/functions.html#aes%5Fdecrypt-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mode,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") padding)
Returns a decrypted value of input.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[aes_decrypt](../../../../org/apache/spark/sql/functions.html#aes%5Fdecrypt-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mode,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") padding,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") aad)
Returns a decrypted value of input using AES in mode with padding.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[aes_encrypt](../../../../org/apache/spark/sql/functions.html#aes%5Fencrypt-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key)
Returns an encrypted value of input.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[aes_encrypt](../../../../org/apache/spark/sql/functions.html#aes%5Fencrypt-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mode)
Returns an encrypted value of input.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[aes_encrypt](../../../../org/apache/spark/sql/functions.html#aes%5Fencrypt-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mode,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") padding)
Returns an encrypted value of input.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[aes_encrypt](../../../../org/apache/spark/sql/functions.html#aes%5Fencrypt-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mode,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") padding,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") iv)
Returns an encrypted value of input.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[aes_encrypt](../../../../org/apache/spark/sql/functions.html#aes%5Fencrypt-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mode,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") padding,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") iv,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") aad)
Returns an encrypted value of input using AES in given mode with the specified padding.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[aggregate](../../../../org/apache/spark/sql/functions.html#aggregate-org.apache.spark.sql.Column-org.apache.spark.sql.Column-scala.Function2-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") expr,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") initialValue, scala.Function2<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> merge)
Applies a binary operator to an initial state and all elements in the array, and reduces this to a single state.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[aggregate](../../../../org/apache/spark/sql/functions.html#aggregate-org.apache.spark.sql.Column-org.apache.spark.sql.Column-scala.Function2-scala.Function1-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") expr,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") initialValue, scala.Function2<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> merge, scala.Function1<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> finish)
Applies a binary operator to an initial state and all elements in the array, and reduces this to a single state.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[any_value](../../../../org/apache/spark/sql/functions.html#any%5Fvalue-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns some value of e for a group of rows.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[any_value](../../../../org/apache/spark/sql/functions.html#any%5Fvalue-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") ignoreNulls)
Aggregate function: returns some value of e for a group of rows.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[any](../../../../org/apache/spark/sql/functions.html#any-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns true if at least one value of e is true.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[approx_count_distinct](../../../../org/apache/spark/sql/functions.html#approx%5Fcount%5Fdistinct-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the approximate number of distinct items in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[approx_count_distinct](../../../../org/apache/spark/sql/functions.html#approx%5Fcount%5Fdistinct-org.apache.spark.sql.Column-double-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, double rsd)
Aggregate function: returns the approximate number of distinct items in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[approx_count_distinct](../../../../org/apache/spark/sql/functions.html#approx%5Fcount%5Fdistinct-java.lang.String-)(String columnName)
Aggregate function: returns the approximate number of distinct items in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[approx_count_distinct](../../../../org/apache/spark/sql/functions.html#approx%5Fcount%5Fdistinct-java.lang.String-double-)(String columnName, double rsd)
Aggregate function: returns the approximate number of distinct items in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[approx_percentile](../../../../org/apache/spark/sql/functions.html#approx%5Fpercentile-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") percentage,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") accuracy)
Aggregate function: returns the approximate percentile of the numeric column col which is the smallest value in the ordered col values (sorted from least to greatest) such that no more than percentage of col values is less than the value or equal to that value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[approxCountDistinct](../../../../org/apache/spark/sql/functions.html#approxCountDistinct-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[approxCountDistinct](../../../../org/apache/spark/sql/functions.html#approxCountDistinct-org.apache.spark.sql.Column-double-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, double rsd)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[approxCountDistinct](../../../../org/apache/spark/sql/functions.html#approxCountDistinct-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[approxCountDistinct](../../../../org/apache/spark/sql/functions.html#approxCountDistinct-java.lang.String-double-)(String columnName, double rsd)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_agg](../../../../org/apache/spark/sql/functions.html#array%5Fagg-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns a list of objects with duplicates.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_append](../../../../org/apache/spark/sql/functions.html#array%5Fappend-org.apache.spark.sql.Column-java.lang.Object-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, Object element)
Returns an ARRAY containing all elements from the source ARRAY as well as the new element.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_compact](../../../../org/apache/spark/sql/functions.html#array%5Fcompact-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column)
Remove all null elements from the given array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_contains](../../../../org/apache/spark/sql/functions.html#array%5Fcontains-org.apache.spark.sql.Column-java.lang.Object-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, Object value)
Returns null if the array is null, true if the array contains value, and false otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_distinct](../../../../org/apache/spark/sql/functions.html#array%5Fdistinct-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Removes duplicate values from the array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_except](../../../../org/apache/spark/sql/functions.html#array%5Fexcept-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col2)
Returns an array of the elements in the first array but not in the second array, without duplicates.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_insert](../../../../org/apache/spark/sql/functions.html#array%5Finsert-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") arr,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pos,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") value)
Adds an item into a given array at a specified position
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_intersect](../../../../org/apache/spark/sql/functions.html#array%5Fintersect-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col2)
Returns an array of the elements in the intersection of the given two arrays, without duplicates.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_join](../../../../org/apache/spark/sql/functions.html#array%5Fjoin-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, String delimiter)
Concatenates the elements of column using the delimiter.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_join](../../../../org/apache/spark/sql/functions.html#array%5Fjoin-org.apache.spark.sql.Column-java.lang.String-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, String delimiter, String nullReplacement)
Concatenates the elements of column using the delimiter.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_max](../../../../org/apache/spark/sql/functions.html#array%5Fmax-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the maximum value in the array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_min](../../../../org/apache/spark/sql/functions.html#array%5Fmin-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the minimum value in the array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_position](../../../../org/apache/spark/sql/functions.html#array%5Fposition-org.apache.spark.sql.Column-java.lang.Object-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, Object value)
Locates the position of the first occurrence of the value in the given array as long.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_prepend](../../../../org/apache/spark/sql/functions.html#array%5Fprepend-org.apache.spark.sql.Column-java.lang.Object-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, Object element)
Returns an array containing value as well as all elements from array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_remove](../../../../org/apache/spark/sql/functions.html#array%5Fremove-org.apache.spark.sql.Column-java.lang.Object-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, Object element)
Remove all elements that equal to element from the given array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_repeat](../../../../org/apache/spark/sql/functions.html#array%5Frepeat-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") left,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") right)
Creates an array containing the left argument repeated the number of times given by the right argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_repeat](../../../../org/apache/spark/sql/functions.html#array%5Frepeat-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int count)
Creates an array containing the left argument repeated the number of times given by the right argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_size](../../../../org/apache/spark/sql/functions.html#array%5Fsize-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the total number of elements in the array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_sort](../../../../org/apache/spark/sql/functions.html#array%5Fsort-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Sorts the input array in ascending order.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_sort](../../../../org/apache/spark/sql/functions.html#array%5Fsort-org.apache.spark.sql.Column-scala.Function2-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, scala.Function2<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> comparator)
Sorts the input array based on the given comparator function.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array_union](../../../../org/apache/spark/sql/functions.html#array%5Funion-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col2)
Returns an array of the elements in the union of the given two arrays, without duplicates.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array](../../../../org/apache/spark/sql/functions.html#array-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Creates a new array column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array](../../../../org/apache/spark/sql/functions.html#array-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
Creates a new array column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array](../../../../org/apache/spark/sql/functions.html#array-java.lang.String-scala.collection.Seq-)(String colName, scala.collection.Seq<String> colNames)
Creates a new array column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[array](../../../../org/apache/spark/sql/functions.html#array-java.lang.String-java.lang.String...-)(String colName, String... colNames)
Creates a new array column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[arrays_overlap](../../../../org/apache/spark/sql/functions.html#arrays%5Foverlap-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") a1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") a2)
Returns true if a1 and a2 have at least one non-null element in common.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[arrays_zip](../../../../org/apache/spark/sql/functions.html#arrays%5Fzip-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... e)
Returns a merged array of structs in which the N-th struct contains all N-th values of input arrays.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[arrays_zip](../../../../org/apache/spark/sql/functions.html#arrays%5Fzip-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> e)
Returns a merged array of structs in which the N-th struct contains all N-th values of input arrays.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[asc_nulls_first](../../../../org/apache/spark/sql/functions.html#asc%5Fnulls%5Ffirst-java.lang.String-)(String columnName)
Returns a sort expression based on ascending order of the column, and null values return before non-null values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[asc_nulls_last](../../../../org/apache/spark/sql/functions.html#asc%5Fnulls%5Flast-java.lang.String-)(String columnName)
Returns a sort expression based on ascending order of the column, and null values appear after non-null values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[asc](../../../../org/apache/spark/sql/functions.html#asc-java.lang.String-)(String columnName)
Returns a sort expression based on ascending order of the column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ascii](../../../../org/apache/spark/sql/functions.html#ascii-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the numeric value of the first character of the string column, and returns the result as an int column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[asin](../../../../org/apache/spark/sql/functions.html#asin-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[asin](../../../../org/apache/spark/sql/functions.html#asin-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[asinh](../../../../org/apache/spark/sql/functions.html#asinh-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[asinh](../../../../org/apache/spark/sql/functions.html#asinh-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[assert_true](../../../../org/apache/spark/sql/functions.html#assert%5Ftrue-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") c)
Returns null if the condition is true, and throws an exception otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[assert_true](../../../../org/apache/spark/sql/functions.html#assert%5Ftrue-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") c,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns null if the condition is true; throws an exception with the error message otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[atan](../../../../org/apache/spark/sql/functions.html#atan-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[atan](../../../../org/apache/spark/sql/functions.html#atan-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[atan2](../../../../org/apache/spark/sql/functions.html#atan2-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") y,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[atan2](../../../../org/apache/spark/sql/functions.html#atan2-org.apache.spark.sql.Column-double-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") y, double xValue)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[atan2](../../../../org/apache/spark/sql/functions.html#atan2-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") y, String xName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[atan2](../../../../org/apache/spark/sql/functions.html#atan2-double-org.apache.spark.sql.Column-)(double yValue,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[atan2](../../../../org/apache/spark/sql/functions.html#atan2-double-java.lang.String-)(double yValue, String xName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[atan2](../../../../org/apache/spark/sql/functions.html#atan2-java.lang.String-org.apache.spark.sql.Column-)(String yName,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[atan2](../../../../org/apache/spark/sql/functions.html#atan2-java.lang.String-double-)(String yName, double xValue)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[atan2](../../../../org/apache/spark/sql/functions.html#atan2-java.lang.String-java.lang.String-)(String yName, String xName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[atanh](../../../../org/apache/spark/sql/functions.html#atanh-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[atanh](../../../../org/apache/spark/sql/functions.html#atanh-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[avg](../../../../org/apache/spark/sql/functions.html#avg-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the average of the values in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[avg](../../../../org/apache/spark/sql/functions.html#avg-java.lang.String-)(String columnName)
Aggregate function: returns the average of the values in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[base64](../../../../org/apache/spark/sql/functions.html#base64-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the BASE64 encoding of a binary column and returns it as a string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bin](../../../../org/apache/spark/sql/functions.html#bin-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
An expression that returns the string representation of the binary value of the given long column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bin](../../../../org/apache/spark/sql/functions.html#bin-java.lang.String-)(String columnName)
An expression that returns the string representation of the binary value of the given long column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bit_and](../../../../org/apache/spark/sql/functions.html#bit%5Fand-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the bitwise AND of all non-null input values, or null if none.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bit_count](../../../../org/apache/spark/sql/functions.html#bit%5Fcount-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the number of bits that are set in the argument expr as an unsigned 64-bit integer, or NULL if the argument is NULL.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bit_get](../../../../org/apache/spark/sql/functions.html#bit%5Fget-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pos)
Returns the value of the bit (0 or 1) at the specified position.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bit_length](../../../../org/apache/spark/sql/functions.html#bit%5Flength-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Calculates the bit length for the specified string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bit_or](../../../../org/apache/spark/sql/functions.html#bit%5For-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the bitwise OR of all non-null input values, or null if none.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bit_xor](../../../../org/apache/spark/sql/functions.html#bit%5Fxor-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the bitwise XOR of all non-null input values, or null if none.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bitmap_bit_position](../../../../org/apache/spark/sql/functions.html#bitmap%5Fbit%5Fposition-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col)
Returns the bit position for the given input column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bitmap_bucket_number](../../../../org/apache/spark/sql/functions.html#bitmap%5Fbucket%5Fnumber-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col)
Returns the bucket number for the given input column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bitmap_construct_agg](../../../../org/apache/spark/sql/functions.html#bitmap%5Fconstruct%5Fagg-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col)
Returns a bitmap with the positions of the bits set from all the values from the input column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bitmap_count](../../../../org/apache/spark/sql/functions.html#bitmap%5Fcount-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col)
Returns the number of set bits in the input bitmap.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bitmap_or_agg](../../../../org/apache/spark/sql/functions.html#bitmap%5For%5Fagg-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col)
Returns a bitmap that is the bitwise OR of all of the bitmaps from the input column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bitwise_not](../../../../org/apache/spark/sql/functions.html#bitwise%5Fnot-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes bitwise NOT (~) of a number.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bitwiseNOT](../../../../org/apache/spark/sql/functions.html#bitwiseNOT-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bool_and](../../../../org/apache/spark/sql/functions.html#bool%5Fand-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns true if all values of e are true.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bool_or](../../../../org/apache/spark/sql/functions.html#bool%5For-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns true if at least one value of e is true.
static <T> [Dataset](../../../../org/apache/spark/sql/Dataset.html "class in org.apache.spark.sql")<T>
[broadcast](../../../../org/apache/spark/sql/functions.html#broadcast-org.apache.spark.sql.Dataset-)([Dataset](../../../../org/apache/spark/sql/Dataset.html "class in org.apache.spark.sql")<T> df)
Marks a DataFrame as small enough for use in broadcast joins.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bround](../../../../org/apache/spark/sql/functions.html#bround-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the value of the column e rounded to 0 decimal places with HALF_EVEN round mode.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bround](../../../../org/apache/spark/sql/functions.html#bround-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int scale)
Round the value of e to scale decimal places with HALF_EVEN round mode if scale is greater than or equal to 0 or at integral part when scale is less than 0.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[btrim](../../../../org/apache/spark/sql/functions.html#btrim-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str)
Removes the leading and trailing space characters from str.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[btrim](../../../../org/apache/spark/sql/functions.html#btrim-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") trim)
Remove the leading and trailing trim characters from str.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bucket](../../../../org/apache/spark/sql/functions.html#bucket-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") numBuckets,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
A transform for any type that partitions by a hash of the input column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[bucket](../../../../org/apache/spark/sql/functions.html#bucket-int-org.apache.spark.sql.Column-)(int numBuckets,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
A transform for any type that partitions by a hash of the input column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[call_function](../../../../org/apache/spark/sql/functions.html#call%5Ffunction-java.lang.String-org.apache.spark.sql.Column...-)(String funcName,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Call a SQL function.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[call_function](../../../../org/apache/spark/sql/functions.html#call%5Ffunction-java.lang.String-scala.collection.Seq-)(String funcName, scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
Call a SQL function.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[call_udf](../../../../org/apache/spark/sql/functions.html#call%5Fudf-java.lang.String-org.apache.spark.sql.Column...-)(String udfName,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Call an user-defined function.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[call_udf](../../../../org/apache/spark/sql/functions.html#call%5Fudf-java.lang.String-scala.collection.Seq-)(String udfName, scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
Call an user-defined function.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[callUDF](../../../../org/apache/spark/sql/functions.html#callUDF-java.lang.String-org.apache.spark.sql.Column...-)(String udfName,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Call an user-defined function.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[callUDF](../../../../org/apache/spark/sql/functions.html#callUDF-java.lang.String-scala.collection.Seq-)(String udfName, scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[cardinality](../../../../org/apache/spark/sql/functions.html#cardinality-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns length of array or map.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[cbrt](../../../../org/apache/spark/sql/functions.html#cbrt-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the cube-root of the given value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[cbrt](../../../../org/apache/spark/sql/functions.html#cbrt-java.lang.String-)(String columnName)
Computes the cube-root of the given column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ceil](../../../../org/apache/spark/sql/functions.html#ceil-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the ceiling of the given value of e to 0 decimal places.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ceil](../../../../org/apache/spark/sql/functions.html#ceil-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") scale)
Computes the ceiling of the given value of e to scale decimal places.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ceil](../../../../org/apache/spark/sql/functions.html#ceil-java.lang.String-)(String columnName)
Computes the ceiling of the given value of e to 0 decimal places.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ceiling](../../../../org/apache/spark/sql/functions.html#ceiling-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the ceiling of the given value of e to 0 decimal places.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ceiling](../../../../org/apache/spark/sql/functions.html#ceiling-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") scale)
Computes the ceiling of the given value of e to scale decimal places.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[char_length](../../../../org/apache/spark/sql/functions.html#char%5Flength-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str)
Returns the character length of string data or number of bytes of binary data.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[character_length](../../../../org/apache/spark/sql/functions.html#character%5Flength-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str)
Returns the character length of string data or number of bytes of binary data.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[chr](../../../../org/apache/spark/sql/functions.html#chr-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") n)
Returns the ASCII character having the binary equivalent to n.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[coalesce](../../../../org/apache/spark/sql/functions.html#coalesce-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... e)
Returns the first column that is not null, or null if all inputs are null.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[coalesce](../../../../org/apache/spark/sql/functions.html#coalesce-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> e)
Returns the first column that is not null, or null if all inputs are null.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[col](../../../../org/apache/spark/sql/functions.html#col-java.lang.String-)(String colName)
Returns a Column based on the given column name.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[collect_list](../../../../org/apache/spark/sql/functions.html#collect%5Flist-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns a list of objects with duplicates.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[collect_list](../../../../org/apache/spark/sql/functions.html#collect%5Flist-java.lang.String-)(String columnName)
Aggregate function: returns a list of objects with duplicates.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[collect_set](../../../../org/apache/spark/sql/functions.html#collect%5Fset-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns a set of objects with duplicate elements eliminated.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[collect_set](../../../../org/apache/spark/sql/functions.html#collect%5Fset-java.lang.String-)(String columnName)
Aggregate function: returns a set of objects with duplicate elements eliminated.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[column](../../../../org/apache/spark/sql/functions.html#column-java.lang.String-)(String colName)
Returns a Column based on the given column name.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[concat_ws](../../../../org/apache/spark/sql/functions.html#concat%5Fws-java.lang.String-org.apache.spark.sql.Column...-)(String sep,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... exprs)
Concatenates multiple input string columns together into a single string column, using the given separator.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[concat_ws](../../../../org/apache/spark/sql/functions.html#concat%5Fws-java.lang.String-scala.collection.Seq-)(String sep, scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> exprs)
Concatenates multiple input string columns together into a single string column, using the given separator.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[concat](../../../../org/apache/spark/sql/functions.html#concat-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... exprs)
Concatenates multiple input columns together into a single column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[concat](../../../../org/apache/spark/sql/functions.html#concat-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> exprs)
Concatenates multiple input columns together into a single column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[contains](../../../../org/apache/spark/sql/functions.html#contains-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") left,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") right)
Returns a boolean.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[conv](../../../../org/apache/spark/sql/functions.html#conv-org.apache.spark.sql.Column-int-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") num, int fromBase, int toBase)
Convert a number in a string column from one base to another.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[convert_timezone](../../../../org/apache/spark/sql/functions.html#convert%5Ftimezone-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") targetTz,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") sourceTs)
Converts the timestamp without time zone sourceTs from the current time zone to targetTz.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[convert_timezone](../../../../org/apache/spark/sql/functions.html#convert%5Ftimezone-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") sourceTz,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") targetTz,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") sourceTs)
Converts the timestamp without time zone sourceTs from the sourceTz time zone to targetTz.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[corr](../../../../org/apache/spark/sql/functions.html#corr-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column2)
Aggregate function: returns the Pearson Correlation Coefficient for two columns.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[corr](../../../../org/apache/spark/sql/functions.html#corr-java.lang.String-java.lang.String-)(String columnName1, String columnName2)
Aggregate function: returns the Pearson Correlation Coefficient for two columns.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[cos](../../../../org/apache/spark/sql/functions.html#cos-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[cos](../../../../org/apache/spark/sql/functions.html#cos-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[cosh](../../../../org/apache/spark/sql/functions.html#cosh-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[cosh](../../../../org/apache/spark/sql/functions.html#cosh-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[cot](../../../../org/apache/spark/sql/functions.html#cot-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[count_distinct](../../../../org/apache/spark/sql/functions.html#count%5Fdistinct-org.apache.spark.sql.Column-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") expr,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... exprs)
Aggregate function: returns the number of distinct items in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[count_distinct](../../../../org/apache/spark/sql/functions.html#count%5Fdistinct-org.apache.spark.sql.Column-scala.collection.Seq-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") expr, scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> exprs)
Aggregate function: returns the number of distinct items in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[count_if](../../../../org/apache/spark/sql/functions.html#count%5Fif-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the number of TRUE values for the expression.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[count_min_sketch](../../../../org/apache/spark/sql/functions.html#count%5Fmin%5Fsketch-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") eps,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") confidence,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") seed)
Returns a count-min sketch of a column with the given esp, confidence and seed.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[count](../../../../org/apache/spark/sql/functions.html#count-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the number of items in a group.
static [TypedColumn](../../../../org/apache/spark/sql/TypedColumn.html "class in org.apache.spark.sql")<Object,Object>
[count](../../../../org/apache/spark/sql/functions.html#count-java.lang.String-)(String columnName)
Aggregate function: returns the number of items in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[countDistinct](../../../../org/apache/spark/sql/functions.html#countDistinct-org.apache.spark.sql.Column-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") expr,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... exprs)
Aggregate function: returns the number of distinct items in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[countDistinct](../../../../org/apache/spark/sql/functions.html#countDistinct-org.apache.spark.sql.Column-scala.collection.Seq-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") expr, scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> exprs)
Aggregate function: returns the number of distinct items in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[countDistinct](../../../../org/apache/spark/sql/functions.html#countDistinct-java.lang.String-scala.collection.Seq-)(String columnName, scala.collection.Seq<String> columnNames)
Aggregate function: returns the number of distinct items in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[countDistinct](../../../../org/apache/spark/sql/functions.html#countDistinct-java.lang.String-java.lang.String...-)(String columnName, String... columnNames)
Aggregate function: returns the number of distinct items in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[covar_pop](../../../../org/apache/spark/sql/functions.html#covar%5Fpop-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column2)
Aggregate function: returns the population covariance for two columns.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[covar_pop](../../../../org/apache/spark/sql/functions.html#covar%5Fpop-java.lang.String-java.lang.String-)(String columnName1, String columnName2)
Aggregate function: returns the population covariance for two columns.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[covar_samp](../../../../org/apache/spark/sql/functions.html#covar%5Fsamp-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column2)
Aggregate function: returns the sample covariance for two columns.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[covar_samp](../../../../org/apache/spark/sql/functions.html#covar%5Fsamp-java.lang.String-java.lang.String-)(String columnName1, String columnName2)
Aggregate function: returns the sample covariance for two columns.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[crc32](../../../../org/apache/spark/sql/functions.html#crc32-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Calculates the cyclic redundancy check value (CRC32) of a binary column and returns the value as a bigint.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[csc](../../../../org/apache/spark/sql/functions.html#csc-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[cume_dist](../../../../org/apache/spark/sql/functions.html#cume%5Fdist--)()
Window function: returns the cumulative distribution of values within a window partition, i.e.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[curdate](../../../../org/apache/spark/sql/functions.html#curdate--)()
Returns the current date at the start of query evaluation as a date column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[current_catalog](../../../../org/apache/spark/sql/functions.html#current%5Fcatalog--)()
Returns the current catalog.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[current_database](../../../../org/apache/spark/sql/functions.html#current%5Fdatabase--)()
Returns the current database.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[current_date](../../../../org/apache/spark/sql/functions.html#current%5Fdate--)()
Returns the current date at the start of query evaluation as a date column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[current_schema](../../../../org/apache/spark/sql/functions.html#current%5Fschema--)()
Returns the current schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[current_timestamp](../../../../org/apache/spark/sql/functions.html#current%5Ftimestamp--)()
Returns the current timestamp at the start of query evaluation as a timestamp column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[current_timezone](../../../../org/apache/spark/sql/functions.html#current%5Ftimezone--)()
Returns the current session local timezone.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[current_user](../../../../org/apache/spark/sql/functions.html#current%5Fuser--)()
Returns the user name of current execution context.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[date_add](../../../../org/apache/spark/sql/functions.html#date%5Fadd-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") start,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days)
Returns the date that is days days after start
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[date_add](../../../../org/apache/spark/sql/functions.html#date%5Fadd-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") start, int days)
Returns the date that is days days after start
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[date_diff](../../../../org/apache/spark/sql/functions.html#date%5Fdiff-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") end,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") start)
Returns the number of days from start to end.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[date_format](../../../../org/apache/spark/sql/functions.html#date%5Fformat-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") dateExpr, String format)
Converts a date/timestamp/string to a value of string in the format specified by the date format given by the second argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[date_from_unix_date](../../../../org/apache/spark/sql/functions.html#date%5Ffrom%5Funix%5Fdate-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days)
Create date from the number of days since 1970-01-01.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[date_part](../../../../org/apache/spark/sql/functions.html#date%5Fpart-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") field,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") source)
Extracts a part of the date/timestamp or interval source.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[date_sub](../../../../org/apache/spark/sql/functions.html#date%5Fsub-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") start,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days)
Returns the date that is days days before start
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[date_sub](../../../../org/apache/spark/sql/functions.html#date%5Fsub-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") start, int days)
Returns the date that is days days before start
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[date_trunc](../../../../org/apache/spark/sql/functions.html#date%5Ftrunc-java.lang.String-org.apache.spark.sql.Column-)(String format,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") timestamp)
Returns timestamp truncated to the unit specified by the format.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[dateadd](../../../../org/apache/spark/sql/functions.html#dateadd-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") start,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days)
Returns the date that is days days after start
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[datediff](../../../../org/apache/spark/sql/functions.html#datediff-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") end,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") start)
Returns the number of days from start to end.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[datepart](../../../../org/apache/spark/sql/functions.html#datepart-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") field,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") source)
Extracts a part of the date/timestamp or interval source.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[day](../../../../org/apache/spark/sql/functions.html#day-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Extracts the day of the month as an integer from a given date/timestamp/string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[dayofmonth](../../../../org/apache/spark/sql/functions.html#dayofmonth-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Extracts the day of the month as an integer from a given date/timestamp/string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[dayofweek](../../../../org/apache/spark/sql/functions.html#dayofweek-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Extracts the day of the week as an integer from a given date/timestamp/string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[dayofyear](../../../../org/apache/spark/sql/functions.html#dayofyear-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Extracts the day of the year as an integer from a given date/timestamp/string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[days](../../../../org/apache/spark/sql/functions.html#days-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
A transform for timestamps and dates to partition data into days.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[decode](../../../../org/apache/spark/sql/functions.html#decode-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") value, String charset)
Computes the first argument into a string from a binary using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16').
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[degrees](../../../../org/apache/spark/sql/functions.html#degrees-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Converts an angle measured in radians to an approximately equivalent angle measured in degrees.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[degrees](../../../../org/apache/spark/sql/functions.html#degrees-java.lang.String-)(String columnName)
Converts an angle measured in radians to an approximately equivalent angle measured in degrees.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[dense_rank](../../../../org/apache/spark/sql/functions.html#dense%5Frank--)()
Window function: returns the rank of rows within a window partition, without any gaps.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[desc_nulls_first](../../../../org/apache/spark/sql/functions.html#desc%5Fnulls%5Ffirst-java.lang.String-)(String columnName)
Returns a sort expression based on the descending order of the column, and null values appear before non-null values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[desc_nulls_last](../../../../org/apache/spark/sql/functions.html#desc%5Fnulls%5Flast-java.lang.String-)(String columnName)
Returns a sort expression based on the descending order of the column, and null values appear after non-null values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[desc](../../../../org/apache/spark/sql/functions.html#desc-java.lang.String-)(String columnName)
Returns a sort expression based on the descending order of the column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[e](../../../../org/apache/spark/sql/functions.html#e--)()
Returns Euler's number.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[element_at](../../../../org/apache/spark/sql/functions.html#element%5Fat-org.apache.spark.sql.Column-java.lang.Object-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, Object value)
Returns element of array at given index in value if column is array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[elt](../../../../org/apache/spark/sql/functions.html#elt-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... inputs)
Returns the n-th input, e.g., returns input2 when n is 2.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[elt](../../../../org/apache/spark/sql/functions.html#elt-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> inputs)
Returns the n-th input, e.g., returns input2 when n is 2.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[encode](../../../../org/apache/spark/sql/functions.html#encode-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") value, String charset)
Computes the first argument into a binary from a string using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16').
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[endswith](../../../../org/apache/spark/sql/functions.html#endswith-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") suffix)
Returns a boolean.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[equal_null](../../../../org/apache/spark/sql/functions.html#equal%5Fnull-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col2)
Returns same result as the EQUAL(=) operator for non-null operands, but returns true if both are null, false if one of the them is null.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[every](../../../../org/apache/spark/sql/functions.html#every-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns true if all values of e are true.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[exists](../../../../org/apache/spark/sql/functions.html#exists-org.apache.spark.sql.Column-scala.Function1-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, scala.Function1<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> f)
Returns whether a predicate holds for one or more elements in the array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[exp](../../../../org/apache/spark/sql/functions.html#exp-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the exponential of the given value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[exp](../../../../org/apache/spark/sql/functions.html#exp-java.lang.String-)(String columnName)
Computes the exponential of the given column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[explode_outer](../../../../org/apache/spark/sql/functions.html#explode%5Fouter-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Creates a new row for each element in the given array or map column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[explode](../../../../org/apache/spark/sql/functions.html#explode-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Creates a new row for each element in the given array or map column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[expm1](../../../../org/apache/spark/sql/functions.html#expm1-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the exponential of the given value minus one.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[expm1](../../../../org/apache/spark/sql/functions.html#expm1-java.lang.String-)(String columnName)
Computes the exponential of the given column minus one.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[expr](../../../../org/apache/spark/sql/functions.html#expr-java.lang.String-)(String expr)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[extract](../../../../org/apache/spark/sql/functions.html#extract-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") field,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") source)
Extracts a part of the date/timestamp or interval source.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[factorial](../../../../org/apache/spark/sql/functions.html#factorial-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the factorial of the given value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[filter](../../../../org/apache/spark/sql/functions.html#filter-org.apache.spark.sql.Column-scala.Function1-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, scala.Function1<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> f)
Returns an array of elements for which a predicate holds in a given array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[filter](../../../../org/apache/spark/sql/functions.html#filter-org.apache.spark.sql.Column-scala.Function2-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, scala.Function2<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> f)
Returns an array of elements for which a predicate holds in a given array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[find_in_set](../../../../org/apache/spark/sql/functions.html#find%5Fin%5Fset-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") strArray)
Returns the index (1-based) of the given string (str) in the comma-delimited list (strArray).
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[first_value](../../../../org/apache/spark/sql/functions.html#first%5Fvalue-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the first value in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[first_value](../../../../org/apache/spark/sql/functions.html#first%5Fvalue-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") ignoreNulls)
Aggregate function: returns the first value in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[first](../../../../org/apache/spark/sql/functions.html#first-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the first value in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[first](../../../../org/apache/spark/sql/functions.html#first-org.apache.spark.sql.Column-boolean-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, boolean ignoreNulls)
Aggregate function: returns the first value in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[first](../../../../org/apache/spark/sql/functions.html#first-java.lang.String-)(String columnName)
Aggregate function: returns the first value of a column in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[first](../../../../org/apache/spark/sql/functions.html#first-java.lang.String-boolean-)(String columnName, boolean ignoreNulls)
Aggregate function: returns the first value of a column in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[flatten](../../../../org/apache/spark/sql/functions.html#flatten-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Creates a single array from an array of arrays.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[floor](../../../../org/apache/spark/sql/functions.html#floor-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the floor of the given value of e to 0 decimal places.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[floor](../../../../org/apache/spark/sql/functions.html#floor-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") scale)
Computes the floor of the given value of e to scale decimal places.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[floor](../../../../org/apache/spark/sql/functions.html#floor-java.lang.String-)(String columnName)
Computes the floor of the given column value to 0 decimal places.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[forall](../../../../org/apache/spark/sql/functions.html#forall-org.apache.spark.sql.Column-scala.Function1-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, scala.Function1<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> f)
Returns whether a predicate holds for every element in the array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[format_number](../../../../org/apache/spark/sql/functions.html#format%5Fnumber-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x, int d)
Formats numeric column x to a format like '#,###,###.##', rounded to d decimal places with HALF_EVEN round mode, and returns the result as a string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[format_string](../../../../org/apache/spark/sql/functions.html#format%5Fstring-java.lang.String-org.apache.spark.sql.Column...-)(String format,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... arguments)
Formats the arguments in printf-style and returns the result as a string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[format_string](../../../../org/apache/spark/sql/functions.html#format%5Fstring-java.lang.String-scala.collection.Seq-)(String format, scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> arguments)
Formats the arguments in printf-style and returns the result as a string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_csv](../../../../org/apache/spark/sql/functions.html#from%5Fcsv-org.apache.spark.sql.Column-org.apache.spark.sql.Column-java.util.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") schema, java.util.Map<String,String> options)
(Java-specific) Parses a column containing a CSV string into a StructType with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_csv](../../../../org/apache/spark/sql/functions.html#from%5Fcsv-org.apache.spark.sql.Column-org.apache.spark.sql.types.StructType-scala.collection.immutable.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[StructType](../../../../org/apache/spark/sql/types/StructType.html "class in org.apache.spark.sql.types") schema, scala.collection.immutable.Map<String,String> options)
Parses a column containing a CSV string into a StructType with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_json](../../../../org/apache/spark/sql/functions.html#from%5Fjson-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") schema)
(Scala-specific) Parses a column containing a JSON string into a MapType with StringType as keys type, StructType or ArrayType of StructTypes with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_json](../../../../org/apache/spark/sql/functions.html#from%5Fjson-org.apache.spark.sql.Column-org.apache.spark.sql.Column-java.util.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") schema, java.util.Map<String,String> options)
(Java-specific) Parses a column containing a JSON string into a MapType with StringType as keys type, StructType or ArrayType of StructTypes with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_json](../../../../org/apache/spark/sql/functions.html#from%5Fjson-org.apache.spark.sql.Column-org.apache.spark.sql.types.DataType-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") schema)
Parses a column containing a JSON string into a MapType with StringType as keys type,StructType or ArrayType with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_json](../../../../org/apache/spark/sql/functions.html#from%5Fjson-org.apache.spark.sql.Column-org.apache.spark.sql.types.DataType-scala.collection.immutable.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") schema, scala.collection.immutable.Map<String,String> options)
(Scala-specific) Parses a column containing a JSON string into a MapType with StringType as keys type, StructType or ArrayType with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_json](../../../../org/apache/spark/sql/functions.html#from%5Fjson-org.apache.spark.sql.Column-org.apache.spark.sql.types.DataType-java.util.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") schema, java.util.Map<String,String> options)
(Java-specific) Parses a column containing a JSON string into a MapType with StringType as keys type, StructType or ArrayType with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_json](../../../../org/apache/spark/sql/functions.html#from%5Fjson-org.apache.spark.sql.Column-java.lang.String-java.util.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, String schema, java.util.Map<String,String> options)
(Java-specific) Parses a column containing a JSON string into a MapType with StringType as keys type, StructType or ArrayType with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_json](../../../../org/apache/spark/sql/functions.html#from%5Fjson-org.apache.spark.sql.Column-java.lang.String-scala.collection.immutable.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, String schema, scala.collection.immutable.Map<String,String> options)
(Scala-specific) Parses a column containing a JSON string into a MapType with StringType as keys type, StructType or ArrayType with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_json](../../../../org/apache/spark/sql/functions.html#from%5Fjson-org.apache.spark.sql.Column-org.apache.spark.sql.types.StructType-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[StructType](../../../../org/apache/spark/sql/types/StructType.html "class in org.apache.spark.sql.types") schema)
Parses a column containing a JSON string into a StructType with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_json](../../../../org/apache/spark/sql/functions.html#from%5Fjson-org.apache.spark.sql.Column-org.apache.spark.sql.types.StructType-scala.collection.immutable.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[StructType](../../../../org/apache/spark/sql/types/StructType.html "class in org.apache.spark.sql.types") schema, scala.collection.immutable.Map<String,String> options)
(Scala-specific) Parses a column containing a JSON string into a StructType with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_json](../../../../org/apache/spark/sql/functions.html#from%5Fjson-org.apache.spark.sql.Column-org.apache.spark.sql.types.StructType-java.util.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[StructType](../../../../org/apache/spark/sql/types/StructType.html "class in org.apache.spark.sql.types") schema, java.util.Map<String,String> options)
(Java-specific) Parses a column containing a JSON string into a StructType with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_unixtime](../../../../org/apache/spark/sql/functions.html#from%5Funixtime-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") ut)
Converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current system time zone in the yyyy-MM-dd HH:mm:ss format.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_unixtime](../../../../org/apache/spark/sql/functions.html#from%5Funixtime-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") ut, String f)
Converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current system time zone in the given format.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_utc_timestamp](../../../../org/apache/spark/sql/functions.html#from%5Futc%5Ftimestamp-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") ts,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") tz)
Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in UTC, and renders that time as a timestamp in the given time zone.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[from_utc_timestamp](../../../../org/apache/spark/sql/functions.html#from%5Futc%5Ftimestamp-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") ts, String tz)
Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in UTC, and renders that time as a timestamp in the given time zone.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[get_json_object](../../../../org/apache/spark/sql/functions.html#get%5Fjson%5Fobject-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, String path)
Extracts json object from a json string based on json path specified, and returns json string of the extracted json object.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[get](../../../../org/apache/spark/sql/functions.html#get-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") index)
Returns element of array at given (0-based) index.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[getbit](../../../../org/apache/spark/sql/functions.html#getbit-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pos)
Returns the value of the bit (0 or 1) at the specified position.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[greatest](../../../../org/apache/spark/sql/functions.html#greatest-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... exprs)
Returns the greatest value of the list of values, skipping null values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[greatest](../../../../org/apache/spark/sql/functions.html#greatest-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> exprs)
Returns the greatest value of the list of values, skipping null values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[greatest](../../../../org/apache/spark/sql/functions.html#greatest-java.lang.String-scala.collection.Seq-)(String columnName, scala.collection.Seq<String> columnNames)
Returns the greatest value of the list of column names, skipping null values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[greatest](../../../../org/apache/spark/sql/functions.html#greatest-java.lang.String-java.lang.String...-)(String columnName, String... columnNames)
Returns the greatest value of the list of column names, skipping null values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[grouping_id](../../../../org/apache/spark/sql/functions.html#grouping%5Fid-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Aggregate function: returns the level of grouping, equals to
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[grouping_id](../../../../org/apache/spark/sql/functions.html#grouping%5Fid-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
Aggregate function: returns the level of grouping, equals to
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[grouping_id](../../../../org/apache/spark/sql/functions.html#grouping%5Fid-java.lang.String-scala.collection.Seq-)(String colName, scala.collection.Seq<String> colNames)
Aggregate function: returns the level of grouping, equals to
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[grouping_id](../../../../org/apache/spark/sql/functions.html#grouping%5Fid-java.lang.String-java.lang.String...-)(String colName, String... colNames)
Aggregate function: returns the level of grouping, equals to
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[grouping](../../../../org/apache/spark/sql/functions.html#grouping-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: indicates whether a specified column in a GROUP BY list is aggregated or not, returns 1 for aggregated or 0 for not aggregated in the result set.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[grouping](../../../../org/apache/spark/sql/functions.html#grouping-java.lang.String-)(String columnName)
Aggregate function: indicates whether a specified column in a GROUP BY list is aggregated or not, returns 1 for aggregated or 0 for not aggregated in the result set.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hash](../../../../org/apache/spark/sql/functions.html#hash-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Calculates the hash code of given columns, and returns the result as an int column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hash](../../../../org/apache/spark/sql/functions.html#hash-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
Calculates the hash code of given columns, and returns the result as an int column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hex](../../../../org/apache/spark/sql/functions.html#hex-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column)
Computes hex value of the given column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[histogram_numeric](../../../../org/apache/spark/sql/functions.html#histogram%5Fnumeric-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") nBins)
Aggregate function: computes a histogram on numeric 'expr' using nb bins.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_sketch_agg](../../../../org/apache/spark/sql/functions.html#hll%5Fsketch%5Fagg-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the updatable binary representation of the Datasketches HllSketch configured with default lgConfigK value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_sketch_agg](../../../../org/apache/spark/sql/functions.html#hll%5Fsketch%5Fagg-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") lgConfigK)
Aggregate function: returns the updatable binary representation of the Datasketches HllSketch configured with lgConfigK arg.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_sketch_agg](../../../../org/apache/spark/sql/functions.html#hll%5Fsketch%5Fagg-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int lgConfigK)
Aggregate function: returns the updatable binary representation of the Datasketches HllSketch configured with lgConfigK arg.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_sketch_agg](../../../../org/apache/spark/sql/functions.html#hll%5Fsketch%5Fagg-java.lang.String-)(String columnName)
Aggregate function: returns the updatable binary representation of the Datasketches HllSketch configured with default lgConfigK value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_sketch_agg](../../../../org/apache/spark/sql/functions.html#hll%5Fsketch%5Fagg-java.lang.String-int-)(String columnName, int lgConfigK)
Aggregate function: returns the updatable binary representation of the Datasketches HllSketch configured with lgConfigK arg.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_sketch_estimate](../../../../org/apache/spark/sql/functions.html#hll%5Fsketch%5Festimate-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") c)
Returns the estimated number of unique values given the binary representation of a Datasketches HllSketch.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_sketch_estimate](../../../../org/apache/spark/sql/functions.html#hll%5Fsketch%5Festimate-java.lang.String-)(String columnName)
Returns the estimated number of unique values given the binary representation of a Datasketches HllSketch.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_union_agg](../../../../org/apache/spark/sql/functions.html#hll%5Funion%5Fagg-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the updatable binary representation of the Datasketches HllSketch, generated by merging previously created Datasketches HllSketch instances via a Datasketches Union instance.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_union_agg](../../../../org/apache/spark/sql/functions.html#hll%5Funion%5Fagg-org.apache.spark.sql.Column-boolean-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, boolean allowDifferentLgConfigK)
Aggregate function: returns the updatable binary representation of the Datasketches HllSketch, generated by merging previously created Datasketches HllSketch instances via a Datasketches Union instance.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_union_agg](../../../../org/apache/spark/sql/functions.html#hll%5Funion%5Fagg-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") allowDifferentLgConfigK)
Aggregate function: returns the updatable binary representation of the Datasketches HllSketch, generated by merging previously created Datasketches HllSketch instances via a Datasketches Union instance.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_union_agg](../../../../org/apache/spark/sql/functions.html#hll%5Funion%5Fagg-java.lang.String-)(String columnName)
Aggregate function: returns the updatable binary representation of the Datasketches HllSketch, generated by merging previously created Datasketches HllSketch instances via a Datasketches Union instance.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_union_agg](../../../../org/apache/spark/sql/functions.html#hll%5Funion%5Fagg-java.lang.String-boolean-)(String columnName, boolean allowDifferentLgConfigK)
Aggregate function: returns the updatable binary representation of the Datasketches HllSketch, generated by merging previously created Datasketches HllSketch instances via a Datasketches Union instance.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_union](../../../../org/apache/spark/sql/functions.html#hll%5Funion-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") c1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") c2)
Merges two binary representations of Datasketches HllSketch objects, using a Datasketches Union object.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_union](../../../../org/apache/spark/sql/functions.html#hll%5Funion-org.apache.spark.sql.Column-org.apache.spark.sql.Column-boolean-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") c1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") c2, boolean allowDifferentLgConfigK)
Merges two binary representations of Datasketches HllSketch objects, using a Datasketches Union object.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_union](../../../../org/apache/spark/sql/functions.html#hll%5Funion-java.lang.String-java.lang.String-)(String columnName1, String columnName2)
Merges two binary representations of Datasketches HllSketch objects, using a Datasketches Union object.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hll_union](../../../../org/apache/spark/sql/functions.html#hll%5Funion-java.lang.String-java.lang.String-boolean-)(String columnName1, String columnName2, boolean allowDifferentLgConfigK)
Merges two binary representations of Datasketches HllSketch objects, using a Datasketches Union object.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hour](../../../../org/apache/spark/sql/functions.html#hour-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Extracts the hours as an integer from a given date/timestamp/string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hours](../../../../org/apache/spark/sql/functions.html#hours-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
A transform for timestamps to partition data into hours.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hypot](../../../../org/apache/spark/sql/functions.html#hypot-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") l,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") r)
Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hypot](../../../../org/apache/spark/sql/functions.html#hypot-org.apache.spark.sql.Column-double-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") l, double r)
Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hypot](../../../../org/apache/spark/sql/functions.html#hypot-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") l, String rightName)
Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hypot](../../../../org/apache/spark/sql/functions.html#hypot-double-org.apache.spark.sql.Column-)(double l,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") r)
Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hypot](../../../../org/apache/spark/sql/functions.html#hypot-double-java.lang.String-)(double l, String rightName)
Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hypot](../../../../org/apache/spark/sql/functions.html#hypot-java.lang.String-org.apache.spark.sql.Column-)(String leftName,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") r)
Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hypot](../../../../org/apache/spark/sql/functions.html#hypot-java.lang.String-double-)(String leftName, double r)
Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[hypot](../../../../org/apache/spark/sql/functions.html#hypot-java.lang.String-java.lang.String-)(String leftName, String rightName)
Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ifnull](../../../../org/apache/spark/sql/functions.html#ifnull-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col2)
Returns col2 if col1 is null, or col1 otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ilike](../../../../org/apache/spark/sql/functions.html#ilike-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pattern)
Returns true if str matches pattern with escapeChar('\') case-insensitively, null if any arguments are null, false otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ilike](../../../../org/apache/spark/sql/functions.html#ilike-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pattern,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") escapeChar)
Returns true if str matches pattern with escapeChar case-insensitively, null if any arguments are null, false otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[initcap](../../../../org/apache/spark/sql/functions.html#initcap-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns a new string column by converting the first letter of each word to uppercase.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[inline_outer](../../../../org/apache/spark/sql/functions.html#inline%5Fouter-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Creates a new row for each element in the given array of structs.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[inline](../../../../org/apache/spark/sql/functions.html#inline-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Creates a new row for each element in the given array of structs.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[input_file_block_length](../../../../org/apache/spark/sql/functions.html#input%5Ffile%5Fblock%5Flength--)()
Returns the length of the block being read, or -1 if not available.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[input_file_block_start](../../../../org/apache/spark/sql/functions.html#input%5Ffile%5Fblock%5Fstart--)()
Returns the start offset of the block being read, or -1 if not available.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[input_file_name](../../../../org/apache/spark/sql/functions.html#input%5Ffile%5Fname--)()
Creates a string column for the file name of the current Spark task.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[instr](../../../../org/apache/spark/sql/functions.html#instr-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str, String substring)
Locate the position of the first occurrence of substr column in the given string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[isnan](../../../../org/apache/spark/sql/functions.html#isnan-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Return true iff the column is NaN.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[isnotnull](../../../../org/apache/spark/sql/functions.html#isnotnull-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col)
Returns true if col is not null, or false otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[isnull](../../../../org/apache/spark/sql/functions.html#isnull-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Return true iff the column is null.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[java_method](../../../../org/apache/spark/sql/functions.html#java%5Fmethod-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Calls a method with reflection.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[java_method](../../../../org/apache/spark/sql/functions.html#java%5Fmethod-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
Calls a method with reflection.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[json_array_length](../../../../org/apache/spark/sql/functions.html#json%5Farray%5Flength-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") jsonArray)
Returns the number of elements in the outermost JSON array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[json_object_keys](../../../../org/apache/spark/sql/functions.html#json%5Fobject%5Fkeys-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") json)
Returns all the keys of the outermost JSON object as an array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[json_tuple](../../../../org/apache/spark/sql/functions.html#json%5Ftuple-org.apache.spark.sql.Column-scala.collection.Seq-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") json, scala.collection.Seq<String> fields)
Creates a new row for a json column according to the given field names.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[json_tuple](../../../../org/apache/spark/sql/functions.html#json%5Ftuple-org.apache.spark.sql.Column-java.lang.String...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") json, String... fields)
Creates a new row for a json column according to the given field names.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[kurtosis](../../../../org/apache/spark/sql/functions.html#kurtosis-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the kurtosis of the values in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[kurtosis](../../../../org/apache/spark/sql/functions.html#kurtosis-java.lang.String-)(String columnName)
Aggregate function: returns the kurtosis of the values in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lag](../../../../org/apache/spark/sql/functions.html#lag-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int offset)
Window function: returns the value that is offset rows before the current row, andnull if there is less than offset rows before the current row.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lag](../../../../org/apache/spark/sql/functions.html#lag-org.apache.spark.sql.Column-int-java.lang.Object-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int offset, Object defaultValue)
Window function: returns the value that is offset rows before the current row, anddefaultValue if there is less than offset rows before the current row.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lag](../../../../org/apache/spark/sql/functions.html#lag-org.apache.spark.sql.Column-int-java.lang.Object-boolean-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int offset, Object defaultValue, boolean ignoreNulls)
Window function: returns the value that is offset rows before the current row, anddefaultValue if there is less than offset rows before the current row.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lag](../../../../org/apache/spark/sql/functions.html#lag-java.lang.String-int-)(String columnName, int offset)
Window function: returns the value that is offset rows before the current row, andnull if there is less than offset rows before the current row.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lag](../../../../org/apache/spark/sql/functions.html#lag-java.lang.String-int-java.lang.Object-)(String columnName, int offset, Object defaultValue)
Window function: returns the value that is offset rows before the current row, anddefaultValue if there is less than offset rows before the current row.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[last_day](../../../../org/apache/spark/sql/functions.html#last%5Fday-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the last day of the month which the given date belongs to.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[last_value](../../../../org/apache/spark/sql/functions.html#last%5Fvalue-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the last value in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[last_value](../../../../org/apache/spark/sql/functions.html#last%5Fvalue-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") ignoreNulls)
Aggregate function: returns the last value in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[last](../../../../org/apache/spark/sql/functions.html#last-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the last value in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[last](../../../../org/apache/spark/sql/functions.html#last-org.apache.spark.sql.Column-boolean-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, boolean ignoreNulls)
Aggregate function: returns the last value in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[last](../../../../org/apache/spark/sql/functions.html#last-java.lang.String-)(String columnName)
Aggregate function: returns the last value of the column in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[last](../../../../org/apache/spark/sql/functions.html#last-java.lang.String-boolean-)(String columnName, boolean ignoreNulls)
Aggregate function: returns the last value of the column in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lcase](../../../../org/apache/spark/sql/functions.html#lcase-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str)
Returns str with all characters changed to lowercase.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lead](../../../../org/apache/spark/sql/functions.html#lead-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int offset)
Window function: returns the value that is offset rows after the current row, andnull if there is less than offset rows after the current row.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lead](../../../../org/apache/spark/sql/functions.html#lead-org.apache.spark.sql.Column-int-java.lang.Object-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int offset, Object defaultValue)
Window function: returns the value that is offset rows after the current row, anddefaultValue if there is less than offset rows after the current row.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lead](../../../../org/apache/spark/sql/functions.html#lead-org.apache.spark.sql.Column-int-java.lang.Object-boolean-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int offset, Object defaultValue, boolean ignoreNulls)
Window function: returns the value that is offset rows after the current row, anddefaultValue if there is less than offset rows after the current row.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lead](../../../../org/apache/spark/sql/functions.html#lead-java.lang.String-int-)(String columnName, int offset)
Window function: returns the value that is offset rows after the current row, andnull if there is less than offset rows after the current row.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lead](../../../../org/apache/spark/sql/functions.html#lead-java.lang.String-int-java.lang.Object-)(String columnName, int offset, Object defaultValue)
Window function: returns the value that is offset rows after the current row, anddefaultValue if there is less than offset rows after the current row.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[least](../../../../org/apache/spark/sql/functions.html#least-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... exprs)
Returns the least value of the list of values, skipping null values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[least](../../../../org/apache/spark/sql/functions.html#least-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> exprs)
Returns the least value of the list of values, skipping null values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[least](../../../../org/apache/spark/sql/functions.html#least-java.lang.String-scala.collection.Seq-)(String columnName, scala.collection.Seq<String> columnNames)
Returns the least value of the list of column names, skipping null values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[least](../../../../org/apache/spark/sql/functions.html#least-java.lang.String-java.lang.String...-)(String columnName, String... columnNames)
Returns the least value of the list of column names, skipping null values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[left](../../../../org/apache/spark/sql/functions.html#left-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") len)
Returns the leftmost len(len can be string type) characters from the string str, if len is less or equal than 0 the result is an empty string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[len](../../../../org/apache/spark/sql/functions.html#len-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the character length of a given string or number of bytes of a binary string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[length](../../../../org/apache/spark/sql/functions.html#length-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the character length of a given string or number of bytes of a binary string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[levenshtein](../../../../org/apache/spark/sql/functions.html#levenshtein-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") l,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") r)
Computes the Levenshtein distance of the two given string columns.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[levenshtein](../../../../org/apache/spark/sql/functions.html#levenshtein-org.apache.spark.sql.Column-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") l,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") r, int threshold)
Computes the Levenshtein distance of the two given string columns if it's less than or equal to a given threshold.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[like](../../../../org/apache/spark/sql/functions.html#like-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pattern)
Returns true if str matches pattern with escapeChar('\'), null if any arguments are null, false otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[like](../../../../org/apache/spark/sql/functions.html#like-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pattern,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") escapeChar)
Returns true if str matches pattern with escapeChar, null if any arguments are null, false otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lit](../../../../org/apache/spark/sql/functions.html#lit-java.lang.Object-)(Object literal)
Creates a Column of literal value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ln](../../../../org/apache/spark/sql/functions.html#ln-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the natural logarithm of the given value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[localtimestamp](../../../../org/apache/spark/sql/functions.html#localtimestamp--)()
Returns the current timestamp without time zone at the start of query evaluation as a timestamp without time zone column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[locate](../../../../org/apache/spark/sql/functions.html#locate-java.lang.String-org.apache.spark.sql.Column-)(String substr,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str)
Locate the position of the first occurrence of substr.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[locate](../../../../org/apache/spark/sql/functions.html#locate-java.lang.String-org.apache.spark.sql.Column-int-)(String substr,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str, int pos)
Locate the position of the first occurrence of substr in a string column, after position pos.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[log](../../../../org/apache/spark/sql/functions.html#log-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the natural logarithm of the given value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[log](../../../../org/apache/spark/sql/functions.html#log-double-org.apache.spark.sql.Column-)(double base,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") a)
Returns the first argument-base logarithm of the second argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[log](../../../../org/apache/spark/sql/functions.html#log-double-java.lang.String-)(double base, String columnName)
Returns the first argument-base logarithm of the second argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[log](../../../../org/apache/spark/sql/functions.html#log-java.lang.String-)(String columnName)
Computes the natural logarithm of the given column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[log10](../../../../org/apache/spark/sql/functions.html#log10-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the logarithm of the given value in base 10.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[log10](../../../../org/apache/spark/sql/functions.html#log10-java.lang.String-)(String columnName)
Computes the logarithm of the given value in base 10.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[log1p](../../../../org/apache/spark/sql/functions.html#log1p-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the natural logarithm of the given value plus one.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[log1p](../../../../org/apache/spark/sql/functions.html#log1p-java.lang.String-)(String columnName)
Computes the natural logarithm of the given column plus one.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[log2](../../../../org/apache/spark/sql/functions.html#log2-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") expr)
Computes the logarithm of the given column in base 2.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[log2](../../../../org/apache/spark/sql/functions.html#log2-java.lang.String-)(String columnName)
Computes the logarithm of the given value in base 2.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lower](../../../../org/apache/spark/sql/functions.html#lower-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Converts a string column to lower case.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lpad](../../../../org/apache/spark/sql/functions.html#lpad-org.apache.spark.sql.Column-int-byte:A-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str, int len, byte[] pad)
Left-pad the binary column with pad to a byte length of len.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[lpad](../../../../org/apache/spark/sql/functions.html#lpad-org.apache.spark.sql.Column-int-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str, int len, String pad)
Left-pad the string column with pad to a length of len.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ltrim](../../../../org/apache/spark/sql/functions.html#ltrim-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Trim the spaces from left end for the specified string value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ltrim](../../../../org/apache/spark/sql/functions.html#ltrim-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, String trimString)
Trim the specified character string from left end for the specified string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_date](../../../../org/apache/spark/sql/functions.html#make%5Fdate-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") year,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") month,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") day)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_dt_interval](../../../../org/apache/spark/sql/functions.html#make%5Fdt%5Finterval--)()
Make DayTimeIntervalType duration.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_dt_interval](../../../../org/apache/spark/sql/functions.html#make%5Fdt%5Finterval-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days)
Make DayTimeIntervalType duration from days.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_dt_interval](../../../../org/apache/spark/sql/functions.html#make%5Fdt%5Finterval-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") hours)
Make DayTimeIntervalType duration from days and hours.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_dt_interval](../../../../org/apache/spark/sql/functions.html#make%5Fdt%5Finterval-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") hours,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mins)
Make DayTimeIntervalType duration from days, hours and mins.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_dt_interval](../../../../org/apache/spark/sql/functions.html#make%5Fdt%5Finterval-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") hours,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mins,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") secs)
Make DayTimeIntervalType duration from days, hours, mins and secs.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_interval](../../../../org/apache/spark/sql/functions.html#make%5Finterval--)()
Make interval.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_interval](../../../../org/apache/spark/sql/functions.html#make%5Finterval-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years)
Make interval from years.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_interval](../../../../org/apache/spark/sql/functions.html#make%5Finterval-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") months)
Make interval from years and months.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_interval](../../../../org/apache/spark/sql/functions.html#make%5Finterval-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") months,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") weeks)
Make interval from years, months and weeks.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_interval](../../../../org/apache/spark/sql/functions.html#make%5Finterval-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") months,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") weeks,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days)
Make interval from years, months, weeks and days.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_interval](../../../../org/apache/spark/sql/functions.html#make%5Finterval-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") months,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") weeks,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") hours)
Make interval from years, months, weeks, days and hours.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_interval](../../../../org/apache/spark/sql/functions.html#make%5Finterval-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") months,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") weeks,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") hours,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mins)
Make interval from years, months, weeks, days, hours and mins.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_interval](../../../../org/apache/spark/sql/functions.html#make%5Finterval-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") months,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") weeks,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") hours,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mins,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") secs)
Make interval from years, months, weeks, days, hours, mins and secs.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_timestamp_ltz](../../../../org/apache/spark/sql/functions.html#make%5Ftimestamp%5Fltz-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") months,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") hours,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mins,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") secs)
Create the current timestamp with local time zone from years, months, days, hours, mins and secs fields.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_timestamp_ltz](../../../../org/apache/spark/sql/functions.html#make%5Ftimestamp%5Fltz-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") months,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") hours,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mins,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") secs,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") timezone)
Create the current timestamp with local time zone from years, months, days, hours, mins, secs and timezone fields.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_timestamp_ntz](../../../../org/apache/spark/sql/functions.html#make%5Ftimestamp%5Fntz-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") months,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") hours,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mins,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") secs)
Create local date-time from years, months, days, hours, mins, secs fields.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_timestamp](../../../../org/apache/spark/sql/functions.html#make%5Ftimestamp-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") months,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") hours,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mins,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") secs)
Create timestamp from years, months, days, hours, mins and secs fields.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_timestamp](../../../../org/apache/spark/sql/functions.html#make%5Ftimestamp-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") months,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") days,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") hours,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mins,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") secs,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") timezone)
Create timestamp from years, months, days, hours, mins, secs and timezone fields.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_ym_interval](../../../../org/apache/spark/sql/functions.html#make%5Fym%5Finterval--)()
Make year-month interval.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_ym_interval](../../../../org/apache/spark/sql/functions.html#make%5Fym%5Finterval-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years)
Make year-month interval from years.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[make_ym_interval](../../../../org/apache/spark/sql/functions.html#make%5Fym%5Finterval-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") years,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") months)
Make year-month interval from years, months.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[map_concat](../../../../org/apache/spark/sql/functions.html#map%5Fconcat-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Returns the union of all the given maps.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[map_concat](../../../../org/apache/spark/sql/functions.html#map%5Fconcat-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
Returns the union of all the given maps.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[map_contains_key](../../../../org/apache/spark/sql/functions.html#map%5Fcontains%5Fkey-org.apache.spark.sql.Column-java.lang.Object-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, Object key)
Returns true if the map contains the key.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[map_entries](../../../../org/apache/spark/sql/functions.html#map%5Fentries-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns an unordered array of all entries in the given map.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[map_filter](../../../../org/apache/spark/sql/functions.html#map%5Ffilter-org.apache.spark.sql.Column-scala.Function2-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") expr, scala.Function2<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> f)
Returns a map whose key-value pairs satisfy a predicate.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[map_from_arrays](../../../../org/apache/spark/sql/functions.html#map%5Ffrom%5Farrays-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") keys,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") values)
Creates a new map column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[map_from_entries](../../../../org/apache/spark/sql/functions.html#map%5Ffrom%5Fentries-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns a map created from the given array of entries.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[map_keys](../../../../org/apache/spark/sql/functions.html#map%5Fkeys-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns an unordered array containing the keys of the map.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[map_values](../../../../org/apache/spark/sql/functions.html#map%5Fvalues-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns an unordered array containing the values of the map.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[map_zip_with](../../../../org/apache/spark/sql/functions.html#map%5Fzip%5Fwith-org.apache.spark.sql.Column-org.apache.spark.sql.Column-scala.Function3-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") left,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") right, scala.Function3<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> f)
Merge two given maps, key-wise into a single map using a function.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[map](../../../../org/apache/spark/sql/functions.html#map-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Creates a new map column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[map](../../../../org/apache/spark/sql/functions.html#map-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
Creates a new map column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[mask](../../../../org/apache/spark/sql/functions.html#mask-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input)
Masks the given string value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[mask](../../../../org/apache/spark/sql/functions.html#mask-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") upperChar)
Masks the given string value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[mask](../../../../org/apache/spark/sql/functions.html#mask-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") upperChar,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") lowerChar)
Masks the given string value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[mask](../../../../org/apache/spark/sql/functions.html#mask-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") upperChar,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") lowerChar,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") digitChar)
Masks the given string value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[mask](../../../../org/apache/spark/sql/functions.html#mask-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") upperChar,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") lowerChar,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") digitChar,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") otherChar)
Masks the given string value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[max_by](../../../../org/apache/spark/sql/functions.html#max%5Fby-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") ord)
Aggregate function: returns the value associated with the maximum value of ord.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[max](../../../../org/apache/spark/sql/functions.html#max-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the maximum value of the expression in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[max](../../../../org/apache/spark/sql/functions.html#max-java.lang.String-)(String columnName)
Aggregate function: returns the maximum value of the column in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[md5](../../../../org/apache/spark/sql/functions.html#md5-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Calculates the MD5 digest of a binary column and returns the value as a 32 character hex string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[mean](../../../../org/apache/spark/sql/functions.html#mean-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the average of the values in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[mean](../../../../org/apache/spark/sql/functions.html#mean-java.lang.String-)(String columnName)
Aggregate function: returns the average of the values in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[median](../../../../org/apache/spark/sql/functions.html#median-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the median of the values in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[min_by](../../../../org/apache/spark/sql/functions.html#min%5Fby-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") ord)
Aggregate function: returns the value associated with the minimum value of ord.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[min](../../../../org/apache/spark/sql/functions.html#min-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the minimum value of the expression in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[min](../../../../org/apache/spark/sql/functions.html#min-java.lang.String-)(String columnName)
Aggregate function: returns the minimum value of the column in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[minute](../../../../org/apache/spark/sql/functions.html#minute-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Extracts the minutes as an integer from a given date/timestamp/string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[mode](../../../../org/apache/spark/sql/functions.html#mode-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the most frequent value in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[monotonically_increasing_id](../../../../org/apache/spark/sql/functions.html#monotonically%5Fincreasing%5Fid--)()
A column expression that generates monotonically increasing 64-bit integers.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[monotonicallyIncreasingId](../../../../org/apache/spark/sql/functions.html#monotonicallyIncreasingId--)()
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[month](../../../../org/apache/spark/sql/functions.html#month-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Extracts the month as an integer from a given date/timestamp/string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[months_between](../../../../org/apache/spark/sql/functions.html#months%5Fbetween-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") end,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") start)
Returns number of months between dates start and end.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[months_between](../../../../org/apache/spark/sql/functions.html#months%5Fbetween-org.apache.spark.sql.Column-org.apache.spark.sql.Column-boolean-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") end,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") start, boolean roundOff)
Returns number of months between dates end and start.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[months](../../../../org/apache/spark/sql/functions.html#months-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
A transform for timestamps and dates to partition data into months.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[named_struct](../../../../org/apache/spark/sql/functions.html#named%5Fstruct-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Creates a struct with the given field names and values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[named_struct](../../../../org/apache/spark/sql/functions.html#named%5Fstruct-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
Creates a struct with the given field names and values.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[nanvl](../../../../org/apache/spark/sql/functions.html#nanvl-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col2)
Returns col1 if it is not NaN, or col2 if col1 is NaN.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[negate](../../../../org/apache/spark/sql/functions.html#negate-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Unary minus, i.e.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[negative](../../../../org/apache/spark/sql/functions.html#negative-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the negated value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[next_day](../../../../org/apache/spark/sql/functions.html#next%5Fday-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") date,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") dayOfWeek)
Returns the first date which is later than the value of the date column that is on the specified day of the week.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[next_day](../../../../org/apache/spark/sql/functions.html#next%5Fday-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") date, String dayOfWeek)
Returns the first date which is later than the value of the date column that is on the specified day of the week.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[not](../../../../org/apache/spark/sql/functions.html#not-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Inversion of boolean expression, i.e.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[now](../../../../org/apache/spark/sql/functions.html#now--)()
Returns the current timestamp at the start of query evaluation.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[nth_value](../../../../org/apache/spark/sql/functions.html#nth%5Fvalue-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int offset)
Window function: returns the value that is the offsetth row of the window frame (counting from 1), and null if the size of window frame is less than offset rows.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[nth_value](../../../../org/apache/spark/sql/functions.html#nth%5Fvalue-org.apache.spark.sql.Column-int-boolean-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int offset, boolean ignoreNulls)
Window function: returns the value that is the offsetth row of the window frame (counting from 1), and null if the size of window frame is less than offset rows.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ntile](../../../../org/apache/spark/sql/functions.html#ntile-int-)(int n)
Window function: returns the ntile group id (from 1 to n inclusive) in an ordered window partition.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[nullif](../../../../org/apache/spark/sql/functions.html#nullif-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col2)
Returns null if col1 equals to col2, or col1 otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[nvl](../../../../org/apache/spark/sql/functions.html#nvl-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col2)
Returns col2 if col1 is null, or col1 otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[nvl2](../../../../org/apache/spark/sql/functions.html#nvl2-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col1,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col2,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col3)
Returns col2 if col1 is not null, or col3 otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[octet_length](../../../../org/apache/spark/sql/functions.html#octet%5Flength-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Calculates the byte length for the specified string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[overlay](../../../../org/apache/spark/sql/functions.html#overlay-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") src,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") replace,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pos)
Overlay the specified portion of src with replace, starting from byte position pos of src.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[overlay](../../../../org/apache/spark/sql/functions.html#overlay-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") src,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") replace,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pos,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") len)
Overlay the specified portion of src with replace, starting from byte position pos of src and proceeding for len bytes.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[parse_url](../../../../org/apache/spark/sql/functions.html#parse%5Furl-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") url,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") partToExtract)
Extracts a part from a URL.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[parse_url](../../../../org/apache/spark/sql/functions.html#parse%5Furl-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") url,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") partToExtract,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key)
Extracts a part from a URL.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[percent_rank](../../../../org/apache/spark/sql/functions.html#percent%5Frank--)()
Window function: returns the relative rank (i.e.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[percentile_approx](../../../../org/apache/spark/sql/functions.html#percentile%5Fapprox-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") percentage,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") accuracy)
Aggregate function: returns the approximate percentile of the numeric column col which is the smallest value in the ordered col values (sorted from least to greatest) such that no more than percentage of col values is less than the value or equal to that value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[percentile](../../../../org/apache/spark/sql/functions.html#percentile-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") percentage)
Aggregate function: returns the exact percentile(s) of numeric column expr at the given percentage(s) with value range in [0.0, 1.0].
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[percentile](../../../../org/apache/spark/sql/functions.html#percentile-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") percentage,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") frequency)
Aggregate function: returns the exact percentile(s) of numeric column expr at the given percentage(s) with value range in [0.0, 1.0].
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[pi](../../../../org/apache/spark/sql/functions.html#pi--)()
Returns Pi.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[pmod](../../../../org/apache/spark/sql/functions.html#pmod-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") dividend,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") divisor)
Returns the positive value of dividend mod divisor.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[posexplode_outer](../../../../org/apache/spark/sql/functions.html#posexplode%5Fouter-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Creates a new row for each element with position in the given array or map column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[posexplode](../../../../org/apache/spark/sql/functions.html#posexplode-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Creates a new row for each element with position in the given array or map column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[position](../../../../org/apache/spark/sql/functions.html#position-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") substr,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str)
Returns the position of the first occurrence of substr in str after position 1.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[position](../../../../org/apache/spark/sql/functions.html#position-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") substr,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") start)
Returns the position of the first occurrence of substr in str after position start.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[positive](../../../../org/apache/spark/sql/functions.html#positive-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[pow](../../../../org/apache/spark/sql/functions.html#pow-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") l,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") r)
Returns the value of the first argument raised to the power of the second argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[pow](../../../../org/apache/spark/sql/functions.html#pow-org.apache.spark.sql.Column-double-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") l, double r)
Returns the value of the first argument raised to the power of the second argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[pow](../../../../org/apache/spark/sql/functions.html#pow-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") l, String rightName)
Returns the value of the first argument raised to the power of the second argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[pow](../../../../org/apache/spark/sql/functions.html#pow-double-org.apache.spark.sql.Column-)(double l,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") r)
Returns the value of the first argument raised to the power of the second argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[pow](../../../../org/apache/spark/sql/functions.html#pow-double-java.lang.String-)(double l, String rightName)
Returns the value of the first argument raised to the power of the second argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[pow](../../../../org/apache/spark/sql/functions.html#pow-java.lang.String-org.apache.spark.sql.Column-)(String leftName,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") r)
Returns the value of the first argument raised to the power of the second argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[pow](../../../../org/apache/spark/sql/functions.html#pow-java.lang.String-double-)(String leftName, double r)
Returns the value of the first argument raised to the power of the second argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[pow](../../../../org/apache/spark/sql/functions.html#pow-java.lang.String-java.lang.String-)(String leftName, String rightName)
Returns the value of the first argument raised to the power of the second argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[power](../../../../org/apache/spark/sql/functions.html#power-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") l,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") r)
Returns the value of the first argument raised to the power of the second argument.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[printf](../../../../org/apache/spark/sql/functions.html#printf-org.apache.spark.sql.Column-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") format,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... arguments)
Formats the arguments in printf-style and returns the result as a string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[printf](../../../../org/apache/spark/sql/functions.html#printf-org.apache.spark.sql.Column-scala.collection.Seq-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") format, scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> arguments)
Formats the arguments in printf-style and returns the result as a string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[product](../../../../org/apache/spark/sql/functions.html#product-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the product of all numerical elements in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[quarter](../../../../org/apache/spark/sql/functions.html#quarter-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Extracts the quarter as an integer from a given date/timestamp/string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[radians](../../../../org/apache/spark/sql/functions.html#radians-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Converts an angle measured in degrees to an approximately equivalent angle measured in radians.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[radians](../../../../org/apache/spark/sql/functions.html#radians-java.lang.String-)(String columnName)
Converts an angle measured in degrees to an approximately equivalent angle measured in radians.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[raise_error](../../../../org/apache/spark/sql/functions.html#raise%5Ferror-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") c)
Throws an exception with the provided error message.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[rand](../../../../org/apache/spark/sql/functions.html#rand--)()
Generate a random column with independent and identically distributed (i.i.d.) samples uniformly distributed in [0.0, 1.0).
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[rand](../../../../org/apache/spark/sql/functions.html#rand-long-)(long seed)
Generate a random column with independent and identically distributed (i.i.d.) samples uniformly distributed in [0.0, 1.0).
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[randn](../../../../org/apache/spark/sql/functions.html#randn--)()
Generate a column with independent and identically distributed (i.i.d.) samples from the standard normal distribution.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[randn](../../../../org/apache/spark/sql/functions.html#randn-long-)(long seed)
Generate a column with independent and identically distributed (i.i.d.) samples from the standard normal distribution.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[random](../../../../org/apache/spark/sql/functions.html#random--)()
Returns a random value with independent and identically distributed (i.i.d.) uniformly distributed values in [0, 1).
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[random](../../../../org/apache/spark/sql/functions.html#random-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") seed)
Returns a random value with independent and identically distributed (i.i.d.) uniformly distributed values in [0, 1).
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[rank](../../../../org/apache/spark/sql/functions.html#rank--)()
Window function: returns the rank of rows within a window partition.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[reduce](../../../../org/apache/spark/sql/functions.html#reduce-org.apache.spark.sql.Column-org.apache.spark.sql.Column-scala.Function2-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") expr,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") initialValue, scala.Function2<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> merge)
Applies a binary operator to an initial state and all elements in the array, and reduces this to a single state.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[reduce](../../../../org/apache/spark/sql/functions.html#reduce-org.apache.spark.sql.Column-org.apache.spark.sql.Column-scala.Function2-scala.Function1-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") expr,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") initialValue, scala.Function2<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> merge, scala.Function1<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> finish)
Applies a binary operator to an initial state and all elements in the array, and reduces this to a single state.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[reflect](../../../../org/apache/spark/sql/functions.html#reflect-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Calls a method with reflection.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[reflect](../../../../org/apache/spark/sql/functions.html#reflect-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
Calls a method with reflection.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regexp_count](../../../../org/apache/spark/sql/functions.html#regexp%5Fcount-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") regexp)
Returns a count of the number of times that the regular expression pattern regexp is matched in the string str.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regexp_extract_all](../../../../org/apache/spark/sql/functions.html#regexp%5Fextract%5Fall-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") regexp)
Extract all strings in the str that match the regexp expression and corresponding to the first regex group index.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regexp_extract_all](../../../../org/apache/spark/sql/functions.html#regexp%5Fextract%5Fall-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") regexp,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") idx)
Extract all strings in the str that match the regexp expression and corresponding to the regex group index.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regexp_extract](../../../../org/apache/spark/sql/functions.html#regexp%5Fextract-org.apache.spark.sql.Column-java.lang.String-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, String exp, int groupIdx)
Extract a specific group matched by a Java regex, from the specified string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regexp_instr](../../../../org/apache/spark/sql/functions.html#regexp%5Finstr-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") regexp)
Searches a string for a regular expression and returns an integer that indicates the beginning position of the matched substring.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regexp_instr](../../../../org/apache/spark/sql/functions.html#regexp%5Finstr-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") regexp,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") idx)
Searches a string for a regular expression and returns an integer that indicates the beginning position of the matched substring.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regexp_like](../../../../org/apache/spark/sql/functions.html#regexp%5Flike-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") regexp)
Returns true if str matches regexp, or false otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regexp_replace](../../../../org/apache/spark/sql/functions.html#regexp%5Freplace-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pattern,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") replacement)
Replace all substrings of the specified string value that match regexp with rep.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regexp_replace](../../../../org/apache/spark/sql/functions.html#regexp%5Freplace-org.apache.spark.sql.Column-java.lang.String-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, String pattern, String replacement)
Replace all substrings of the specified string value that match regexp with rep.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regexp_substr](../../../../org/apache/spark/sql/functions.html#regexp%5Fsubstr-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") regexp)
Returns the substring that matches the regular expression regexp within the string str.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regexp](../../../../org/apache/spark/sql/functions.html#regexp-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") regexp)
Returns true if str matches regexp, or false otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regr_avgx](../../../../org/apache/spark/sql/functions.html#regr%5Favgx-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") y,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x)
Aggregate function: returns the average of the independent variable for non-null pairs in a group, where y is the dependent variable and x is the independent variable.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regr_avgy](../../../../org/apache/spark/sql/functions.html#regr%5Favgy-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") y,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x)
Aggregate function: returns the average of the independent variable for non-null pairs in a group, where y is the dependent variable and x is the independent variable.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regr_count](../../../../org/apache/spark/sql/functions.html#regr%5Fcount-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") y,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x)
Aggregate function: returns the number of non-null number pairs in a group, where y is the dependent variable and x is the independent variable.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regr_intercept](../../../../org/apache/spark/sql/functions.html#regr%5Fintercept-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") y,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x)
Aggregate function: returns the intercept of the univariate linear regression line for non-null pairs in a group, where y is the dependent variable andx is the independent variable.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regr_r2](../../../../org/apache/spark/sql/functions.html#regr%5Fr2-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") y,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x)
Aggregate function: returns the coefficient of determination for non-null pairs in a group, where y is the dependent variable and x is the independent variable.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regr_slope](../../../../org/apache/spark/sql/functions.html#regr%5Fslope-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") y,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x)
Aggregate function: returns the slope of the linear regression line for non-null pairs in a group, where y is the dependent variable and x is the independent variable.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regr_sxx](../../../../org/apache/spark/sql/functions.html#regr%5Fsxx-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") y,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x)
Aggregate function: returns REGR_COUNT(y, x) * VAR_POP(x) for non-null pairs in a group, where y is the dependent variable and x is the independent variable.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regr_sxy](../../../../org/apache/spark/sql/functions.html#regr%5Fsxy-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") y,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x)
Aggregate function: returns REGR_COUNT(y, x) * COVAR_POP(y, x) for non-null pairs in a group, where y is the dependent variable and x is the independent variable.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[regr_syy](../../../../org/apache/spark/sql/functions.html#regr%5Fsyy-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") y,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x)
Aggregate function: returns REGR_COUNT(y, x) * VAR_POP(y) for non-null pairs in a group, where y is the dependent variable and x is the independent variable.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[repeat](../../../../org/apache/spark/sql/functions.html#repeat-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str, int n)
Repeats a string column n times, and returns it as a new string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[replace](../../../../org/apache/spark/sql/functions.html#replace-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") src,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") search)
Replaces all occurrences of search with replace.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[replace](../../../../org/apache/spark/sql/functions.html#replace-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") src,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") search,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") replace)
Replaces all occurrences of search with replace.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[reverse](../../../../org/apache/spark/sql/functions.html#reverse-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns a reversed string or an array with reverse order of elements.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[right](../../../../org/apache/spark/sql/functions.html#right-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") len)
Returns the rightmost len(len can be string type) characters from the string str, if len is less or equal than 0 the result is an empty string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[rint](../../../../org/apache/spark/sql/functions.html#rint-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the double value that is closest in value to the argument and is equal to a mathematical integer.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[rint](../../../../org/apache/spark/sql/functions.html#rint-java.lang.String-)(String columnName)
Returns the double value that is closest in value to the argument and is equal to a mathematical integer.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[rlike](../../../../org/apache/spark/sql/functions.html#rlike-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") regexp)
Returns true if str matches regexp, or false otherwise.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[round](../../../../org/apache/spark/sql/functions.html#round-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the value of the column e rounded to 0 decimal places with HALF_UP round mode.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[round](../../../../org/apache/spark/sql/functions.html#round-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int scale)
Round the value of e to scale decimal places with HALF_UP round mode if scale is greater than or equal to 0 or at integral part when scale is less than 0.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[row_number](../../../../org/apache/spark/sql/functions.html#row%5Fnumber--)()
Window function: returns a sequential number starting at 1 within a window partition.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[rpad](../../../../org/apache/spark/sql/functions.html#rpad-org.apache.spark.sql.Column-int-byte:A-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str, int len, byte[] pad)
Right-pad the binary column with pad to a byte length of len.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[rpad](../../../../org/apache/spark/sql/functions.html#rpad-org.apache.spark.sql.Column-int-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str, int len, String pad)
Right-pad the string column with pad to a length of len.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[rtrim](../../../../org/apache/spark/sql/functions.html#rtrim-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Trim the spaces from right end for the specified string value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[rtrim](../../../../org/apache/spark/sql/functions.html#rtrim-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, String trimString)
Trim the specified character string from right end for the specified string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[schema_of_csv](../../../../org/apache/spark/sql/functions.html#schema%5Fof%5Fcsv-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") csv)
Parses a CSV string and infers its schema in DDL format.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[schema_of_csv](../../../../org/apache/spark/sql/functions.html#schema%5Fof%5Fcsv-org.apache.spark.sql.Column-java.util.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") csv, java.util.Map<String,String> options)
Parses a CSV string and infers its schema in DDL format using options.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[schema_of_csv](../../../../org/apache/spark/sql/functions.html#schema%5Fof%5Fcsv-java.lang.String-)(String csv)
Parses a CSV string and infers its schema in DDL format.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[schema_of_json](../../../../org/apache/spark/sql/functions.html#schema%5Fof%5Fjson-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") json)
Parses a JSON string and infers its schema in DDL format.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[schema_of_json](../../../../org/apache/spark/sql/functions.html#schema%5Fof%5Fjson-org.apache.spark.sql.Column-java.util.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") json, java.util.Map<String,String> options)
Parses a JSON string and infers its schema in DDL format using options.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[schema_of_json](../../../../org/apache/spark/sql/functions.html#schema%5Fof%5Fjson-java.lang.String-)(String json)
Parses a JSON string and infers its schema in DDL format.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sec](../../../../org/apache/spark/sql/functions.html#sec-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[second](../../../../org/apache/spark/sql/functions.html#second-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Extracts the seconds as an integer from a given date/timestamp/string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sentences](../../../../org/apache/spark/sql/functions.html#sentences-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") string)
Splits a string into arrays of sentences, where each sentence is an array of words.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sentences](../../../../org/apache/spark/sql/functions.html#sentences-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") string,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") language,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") country)
Splits a string into arrays of sentences, where each sentence is an array of words.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sequence](../../../../org/apache/spark/sql/functions.html#sequence-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") start,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") stop)
Generate a sequence of integers from start to stop, incrementing by 1 if start is less than or equal to stop, otherwise -1.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sequence](../../../../org/apache/spark/sql/functions.html#sequence-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") start,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") stop,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") step)
Generate a sequence of integers from start to stop, incrementing by step.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[session_window](../../../../org/apache/spark/sql/functions.html#session%5Fwindow-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") timeColumn,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") gapDuration)
Generates session window given a timestamp specifying column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[session_window](../../../../org/apache/spark/sql/functions.html#session%5Fwindow-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") timeColumn, String gapDuration)
Generates session window given a timestamp specifying column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sha](../../../../org/apache/spark/sql/functions.html#sha-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col)
Returns a sha1 hash value as a hex string of the col.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sha1](../../../../org/apache/spark/sql/functions.html#sha1-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Calculates the SHA-1 digest of a binary column and returns the value as a 40 character hex string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sha2](../../../../org/apache/spark/sql/functions.html#sha2-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int numBits)
Calculates the SHA-2 family of hash functions of a binary column and returns the value as a hex string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[shiftleft](../../../../org/apache/spark/sql/functions.html#shiftleft-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int numBits)
Shift the given value numBits left.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[shiftLeft](../../../../org/apache/spark/sql/functions.html#shiftLeft-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int numBits)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[shiftright](../../../../org/apache/spark/sql/functions.html#shiftright-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int numBits)
(Signed) shift the given value numBits right.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[shiftRight](../../../../org/apache/spark/sql/functions.html#shiftRight-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int numBits)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[shiftrightunsigned](../../../../org/apache/spark/sql/functions.html#shiftrightunsigned-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int numBits)
Unsigned shift the given value numBits right.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[shiftRightUnsigned](../../../../org/apache/spark/sql/functions.html#shiftRightUnsigned-org.apache.spark.sql.Column-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, int numBits)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[shuffle](../../../../org/apache/spark/sql/functions.html#shuffle-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns a random permutation of the given array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sign](../../../../org/apache/spark/sql/functions.html#sign-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the signum of the given value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[signum](../../../../org/apache/spark/sql/functions.html#signum-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the signum of the given value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[signum](../../../../org/apache/spark/sql/functions.html#signum-java.lang.String-)(String columnName)
Computes the signum of the given column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sin](../../../../org/apache/spark/sql/functions.html#sin-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sin](../../../../org/apache/spark/sql/functions.html#sin-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sinh](../../../../org/apache/spark/sql/functions.html#sinh-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sinh](../../../../org/apache/spark/sql/functions.html#sinh-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[size](../../../../org/apache/spark/sql/functions.html#size-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns length of array or map.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[skewness](../../../../org/apache/spark/sql/functions.html#skewness-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the skewness of the values in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[skewness](../../../../org/apache/spark/sql/functions.html#skewness-java.lang.String-)(String columnName)
Aggregate function: returns the skewness of the values in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[slice](../../../../org/apache/spark/sql/functions.html#slice-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") start,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") length)
Returns an array containing all the elements in x from index start (or starting from the end if start is negative) with the specified length.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[slice](../../../../org/apache/spark/sql/functions.html#slice-org.apache.spark.sql.Column-int-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x, int start, int length)
Returns an array containing all the elements in x from index start (or starting from the end if start is negative) with the specified length.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[some](../../../../org/apache/spark/sql/functions.html#some-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns true if at least one value of e is true.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sort_array](../../../../org/apache/spark/sql/functions.html#sort%5Farray-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Sorts the input array for the given column in ascending order, according to the natural ordering of the array elements.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sort_array](../../../../org/apache/spark/sql/functions.html#sort%5Farray-org.apache.spark.sql.Column-boolean-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, boolean asc)
Sorts the input array for the given column in ascending or descending order, according to the natural ordering of the array elements.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[soundex](../../../../org/apache/spark/sql/functions.html#soundex-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the soundex code for the specified expression.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[spark_partition_id](../../../../org/apache/spark/sql/functions.html#spark%5Fpartition%5Fid--)()
Partition ID.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[split_part](../../../../org/apache/spark/sql/functions.html#split%5Fpart-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") delimiter,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") partNum)
Splits str by delimiter and return requested part of the split (1-based).
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[split](../../../../org/apache/spark/sql/functions.html#split-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str, String pattern)
Splits str around matches of the given pattern.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[split](../../../../org/apache/spark/sql/functions.html#split-org.apache.spark.sql.Column-java.lang.String-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str, String pattern, int limit)
Splits str around matches of the given pattern.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sqrt](../../../../org/apache/spark/sql/functions.html#sqrt-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Computes the square root of the specified float value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sqrt](../../../../org/apache/spark/sql/functions.html#sqrt-java.lang.String-)(String colName)
Computes the square root of the specified float value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[stack](../../../../org/apache/spark/sql/functions.html#stack-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Separates col1, ..., colk into n rows.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[stack](../../../../org/apache/spark/sql/functions.html#stack-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
Separates col1, ..., colk into n rows.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[startswith](../../../../org/apache/spark/sql/functions.html#startswith-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") prefix)
Returns a boolean.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[std](../../../../org/apache/spark/sql/functions.html#std-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: alias for stddev_samp.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[stddev_pop](../../../../org/apache/spark/sql/functions.html#stddev%5Fpop-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the population standard deviation of the expression in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[stddev_pop](../../../../org/apache/spark/sql/functions.html#stddev%5Fpop-java.lang.String-)(String columnName)
Aggregate function: returns the population standard deviation of the expression in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[stddev_samp](../../../../org/apache/spark/sql/functions.html#stddev%5Fsamp-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the sample standard deviation of the expression in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[stddev_samp](../../../../org/apache/spark/sql/functions.html#stddev%5Fsamp-java.lang.String-)(String columnName)
Aggregate function: returns the sample standard deviation of the expression in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[stddev](../../../../org/apache/spark/sql/functions.html#stddev-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: alias for stddev_samp.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[stddev](../../../../org/apache/spark/sql/functions.html#stddev-java.lang.String-)(String columnName)
Aggregate function: alias for stddev_samp.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[str_to_map](../../../../org/apache/spark/sql/functions.html#str%5Fto%5Fmap-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") text)
Creates a map after splitting the text into key/value pairs using delimiters.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[str_to_map](../../../../org/apache/spark/sql/functions.html#str%5Fto%5Fmap-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") text,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pairDelim)
Creates a map after splitting the text into key/value pairs using delimiters.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[str_to_map](../../../../org/apache/spark/sql/functions.html#str%5Fto%5Fmap-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") text,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pairDelim,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") keyValueDelim)
Creates a map after splitting the text into key/value pairs using delimiters.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[struct](../../../../org/apache/spark/sql/functions.html#struct-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Creates a new struct column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[struct](../../../../org/apache/spark/sql/functions.html#struct-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
Creates a new struct column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[struct](../../../../org/apache/spark/sql/functions.html#struct-java.lang.String-scala.collection.Seq-)(String colName, scala.collection.Seq<String> colNames)
Creates a new struct column that composes multiple input columns.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[struct](../../../../org/apache/spark/sql/functions.html#struct-java.lang.String-java.lang.String...-)(String colName, String... colNames)
Creates a new struct column that composes multiple input columns.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[substr](../../../../org/apache/spark/sql/functions.html#substr-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pos)
Returns the substring of str that starts at pos, or the slice of byte array that starts at pos.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[substr](../../../../org/apache/spark/sql/functions.html#substr-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") pos,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") len)
Returns the substring of str that starts at pos and is of length len, or the slice of byte array that starts at pos and is of length len.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[substring_index](../../../../org/apache/spark/sql/functions.html#substring%5Findex-org.apache.spark.sql.Column-java.lang.String-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str, String delim, int count)
Returns the substring from string str before count occurrences of the delimiter delim.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[substring](../../../../org/apache/spark/sql/functions.html#substring-org.apache.spark.sql.Column-int-int-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str, int pos, int len)
Substring starts at pos and is of length len when str is String type or returns the slice of byte array that starts at pos in byte and is of length len when str is Binary type
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sum_distinct](../../../../org/apache/spark/sql/functions.html#sum%5Fdistinct-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the sum of distinct values in the expression.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sum](../../../../org/apache/spark/sql/functions.html#sum-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the sum of all values in the expression.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sum](../../../../org/apache/spark/sql/functions.html#sum-java.lang.String-)(String columnName)
Aggregate function: returns the sum of all values in the given column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sumDistinct](../../../../org/apache/spark/sql/functions.html#sumDistinct-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[sumDistinct](../../../../org/apache/spark/sql/functions.html#sumDistinct-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[tan](../../../../org/apache/spark/sql/functions.html#tan-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[tan](../../../../org/apache/spark/sql/functions.html#tan-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[tanh](../../../../org/apache/spark/sql/functions.html#tanh-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[tanh](../../../../org/apache/spark/sql/functions.html#tanh-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[timestamp_micros](../../../../org/apache/spark/sql/functions.html#timestamp%5Fmicros-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Creates timestamp from the number of microseconds since UTC epoch.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[timestamp_millis](../../../../org/apache/spark/sql/functions.html#timestamp%5Fmillis-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Creates timestamp from the number of milliseconds since UTC epoch.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[timestamp_seconds](../../../../org/apache/spark/sql/functions.html#timestamp%5Fseconds-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Converts the number of seconds from the Unix epoch (1970-01-01T00:00:00Z) to a timestamp.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_binary](../../../../org/apache/spark/sql/functions.html#to%5Fbinary-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Converts the input e to a binary value based on the default format "hex".
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_binary](../../../../org/apache/spark/sql/functions.html#to%5Fbinary-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") format)
Converts the input e to a binary value based on the supplied format.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_char](../../../../org/apache/spark/sql/functions.html#to%5Fchar-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") format)
Convert e to a string based on the format.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_csv](../../../../org/apache/spark/sql/functions.html#to%5Fcsv-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Converts a column containing a StructType into a CSV string with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_csv](../../../../org/apache/spark/sql/functions.html#to%5Fcsv-org.apache.spark.sql.Column-java.util.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, java.util.Map<String,String> options)
(Java-specific) Converts a column containing a StructType into a CSV string with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_date](../../../../org/apache/spark/sql/functions.html#to%5Fdate-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Converts the column into DateType by casting rules to DateType.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_date](../../../../org/apache/spark/sql/functions.html#to%5Fdate-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, String fmt)
Converts the column into a DateType with a specified format
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_json](../../../../org/apache/spark/sql/functions.html#to%5Fjson-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Converts a column containing a StructType, ArrayType or a MapType into a JSON string with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_json](../../../../org/apache/spark/sql/functions.html#to%5Fjson-org.apache.spark.sql.Column-scala.collection.immutable.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, scala.collection.immutable.Map<String,String> options)
(Scala-specific) Converts a column containing a StructType, ArrayType or a MapType into a JSON string with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_json](../../../../org/apache/spark/sql/functions.html#to%5Fjson-org.apache.spark.sql.Column-java.util.Map-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, java.util.Map<String,String> options)
(Java-specific) Converts a column containing a StructType, ArrayType or a MapType into a JSON string with the specified schema.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_number](../../../../org/apache/spark/sql/functions.html#to%5Fnumber-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") format)
Convert string 'e' to a number based on the string format 'format'.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_timestamp_ltz](../../../../org/apache/spark/sql/functions.html#to%5Ftimestamp%5Fltz-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") timestamp)
Parses the timestamp expression with the default format to a timestamp without time zone.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_timestamp_ltz](../../../../org/apache/spark/sql/functions.html#to%5Ftimestamp%5Fltz-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") timestamp,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") format)
Parses the timestamp expression with the format expression to a timestamp without time zone.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_timestamp_ntz](../../../../org/apache/spark/sql/functions.html#to%5Ftimestamp%5Fntz-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") timestamp)
Parses the timestamp expression with the default format to a timestamp without time zone.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_timestamp_ntz](../../../../org/apache/spark/sql/functions.html#to%5Ftimestamp%5Fntz-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") timestamp,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") format)
Parses the timestamp_str expression with the format expression to a timestamp without time zone.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_timestamp](../../../../org/apache/spark/sql/functions.html#to%5Ftimestamp-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") s)
Converts to a timestamp by casting rules to TimestampType.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_timestamp](../../../../org/apache/spark/sql/functions.html#to%5Ftimestamp-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") s, String fmt)
Converts time string with the given pattern to timestamp.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_unix_timestamp](../../../../org/apache/spark/sql/functions.html#to%5Funix%5Ftimestamp-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the UNIX timestamp of the given time.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_unix_timestamp](../../../../org/apache/spark/sql/functions.html#to%5Funix%5Ftimestamp-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") format)
Returns the UNIX timestamp of the given time.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_utc_timestamp](../../../../org/apache/spark/sql/functions.html#to%5Futc%5Ftimestamp-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") ts,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") tz)
Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in the given time zone, and renders that time as a timestamp in UTC.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_utc_timestamp](../../../../org/apache/spark/sql/functions.html#to%5Futc%5Ftimestamp-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") ts, String tz)
Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in the given time zone, and renders that time as a timestamp in UTC.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[to_varchar](../../../../org/apache/spark/sql/functions.html#to%5Fvarchar-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") format)
Convert e to a string based on the format.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[toDegrees](../../../../org/apache/spark/sql/functions.html#toDegrees-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[toDegrees](../../../../org/apache/spark/sql/functions.html#toDegrees-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[toRadians](../../../../org/apache/spark/sql/functions.html#toRadians-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[toRadians](../../../../org/apache/spark/sql/functions.html#toRadians-java.lang.String-)(String columnName)
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[transform_keys](../../../../org/apache/spark/sql/functions.html#transform%5Fkeys-org.apache.spark.sql.Column-scala.Function2-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") expr, scala.Function2<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> f)
Applies a function to every key-value pair in a map and returns a map with the results of those applications as the new keys for the pairs.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[transform_values](../../../../org/apache/spark/sql/functions.html#transform%5Fvalues-org.apache.spark.sql.Column-scala.Function2-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") expr, scala.Function2<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> f)
Applies a function to every key-value pair in a map and returns a map with the results of those applications as the new values for the pairs.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[transform](../../../../org/apache/spark/sql/functions.html#transform-org.apache.spark.sql.Column-scala.Function1-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, scala.Function1<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> f)
Returns an array of elements after applying a transformation to each element in the input array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[transform](../../../../org/apache/spark/sql/functions.html#transform-org.apache.spark.sql.Column-scala.Function2-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column, scala.Function2<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> f)
Returns an array of elements after applying a transformation to each element in the input array.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[translate](../../../../org/apache/spark/sql/functions.html#translate-org.apache.spark.sql.Column-java.lang.String-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") src, String matchingString, String replaceString)
Translate any character in the src by a character in replaceString.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[trim](../../../../org/apache/spark/sql/functions.html#trim-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Trim the spaces from both ends for the specified string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[trim](../../../../org/apache/spark/sql/functions.html#trim-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e, String trimString)
Trim the specified character from both ends for the specified string column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[trunc](../../../../org/apache/spark/sql/functions.html#trunc-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") date, String format)
Returns date truncated to the unit specified by the format.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_add](../../../../org/apache/spark/sql/functions.html#try%5Fadd-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") left,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") right)
Returns the sum of left and right and the result is null on overflow.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_aes_decrypt](../../../../org/apache/spark/sql/functions.html#try%5Faes%5Fdecrypt-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key)
Returns a decrypted value of input.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_aes_decrypt](../../../../org/apache/spark/sql/functions.html#try%5Faes%5Fdecrypt-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mode)
Returns a decrypted value of input.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_aes_decrypt](../../../../org/apache/spark/sql/functions.html#try%5Faes%5Fdecrypt-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mode,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") padding)
Returns a decrypted value of input.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_aes_decrypt](../../../../org/apache/spark/sql/functions.html#try%5Faes%5Fdecrypt-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") input,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") key,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") mode,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") padding,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") aad)
This is a special version of aes_decrypt that performs the same operation, but returns a NULL value instead of raising an error if the decryption cannot be performed.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_avg](../../../../org/apache/spark/sql/functions.html#try%5Favg-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the mean calculated from values of a group and the result is null on overflow.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_divide](../../../../org/apache/spark/sql/functions.html#try%5Fdivide-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") dividend,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") divisor)
Returns dividend / divisor.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_element_at](../../../../org/apache/spark/sql/functions.html#try%5Felement%5Fat-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") value)
(array, index) - Returns element of array at given (1-based) index.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_multiply](../../../../org/apache/spark/sql/functions.html#try%5Fmultiply-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") left,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") right)
Returns left * right and the result is null on overflow.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_subtract](../../../../org/apache/spark/sql/functions.html#try%5Fsubtract-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") left,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") right)
Returns left - right and the result is null on overflow.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_sum](../../../../org/apache/spark/sql/functions.html#try%5Fsum-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the sum calculated from values of a group and the result is null on overflow.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_to_binary](../../../../org/apache/spark/sql/functions.html#try%5Fto%5Fbinary-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
This is a special version of to_binary that performs the same operation, but returns a NULL value instead of raising an error if the conversion cannot be performed.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_to_binary](../../../../org/apache/spark/sql/functions.html#try%5Fto%5Fbinary-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") format)
This is a special version of to_binary that performs the same operation, but returns a NULL value instead of raising an error if the conversion cannot be performed.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_to_number](../../../../org/apache/spark/sql/functions.html#try%5Fto%5Fnumber-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") format)
Convert string e to a number based on the string format format.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_to_timestamp](../../../../org/apache/spark/sql/functions.html#try%5Fto%5Ftimestamp-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") s)
Parses the s to a timestamp.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[try_to_timestamp](../../../../org/apache/spark/sql/functions.html#try%5Fto%5Ftimestamp-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") s,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") format)
Parses the s with the format to a timestamp.
static <T> [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[typedlit](../../../../org/apache/spark/sql/functions.html#typedlit-T-scala.reflect.api.TypeTags.TypeTag-)(T literal, scala.reflect.api.TypeTags.TypeTag<T> evidence$2)
Creates a Column of literal value.
static <T> [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[typedLit](../../../../org/apache/spark/sql/functions.html#typedLit-T-scala.reflect.api.TypeTags.TypeTag-)(T literal, scala.reflect.api.TypeTags.TypeTag<T> evidence$1)
Creates a Column of literal value.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[typeof](../../../../org/apache/spark/sql/functions.html#typeof-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") col)
Return DDL-formatted type string for the data type of the input.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[ucase](../../../../org/apache/spark/sql/functions.html#ucase-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str)
Returns str with all characters changed to uppercase.
static <IN,BUF,OUT> [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udaf](../../../../org/apache/spark/sql/functions.html#udaf-org.apache.spark.sql.expressions.Aggregator-org.apache.spark.sql.Encoder-)([Aggregator](../../../../org/apache/spark/sql/expressions/Aggregator.html "class in org.apache.spark.sql.expressions")<IN,BUF,OUT> agg,[Encoder](../../../../org/apache/spark/sql/Encoder.html "interface in org.apache.spark.sql")<IN> inputEncoder)
Obtains a UserDefinedFunction that wraps the given Aggregator so that it may be used with untyped Data Frames.
static <IN,BUF,OUT> [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udaf](../../../../org/apache/spark/sql/functions.html#udaf-org.apache.spark.sql.expressions.Aggregator-scala.reflect.api.TypeTags.TypeTag-)([Aggregator](../../../../org/apache/spark/sql/expressions/Aggregator.html "class in org.apache.spark.sql.expressions")<IN,BUF,OUT> agg, scala.reflect.api.TypeTags.TypeTag<IN> evidence$3)
Obtains a UserDefinedFunction that wraps the given Aggregator so that it may be used with untyped Data Frames.
static <RT> [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-scala.Function0-scala.reflect.api.TypeTags.TypeTag-)(scala.Function0<RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$4)
Defines a Scala closure of 0 arguments as user-defined function (UDF).
static <RT,A1> [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-scala.Function1-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-)(scala.Function1<A1,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$5, scala.reflect.api.TypeTags.TypeTag<A1> evidence$6)
Defines a Scala closure of 1 arguments as user-defined function (UDF).
static <RT,A1,A2,A3,A4,A5,A6,A7,A8,A9,A10> [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-scala.Function10-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-)(scala.Function10<A1,A2,A3,A4,A5,A6,A7,A8,A9,A10,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$59, scala.reflect.api.TypeTags.TypeTag<A1> evidence$60, scala.reflect.api.TypeTags.TypeTag<A2> evidence$61, scala.reflect.api.TypeTags.TypeTag<A3> evidence$62, scala.reflect.api.TypeTags.TypeTag<A4> evidence$63, scala.reflect.api.TypeTags.TypeTag<A5> evidence$64, scala.reflect.api.TypeTags.TypeTag<A6> evidence$65, scala.reflect.api.TypeTags.TypeTag<A7> evidence$66, scala.reflect.api.TypeTags.TypeTag<A8> evidence$67, scala.reflect.api.TypeTags.TypeTag<A9> evidence$68, scala.reflect.api.TypeTags.TypeTag<A10> evidence$69)
Defines a Scala closure of 10 arguments as user-defined function (UDF).
static <RT,A1,A2> [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-scala.Function2-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-)(scala.Function2<A1,A2,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$7, scala.reflect.api.TypeTags.TypeTag<A1> evidence$8, scala.reflect.api.TypeTags.TypeTag<A2> evidence$9)
Defines a Scala closure of 2 arguments as user-defined function (UDF).
static <RT,A1,A2,A3> [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-scala.Function3-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-)(scala.Function3<A1,A2,A3,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$10, scala.reflect.api.TypeTags.TypeTag<A1> evidence$11, scala.reflect.api.TypeTags.TypeTag<A2> evidence$12, scala.reflect.api.TypeTags.TypeTag<A3> evidence$13)
Defines a Scala closure of 3 arguments as user-defined function (UDF).
static <RT,A1,A2,A3,A4> [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-scala.Function4-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-)(scala.Function4<A1,A2,A3,A4,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$14, scala.reflect.api.TypeTags.TypeTag<A1> evidence$15, scala.reflect.api.TypeTags.TypeTag<A2> evidence$16, scala.reflect.api.TypeTags.TypeTag<A3> evidence$17, scala.reflect.api.TypeTags.TypeTag<A4> evidence$18)
Defines a Scala closure of 4 arguments as user-defined function (UDF).
static <RT,A1,A2,A3,A4,A5> [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-scala.Function5-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-)(scala.Function5<A1,A2,A3,A4,A5,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$19, scala.reflect.api.TypeTags.TypeTag<A1> evidence$20, scala.reflect.api.TypeTags.TypeTag<A2> evidence$21, scala.reflect.api.TypeTags.TypeTag<A3> evidence$22, scala.reflect.api.TypeTags.TypeTag<A4> evidence$23, scala.reflect.api.TypeTags.TypeTag<A5> evidence$24)
Defines a Scala closure of 5 arguments as user-defined function (UDF).
static <RT,A1,A2,A3,A4,A5,A6> [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-scala.Function6-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-)(scala.Function6<A1,A2,A3,A4,A5,A6,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$25, scala.reflect.api.TypeTags.TypeTag<A1> evidence$26, scala.reflect.api.TypeTags.TypeTag<A2> evidence$27, scala.reflect.api.TypeTags.TypeTag<A3> evidence$28, scala.reflect.api.TypeTags.TypeTag<A4> evidence$29, scala.reflect.api.TypeTags.TypeTag<A5> evidence$30, scala.reflect.api.TypeTags.TypeTag<A6> evidence$31)
Defines a Scala closure of 6 arguments as user-defined function (UDF).
static <RT,A1,A2,A3,A4,A5,A6,A7> [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-scala.Function7-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-)(scala.Function7<A1,A2,A3,A4,A5,A6,A7,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$32, scala.reflect.api.TypeTags.TypeTag<A1> evidence$33, scala.reflect.api.TypeTags.TypeTag<A2> evidence$34, scala.reflect.api.TypeTags.TypeTag<A3> evidence$35, scala.reflect.api.TypeTags.TypeTag<A4> evidence$36, scala.reflect.api.TypeTags.TypeTag<A5> evidence$37, scala.reflect.api.TypeTags.TypeTag<A6> evidence$38, scala.reflect.api.TypeTags.TypeTag<A7> evidence$39)
Defines a Scala closure of 7 arguments as user-defined function (UDF).
static <RT,A1,A2,A3,A4,A5,A6,A7,A8> [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-scala.Function8-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-)(scala.Function8<A1,A2,A3,A4,A5,A6,A7,A8,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$40, scala.reflect.api.TypeTags.TypeTag<A1> evidence$41, scala.reflect.api.TypeTags.TypeTag<A2> evidence$42, scala.reflect.api.TypeTags.TypeTag<A3> evidence$43, scala.reflect.api.TypeTags.TypeTag<A4> evidence$44, scala.reflect.api.TypeTags.TypeTag<A5> evidence$45, scala.reflect.api.TypeTags.TypeTag<A6> evidence$46, scala.reflect.api.TypeTags.TypeTag<A7> evidence$47, scala.reflect.api.TypeTags.TypeTag<A8> evidence$48)
Defines a Scala closure of 8 arguments as user-defined function (UDF).
static <RT,A1,A2,A3,A4,A5,A6,A7,A8,A9> [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-scala.Function9-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-scala.reflect.api.TypeTags.TypeTag-)(scala.Function9<A1,A2,A3,A4,A5,A6,A7,A8,A9,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$49, scala.reflect.api.TypeTags.TypeTag<A1> evidence$50, scala.reflect.api.TypeTags.TypeTag<A2> evidence$51, scala.reflect.api.TypeTags.TypeTag<A3> evidence$52, scala.reflect.api.TypeTags.TypeTag<A4> evidence$53, scala.reflect.api.TypeTags.TypeTag<A5> evidence$54, scala.reflect.api.TypeTags.TypeTag<A6> evidence$55, scala.reflect.api.TypeTags.TypeTag<A7> evidence$56, scala.reflect.api.TypeTags.TypeTag<A8> evidence$57, scala.reflect.api.TypeTags.TypeTag<A9> evidence$58)
Defines a Scala closure of 9 arguments as user-defined function (UDF).
static [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-java.lang.Object-org.apache.spark.sql.types.DataType-)(Object f,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") dataType)
static [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-org.apache.spark.sql.api.java.UDF0-org.apache.spark.sql.types.DataType-)([UDF0](../../../../org/apache/spark/sql/api/java/UDF0.html "interface in org.apache.spark.sql.api.java")<?> f,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") returnType)
Defines a Java UDF0 instance as user-defined function (UDF).
static [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-org.apache.spark.sql.api.java.UDF1-org.apache.spark.sql.types.DataType-)([UDF1](../../../../org/apache/spark/sql/api/java/UDF1.html "interface in org.apache.spark.sql.api.java")<?,?> f,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") returnType)
Defines a Java UDF1 instance as user-defined function (UDF).
static [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-org.apache.spark.sql.api.java.UDF10-org.apache.spark.sql.types.DataType-)([UDF10](../../../../org/apache/spark/sql/api/java/UDF10.html "interface in org.apache.spark.sql.api.java")<?,?,?,?,?,?,?,?,?,?,?> f,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") returnType)
Defines a Java UDF10 instance as user-defined function (UDF).
static [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-org.apache.spark.sql.api.java.UDF2-org.apache.spark.sql.types.DataType-)([UDF2](../../../../org/apache/spark/sql/api/java/UDF2.html "interface in org.apache.spark.sql.api.java")<?,?,?> f,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") returnType)
Defines a Java UDF2 instance as user-defined function (UDF).
static [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-org.apache.spark.sql.api.java.UDF3-org.apache.spark.sql.types.DataType-)([UDF3](../../../../org/apache/spark/sql/api/java/UDF3.html "interface in org.apache.spark.sql.api.java")<?,?,?,?> f,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") returnType)
Defines a Java UDF3 instance as user-defined function (UDF).
static [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-org.apache.spark.sql.api.java.UDF4-org.apache.spark.sql.types.DataType-)([UDF4](../../../../org/apache/spark/sql/api/java/UDF4.html "interface in org.apache.spark.sql.api.java")<?,?,?,?,?> f,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") returnType)
Defines a Java UDF4 instance as user-defined function (UDF).
static [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-org.apache.spark.sql.api.java.UDF5-org.apache.spark.sql.types.DataType-)([UDF5](../../../../org/apache/spark/sql/api/java/UDF5.html "interface in org.apache.spark.sql.api.java")<?,?,?,?,?,?> f,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") returnType)
Defines a Java UDF5 instance as user-defined function (UDF).
static [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-org.apache.spark.sql.api.java.UDF6-org.apache.spark.sql.types.DataType-)([UDF6](../../../../org/apache/spark/sql/api/java/UDF6.html "interface in org.apache.spark.sql.api.java")<?,?,?,?,?,?,?> f,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") returnType)
Defines a Java UDF6 instance as user-defined function (UDF).
static [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-org.apache.spark.sql.api.java.UDF7-org.apache.spark.sql.types.DataType-)([UDF7](../../../../org/apache/spark/sql/api/java/UDF7.html "interface in org.apache.spark.sql.api.java")<?,?,?,?,?,?,?,?> f,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") returnType)
Defines a Java UDF7 instance as user-defined function (UDF).
static [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-org.apache.spark.sql.api.java.UDF8-org.apache.spark.sql.types.DataType-)([UDF8](../../../../org/apache/spark/sql/api/java/UDF8.html "interface in org.apache.spark.sql.api.java")<?,?,?,?,?,?,?,?,?> f,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") returnType)
Defines a Java UDF8 instance as user-defined function (UDF).
static [UserDefinedFunction](../../../../org/apache/spark/sql/expressions/UserDefinedFunction.html "class in org.apache.spark.sql.expressions")
[udf](../../../../org/apache/spark/sql/functions.html#udf-org.apache.spark.sql.api.java.UDF9-org.apache.spark.sql.types.DataType-)([UDF9](../../../../org/apache/spark/sql/api/java/UDF9.html "interface in org.apache.spark.sql.api.java")<?,?,?,?,?,?,?,?,?,?> f,[DataType](../../../../org/apache/spark/sql/types/DataType.html "class in org.apache.spark.sql.types") returnType)
Defines a Java UDF9 instance as user-defined function (UDF).
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[unbase64](../../../../org/apache/spark/sql/functions.html#unbase64-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Decodes a BASE64 encoded string column and returns it as a binary column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[unhex](../../../../org/apache/spark/sql/functions.html#unhex-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column)
Inverse of hex.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[unix_date](../../../../org/apache/spark/sql/functions.html#unix%5Fdate-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the number of days since 1970-01-01.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[unix_micros](../../../../org/apache/spark/sql/functions.html#unix%5Fmicros-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the number of microseconds since 1970-01-01 00:00:00 UTC.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[unix_millis](../../../../org/apache/spark/sql/functions.html#unix%5Fmillis-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the number of milliseconds since 1970-01-01 00:00:00 UTC.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[unix_seconds](../../../../org/apache/spark/sql/functions.html#unix%5Fseconds-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the number of seconds since 1970-01-01 00:00:00 UTC.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[unix_timestamp](../../../../org/apache/spark/sql/functions.html#unix%5Ftimestamp--)()
Returns the current Unix timestamp (in seconds) as a long.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[unix_timestamp](../../../../org/apache/spark/sql/functions.html#unix%5Ftimestamp-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") s)
Converts time string in format yyyy-MM-dd HH:mm:ss to Unix timestamp (in seconds), using the default timezone and the default locale.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[unix_timestamp](../../../../org/apache/spark/sql/functions.html#unix%5Ftimestamp-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") s, String p)
Converts time string with given pattern to Unix timestamp (in seconds).
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[unwrap_udt](../../../../org/apache/spark/sql/functions.html#unwrap%5Fudt-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") column)
Unwrap UDT data type column into its underlying type.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[upper](../../../../org/apache/spark/sql/functions.html#upper-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Converts a string column to upper case.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[url_decode](../../../../org/apache/spark/sql/functions.html#url%5Fdecode-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str)
Decodes a str in 'application/x-www-form-urlencoded' format using a specific encoding scheme.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[url_encode](../../../../org/apache/spark/sql/functions.html#url%5Fencode-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") str)
Translates a string into 'application/x-www-form-urlencoded' format using a specific encoding scheme.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[user](../../../../org/apache/spark/sql/functions.html#user--)()
Returns the user name of current execution context.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[uuid](../../../../org/apache/spark/sql/functions.html#uuid--)()
Returns an universally unique identifier (UUID) string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[var_pop](../../../../org/apache/spark/sql/functions.html#var%5Fpop-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the population variance of the values in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[var_pop](../../../../org/apache/spark/sql/functions.html#var%5Fpop-java.lang.String-)(String columnName)
Aggregate function: returns the population variance of the values in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[var_samp](../../../../org/apache/spark/sql/functions.html#var%5Fsamp-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: returns the unbiased variance of the values in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[var_samp](../../../../org/apache/spark/sql/functions.html#var%5Fsamp-java.lang.String-)(String columnName)
Aggregate function: returns the unbiased variance of the values in a group.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[variance](../../../../org/apache/spark/sql/functions.html#variance-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Aggregate function: alias for var_samp.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[variance](../../../../org/apache/spark/sql/functions.html#variance-java.lang.String-)(String columnName)
Aggregate function: alias for var_samp.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[version](../../../../org/apache/spark/sql/functions.html#version--)()
Returns the Spark version.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[weekday](../../../../org/apache/spark/sql/functions.html#weekday-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Returns the day of the week for date/timestamp (0 = Monday, 1 = Tuesday, ..., 6 = Sunday).
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[weekofyear](../../../../org/apache/spark/sql/functions.html#weekofyear-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Extracts the week number as an integer from a given date/timestamp/string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[when](../../../../org/apache/spark/sql/functions.html#when-org.apache.spark.sql.Column-java.lang.Object-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") condition, Object value)
Evaluates a list of conditions and returns one of multiple possible result expressions.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[width_bucket](../../../../org/apache/spark/sql/functions.html#width%5Fbucket-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") v,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") min,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") max,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") numBucket)
Returns the bucket number into which the value of this expression would fall after being evaluated.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[window_time](../../../../org/apache/spark/sql/functions.html#window%5Ftime-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") windowColumn)
Extracts the event time from the window column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[window](../../../../org/apache/spark/sql/functions.html#window-org.apache.spark.sql.Column-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") timeColumn, String windowDuration)
Generates tumbling time windows given a timestamp specifying column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[window](../../../../org/apache/spark/sql/functions.html#window-org.apache.spark.sql.Column-java.lang.String-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") timeColumn, String windowDuration, String slideDuration)
Bucketize rows into one or more time windows given a timestamp specifying column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[window](../../../../org/apache/spark/sql/functions.html#window-org.apache.spark.sql.Column-java.lang.String-java.lang.String-java.lang.String-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") timeColumn, String windowDuration, String slideDuration, String startTime)
Bucketize rows into one or more time windows given a timestamp specifying column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[xpath_boolean](../../../../org/apache/spark/sql/functions.html#xpath%5Fboolean-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") p)
Returns true if the XPath expression evaluates to true, or if a matching node is found.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[xpath_double](../../../../org/apache/spark/sql/functions.html#xpath%5Fdouble-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") p)
Returns a double value, the value zero if no match is found, or NaN if a match is found but the value is non-numeric.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[xpath_float](../../../../org/apache/spark/sql/functions.html#xpath%5Ffloat-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") p)
Returns a float value, the value zero if no match is found, or NaN if a match is found but the value is non-numeric.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[xpath_int](../../../../org/apache/spark/sql/functions.html#xpath%5Fint-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") p)
Returns an integer value, or the value zero if no match is found, or a match is found but the value is non-numeric.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[xpath_long](../../../../org/apache/spark/sql/functions.html#xpath%5Flong-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") p)
Returns a long integer value, or the value zero if no match is found, or a match is found but the value is non-numeric.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[xpath_number](../../../../org/apache/spark/sql/functions.html#xpath%5Fnumber-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") p)
Returns a double value, the value zero if no match is found, or NaN if a match is found but the value is non-numeric.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[xpath_short](../../../../org/apache/spark/sql/functions.html#xpath%5Fshort-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") p)
Returns a short integer value, or the value zero if no match is found, or a match is found but the value is non-numeric.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[xpath_string](../../../../org/apache/spark/sql/functions.html#xpath%5Fstring-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") p)
Returns the text contents of the first xml node that matches the XPath expression.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[xpath](../../../../org/apache/spark/sql/functions.html#xpath-org.apache.spark.sql.Column-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") x,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") p)
Returns a string array of values within the nodes of xml that match the XPath expression.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[xxhash64](../../../../org/apache/spark/sql/functions.html#xxhash64-org.apache.spark.sql.Column...-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")... cols)
Calculates the hash code of given columns using the 64-bit variant of the xxHash algorithm, and returns the result as a long column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[xxhash64](../../../../org/apache/spark/sql/functions.html#xxhash64-scala.collection.Seq-)(scala.collection.Seq<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> cols)
Calculates the hash code of given columns using the 64-bit variant of the xxHash algorithm, and returns the result as a long column.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[year](../../../../org/apache/spark/sql/functions.html#year-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
Extracts the year as an integer from a given date/timestamp/string.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[years](../../../../org/apache/spark/sql/functions.html#years-org.apache.spark.sql.Column-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") e)
A transform for timestamps and dates to partition data into years.
static [Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")
[zip_with](../../../../org/apache/spark/sql/functions.html#zip%5Fwith-org.apache.spark.sql.Column-org.apache.spark.sql.Column-scala.Function2-)([Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") left,[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql") right, scala.Function2<[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql"),[Column](../../../../org/apache/spark/sql/Column.html "class in org.apache.spark.sql")> f)
Merge two given arrays, element-wise, into a single array using a function.