Class DataFrameExtensions

Namespace

SparkTest.NET.Extensions

Assembly

SparkTest.NET.dll

Extension methods for working with the DataFrame API

public static class DataFrameExtensions

Inheritance

object

DataFrameExtensions

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Methods

AsSparkType(Type)

Converts a type to its spark type

AsStructType(Type)

Creates a struct type from a given Type

AsStructType<T>(T)

Creates a struct type from a given T

CreateDataFrameFromData<TData>(SparkSession, IEnumerable<TData>)

Creates a DataFrame from a given TData loaded into spark as a JSON file

CreateDataFrameFromData<TData>(SparkSession, TData, params TData[])

Creates a DataFrame from a given TData loaded into spark as a JSON file

CreateEmptyFrame(SparkSession)

Creates an empty DataFrame

CreateStructType<T>()

Creates a struct type from a given T

Debug(DataFrame, int, int, bool)

Debugs a provided DataFrame

PrintSchemaString(DataFrame)

Prints the schema to the console in a nice tree format

ReIndexExplainPlan(string, bool)

Replaces the non-deterministic parts of an explain plan result

ShowMdString(DataFrame, int, int, bool, bool, bool, bool)

Displays rows of the DataFrame in tabular form in common mark format

ShowString(DataFrame, int, int, bool)

Displays rows of the DataFrame in tabular form