# R Interface to Apache Spark

## Documentation for package ‘sparklyr’ version 0.5.6

- DESCRIPTION file.

## Help Pages

C E F H I J L M N R S T

## -- C --

compile_package_jars | Compile Scala sources into a Java Archive (jar)
connection_config | Read configuration values for a connection
copy_to.spark_connection | Copy an R Data Frame to Spark

## -- E --

ensure | Enforce Specific Structure for R Objects
ensure_scalar_boolean | Enforce Specific Structure for R Objects
ensure_scalar_character | Enforce Specific Structure for R Objects
ensure_scalar_double | Enforce Specific Structure for R Objects
ensure_scalar_integer | Enforce Specific Structure for R Objects

## -- F --

find_scalac | Discover the Scala Compiler
ft_binarizer | Feature Transformation - Binarizer
ft_bucketizer | Feature Transformation - Bucketizer
ft_discrete_cosine_transform | Feature Transformation - Discrete Cosine Transform (DCT)
ft_elementwise_product | Feature Transformation - ElementwiseProduct
ft_index_to_string | Feature Transformation - IndexToString
ft_one_hot_encoder | Feature Transformation - OneHotEncoder
ft_quantile_discretizer | Feature Transformation - QuantileDiscretizer
ft_regex_tokenizer | Feature Tranformation - RegexTokenizer
ft_sql_transformer | Feature Transformation - SQLTransformer
ft_string_indexer | Feature Transformation - StringIndexer
ft_tokenizer | Feature Tranformation - Tokenizer
ft_vector_assembler | Feature Transformation - VectorAssembler

## -- H --

hive_context | Access the Spark API

## -- I --

invoke | Invoke a Method on a JVM Object
invoke_new | Invoke a Method on a JVM Object
invoke_static | Invoke a Method on a JVM Object

## -- J --

java_context | Access the Spark API

## -- L --

livy_config | Create a Spark Configuration for Livy
livy_service_start | Start Livy
livy_service_stop | Start Livy

## -- M --

ml_als_factorization | Spark ML - Alternating Least Squares (ALS) matrix factorization.
ml_binary_classification_eval | Spark ML - Binary Classification Evaluator
ml_classification_eval | Spark ML - Classification Evaluator
ml_create_dummy_variables | Create Dummy Variables
ml_decision_tree | Spark ML - Decision Trees
ml_generalized_linear_regression | Spark ML - Generalized Linear Regression
ml_gradient_boosted_trees | Spark ML - Gradient-Boosted Tree
ml_kmeans | Spark ML - K-Means Clustering
ml_lda | Spark ML - Latent Dirichlet Allocation
ml_linear_regression | Spark ML - Linear Regression
ml_load | Save / Load a Spark ML Model Fit
ml_logistic_regression | Spark ML - Logistic Regression
ml_model | Create an ML Model Object
ml_multilayer_perceptron | Spark ML - Multilayer Perceptron
ml_naive_bayes | Spark ML - Naive-Bayes
ml_one_vs_rest | Spark ML - One vs Rest
ml_options | Options for Spark ML Routines
ml_pca | Spark ML - Principal Components Analysis
ml_prepare_dataframe | Prepare a Spark DataFrame for Spark ML Routines
ml_prepare_features | Pre-process the Inputs to a Spark ML Routine
ml_prepare_inputs | Pre-process the Inputs to a Spark ML Routine
ml_prepare_response_features_intercept | Pre-process the Inputs to a Spark ML Routine
ml_random_forest | Spark ML - Random Forests
ml_save | Save / Load a Spark ML Model Fit
ml_saveload | Save / Load a Spark ML Model Fit
ml_survival_regression | Spark ML - Survival Regression
ml_tree_feature_importance | Spark ML - Feature Importance for Tree Models

## -- N --

na.replace | Replace Missing Values in Objects

## -- R --

registered_extensions | Register a Package that Implements a Spark Extension
register_extension | Register a Package that Implements a Spark Extension

## -- S --

sdf-saveload | Save / Load a Spark DataFrame
sdf_copy_to | Copy an Object into Spark
sdf_import | Copy an Object into Spark
sdf_load_parquet | Save / Load a Spark DataFrame
sdf_load_table | Save / Load a Spark DataFrame
sdf_mutate | Mutate a Spark DataFrame
sdf_mutate_ | Mutate a Spark DataFrame
sdf_partition | Partition a Spark Dataframe
sdf_persist | Persist a Spark DataFrame
sdf_predict | Model Predictions with Spark DataFrames
sdf_quantile | Compute (Approximate) Quantiles with a Spark DataFrame
sdf_read_column | Read a Column from a Spark DataFrame
sdf_register | Register a Spark DataFrame
sdf_sample | Randomly Sample Rows from a Spark DataFrame
sdf_save_parquet | Save / Load a Spark DataFrame
sdf_save_table | Save / Load a Spark DataFrame
sdf_schema | Read the Schema of a Spark DataFrame
sdf_sort | Sort a Spark DataFrame
sdf_with_unique_id | Add a Unique ID Column to a Spark DataFrame
spark-api | Access the Spark API
spark-connections | Manage Spark Connections
spark_available_versions | Download and install various versions of Spark
spark_compilation_spec | Define a Spark Compilation Specification
spark_config | Read Spark Configuration
spark_connect | Manage Spark Connections
spark_connection | Retrieve the Spark Connection Associated with an R Object
spark_connection_is_open | Manage Spark Connections
spark_context | Access the Spark API
spark_dataframe | Retrieve a Spark DataFrame
spark_default_compilation_spec | Default Compilation Specification for Spark Extensions
spark_dependency | Define a Spark dependency
spark_disconnect | Manage Spark Connections
spark_disconnect_all | Manage Spark Connections
spark_install | Download and install various versions of Spark
spark_installed_versions | Download and install various versions of Spark
spark_install_dir | Download and install various versions of Spark
spark_install_tar | Download and install various versions of Spark
spark_jobj | Retrieve a Spark JVM Object Reference
spark_load_table | Load a Spark Table into a Spark DataFrame.
spark_log | View Entries in the Spark Log
spark_read_csv | Read a CSV file into a Spark DataFrame
spark_read_json | Read a JSON file into a Spark DataFrame
spark_read_parquet | Read a Parquet file into a Spark DataFrame
spark_save_table | Saves a Spark DataFrame as a Spark table
spark_session | Access the Spark API
spark_uninstall | Download and install various versions of Spark
spark_version | Get the Spark Version Associated with a Spark Connection
spark_version_from_home | Get the Spark Version Associated with a Spark Installation
spark_web | Open the Spark web interface
spark_write_csv | Write a Spark DataFrame to a CSV
spark_write_json | Write a Spark DataFrame to a JSON file
spark_write_parquet | Write a Spark DataFrame to a Parquet file

## -- T --

tbl_cache | Cache a Spark Table
tbl_uncache | Uncache a Spark Table
top_n | Select top (or bottom) n rows (by value)