livy_config {sparklyr} | R Documentation

## Create a Spark Configuration for Livy

### Description

Create a Spark Configuration for Livy

### Usage

    livy_config(config = spark_config(), username = NULL, password = NULL,
      custom_headers = list(`X-Requested-By` = "sparklyr"), ...)

### Arguments

`config` | Optional base configuration
`username` | The username to use in the Authorization header
`password` | The password to use in the Authorization header
`custom_headers` | List of custom headers to append to http requests. Defaults to `list("X-Requested-By" = "sparklyr")`.
`...` | additional Livy session parameters

### Details

Extends a Spark `"spark_config"` configuration with settings for Livy. For instance, `"username"` and `"password"` define the basic authentication settings for a Livy session.

The default value of `"custom_headers"` is set to `list("X-Requested-By" = "sparklyr")` in order to facilitate connection to Livy servers with CSRF protection enabled.

Additional parameters for Livy sessions are:

`proxy_user`

  User to impersonate when starting the session

`jars`

  jars to be used in this session

`py_files`

  Python files to be used in this session

`files`

  files to be used in this session

`driver_memory`

  Amount of memory to use for the driver process

`driver_cores`

  Number of cores to use for the driver process

`executor_memory`

  Amount of memory to use per executor process

`executor_cores`

  Number of cores to use for each executor

`num_executors`

  Number of executors to launch for this session

`archives`

  Archives to be used in this session

`queue`

  The name of the YARN queue to which submitted

`queue`

  The name of this session

`heartbeat_timeout`

  Timeout in seconds to which session be orphaned

### Value

Named list with configuration data

[Package sparklyr version 0.6.3 Index]