Installation - Advanced Configuration

Overview

The installation process described on the Installation documentation is appropriate for most environments and use cases. However, you might want to customize Anaconda platform settings related to security, network, installation paths, or other settings using the process described on this page.

Cluster profiles are used to define information about a cluster before it is created, including the cluster name, nodes in the cluster, and other platform component options. Cluster files are located in the ~/.continuum/adam/profile.d/ directory.

Cluster definitions are used to identify information about a running cluster, including the cluster name, nodes in the cluster, and other platform component options. Cluster definitions are located in the ~/.continuum/adam/cluster.d/ directory.

Cluster Profile - Reference

A cluster profile located at ~/.continuum/adam/profile.d/my-cluster.yaml with all configurable settings is shown below:

name: my-cluster
provider: bare

bare:
  username: centos
  # password: anaconda  # Optional
  port: 22
  keypair: ~/.ssh/my-private-key
  nodes:
  - host: node1.company.com
  - host: node2.company.com
  - host: node3.company.com
  - host: node4.company.com

security:
  flush_iptables: false
  selinux_context: false
  selinux_permissive: false

network:
  http_proxy: http://server:port/
  # https_proxy: http://server:port/

system:
  tmp_dir: /tmp

plugins:
  accelerate:
    accelerate_version: null
    enabled: false

  conda:
    anaconda_hash: md5=a0d1fbe47014b71c6764d76fb403f217
    anaconda_url: http://repo.continuum.io/archive/Anaconda2-4.2.0-Linux-x86_64.sh#adam
    channel_alias: https://conda.anaconda.org/
    channels:
    - defaults
    - anaconda-adam
    conda_canary: false
    enabled: true
    miniconda_hash: md5=c8b836baaa4ff89192947e4b1a70b07e
    miniconda_url: http://repo.continuum.io/miniconda/Miniconda2-4.2.12-Linux-x86_64.sh#adam
    rootdir: /opt/continuum
    ssl_verify: null

  dask:
    bokeh_port: 8787
    bokeh_whitelist: '''*'''
    enabled: false
    host: 0.0.0.0
    http_port: 9786
    nprocs: 1
    port: 8786

  enterprise-notebooks:
    admin_email: admin@yourdomain.com
    admin_password: anaconda
    admin_user: wakari
    directory: /opt/wakari
    elasticsearch_fn: elasticsearch-1.7.2.noarch.rpm
    enabled: false
    enterprise_notebooks_version: 4.0.0
    gateway_port: 8089
    java_fn: jre-8u65-linux-x64.rpm
    mongodb_fn: mongodb-org-2.6.8-1.x86_64.rpm
    mongodb_mongos_fn: mongodb-org-mongos-2.6.8-1.x86_64.rpm
    mongodb_server_fn: mongodb-org-server-2.6.8-1.x86_64.rpm
    mongodb_shell_fn: mongodb-org-shell-2.6.8-1.x86_64.rpm
    mongodb_tools_fn: mongodb-org-tools-2.6.8-1.x86_64.rpm
    mongodb_version: 2.6.8
    nginx_fn: nginx-1.6.2-1.el6.ngx.x86_64.rpm
    root_download_url: https://820451f3d8380952ce65-4cc6343b423784e82fd202bb87cf87cf.ssl.cf1.rackcdn.com/
    user: wakari

  jupyter:
    dashboards_server:
      enabled: false
      ip: 0.0.0.0
      port: 3000
      user: anaconda
    enabled: false
    kernel_gateway:
      enabled: false
      ip: 0.0.0.0
      port: 7000
      user: anaconda
    notebook:
      directory: ~/notebooks
      enabled: false
      ip: 0.0.0.0
      password: ''
      port: 8888
      user: anaconda

  mosaic:
    anaconda_mosaic_version: 1.3.1
    blaze_host: 0.0.0.0
    blaze_port: 6363
    directory: /mosaic
    enabled: false
    environment_name: mosaic
    host: 0.0.0.0
    notebook_host: '''*'''
    notebook_port: 8001
    port: 8000
    python_version: 3.4.*
    token: an-dae1acca-da25-47ba-8766-dd42ecf5e03e
    user: anaconda
    whitelist: '''*'''

  repository:
    binstar_server_version: 2.21.0
    channel: main
    email: youremail@continuum.io
    enabled: false
    mongodb_fn: mongodb-org-2.6.8-1.x86_64.rpm
    mongodb_mongos_fn: mongodb-org-mongos-2.6.8-1.x86_64.rpm
    mongodb_server_fn: mongodb-org-server-2.6.8-1.x86_64.rpm
    mongodb_shell_fn: mongodb-org-shell-2.6.8-1.x86_64.rpm
    mongodb_tools_fn: mongodb-org-tools-2.6.8-1.x86_64.rpm
    mongodb_version: 2.6.9
    password: anaconda
    port: 8080
    root_download_url: https://820451f3d8380952ce65-4cc6343b423784e82fd202bb87cf87cf.ssl.cf1.rackcdn.com/
    server_user: anaconda-server
    superuser: superuser
    token: qu-a49e2e69-1047-4eab-a879-a2ee9c198381

  salt_settings:
    acl:
    - anaconda
    job_pub_port: 14505
    minion_pub_port: 14510
    minion_pull_port: 14511
    minion_ret_port: 14506
    rest_port: 18000
    salt_groupname: anaconda
    salt_password: anaconda
    salt_username: anaconda
    service_scripts: false

Cluster Name

This setting specifies the name of the cluster and is used by the -n option in adam commands. For example, to create a new cluster named my-cluster from a profile called cluster, you would use the command:

$ adam up -n my-cluster cluster

SSH Authentication

To connect to remote machines via SSH, Anaconda Adam requires a valid username, port (default: 22), and an authentication method. The following SSH authentication methods are supported in Anaconda Adam:

  • Password
  • Key pair (full path to your private key)
  • Encrypted key pair (full path to your public key and ssh-agent)

Refer to the Requirements page for more information about system and account requirements.

For example, to use SSH with a key pair:

bare:
  username: centos
  keypair: /full/path/to/key.rsa

Or, to use SSH with a username and password:

bare:
  username: centos
  password: anaconda

Or, to use SSH with an encrypted key pair, the private key must first be added to a running ssh-agent. Use the agent_pubkey setting in the cluster profile to specify the full path to the associated public key. For example:

bare:
  username: centos
  agent_pubkey: /full/path/to/key.pub

Node Specification

This setting defines the nodes that exist within the cluster. By default, the first node in the list of nodes will be defined as the head node, and the remaining nodes will be defined as compute nodes.

Each host should be able to connect to the other hosts via the specified FQDN or IP address. It is recommended that you use the FQDN of the nodes that matches the hostname on each machine.

bare:
  nodes:
  - host: node1.company.com
  - host: node2.company.com
  - host: node3.company.com
  - host: node4.company.com

Security and Network Settings

These settings allow you to configure iptabes and SELinux.

To flush iptables (default: false):

security:
  flush_iptables: false

To set SELinux to permissive (default: false):

security:
  selinux_permissive: false

To set SELinux contexts (default: false):

security:
  selinux_context: false

Proxy Configuration

You can install Anaconda Adam to nodes that access the Internet via an HTTP or HTTPS proxy by using the network setting. To specify an HTTP proxy:

network:
  http_proxy: http://server:port/

or, to specify an HTTPS proxy:

network:
  https_proxy: http://server:port/

Conda Settings

These settings are related to the installation of Anaconda/Miniconda across the cluster using Anaconda Scale.

conda:
    anaconda_hash: md5=a0d1fbe47014b71c6764d76fb403f217
    anaconda_url: http://repo.continuum.io/archive/Anaconda2-4.2.0-Linux-x86_64.sh#adam
    channel_alias: https://conda.anaconda.org/
    channels:
    - defaults
    - anaconda-adam
    enabled: true
    miniconda_hash: md5=c8b836baaa4ff89192947e4b1a70b07e
    miniconda_url: http://repo.continuum.io/miniconda/Miniconda2-4.2.12-Linux-x86_64.sh#adam
    rootdir: /opt/continuum
    ssl_verify: False

Above are the default settings for configuring conda. The conda setting for ssl_verify can be set to one of three options:

  • False: no SSL verification (default),
  • True: SSL verification is used and conda verifies certificates for SSL connections,
  • Cert Path: string path to a certificate to be used to verify SSL connections

Salt Settings

These settings allow you to configure the network options, access control list, and API user credentials used by Salt, which is the configuration management system used by Anaconda Adam. To specify the ports used by the Salt master, minions, and REST API:

plugins:
  salt_settings:
    job_pub_port: 14505
    minion_pub_port: 14510
    minion_pull_port: 14511
    minion_ret_port: 14506
    rest_port: 18000

To specify non-root system users that can execute Salt commands via an access control list:

plugins:
  salt_settings:
    acl:
    - anaconda

To specify the username, password, and group that gets created across the cluster and owns the directory and files located in the root installation directory (default /opt/continuum):

plugins:
  salt_settings:
    salt_username: anaconda
    salt_password: anaconda
    salt_groupname: anaconda

If you do not want to store a clear text password in the cluster profile or cluster definition file, you can set the salt_password to an empty string:

plugins:
  salt_settings:
    salt_username: anaconda
    salt_password: ''
    salt_groupname: anaconda

In this case, Anaconda Adam will prompt for a password in the CLI and only store the password in memory for the duration of the command/job.

System Settings

These settings allow you to configure system options.

To specify the download directory where temporary installers are downloaded to (default: /tmp):

system:
  tmp_dir: /tmp