Client API

Client API¶

Gateway¶

class dask_gateway.Gateway(address=None, proxy_address=None, public_address=None, auth=None, asynchronous=False, loop=None)¶

A client for a Dask Gateway Server.

Parameters

address (str, optional) – The address to the gateway server.
proxy_address (str, int, optional) – The address of the scheduler proxy server. Defaults to address if not provided. If an int, it’s used as the port, with the host/ip taken from address. Provide a full address if a different host/ip should be used.
public_address (str, optional) – The address to the gateway server, as accessible from a web browser. This will be used as the root of all browser-facing links (e.g. the dask dashboard). Defaults to address if not provided.
auth (GatewayAuth, optional) – The authentication method to use.
asynchronous (bool, optional) – If true, starts the client in asynchronous mode, where it can be used in other async code.
loop (IOLoop, optional) – The IOLoop instance to use. Defaults to the current loop in asynchronous mode, otherwise a background loop is started.

adapt_cluster(cluster_name, minimum=None, maximum=None, active=True, **kwargs)¶

Configure adaptive scaling for a cluster.

Parameters

cluster_name (str) – The cluster name.
minimum (int, optional) – The minimum number of workers to scale to. Defaults to 0.
maximum (int, optional) – The maximum number of workers to scale to. Defaults to infinity.
active (bool, optional) – If True (default), adaptive scaling is activated. Set to False to deactivate adaptive scaling.

close()¶: Close the gateway client

cluster_options(use_local_defaults=True, **kwargs)¶

Get the available cluster configuration options.

Parameters: use_local_defaults (bool, optional) – Whether to use any default options from the local configuration. Default is True, set to False to use only the server-side defaults.
Returns: cluster_options – A dict of cluster options.
Return type: dask_gateway.options.Options

connect(cluster_name, shutdown_on_close=False)¶

Connect to a submitted cluster.

Parameters

cluster_name (str) – The cluster to connect to.
shutdown_on_close (bool, optional) – If True, the cluster will be automatically shutdown on close. Default is False.

Returns

cluster

Return type

GatewayCluster

get_cluster(cluster_name, **kwargs)¶

Get information about a specific cluster.

Parameters: cluster_name (str) – The cluster name.
Returns: report
Return type: ClusterReport

get_versions()¶

Return version info for the server and client

Returns: version_info
Return type: dict

list_clusters(status=None, **kwargs)¶

List clusters for this user.

Parameters: status (ClusterStatus, str, or list, optional) – The cluster status (or statuses) to select. Valid options are ‘pending’, ‘running’, ‘stopping’, ‘stopped’, ‘failed’. By default selects active clusters (‘pending’, ‘running’).
Returns: clusters
Return type: list of ClusterReport

new_cluster(cluster_options=None, shutdown_on_close=True, **kwargs)¶

Submit a new cluster to the gateway, and wait for it to be started.

Same as calling submit and connect in one go.

Parameters

cluster_options (dask_gateway.options.Options, optional) – An Options object describing the desired cluster configuration.
shutdown_on_close (bool, optional) – If True (default), the cluster will be automatically shutdown on close. Set to False to have cluster persist until explicitly shutdown.
**kwargs – Additional cluster configuration options. If cluster_options is provided, these are applied afterwards as overrides. Available options are specific to each deployment of dask-gateway, see cluster_options for more information.

Returns

cluster

Return type

GatewayCluster

scale_cluster(cluster_name, n, **kwargs)¶

Scale a cluster to n workers.

Parameters

cluster_name (str) – The cluster name.
n (int) – The number of workers to scale to.

stop_cluster(cluster_name, **kwargs)¶

Stop a cluster.

Parameters: cluster_name (str) – The cluster name.

submit(cluster_options=None, **kwargs)¶

Submit a new cluster to be started.

This returns quickly with a cluster_name, which can later be used to connect to the cluster.

Parameters

cluster_options (dask_gateway.options.Options, optional) – An Options object describing the desired cluster configuration.
**kwargs – Additional cluster configuration options. If cluster_options is provided, these are applied afterwards as overrides. Available options are specific to each deployment of dask-gateway, see cluster_options for more information.

Returns

cluster_name – The cluster name.

Return type

str

GatewayCluster¶

class dask_gateway.GatewayCluster(address=None, proxy_address=None, public_address=None, auth=None, cluster_options=None, shutdown_on_close=True, asynchronous=False, loop=None, **kwargs)¶

A dask-gateway cluster.

Parameters

address (str, optional) – The address to the gateway server.
proxy_address (str, int, optional) – The address of the scheduler proxy server. If an int, it’s used as the port, with the host/ip taken from address. Provide a full address if a different host/ip should be used.
public_address (str, optional) – The address to the gateway server, as accessible from a web browser. This will be used as the root of all browser-facing links (e.g. the dask dashboard). Defaults to address if not provided.
auth (GatewayAuth, optional) – The authentication method to use.
cluster_options (mapping, optional) – A mapping of cluster options to use to start the cluster.
shutdown_on_close (bool, optional) – If True (default), the cluster will be automatically shutdown on close.
asynchronous (bool, optional) – If true, starts the cluster in asynchronous mode, where it can be used in other async code.
loop (IOLoop, optional) – The IOLoop instance to use. Defaults to the current loop in asynchronous mode, otherwise a background loop is started.
**kwargs – Additional cluster configuration options. If cluster_options is provided, these are applied afterwards as overrides. Available options are specific to each deployment of dask-gateway, see Gateway.cluster_options for more information.

adapt(minimum=None, maximum=None, active=True, **kwargs)¶

Configure adaptive scaling for the cluster.

Parameters

minimum (int, optional) – The minimum number of workers to scale to. Defaults to 0.
maximum (int, optional) – The maximum number of workers to scale to. Defaults to infinity.
active (bool, optional) – If True (default), adaptive scaling is activated. Set to False to deactivate adaptive scaling.

close(shutdown=None)¶

Close the cluster object.

Parameters: shutdown (bool, optional) – Whether to shutdown the cluster. If True, the cluster will be shutdown, if False only the local state will be cleaned up. Defaults to the value of shutdown_on_close, set to True or False to override.

classmethod from_name(cluster_name, shutdown_on_close=False, address=None, proxy_address=None, public_address=None, auth=None, asynchronous=False, loop=None)¶

Connect to a submitted cluster.

Parameters

cluster_name (str) – The cluster to connect to.
shutdown_on_close (bool, optional) – If True, the cluster will be automatically shutdown on close. Default is False.
**kwargs – Additional parameters to pass to the GatewayCluster constructor. See the docstring for more information.

Returns

cluster

Return type

GatewayCluster

get_client(set_as_default=True)¶

Get a Client for this cluster.

Returns: client
Return type: dask.distributed.Client

scale(n, **kwargs)¶

Scale the cluster to n workers.

Parameters: n (int) – The number of workers to scale to.

shutdown()¶: Shutdown this cluster. Alias for close(shutdown=True).

Options¶

class dask_gateway.options.Options(*fields)¶

Options to submit to dask-gateway.

A mutable-mapping that describes all cluster options available for user’s to set. Options can be modified programatically, or via a widget when using a web interface (e.g. Jupyter Notebooks) with ipywidgets installed.

Examples

Options objects are normally created via the Gateway.cluster_options method:

>>> options = gateway.cluster_options()  

Available options can then be accessed or set via attribute or key:

>>> options.worker_cores  
1
>>> options["worker_cores"]  
1
>>> options.worker_cores = 2  
>>> options.worker_cores  
2

Accessing invalid options error appropriately:

>>> options.not_a_valid_option = 'myvalue'  
Traceback (most recent call last):
    ...
AttributeError: No option 'not_a_valid_option' available

Errors are also raised if values being set are invalid (e.g. wrong type, out-of-bounds, etc…).

Authentication¶

class dask_gateway.auth.GatewayAuth(**kwargs)¶: Base class for authenticating clients in dask-gateway

class dask_gateway.auth.BasicAuth(username=None, password=None)¶: Attaches HTTP Basic Authentication to the given Request object.

class dask_gateway.auth.KerberosAuth(**kwargs)¶: Authenticate with kerberos

class dask_gateway.auth.JupyterHubAuth(api_token=None)¶: Uses JupyterHub API tokens to authenticate

Exceptions¶

class dask_gateway.GatewayClusterError¶: Exception related to starting/stopping/scaling of a gateway cluster

class dask_gateway.GatewayServerError¶

Exception related to the operation of the gateway server.

Indicates an internal error in the gateway server.

Cluster Resource Limits

Configuration Reference

Dask Gateway 2024.1.0 documentation

Client API

Contents

Client API¶

Gateway¶

GatewayCluster¶

Options¶

Authentication¶

Exceptions¶