Before you can start using on-demand Spark clusters on Domino, you must enable and configure the functionality on your deployment.
Your Domino administrator must set ShortLived.SparkClustersEnabled
to true to enable on-demand Spark functionality.
By default, Domino does not come with a Spark compatible compute environment that can be used for the components of the cluster. Without at least one such environment available, you cannot create a cluster.
When using on-demand Spark in Domino, you need one environment for the Spark cluster (base or worker environment) and one environment for the workspace/job execution (compute environment).
Create a new base Spark cluster environment
-
Follow the instructions to create an environment.
-
In the Base image section, select Custom Image and specify an image URI that points to a deployable Spark image.