You set the resource quotas for the hardware specifications of the pods that host Model APIs. A resource quota determines the CPU and memory resources available to the Model that uses it. Resource quotas are different than the hardware tiers Domino Runs use.
-
From the admin home, go to Advanced > Resource Quotas.
-
Click New to create a resource quota, or click Edit to modify an existing resource quota or set a default resource quota. Resource quotas cannot be permanently deleted.
-
When you create or edit a Resource quotas, set the following properties:
-
Visible: Leave this checkbox selected for a resource quota to be listed in the menu for users publishing Models. Clear the checkbox to make a resource quota unavailable for use.
-
Memory (GB) - Request: The amount of RAM reserved for a model with this quota.
-
Memory (GB) - Limit: If the hosting node has RAM available, a model running this quota can make use of additional memory up to this limit.
-
CPU (# of Cores) - Request: The number of cores reserved for a Model with this quota.
-
CPU (# of Cores) - Limit: If the hosting node has idle cores available, a model running this quota can use additional cores up to this limit.
-
Default: The resource quota with this set to
truewill be used for all newly published Models by default. -
GPU: Select this checkbox to request GPU resources. When the checkbox is selected, you must enter the GPU Resource Name and Number of requested GPUs.
The Request values for CPU Cores and Memory, and GPUs are thresholds used to determine whether a node has capacity to host an execution pod. These requested resources are effectively reserved for the pod.
The Limit values control the amount of resources a pod can use beyond the amount requested. If there’s additional headroom on the node, the pod can use resources up to this limit.
Caution -
Node Pool: Select the type of the node pool for the deployment.
-
If you select Default, the node pool defaults to the value that your administrator set in the
com.cerebro.domino.modelManager.nodeSelectorLabelValueorcom.cerebro.domino.modelManager.nodeSelectorLabelValue.gpukey, depending on whether you are using GPUs or not. -
If you select Custom, enter the name of the node pool.
Your users can now set the Resource Quotes for their deployments. To do this, they must go to the Model’s deployment page. Then, go to the Compute resources per instance menu in the Resource Quota section to set resource quotas.
-
-
-
Click Create.