About Domino
Domino Data LabKnowledge BaseData Science BlogTraining

Scale deployed Model APIs

To scale the performance of Model APIs in Domino, you can configure model resource quotas, or increase the degree of parallelism.

Configure resource quotas

You set the resource quotas for the hardware specifications of the pods that host Model APIs. A resource quota determines the CPU and memory resources available to the model that uses it. Resource quotas are different than the hardware tiers Domino Runs use.

  1. From the admin home, go to Advanced > Resource Quotas.

  2. Click New to create a resource quota, or click Edit to modify an existing resource quota or set a default resource quota. Resource quotas cannot be permanently deleted.

  3. When you create or edit resource quotas, set the following properties:

    • Visible: Leave this checkbox selected for a resource quota to be listed in the menu for users publishing models. Clear the checkbox to make a resource quota unavailable for use.

    • Memory (GB) - Request: The amount of RAM reserved for a model with this quota.

    • Memory (GB) - Limit: If the hosting node has RAM available, a model running this quota can make use of additional memory up to this limit.

    • CPU (# of Cores) - Request: The number of cores reserved for a model with this quota.

    • CPU (# of Cores) - Limit: If the hosting node has idle cores available, a model running this quota can use additional cores up to this limit.

    • Default: The resource quota with this set to true will be used for all newly published models by default.

      Your users can now set the resource quotas for their deployments. To do this, they must go to the model’s deployment page. Then, go to the Compute resources per instance menu in the Resource Quota section to set resource quotas.

      Deployment configuration

  4. Click Create.

Scale Model APIs

To scale all Python Model APIs, set the degree of parallelism.

Note
 Only synchronous models support this.

  1. Go to Admin > Advanced > Central Config.

  2. Set com.cerebro.domino.modelmanager.uWsgi.workerCount to a value greater than its default value of 1 to increase the uWSGI worker count. See the uWSGI documentation for more information.

  3. The system shows the following message: Changes here do not take effect until services are restarted. Click here to restart services.

  4. Click here to restart the services.

Next steps

Learn more about Domino hardware tiers.