Configuration override

Data Source configurations are set in the Data / Data Sources panel of the Domino UI. You can update it there for any permanent change.

Some configuration attributes can be overridden locally in the API. A sample of Data Source type configuration is described in the Classes section below.

Usage

You can create a configuration override with any of the config classes and update your Data Source entity:

from domino.data_sources import DataSourceClient, SnowflakeConfig

snowflake = DataSourceClient().get_datasource("snowflake-prod")

# Build a override config with a different warehouse than configured in Domino
config_xxl = SnowflakeConfig(warehouse="compute-xxl")

# Local update with no permanent change
snowflake.update(config=config_xxl)
res = snowflake.query("SELECT COUNT(*) FROM very_large_table")

# Override can also be used for temporary credentials
snowflake.update(config=SnowflakeConfig(username="admin", password="<password>"))
res = snowflake.query("SELECT secret_data FROM secret_table LIMIT 10")

To remove the configuration override, simply reset it:

from domino.data_sources import DataSourceClient, SnowflakeConfig

snowflake = DataSourceClient().get_datasource("snowflake-prod")
# Update to dev database
snowflake.update(SnowflakeConfig(database="dev"))

# Reset to default values
snowflake.reset_config()
res = snowflake.query("SELECT * FROM prod_table")

Starting from Domino 6.1.1, relational Data Sources support an optional datetime precision field, which is overridable in the Data API for version 6.3.0 and above. Adjusting the precision prevents extreme datetimes from overflowing. The precision can be set to "seconds", "milliseconds", "microseconds", or "nanoseconds", with "nanoseconds" as the default.

from domino.data_sources import DataSourceClient, SnowflakeConfig

snowflake = DataSourceClient().get_datasource("snowflake-prod")

# Update to datetime precision to milliseconds to support extremely large datetime values
snowflake.update(SnowflakeConfig(datetime_precision="milliseconds"))

res = snowflake.query("SELECT * FROM prod_table")

Classes

Here are examples of Data Sources and their configurable override options.

ADLS

class domino_data.data_sources.ADLSConfig(*, container=None, access_key=None)[source]

ADLS Data Source configuration.

  • access_key: Optional[str]

  • container: Optional[str]

BigQuery

class domino_data.data_sources.BigQueryConfig(*, gcp_project_id=None)[source]

BigQuery Data Source configuration.

  • datetime_precision: Optional[str]
    For dominodatalab-data>=6.3.0, valid values are: "seconds", "milliseconds", "microseconds", or "nanoseconds" (default: "nanoseconds").

  • gcp_project_id: Optional[str]

  • private_key_json: Optional[str]

GCS

class domino_data.data_sources.GCSConfig(*, bucket=None, private_key_json=None)[source]

GCS Data Source configuration.

  • bucket: Optional[str]

  • private_key_json: Optional[str]

GenericS3

class domino_data.data_sources.GenericS3Config(*, bucket=None, host=None, region=None, aws_access_key_id=None, aws_secret_access_key=None)[source]

Generic S3 Data Source configuration.

  • aws_access_key_id: Optional[str]

  • aws_secret_access_key: Optional[str]

  • bucket: Optional[str]

  • subfolder: Optional[str]

  • host: Optional[str]

  • region: Optional[str]

MySQL

class domino_data.data_sources.MySQLConfig(*, database=None, password=None, username=None, aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None)[source]

MySQL Data Source configuration.

  • aws_access_key_id: Optional[str]

  • aws_secret_access_key: Optional[str]

  • aws_session_token: Optional[str]

  • database: Optional[str]

  • datetime_precision: Optional[str]
    For dominodatalab-data>=6.3.0, valid values are: "seconds", "milliseconds", "microseconds", or "nanoseconds" (default: "nanoseconds").

  • password: Optional[str]

  • region: Optional[str]

  • username: Optional[str]

Oracle

class domino_data.data_sources.OracleConfig(*, database=None, password=None, username=None)[source]

Oracle Data Source configuration.

  • database: Optional[str]

  • datetime_precision: Optional[str]
    For dominodatalab-data>=6.3.0, valid values are: "seconds", "milliseconds", "microseconds", or "nanoseconds" (default: "nanoseconds").

  • password: Optional[str]

  • username: Optional[str]

PostgreSQL

class domino_data.data_sources.PostgreSQLConfig(*, database=None, password=None, username=None, aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None)[source]

PostgreSQL Data Source configuration.

  • aws_access_key_id: Optional[str]

  • aws_secret_access_key: Optional[str]

  • aws_session_token: Optional[str]

  • database: Optional[str]

  • datetime_precision: Optional[str]
    For dominodatalab-data>=6.3.0, valid values are: "seconds", "milliseconds", "microseconds", or "nanoseconds" (default: "nanoseconds").

  • password: Optional[str]

  • region: Optional[str]

  • username: Optional[str]

Redshift

class domino_data.data_sources.RedshiftConfig(*, database=None, password=None, username=None, aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None)[source]

Redshift Data Source configuration.

  • aws_access_key_id: Optional[str]

  • aws_secret_access_key: Optional[str]

  • aws_session_token: Optional[str]

  • database: Optional[str]

  • datetime_precision: Optional[str]
    For dominodatalab-data>=6.3.0, valid values are: "seconds", "milliseconds", "microseconds", or "nanoseconds" (default: "nanoseconds").

  • password: Optional[str]

  • region: Optional[str]

  • username: Optional[str]

S3

class domino_data.data_sources.S3Config(profile=None, *, bucket=None, region=None, aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None)

S3 Data Source configuration.

  • aws_access_key_id: Optional[str]

  • aws_secret_access_key: Optional[str]

  • aws_session_token: Optional[str]

  • bucket: Optional[str]

  • subfolder: optional[str]

  • region: Optional[str]

SQLServer

class domino_data.data_sources.SQLServerConfig(*, database=None, password=None, username=None)

SQL Server datasource configuration.

  • database: Optional[str]

  • datetime_precision: Optional[str]
    For dominodatalab-data>=6.3.0, valid values are: "seconds", "milliseconds", "microseconds", or "nanoseconds" (default: "nanoseconds").

  • password: Optional[str]

  • username: Optional[str]

Snowflake

class domino_data.data_sources.SnowflakeConfig(*, database=None, schema=None, warehouse=None, role=None, password=None, username=None, token=None)

Snowflake datasource configuration.

  • database: Optional[str]

  • datetime_precision: Optional[str]
    For dominodatalab-data>=6.3.0, valid values are: "seconds", "milliseconds", "microseconds", or "nanoseconds" (default: "nanoseconds").

  • password: Optional[str]

  • role: Optional[str]

  • schema: Optional[str]

  • token: Optional[str]

  • username: Optional[str]

  • warehouse: Optional[str]