Domino can connect to your external data sources, such as databases, data warehouses, and data lakes.
Domino automatically creates an audit trail for Domino Data Sources to let you audit and track all Data Source-related activities.
There are two ways to connect your external data:
-
Domino Data Source connectors are available for some types of data sources. Data Source connectors are the simplest way to connect to supported data sources. It takes just a few clicks to create a Data Source, and you can re-use it in any project.
-
Direct connections are available for all types of data sources. This approach is useful if you want to use a data source that is not supported by a Domino connector, or if you want to use a different version of the connector than the one that is installed on the Domino server. However, direct connections require more configuration than Domino connectors and are not officially supported.
|
Note
|
In a Domino Nexus deployment, Data Sources can be accessed on both the Local and remote data planes, with the exception of Starburst Trino.
Data sources may not be usable in every data plane due to network restrictions.
|
The table below compares the two methods for accessing external data sources:
| Domino Data Source | Direct connection |
|---|---|
Fetch data from the Data Source into a data frame using SQL. | Fetch data using native library functions. |
Optimized for read-heavy workflows. Data sources support synchronous read/write operations. However, Data Source connectors don’t return metadata about the results of a write operation, such as the number of rows affected. | Write-heavy workflows. Direct connections can support asynchronous read/write operations |
Easy to set up and use. Once configured a few simple steps or programmatically (such as with Domino Code Assist), you can embed the Data Sources into your notebooks and fetch data into your data frames through SQL. | Use of specialized library functions is a priority. This can be important in non-traditional data sources or in data lakes, where you might rely on specialized functions to process different shapes of data to embed them into your CI/CD pipelines. |
Sharing is a priority. Domino Data Sources are easy to share with other users across Domino projects. They also support service accounts that authenticate all users, in addition to individual accounts where users provide their own credentials. | Sharing is not a priority. You must set up connectivity to your Data Sources on your own and explicitly share them with other users. |
Easy security configuration. Domino Data Sources allow for secure safe-keeping of individual and service account credentials in HashiCorp Vault; there is no risk of those secrets being shared with or leaked to other users. | Secure connectivity must be established through custom solutions using Kubernetes secrets or through integration with a centralized vault/safe-keeping mechanism. |
Portability is not a priority. Domino Data Sources do not prioritize support for integrated workflows/pipelines that pull/process data outside of Domino. Although you can call the Domino Data API from outside of Domino, it does not offer the same benefits and safeguards that native libraries provide when talking directly to services outside of Domino. | Portability is a priority. If you need to reuse your code outside of Domino or you want broad functionality when communicating with your data services from outside of Domino, using native libraries might be preferable. |
Domino comes with connectors for many external data sources. See full list of supported connectors in the left navigation under Data Source Connectors.