Git-based Projects with CodeSync

Note

We recommend using Git-based projects in Domino to take advantage of our CodeSync technology if:

  • you have at least a beginner’s proficiency with Git,

  • you have experience using hosted version control (GitHub, Bitbucket, etc.), and

  • you frequently collaborate with multiple data scientists on projects.

To learn more about common workflows using Git and branching, we recommend the following resources: Git Feature Branch Workflow.

If you’re not familiar with Git or hosted version control, or if you’re primarily looking to reproduce data science research, then a project that uses the Domino File System (DFS) may be a better fit for you.

Git-based projects with CodeSync are a beta feature. If you’d like to provide feedback on Git-based projects, please file a ticket.

Overview

Git-based projects provide a full CodeSync experience for your code by using Git and a Git service provider of your choice. Integrated CodeSync technology ensures that all of the common Git workflows, like committing, pushing changes, and more, are available to you natively within workspaces launched in Git-based projects. This makes it easy for you to engage in version controlled, code-based collaboration with fellow project team members, all from within Domino. Git-based projects also organize your projects’ assets as either Code, Data, or Artifacts, an organizational structure intended to support common data science workflows.




Create a Git-based project

If you plan on using a private Git repository to store your code, then you’ll first need to add the corresponding Git credentials in your Domino account settings prior to creating your project. After adding credentials, you’ll be able to easily create a Git-based project in Domino, thereby enabling CodeSync experience.

If you plan on using a public Git repository to store your code, then you won’t need to add any Git credentials.

To create a Git-based project:

  1. Click on Projects in the Domino sidebar menu and then click the New Project button. A project creation modal will appear.

gbp-2

  1. Enter a name for your project.

  2. Set your project’s visibility.

  3. Under “Code Repository”, select “Git (beta)”.

  4. Under “Git Repository”, enter the URL of the Git repository that you’ll be using for your project.

Attention

If the repository you’re using to store your code contains one or more files exceeding 2 GB in size, then Domino will create your Git-based project but will not use the repository for your project. Please note that your Git service provider may also impose size limits on individual files.

You can use the following tool to check the total size of a Git repository, as well as the size of individual files within the repository: git sizer.

  1. Under “Git Credentials”, select the credentials associated with the Git repository your project will use. If the repository is public, then you won’t need to enter any credentials.

  2. Under “Git Service Provider”, select your Git service provider.

  3. Click Create Project.

gbp-3




Code, Data, & Artifacts

In Domino, the Domino File System (DFS) is the traditional way of storing a project’s assets. DFS-based projects organize all of your project’s assets as either Data or Files. Git-based projects, however, organize your project’s assets as either Code, Data, or Artifacts, and apply CodeSync experience to Code assets.

Code – This section of your Git-based project organizes and lists all of the Git-based repositories used to store your project’s code, as well as any additional imported repositories. For more information, see Git repositories in Domino. Files within any of these repositories can be accessed from within a Domino workspace via CodeSync technology.

The common Git workflows, like committing, pushing, pulling, and more, are available to you when interacting with your code from within a Domino workspace. For more information, see Using Git in your workspace.

Data – Similar to DFS projects, this section of your Git-based project organizes and lists all data sources used in your project, including Domino datasets, external data volumes, and dataset scratch spaces. For more information on how to use data with your project, please refer to the Domino datasets documentation.

Artifacts – Git-based projects introduce “Artifacts”. Artifacts are typically results or products from your research and analysis, like plots, charts, serialized models, and more. You can organize these outputs in this section, as well as import artifacts from other projects.

gbp-1




Running jobs

Warning

If you run a Job in a Git-based project, CodeSync ensures that only artifacts will be automatically synced and saved to the Domino File System (DFS). Code, on the other hand, will not be automatically synced / pushed to the Git repository being used for the Git-based project. This is intentional and intended to support the “Code”, “Data”, and “Artifacts” workflow. To learn more, see running jobs.




Working with artifacts in your workspace

Attention

All files in Artifacts are saved exclusively to the Domino File System (DFS). If you do not want to save a particular asset to the Domino File System, we recommend that you do not save it as an artifact. To learn more, see Syncing your work to Domino.

Artifacts are results from your research, like plots, charts, serialized models, and more. In Domino, you can save these results in the Artifacts section of your project.

Saving artifacts and pushing changes

  1. Click on the File Changes option in the sidebar of your workspace.

  2. Under “Artifacts”, view changes by expanding “File Changes”.

  3. Enter a commit message.

  4. Click Sync to Domino. Domino will save your artifacts to the Domino File System (DFS).

gbp-8

Pulling changes

To pull the latest artifacts (from the Domino File System) into your workspace:

  1. Click the File Changes option in the sidebar menu of your workspace.

  2. Under the “Artifacts” section, click Pull. Domino will pull the latest changes into your workspace.

gbp-9