Read/Write to an object store

The API supports object store type datasources (such as S3) and allows for easy retrieval and upload of objects.

Note
 The following APIs are only available when using this type of datasource.

List

Get the datasource from the client:

 from domino.data_sources import DataSourceClient

s3_dev = DataSourceClient().get_datasource("s3-dev")
 library(DominoDataR)
client <- DominoDataR::datasource_client()

You can list objects available in the datasource. You can also specify a prefix:

 objects = s3_dev.list_objects()

objects_under_path = s3_dev.list_objects("path_prefix")
 keys <- DominoDataR::list_keys(client, "s3-dev")

keys_under_path <- DominoDataR::list_keys(client, "s3-dev", "path_prefix")

By default the number of returned objects is limited by the underlying datasource. You can specify how many keys you want as an optional parameter:

 objects = s3_dev.list_objects(page_size = 1500)
 keys = DominoDataR::list_keys(client, "s3-dev", page_size = 1500)

Read

You can get object content, without having to create object entities, by using the datasource API and specifying the Object key name:

 # Get content as binary
content = s3_dev.get("key")

# Download content to file
s3_dev.download_file("key", "./path/to/local/file")

# Download content to file-like object
f = io.BytesIO()
s3_dev.download_fileobj("key", f)
 # Get content as raw vector
content <- DominoDataR::get_object(client, "s3-dev", "key")

# Get content as character vector
content <- DominoDataR::get_object(client, "s3-dev", "key", as = "text")

# Download content to file
DominoDataR::save_object(client, "s3-dev", "key", "./path/to/local/file")

You can also get the datasource entity content from an object entity (Python only):

 # Key object
my_key = s3_dev.Object("key")

# Get content as binary
content = my_key.get()

# Download content to file
my_key.download_file("./path/to/local/file")

# Download content to file-like object
f = io.BytesIO()
my_key.download_fileobj(f)

Write

Similar to the read/get APIs, you can also write data to a specific object key. From the datasource:

 # Put binary content to given object key
s3_dev.put("key", b"content")

# Upload file content to specified object key
s3_dev.upload_file("key", "./path/to/local/file")

# Upload file-like content to specified object key
f = io.BytesIO(b"content")
s3_dev.upload_fileobj("key", f)
 # Put raw or character vector to object key
DominoDataR::put_object(client, "s3-dev", "key", what = "content")

# Upload file content to specified object key
DominoDataR::upload_object(client, "s3-dev", "key", file = "./path/to/local/file")

You can also write from the object entity (Python only).

 # Key object
my_key = s3_dev.Object("key")

# Put content as binary
my_key.put(b"content")

# Upload content from file
my_key.upload_file("./path/to/local/file")

# Upload content from file-like object
f = io.BytesIO()
my_key.upload_fileobj(f)