Product DocumentationAppearance
S3 Object Storage URIs
The ybload client supports several patterns for Amazon S3 and S3-compatible URIs. All of these patterns identify the location of the source data for a bulk load operation.
| URI Pattern | Loads | Example |
|---|---|---|
s3://BUCKET | All keys (files) in the bucket | s3://premdb |
s3://BUCKET/KEY | Current version of the named key | s3://premdb/team01.csv |
s3://BUCKET/KEY?versionId=VERSION | Specific version of the named key | s3://premdb/team01.csv?versionId=w_B5qT8s5MkiT09.IRHay0lW.PycsHTS |
s3://BUCKET/KEY_PREFIX | All keys that match the prefix | s3://premdb/team |
You can specify multiple URIs on the ybload command line if you want to identify multiple source files explicitly. However, the sources must belong to the same S3 endpoint.
A valid URI must not contain a space character unless it is encoded with the %20 escape sequence. For example, if you upload a season.csv source file into ybbobr/premdb/new files, the following URI will work:
s3://ybbobr/premdb/new%20files/season.csvNote: The following "path-based" URI patterns are not supported:
http://SUB.s3.DOMAIN/BUCKET/KEY (or https)
http://s3.DOMAIN/BUCKET/KEY (or https)URI Parameters
As an alternative to specifying ybload --object-store options, you can specify the S3 configuration in the form of URI parameters. To use a URI parameter, drop the --object-store- prefix from the ybload option name. For example, the --object-store-endpoint option is specified in the following URI as endpoint=:
s3://my_bucket/my_key?endpoint=http://nycloadsrv1.nyc.yellowbrick.io:9000Note: If you provide multiple source URIs in a single ybload command, only the first URI will be checked for configuration options.
Parent topic:Loading from Amazon S3