Product Documentation

Appearance

7.1.2
7.0.2
6.9.3
5.4.17
5.2.31

Setting Up an Unload ​

This section summarizes the steps required to unload data from a Yellowbrick database.

  1. Make sure the target Yellowbrick database is up and running.

  2. Determine the output destination and the format of the unloaded files.

  3. Run the ybunload command as a user with SELECT privileges on the tables referenced in the command.

You must define a few essential pieces of information in the command. You can also specify a range of other processing, formatting, and logging options, as needed.

ybunload [options] -t table --format -o destination

  • [options]: You will need to set some database connection values if they are not already set with environment variables. Other options depend on the nature of the data and related formatting requirements for the load.

  • table: the target table for the unload. Alternatively, you can unload the results of a query, using the -s (select) option instead of -t.

  • --format: csv (comma-delimited by default), text (tab-delimited by default), or parquet (binary)

    Note: Azure object storage only supports unloaded data in parquet format.

  • -o destination: A local file system where the client user has write permissions or a supported object storage system where the client user has credentials to upload files. (See Unloading Data to Object Storage. You can also unload data to stdout.

    You can specify a prefix for the output files so that they are named consistently and uniquely for a given unload.

    Compressed GZIP (.gz) output files are supported.

For more details, see ybunload Options, Common Options in ybtools, and ybunload Examples.

Tip: If you are regularly unloading data into the same directory location with the same prefix, use the --truncate-existing option to remove files from previous runs. Otherwise, the output directory may contain a mix of unload files from different runs, or the unload may fail with an error.

  1. Query the following system views to track unload progress and look at completed unload operations:

Parent topic:Unloading Data