Product Documentation

Appearance

All platforms

Monitoring Load Operations ​

You can monitor active bulk load operations in Yellowbrick Manager. The Load Assistant reports progress periodically, with a brief summary when the load is complete. See Loading a Table via the Load Assistant.

Regardless of the method you choose for loading tables, you can filter the rows under Query Activity and switch between Active and Historical views. Filter on ybulkload to see information about bulk load operations. For example:

Select a query ID, then click Details to see statistics and other information about a given load.

Note that, by default, rejected rows do not stop a load from running to completion. They are logged to a file (and its location is reported).

You can also run queries against the following system views:

For example, assume that user bobr runs the following load of 25 million rows into the newmatchstats table:

$ ybload -d premdb --username bobr -t newmatchstats -W --format csv /home/brumsby/newmatchstats25mil.csv
Password for user bobr: 
...
13:30:44.841 [ INFO] Flushing last 6818022 rows (of 24785280 total) in transaction #1 for newmatchstats
READ:600.1MB(94.58MB/s). ROWS G/B: 24785280/0( 3.73M/s). WRITE:661.8MB(104.3MB/s).  TIME E/R:   0:00:06/  0:00:0013:30:45.153 [ INFO] Committing 24785280 rows into transaction #1 for newmatchstats ...
13:30:46.535 [ INFO] Committed transaction #1 after a total of 693987840 bytes and 24785280 good rows for newmatchstats
13:30:46.560 [ INFO] READ:600.1MB(77.65MB/s). ROWS G/B: 24785280/0( 3.06M/s). WRITE:661.8MB(85.64MB/s).  TIME E/R:   0:00:07/  0:00:00
13:30:46.561 [ INFO] SUCCESSFUL BULK LOAD: Loaded 24785280 good rows in   0:00:07 (READ: 77.65MB/s WRITE: 85.64MB/s)

Using the transaction ID (transaction_first=slq.transaction_id), the following join of sys.log_load and sys.log_query returns detailed information about that load:

premdb=# select sll.table_name, sll.username, sll.end_time, sll.transaction_first, sll.inserted_rows, substr(slq.query_text,1,50) command
from sys.log_load sll join sys.log_query slq on sll.transaction_first=slq.transaction_id 
where sll.username='bobr' and sll.table_name='newmatchstats' 
order by 4 desc;
  table_name   | username |          end_time          | transaction_first | inserted_rows |                      command                       
---------------+----------+----------------------------+-------------------+---------------+----------------------------------------------------
 newmatchstats | bobr     | 2020-10-30 13:30:46.579-07 |             96045 |      24785280 | YBULKLOAD INTO "premdb"."public"."newmatchstats" (
(1 row)

See also Delete, Update, and Upsert Examples.

Note that the application_name for a ybload operation, available in the sys.query and sys.log_query views, is ybload <version>. For example:

ybload 6.0.0-a8363f32.1792