sys.log_load

This system view logs information about completed bulk load (ybload) operations. The sys.load view contains the same information for ybload operations that are in progress.


Column Name	Data Type	Description
session_key	text	Unique authentication key for the `ybload` operation. This is different from the session ID, which is the ID tracked for the user session in the `sys.session` and `sys.log_session` views. (A unique query ID is not assigned until the bulk load transaction is executed.)
state	text	State of the load operation, such as `STARTING`, `DONE`, or `ERROR`.
error_string	text	Error message for a failed load.
table_id	bigint	The unique ID for the table. Tables in different schemas within the same database may have the same name, so you can use this ID to uniquely identify a table.
schema_id	bigint	Unique ID of the schema that contains the table being loaded.
database_id	bigint	Unique ID of the database that contains the table being loaded.
user_id	bigint	Unique ID of the user who ran the `ybload` command.
session_id	bigint	Session ID; foreign key to the `sys.session` and `sys.log_session` views.
table_name	text	Name of the table being loaded.
schema_name	text	Name of the schema that the table belongs to.
database_name	text	Name of the database that the table belongs to.
user_name	text	Name of the user who ran the `ybload` command.
client_hostname	text	Host name of the client system where the load was started.
client_username	text	Client OS user running the `ybload` command.
start_time	timestamptz	When the `ybload` command started running.
end_time	timestamptz	When the `ybload` command completed its execution.
elapsed_ms	decimal(18,3)	Duration of the load, in milliseconds.
remaining_ms	decimal(18,3)	Estimated number of milliseconds remaining to complete the load. (This value is always 0.0 for completed loads in this view.)
transaction_first	bigint	First transaction ID for the load. Bulk load operations may be split into multiple transactions.
transaction_last	bigint	Last transaction ID for the load. Bulk load operations may be split into multiple transactions.
num_transactions	bigint	Total number of transactions for the load operation.
parsed_rows	bigint	Number of rows that have been parsed.
parsed_bytes	bigint	Amount of table data read from the source file (in uncompressed bytes).
parsed_rows_per_second	double precision	Number of rows parsed per second.
parsed_bytes_per_second	double precision	Amount of table data read from the source file per second (in uncompressed bytes).
inserted_rows	bigint	The number of rows that were loaded successfully into the table.
inserted_bytes	bigint	Amount of table data written to the table (in compressed bytes). `inserted_bytes` may be greater than `parsed_bytes`, given the overhead of the size of a shard (unit of data storage).
inserted_rows_per_second	double precision	Number of rows that were loaded successfully per second.
inserted_bytes_per_second	double precision	Amount of table data written to the table per second, in bytes.
error_rows	bigint	The number of rows that could not be loaded into the table (see the `.bad` rows file for details).
sent_bytes	bigint	Number of bytes transferred over the network during the load.
sent_bytes_per_second	double precision	Number of bytes transferred per second during the load.

Examples

The following example shows information for a load of 26 million rows into the newmatchstats table. (Expanded ybsql display is used in this example for readability.)

premdb=# select * from sys.log_load where table_name = 'newmatchstats';
-[ RECORD 1 ]-------------+-----------------------------------------------------------------
session_key               | A0iOTdPuAXDBlGY_IJcXrFZXoxncCDlG1Spo5477TVa9P-Y9IE8k30FarQ9iCx1e
state                     | DONE
error_string              | [NULL]
table_id                  | 16496
schema_id                 | 2200
database_id               | 16404
user_id                   | 16415
session_id                | 18913
table_name                | newmatchstats
schema_name               | public
database_name             | premdb
user_name                 | bobr
client_hostname           | ybload100
client_username           | ybuser100
start_time                | 2018-03-26 16:42:09.233-07
end_time                  | 2018-03-26 16:42:16.012-07
elapsed_ms                | 6779.000
remaining_ms              | 0.000
transaction_first         | 179110
transaction_last          | 179110
num_transactions          | 1
parsed_rows               | 25990128
parsed_bytes              | 659797724
parsed_rows_per_second    | 3820717.46658389
parsed_bytes_per_second   | 96994546.8717621
inserted_rows             | 25990128
inserted_bytes            | 727723568
inserted_rows_per_second  | 3833917.66216401
inserted_bytes_per_second | 107349692.180362
error_rows                | 0
sent_bytes                | 727723568
sent_bytes_per_second     | 106980086.712245

premdb=# select count(*) from newmatchstats;
  count   
----------
 25990128
(1 row)

The following example returns the duration, in seconds, of the same load into the same table:

premdb=# select table_id, start_time, end_time, trunc(elapsed_ms/1000,2) as duration, inserted_rows, error_rows, inserted_bytes 
from sys.log_load where table_name='newmatchstats' and session_id=18913;
 table_id |         start_time         |          end_time          | duration | inserted_rows | error_rows | inserted_bytes 
----------+----------------------------+----------------------------+----------+---------------+------------+----------------
    16496 | 2018-03-26 16:42:09.233-07 | 2018-03-26 16:42:16.012-07 |     6.77 |      25990128 |          0 |      727723568
(1 row)