Skip to content

log

Return log file entries that were recorded in the past 1 to 24 hours or tail one of the logs. You can return logs specific to a single compute blade, the database, the keystore, or the local manager node.

log blade [ 1-24 | all | tail ] [ <chassis>/<bay> | <uuid> | all ] [ <keyword> | "<search_string>" ]
log database [ 1-24 | all | tail ] [ <keyword> | "<search_string>" ]
log install [ <keyword> | "<search_string>" ]
log keystore [ 1-24 | all | tail ] [ <keyword> | "<search_string>" ]
log manager [ 1-24 | all | tail ] [ <keyword> | "<search_string>" ]
log ybcli [ 1-24 | all | tail ] [ <keyword> | "<search_string>" ]

Note: In all cases, 1-24 refers to a specific hour from 1 to 24, where 1 is the last hour and 24 is the last 24 hours. For <chassis>/<bay>, specify a chassis number and a bay number. For example: 2/1, 1/15 (without spaces).

All of the log commands have a search feature. You can append a keyword or a text string to the end of any log command, and only the log lines that contain the keyword or string are returned. A single keyword does not require quotes, but a string with multiple words does require quotes. The search is not case-sensitive. A single-keyword search will find words with a matching stem. See Log Search Examples.

blade

Return the log for a specific blade or all blades. For a specific blade, use its chassis and bay number or its UUID. (Use the status blade command to get the UUID.) For example:

YBCLI (PRIMARY)> log blade tail 0/1

Press ctrl-c to stop tailing log....
2018-07-10T17:53:42.042404-07:00 ybos-00000000-0000-0000-0000-38B8EBD00C1C YBDB[25486]:  23465685 Warn           kernel.Work: waiting for resource RDMA Req: Send Req O on proc 28
2018-07-10T17:53:42.042423-07:00 ybos-00000000-0000-0000-0000-38B8EBD00C1C YBDB[25486]:  23465685 Warn           kernel.Work: Cleaned up resource Timeout trackable on proc 21
2018-07-10T17:53:42.042434-07:00 ybos-00000000-0000-0000-0000-38B8EBD00C1C YBDB[25486]:  23465685 Warn           kernel.Work: waiting for resource RDMA Req: Send Req O on proc 22
...

To return logs for all blades (for all hours), use the following command:

log blade all all

You can also specify a number of hours for all blades. For example:

log blade 1 all
database

Return the log for the database. This command must be run on the primary manager node.

YBCLI (PRIMARY)> log database tail

Press ctrl-c to stop tailing log....
	LEFT OUTER JOIN sys."vt_flush_tables" f
	  ON f."tableId" = c.oid
	 AND f."databaseName" = current_database()
	 AND NOT f."expired"
	LEFT OUTER JOIN pg_catalog.pg_stat_all_tables t
	  ON t."relid" = c."oid"
	WHERE c."relkind" = 'r'
	;
2017-03-14 15:05:03.788 PDT 26246 FATAL:  password authentication failed for user "ybd"
2017-03-14 15:05:03.788 PDT 26246 DETAIL:  Connection matched pg_hba.conf line 96: "host    all             all             0.0.0.0/0               md5"
...
install

Return the log for when the installer was last run on the system. For example:

YBCLI(31444) (PRIMARY - yb100-mgr1)> log install

The following output is the contents of the log when the Yellowbrick installer was last run on this system.

Yellowbrick Data appliance installer v2.0.0
Installing: v0.0.1-11160
Installing using the following parameters: {'core': False, 'fieldreset': False, 'force': False, 'rpmreinstall': False, 'ignoreoff': False, 'factory': True, 'nobackup': False, 'yy': True, 'rpmonly': False, 'rpmdisable': False, 'y': False, 'backup_dir': None, 'reinstall': False, 'fixperms': False}

Blackout tenant  : YB
Blackout cluster : yb00
Blackout duration: 3600
stderr filename  : /root/YB-yb00-create-blackout-stderr.out

Blackout UUID    : 2d46bf92-d500-4e23-845e-1b4e1d6ff5fc

Starting backing up existing install
Backup location: /mnt/ybdata/backup/backup-2019_02_22-0534-59/
Backup complete
Installing YBD appliance version: 0.0.1-11160-release...
...
...
System upgrade complete
Install complete

End of the install log.
keystore

Return the keystore/encryption audit log (on systems with encrypted drives). For example:

YBCLI (PRIMARY)> log keystore tail

Press ctrl-c to stop tailing log....
2017-10-25T13:59:34.415317-07:00 yb00-mgr1 journal: YBCLI user ybdadmin (10.20.12.24): keystore: Executing -> keystore init
2017-10-25T13:59:34.419245-07:00 yb00-mgr1 journal: YBCLI user ybdadmin (10.20.12.24): keystore: Requires authentication
2017-10-25T13:59:57.254574-07:00 yb00-mgr1 journal: YBCLI user ybdadmin (10.20.12.24): keystore: Executing -> keystore init
2017-10-25T13:59:57.258328-07:00 yb00-mgr1 journal: YBCLI user ybdadmin (10.20.12.24): keystore: Requires authentication
2017-10-25T14:00:37.265908-07:00 yb00-mgr1 journal: YBCLI user ybdadmin (10.20.12.24): keystore: Executing -> keystore init
2017-10-25T14:00:37.270577-07:00 yb00-mgr1 journal: YBCLI user ybdadmin (10.20.12.24): keystore: Requires authentication
2017-10-25T14:04:32.013941-07:00 yb00-mgr1 journal: YBCLI user ybdadmin (10.20.12.24): keystore: Executing -> keystore init
2017-10-25T14:04:32.016892-07:00 yb00-mgr1 journal: YBCLI user ybdadmin (10.20.12.24): keystore: Requires authentication
2017-10-25T14:04:37.550317-07:00 yb00-mgr1 journal: YBCLI user ybdadmin (10.20.12.24): Keystore: Executing -> keystore unlock
2017-10-25T14:04:37.552872-07:00 yb00-mgr1 journal: YBCLI user ybdadmin (10.20.12.24): Keystore: Already unlocked
...
manager

Return or tail the log for the local manager node. For example:

YBCLI (SECONDARY)> log manager all

2017-10-31T13:01:01.639817-07:00 yb00-mgr1 rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="1383" x-info="http://www.rsyslog.com"] rsyslogd was HUPed
2017-10-31T13:01:01.973358-07:00 yb00-mgr1 crmd[3756]:  notice: Operation ybdkeystore_stop_0: ok (node=yb00-mgr1.yellowbrick.io, call=1421, rc=0, cib-update=3539, confirmed=true)
2017-10-31T13:01:01.974923-07:00 yb00-mgr1 crmd[3756]:  notice: Initiating action 122: start ybdkeystore_start_0 on yb00-mgr1.yellowbrick.io (local)
2017-10-31T13:01:01.975659-07:00 yb00-mgr1 systemd: Reloading.
...
YBCLI (SECONDARY)> log manager tail

Press ctrl-c to stop tailing log....
2017-11-27T19:27:54.168847+01:00 yb00-mgr1 crmd[3725]:  notice: Operation ybdfs_notify_0: ok (node=yb00-mgr1.yellowbrick.io, call=5315, rc=0, cib-update=0, confirmed=true)
2017-11-27T19:27:54.225340+01:00 yb00-mgr1 crmd[3725]:   error: pcmkRegisterNode: Triggered assert at xml.c:594 : node->type == XML_ELEMENT_NODE
2017-11-27T19:27:54.569690+01:00 yb00-mgr1 ntpd[7901]: Deleting interface #35 BLADE_bond, 192.168.10.1#123, interface stats: received=0, sent=0, dropped=0, active_time=24477 secs
2017-11-27T19:27:54.729028+01:00 yb00-mgr1 systemd: Stopped YellowBrick Vault.
...
ybcli

Return the ybcli command history. For example:

YBCLI (PRIMARY - yb100-mgr0)> log ybcli tail
Press ctrl-c to stop tailing log....
2018-12-05 05:35:38 :  INFO : Executing -> system appliance version
2018-12-05 05:35:38 :  INFO : Completed -> system appliance version
2018-12-05 05:35:43 :  INFO : Executing -> health cmp all
2018-12-05 05:35:45 :  INFO : Completed -> health cmp all
2018-12-05 11:44:38 :  INFO : YBCLI 2.0.0-9737 started
2018-12-05 11:44:47 :  INFO : Executing -> help log
2018-12-05 11:45:41 :  INFO : Executing -> log ybcli tail
...

Log Search Examples

The following examples show how you can search through the logs by specifying a single keyword or a longer search string. The first example searches the manager logs for the word installed. Quotes are not required:

YBCLI(16789) (PRIMARY - yb00-mgr0)> log manager 24 installed

2019-04-16T13:41:34.155927-07:00 yb00-mgr0 installer2[10836]: Current version installed: v0.0.1-12607
2019-04-16T14:41:37.304598-07:00 yb00-mgr0 installer2[10894]: Current version installed: v0.0.1-12607
2019-04-16T14:44:41.598975-07:00 yb00-mgr0 installer2[10894]: New YBD appliance package installed. Continuing

The second example searches the manager logs for the string Current version installed. Quotes are required:

YBCLI(16789) (PRIMARY - yb00-mgr0)> log manager 24 "Current version installed"

2019-04-16T13:41:34.155927-07:00 yb00-mgr0 installer2[10836]: Current version installed: v0.0.1-12607
2019-04-16T14:41:37.304598-07:00 yb00-mgr0 installer2[10894]: Current version installed: v0.0.1-12607

The third example searches the database logs for the string System not ready:

YBCLI(16789) (PRIMARY - yb00-mgr0)> log database all "System not ready"

2019-04-16 14:50:59.330 PDT 7581 0 WM002 WARNING:  System not ready
2019-04-16 14:50:59.601 PDT 7600 0 WM002 WARNING:  System not ready
2019-04-16 14:51:00.790 PDT 7869 0 WM002 WARNING:  System not ready
...

The fourth example searches the logs for a specific blade and returns all entries that contain fail, Fail, and words that begin with fail or Fail:

YBCLI(16789) (PRIMARY - yb00-mgr0)> log blade 24 1/1 fail

Chassis:  1
-----------
2019-04-16T14:44:42.960413-07:00 ybos-00000000-0000-0000-0000-38B8EBD0061D YBDB[4423]:         1 Inform    kernel.LoggingDumper: Cluster Status: DCS: Failed to register with the cluster manager :DCS. Retrying... KE023|Missing registerWorker() response
2019-04-16T14:44:42.960413-07:00 ybos-00000000-0000-0000-0000-38B8EBD0061D YBDB[4423]:         1 Inform    kernel.LoggingDumper: Cluster Status: DCS: Failed to register with the cluster manager :DCS. Retrying... KE023|Missing registerWorker() response
...
2019-04-16T14:47:52.951151-07:00 ybos-00000000-0000-0000-0000-38B8EBD0061D kernel:  [    0.699306] pci 0000:01:00.0: BAR 6: failed to assign [mem size 0x00100000 pref]
2019-04-16T14:47:52.951151-07:00 ybos-00000000-0000-0000-0000-38B8EBD0061D kernel:  [    0.699402] pci 0000:ff:12.0: BAR 2: failed to assign [mem size 0x00000040]
...
2019-04-16T14:47:52.951730-07:00 ybos-00000000-0000-0000-0000-38B8EBD0061D kernel:  [    9.117878] mgmt_bond: Setting fail_over_mac to active (1)
2019-04-16T14:47:52.951730-07:00 ybos-00000000-0000-0000-0000-38B8EBD0061D kernel:  [    9.176539] psmouse serio1: Failed to reset mouse on isa0060/serio1
Press any key for next section. 'q' to stop 
...

Parent topic:ybcli Reference