Product Documentation

Appearance

All platforms

ybrelay Options ​

The ybrelay service can run without options; however, many options are available, as described in this section. This section also covers options available for the ybrelay-init script.

A simple start command would identify the work directory (-w or --workdir option):

$ ybrelay -w ./work
14:23:34.123 [ INFO] ABOUT CLIENT:
   app.cli_args         = "-w" "./work"
   app.name_and_version = "Yellowbrick Data Relay version 3.1.0-616"
   java.home            = "/usr/lib/jvm/java-11-oracle/jre"
   java.version         = "11"
   jvm.memory           = "491.00 MB (max=7.11 GB)"
   jvm.name_and_version = "Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)"
   jvm.options          = "-Xms512m, -Xmx8g, -ea, -Dapp.name=ybrelay, -Dapp.pid=11403, -Dapp.repo=/opt/ybtools/3.1.0.616/lib, -Dapp.home=/opt/ybtools/3.1.0.616, -Dbasedir=/opt/ybtools/3.1.0.616"
   jvm.vendor           = "Oracle Corporation"
   os.name_and_version  = "Linux 4.4.0-31-generic (amd64)"

14:23:34.125 [ INFO] Yellowbrick log level: DEBUG
...

Startup options may be specified in the ybrelay.options file.

Contents of the ybrelay.options File ​

The default ybrelay.options file, found in the /opt/ybtools/<version>/config directory, contains the following entries:

--workdir
/var/lib/ybrelay/work
--logdir
/var/log/ybrelay/logs
--log-level
WARN
--logfile
/var/log/ybrelay/ybrelay.log
--logfile-log-level
DEBUG
--logfile-backups
10
--logfile-size
10MB

The options in the file are defined in the following sections.

Common Options ​

The following ybrelay options are common to several ybtools clients.

@file

Specify a file that includes a set of ybrelay options and values to use. Options in the file must be specified in the following form (using separate lines for each option name and value):

option1
  value for option1
  option2
  value for option2
  ...
--cacert STRING

Customize trust with secured communication; use this option in combination with the --secured option. Enter the file name of a custom PEM-encoded certificate or the file name and password for a Java KeyStore (JKS).

For PEM format, the file must be named with a .pem, .cert, .cer, .crt, or .key extension. For example:

--cacert cacert.pem

For JKS format, files are always password-protected. Use the following format:

--cacert yellowbrick.jks:changeit

where the : character separates the file name from the password of the keystore.

See also Secure Connections for Java-based ybtools.

--disable-trust, -k

Disable SSL/TLS trust when using secured communications. Trust is enabled by default. See also Verifying SSL/TLS Encryption.

Important: This option is not supported for use on production systems and is only recommended for testing purposes. It may be useful to disable trust during testing, then enable it when a formal signed certificate is installed on the system.

-?, --help, --help-advanced

Return basic (or advanced) usage information for the ybrelay command and its options.

--java-version

Return the Java version that is running on the client system. The client tools require the 64-bit version of Java 8 (also known as Java 1.8). Java 9 and 10 are not supported.

--log-level

Specify the logging level for the default console output. The default level is WARN. (Use the --logfile-log-level option to specify the logging level for a named log file.)

Using the default WARN setting reduces the logs that ybrelay writes to stderr.

--logfile

Specify the name and location of a log file for ybrelay operations. If the specified file already exists, it will be truncated. If this option is not specified, no log file is written. When you specify this --logfile option, also specify a --logfile-log-level value other than OFF.

--logfile-log-level

Specify the logging level for a given log file (as defined with the --logfile option). If the level is not specified, it defaults to the --log-level value. You must specify a --logfile-log-level value other than OFF when you specify the --logfile option.

-q, --quiet

Do not write any output to the console. If --quiet is specified, you must also specify --logfile.

--secured

Use SSL/TLS to secure all communications. The default is not secured. See also Verifying SSL/TLS Encryption.

--version

Display the version of ybrelay you are running (as part of ybtools). This option is not intended to be combined with other options. For example:

$ ybrelay --version
Yellowbrick Data Relay version 5.1.0-33316

ybrelay Service Options ​

The following options are specific to the ybrelay service.

--address STRING, -a STRING

Listen address. Default: 0.0.0.0.

--connection-timeout-seconds NUMBER

Connection timeout (in seconds). Default: 120.

--idle-timeout NUMBER

Detect idle timeout for aborted loads (in minutes). Default: 5

--keystore FILENAME[:PASSWORD]

Specify the path to the Java key store that includes the TLS private key and certificate. For example:

--keystore mykeystore.jks:******
--log-retention-days NUMBER

Set the number of days to retain stored log files (in /var/log/ybrelay/logs or the specified logdir). Default: 14. This directory can grow large if a large number of ybload logs are generated. For example, if ybrelay is running a distinct ybload operation every minute, at least 2880 files are generated daily.

--logdir STRING, -l STRING

Specify a path where log data is created and managed. Default: /var/lib/ybrelay/work.

--logfile-backups NUMBER

Set the maximum number of log file backups. Default: 10. The ybrelay.log file is managed with the --logfile-backups and --logfile-size values, which set the rotation policies for this log file.

--logfile-size STRING

Set the maximum log file size. Default: 100MB.

--port NUMBER, -p NUMBER

Listen port. Default: 21212.

--queuesizebytes NUMBER, -s NUMBER

Set the queue size, in bytes (per load job). Default: 536870912.

--read-timeout-seconds NUMBER

Set the read timeout (in seconds). Default: 0.

--tasks NUMBER, -t NUMBER

Set the task count (for parallelism). Default: 4.

--workdir STRING, -w STRING

Set the path to create and manage temporary data. Default: /var/lib/ybrelay/work.

--write-timeout-seconds NUMBER

Set the write timeout (in seconds). Default: 0.

--ybload STRING, -y STRING

Set the path to the ybload client.

--ybload-args STRING

Set the default arguments to ybload. Default:

--no-trim-white --log-level WARN --logfile-log-level DEBUG
--ybload-vmargs STRING

Set VM arguments to ybload (if --ybload path is not used).

ybrelay-init Options ​

You can specify the following options when you run ybrelay-init:

--disable-trust

Use the ybrelay service --disable-trust option for the systemd service.

--force

Confirm that you want to overwrite an existing configuration of the ybrelay service. For example, you must use this option if you want to change a previous keystore configuration.

--help

Display help text for the ybrelay-init options.

--keystore

Enable use of the ybrelay service --keystore option for the systemd service. If you intend to use ybrelay in TLS mode, you must enable the --keystore option here, then use the --keystore option when you run the Spark job.

--prompt-password

Prompt the user for a password: for use in conjunction with the --keystore option. For example:

ybrelay-init --keystore ybkeystore.jks --prompt-password