ybload Field Options

The ybload command supports advanced options for parsing fields with different data types. This section covers these options, which you specify in JSON format.

There are three ways to specify field options:
  • On a per-field basis, naming specific fields either by ordinal position in the file or by destination column name in the table.
  • On a per-type basis, defining behavior for all of the fields that have a given data type.
  • Using default settings, when no per-field or per-type settings are defined.

Per-field options, if set, override per-type options. Per-field and per-type options, if set, override default field options.

Default Field Options (--default-field-options)

Use --default-field-options '{"nullmarker": STRING}' to specify field options for fields that do not have their own per-field options or per-type options.

Default: '{}'

Field Options per Data Type

For every data type that Yellowbrick supports, you can define parsing behavior for all fields of that type. These options are used in the absence of option settings for specific fields.

The integer types (SMALLINT, INTEGER, BIGINT) all support an inhex option. When you use this option for one or more fields, ybload parses the incoming data in those fields as hexadecimal numbers (instead of decimal values).

  • BIGINT: --bigint-field-options '{"nullmarker": STRING, "inhex": "TRUE" | "FALSE"}'

    Default: '{"inhex":false}'

  • BOOLEAN: --boolean-field-options '{"nullmarker": STRING}'

    Default: '{}'

  • CHAR and VARCHAR: --char-field-options '{"nullmarker": STRING, "emptymarker": STRING, "onZeroChar": REMOVE | ERROR, "convertAsciiControl": BOOLEAN, "convertCEscape": BOOLEAN}'

    Default: '{}'

  • DATE: --date-field-options '{"nullmarker": STRING, "y2base": NUMBER, "formats": [<DateFormat>...]}'

    Default: '{"formats":[{"style":"YMD","delim":"auto"},{"style":"DMONY","delim":"auto"},{"style":"MONDY","delim":"auto"}]}'

    See ybload Date Formats.

  • DECIMAL: --decimal-field-options '{"nullmarker": STRING}'

    Default: '{}'

  • DOUBLE: --double-field-options '{"nullmarker": STRING}'

    Default: '{}'

  • INTEGER: --integer-field-options '{"nullmarker": STRING, "inhex": "TRUE" | "FALSE"}'

    Default: '{"inhex":false}'

  • IPV4: --ipv4-field-options '{"nullmarker": STRING}'

    Default: '{}'

  • IPV6: --ipv6-field-options '{"nullmarker": STRING}'

    Default: '{}'

  • MACADDR: --macaddr-field-options '{"nullmarker": STRING}'

    Default: '{}'

  • MACADDR8: --macaddr8-field-options '{"nullmarker": STRING}'

    Default: '{}'

  • REAL: --real-field-options '{"nullmarker": STRING}'

    Default: '{}'

  • SMALLINT: --smallint-field-options '{"nullmarker": STRING, "inhex": "TRUE" | "FALSE"}'

    Default: '{"inhex":false}'

  • TIME: --time-field-options '{"nullmarker": STRING, "timeformats": [<TimeFormat>...]}'

    Default: '{"timeformats":[{"style":"HMSs","delim":"auto"},{"style":"HMS","delim":"auto"},{"style":"HM","delim":"auto"}]}'

    See ybload Time Formats.

  • TIMESTAMP WITHOUT TIME ZONE: '{"nullmarker": STRING, "y2base": NUMBER, "dateformats": [<DateFormat>...], "timeformats": [<TimeFormat>...]}'

    Unspecified <TimestampFieldOptions> fields are populated from the matching fields in --date-field-options.

    Default: '{"timeformats":[{"style":"HMSs","delim":"auto"},{"style":"HMS","delim":"auto"},{"style":"HM","delim":"auto"},{"style":"None","delim":"auto"}]}'

    See ybload Timestamp Formats and ybload Time Formats.

  • TIMESTAMPTZ: '{"nullmarker": STRING, "y2base": NUMBER, "dateformats": [<DateFormat>...], "timeformats": [<TimeFormat>...], "timezone": STRING}'

    Unspecified <TimestampTzFieldOptions> fields are populated from the matching fields in --timestamp-field-options.

    Default: '{}'

    See ybload Timestamp Formats and ybload Time Formats.

  • UUID: --uuid-field-options '{"nullmarker": STRING}'

    Default: '{}'

Options per Field (--per-field-options)

You can specify --per-field-options multiple times in a single ybload command. The JSON object keys are either source field numbers (1-based) or destination column names. The value for each key must be a <...FieldOptions> object that is appropriate for the data type of the field.

The following example sets DATE field options specifically for the salesdate field: 
--per-field-options {"salesdate":{"nullmarker":"--NODATE--","y2base":2000,"formats":[{"style":"MONDY2","delim":"_"}]}}
The following example accounts for field 5 in the source file: 
--per-field-options {"5":{"nullmarker":"?"}}
The following example applies to field 1, an integer field, and specifies that ybload should parse the field values as hexadecimal numbers. 
--per-field-options {"1": {"inhex": true}}