Product Documentation

Appearance

EE: All cloud platforms

Yellowbrick Operator ​

Yellowbrick Operator provides lifecycle for Yellowbrick instances, including managing their creation, destruction, suspend/resume operations, upgrades and configuration. The operator monitors changes to Custom Resource Definitions (CRDs) by provisioning and managing Kubernetes resources that support Yellowbrick instances.

Custom Resource Definitions ​

The following describes each of the custom resource definitions managed by Yellowbrick Operator.

NameCRDDescription
YBInstanceybinstances.cluster.yellowbrick.ioThe Yellowbrick instance specification and status
YBInstanceTaskybinstancetasks.cluster.yellowbrick.ioTasks that perform lifecycle operations on Yellowbrick instances
YBVersionybversions.cluster.yellowbrick.ioVersion uploaded by Yellowbrick Deployer, containing registry location and semver
YBNodeGroupybnodegroups.cluster.yellowbrick.ioThe hardware node group used by Kubernetes in a cloud provider to configure how to acquire and configure compute infrastructure
YBHwInstanceTypeybhwinstancetypes.cluster.yellowbrick.ioThe hardware instance types supported by Yellowbrick compute clusters and their configuration
YBSharedServiceybsharedservices.cluster.yellowbrick.ioThe parameters for scaling between different Yellowbrick instance shared service configurations
YBExternalInstanceybexternalinstances.cluster.yellowbrick.ioRegistration of an unmanaged Yellowbrick instance to be manageable by Yellowbrick Manager
YBStorageProvisionRequestybstorageprovisionrequests.cluster.yellowbrick.ioDescribes a storage provision request made for an object storage location on behalf of a Yellowbrick instance

Usage ​

Refer to Kubectl Administration to see some common commands you can use to manage Yellowbrick instances.

Use Cases ​

Listing Versions ​

% kubectl get -A ybversion

NAME                   AGE
7.0.1-58367.66d9e03e   2d
7.0.1-58423.ec33cf36   4d22h
7.0.1-58503.7c5c6d83   2d4h
7.0.1-58583.7dae3b48   24h

Creating an Instance ​

Using the following:

  • the target namespace for the instance $instanceNamespace
  • the name of the instance $instanceName
  • the version of the instance $version (see listing versions above)
% kubectl -n $instanceNamespace apply -f - <<-EOF
apiVersion: cluster.yellowbrick.io/v1
kind: YBInstance
metadata:
  name: $instanceName
spec:
  version: $version
  sharedServicesType: standard
  storageManaged: true
EOF

Adjust the .spec parameters according to your requirements.

Destroying an Instance ​

Normally, a custom resource can be removed with kubectl -n [namespace] delete [type] [name]; however Yellowbrick Operator prevents accidental deletion of an entire Yellowbrick instance through a finalizer and custom annotation which must be provided on the instance to proceed with deleting the object storage and catalog / rowstore volumes for the instance.

In order to annotate the Yellowbrick instance for deletion, use the following:

  • the target namespace for the instance $instanceNamespace
  • the name of the instance $instanceName
% id=$(kubectl -n $instanceNamespace get ybinstance $instanceName -o jsonpath='{.metadata.uid}' 2>/dev/null)
% kubectl -n $instanceNamespace patch ybinstance $instanceName -p "{\"metadata\":{\"annotations\":{\"yellowbrick.io/delete-confirmation\":\"$id\"}}}" --type=merge

This prepares the instance for deletion, which is then done with this command:

% kubectl -n $instanceNamespace delete ybinstance $instanceName

Obtain the Details for an Instance ​

Given:

  • the namespace for the instance $instanceNamespace
  • the name of the instance $instanceName
% kubectl get -n $instanceNamespace ybinstance $instanceName -o yaml
Example Output
apiVersion: cluster.yellowbrick.io/v1
kind: YBInstance
metadata:
  creationTimestamp: "2024-09-08T00:04:07Z"
  finalizers:
  - yellowbrick.io/delete-confirmation
  generation: 27
  name: kw-ybd-39497
  namespace: yellowbrick-operator
  resourceVersion: "1877140644"
  uid: b4990f68-78c1-4580-b25c-85ff9d41487c
spec:
  requestedState: Suspended
  sharedServicesType: standard
  storageManaged: true
  version: 7.0.1-58423.ec33cf36
status:
  currentRequestedState: Suspended
  currentSharedServicesType: standard
  currentState: Suspended
  currentVersion: 7.0.1-58423.ec33cf36
  dependencyStatus:
    YBStorageProvisionRequest-kw-ybd-39497-default:
      message: Provisioning completed
      provisionState: OK
      timestamp: "2024-09-08T00:04:08Z"
  instanceTaskInProgress:
    name: ""
    namespace: ""
  instanceTaskLast:
    name: kw-ybd-39497-suspend-8qgt9
    namespace: yellowbrick-operator
    operation: Suspend
  lastState: Suspending
  stateChangeTimestamp: "2024-09-11T10:48:48Z"
  storageLocations:
    default:
      endpoint: http://minio.ybinst.svc.cluster.local.:80
      path: yb-yellowbrick-operator-kw-ybd-39497-default-babb63-5a9b47
      region: us-east-1
      type: s3

Listing the Tasks of an Instance ​

Given:

  • the namespace for the instance $instanceNamespace
  • the name of the instance $instanceName
% k get -n $instanceNamespace get ybinstancetask |grep $instanceName

NAME                         INSTANCE NAMESPACE     INSTANCE NAME  OPERATION  STATE       AGE     COMPLETED              RESULT
kw-test-10-create-230f60b3   yellowbrick-operator   kw-test-10     Create     Completed   145d    2024-04-18T17:42:12Z   Success
kw-test-10-delete-230f60b3   yellowbrick-operator   kw-test-10     Delete     Completed   145d    2024-04-18T17:45:28Z   Success
kw-test-10-resume-crgph      yellowbrick-operator   kw-test-10     Resume     Completed   145d    2024-04-18T17:43:52Z   Success

CRD Reference ​

YBInstance ​

Custom Resource Definition
yaml
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.14.0
  name: ybinstances.cluster.yellowbrick.io
spec:
  group: cluster.yellowbrick.io
  names:
    kind: YBInstance
    listKind: YBInstanceList
    plural: ybinstances
    singular: ybinstance
  scope: Namespaced
  versions:
  - additionalPrinterColumns:
    - description: The Yellowbrick instance version
      jsonPath: .status.currentVersion
      name: Version
      type: string
    - description: The Yellowbrick instance shard services type
      jsonPath: .status.currentSharedServicesType
      name: Shared Services
      type: string
    - description: The age of the Yellowbrick instance
      jsonPath: .metadata.creationTimestamp
      name: Age
      type: date
    - description: The current state of the Yellowbrick instance
      jsonPath: .status.currentState
      name: State
      type: string
    - description: The current operation for the Yellowbrick instance
      jsonPath: .status.instanceTaskInProgress.operation
      name: Current Operation
      type: string
    - description: The last operation timestamp for the Yellowbrick instance
      jsonPath: .status.stateChangeTimestamp
      name: Changed
      type: date
    - description: The last operation for the Yellowbrick instance
      jsonPath: .status.instanceTaskLast.operation
      name: Last Operation
      type: string
    - description: Last error for the Yellowbrick instance, if any
      jsonPath: .status.error
      name: Error
      priority: 1
      type: string
    name: v1
    schema:
      openAPIV3Schema:
        properties:
          apiVersion:
            description: |-
              APIVersion defines the versioned schema of this representation of an object.
              Servers should convert recognized schemas to the latest internal value, and
              may reject unrecognized values.
              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
            type: string
          kind:
            description: |-
              Kind is a string value representing the REST resource this object represents.
              Servers may infer this from the endpoint the client submits requests to.
              Cannot be updated.
              In CamelCase.
              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
            type: string
          metadata:
            type: object
          spec:
            properties:
              allowlistIpCidrs:
                items:
                  type: string
                type: array
              featureFlags:
                additionalProperties:
                  type: string
                type: object
              networkType:
                type: string
              options:
                additionalProperties:
                  type: string
                type: object
              requestedState:
                enum:
                - Suspended
                - Running
                - Restart
                type: string
              serviceAnnotations:
                additionalProperties:
                  type: string
                description: Service Properties
                type: object
              sharedServicesType:
                type: string
              storageManaged:
                type: boolean
              values:
                type: string
              version:
                type: string
            required:
            - sharedServicesType
            - version
            type: object
          status:
            properties:
              currentAllowlistIpCidrs:
                items:
                  type: string
                type: array
              currentRequestedState:
                enum:
                - Suspended
                - Running
                - Restart
                type: string
              currentSharedServicesType:
                type: string
              currentState:
                enum:
                - Requested
                - Creating
                - Created
                - Suspending
                - Suspended
                - Running
                - Resuming
                - Deleting
                - Error
                type: string
              currentVersion:
                type: string
              dependencyStatus:
                additionalProperties:
                  properties:
                    message:
                      type: string
                    provisionState:
                      enum:
                      - Initial
                      - OK
                      - Error
                      type: string
                    timestamp:
                      type: string
                  type: object
                type: object
              detail:
                type: string
              error:
                type: string
              instanceIdOriginal:
                description: |-
                  UID is a type that holds unique ID values, including UUIDs.  Because we
                  don't ONLY use UUIDs, this is an alias to string.  Being a type captures
                  intent and helps make sure that UIDs and names do not get conflated.
                type: string
              instanceNameOriginal:
                type: string
              instanceTaskInProgress:
                properties:
                  name:
                    type: string
                  namespace:
                    type: string
                  operation:
                    description: YBInstanceTaskOperation defines the allowable operations
                      for a YBInstanceTask
                    enum:
                    - Create
                    - Delete
                    - Resume
                    - Suspend
                    - SetVersion
                    - Diagnostics
                    - ChangeSharedServices
                    - Migrate
                    type: string
                required:
                - name
                - namespace
                type: object
              instanceTaskLast:
                properties:
                  name:
                    type: string
                  namespace:
                    type: string
                  operation:
                    description: YBInstanceTaskOperation defines the allowable operations
                      for a YBInstanceTask
                    enum:
                    - Create
                    - Delete
                    - Resume
                    - Suspend
                    - SetVersion
                    - Diagnostics
                    - ChangeSharedServices
                    - Migrate
                    type: string
                required:
                - name
                - namespace
                type: object
              lastState:
                enum:
                - Requested
                - Creating
                - Created
                - Suspending
                - Suspended
                - Running
                - Resuming
                - Deleting
                - Error
                type: string
              stateChangeTimestamp:
                format: date-time
                type: string
              storageLocations:
                additionalProperties:
                  properties:
                    endpoint:
                      type: string
                    path:
                      description: |-
                        Ideally: path is of form similar to: ybexstor-{ namespace }-{ instanceName }-{ storageName }
                        Example:  ybexstor-yellowbrick-dw-yb-inst1-default
                        Where:
                           prefix:        ybexstor-
                           namespace:     yellowbrick-dw
                           instanceName:  yb-inst1
                           storageName:   default
                      type: string
                    region:
                      type: string
                    type:
                      type: string
                  required:
                  - endpoint
                  - path
                  - region
                  - type
                  type: object
                type: object
            required:
            - dependencyStatus
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}

YBInstanceTask ​

Custom Resource Definition
yaml
spec:
  conversion:
    strategy: None
  group: cluster.yellowbrick.io
  names:
    kind: YBInstanceTask
    listKind: YBInstanceTaskList
    plural: ybinstancetasks
    singular: ybinstancetask
  scope: Namespaced
  versions:
  - additionalPrinterColumns:
    - description: 'The namespace of the Yellowbrick instance '
      jsonPath: .spec.instanceNamespace
      name: Instance Namespace
      type: string
    - description: 'The name of the Yellowbrick instance '
      jsonPath: .spec.instanceName
      name: Instance Name
      type: string
    - description: The name of the Yellowbrick instance operation
      jsonPath: .spec.operation
      name: Operation
      type: string
    - description: The current state of the Yellowbrick instance operation
      jsonPath: .status.state
      name: State
      type: string
    - description: The submittted timestamp for the Yellowbrick instance operation
      jsonPath: .metadata.creationTimestamp
      name: Age
      type: date
    - description: The completion timestamp for the Yellowbrick instance operation
      jsonPath: .status.completionTimestamp
      name: Completed
      type: string
    - description: The result of the Yellowbrick instance operation
      jsonPath: .status.result
      name: Result
      type: string
    - description: The summary of the Yellowbrick instance operation
      jsonPath: .status.summary
      name: Summary
      priority: 1
      type: string
    name: v1
    schema:
      openAPIV3Schema:
        properties:
          apiVersion:
            description: |-
              APIVersion defines the versioned schema of this representation of an object.
              Servers should convert recognized schemas to the latest internal value, and
              may reject unrecognized values.
              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
            type: string
          kind:
            description: |-
              Kind is a string value representing the REST resource this object represents.
              Servers may infer this from the endpoint the client submits requests to.
              Cannot be updated.
              In CamelCase.
              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
            type: string
          metadata:
            type: object
          spec:
            description: YBInstanceTaskSpec defines the desired state of YBInstanceTask
            properties:
              cancel:
                type: boolean
              diagsInput:
                properties:
                  disableInfrastructureLogs:
                    type: boolean
                  enableWorkerDump:
                    type: boolean
                  endTime:
                    format: date-time
                    type: string
                  note:
                    type: string
                  startTime:
                    format: date-time
                    type: string
                type: object
              instanceName:
                type: string
              instanceNamespace:
                type: string
              migrateInput:
                properties:
                  disks:
                    additionalProperties:
                      type: object
                      x-kubernetes-preserve-unknown-fields: true
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  instanceId:
                    description: If migration succeeds, these will be recorded in
                      YBInstance.Spec.OriginalInstanceId and YBInstance.Spec.OriginalInstanceName
                    type: string
                  instanceName:
                    type: string
                required:
                - disks
                - instanceId
                - instanceName
                type: object
              operation:
                description: YBInstanceTaskOperation defines the allowable operations
                  for a YBInstanceTask
                enum:
                - Create
                - Delete
                - Resume
                - Suspend
                - SetVersion
                - Diagnostics
                - ChangeSharedServices
                - Migrate
                type: string
              options:
                additionalProperties:
                  type: string
                type: object
              timeout:
                type: string
            required:
            - instanceName
            - instanceNamespace
            - operation
            type: object
          status:
            properties:
              completionTimestamp:
                format: date-time
                type: string
              detail:
                type: string
              diagsLocation:
                properties:
                  bucket:
                    type: string
                  container:
                    type: string
                  path:
                    type: string
                type: object
              lastReportedTimestamp:
                format: date-time
                type: string
              progress:
                type: string
              result:
                enum:
                - Success
                - Error
                type: string
              results:
                additionalProperties:
                  type: string
                type: object
              state:
                enum:
                - Running
                - Error
                - Abandoned
                - Completed
                - Queued
                type: string
              summary:
                type: string
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}

YBVersion ​

Custom Resource Definition
yaml
spec:
  conversion:
    strategy: None
  group: cluster.yellowbrick.io
  names:
    kind: YBVersion
    listKind: YBVersionList
    plural: ybversions
    singular: ybversion
  scope: Namespaced
  versions:
  - name: v1
    schema:
      openAPIV3Schema:
        properties:
          apiVersion:
            description: |-
              APIVersion defines the versioned schema of this representation of an object.
              Servers should convert recognized schemas to the latest internal value, and
              may reject unrecognized values.
              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
            type: string
          kind:
            description: |-
              Kind is a string value representing the REST resource this object represents.
              Servers may infer this from the endpoint the client submits requests to.
              Cannot be updated.
              In CamelCase.
              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
            type: string
          metadata:
            type: object
          spec:
            properties:
              buildTimestamp:
                type: string
              chart:
                type: string
              registry:
                type: string
              tag:
                type: string
              values:
                type: string
            required:
            - chart
            - registry
            type: object
          status:
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}