Reserved Node Capacity ​

The Reserved field in the Create Data Warehouse Instance screen is a means of reserving and spinning up the specified number of compute nodes when an instance is created, as opposed to waiting for nodes to spin up when compute clusters are created and started. Reserved nodes are shared among all the compute clusters that are created for a single Yellowbrick instance, and they are immediately available for use when those clusters start running and doing work. (Reserved nodes that are not part of a running cluster remain available but are not billed.)

The Limit field sets the maximum node capacity for the instance, while the Reserved field sets the reserved capacity within that limit. The limit per instance must be greater than or equal to the number of reserved nodes.

This approach to reserving nodes is driven by K8S auto-scaling, which causes AWS to spin up nodes during instance creation, before any clusters have been created for the instance. The reserved nodes feature is independent from, but complementary to, the creation of on-demand capacity reservations (ODCRs) in AWS.

There is no direct integration between these two features, but you can use Yellowbrick reserved nodes or ODCRs (or both) to expedite the availability of compute resources. However, note that ODCRs belong to an AWS account and have a global application. This means that they are available to all Yellowbrick data warehouse instances as well as associated Yellowbrick services that run in their own EC2 instances (shared services, bulk load servers, and so on). Also, ODCRs are available (in theory) to non-Yellowbrick software that may be driven from the same AWS account. The presence of an ODCR in AWS facilitates the provisioning of reserved nodes via K8S auto-scaling. Nodes reserved explicitly for a Yellowbrick instance in Yellowbrick Manager apply only to that instance and only to the nodes created for compute clusters.

Availability of hardware resources in AWS may never be guaranteed, but ODCRs and reserved nodes both make compute resources more immediately available. In order to decide how many reserved nodes or ODCRs your Yellowbrick deployment is going to benefit from in practice, consider the following questions:

• How many Yellowbrick data warehouse instances are you going to create?
• How many compute clusters will be created for each instance, and what instance types and number of nodes will you allocate?
• How dynamic is the behavior of your cloud operations in Yellowbrick? Will clusters be routinely suspended and resumed, or expanded and shrunken? How variable is the general use of compute clusters going to be? In other words, how many compute nodes need to be readily available at any given time?
• What will the additional cost overhead be if you create ODCRs?

Note: ODCRs are of significant value as a boot-strapping mechanism during initial Yellowbrick installation. You can use ODCRs to make sure that sufficient hardware resources are available for creating the software stack and running all of its services. Once Yellowbrick is installed, you may to decide to drop the ODCRs and rely on reserved nodes, or you may wish to use the two mechanisms in a complementary way.

Parent topic:Data Warehouse Instances