Boundaries of Competence: Shared-data Clusters
Each of the features below lists the version number that they were added in. If you are deploying a new cluster please deploy the latest patch release of version 3.2 or higher.
Overview
Shared-data StarRocks clusters feature a disaggregated storage and compute architecture. This allows data to be stored in remote storage, leading to lower storage costs, optimized resource isolation, and better service elasticity compared to a shared-nothing cluster.
This document outlines the boundaries of competence for shared-data clusters, covering deployment methods, storage configurations, caching mechanisms, Compaction, Primary Key table functionalities, and performance test results.
Deployment
Shared-data clusters support deployments on physical/virtual machines and on Kubernetes via Operator.
Both deployment solutions have the following limitations:
- Mixed deployment of shared-nothing and shared-data mode is not supported.
- The transformation from a shared-nothing cluster to a shared-data cluster or vice versa is not supported.
- Heterogeneous deployments are not supported, meaning the hardware specifications of all CN nodes within a cluster must be the same.
StarRocks Kubernetes Operator
StarRocks offers the StarRocks Kubernetes Operator for shared-data deployment on Kubernetes.
You can scale shared-data clusters by the following methods:
- Manual operations.
- Automatic scaling using Kubernetes HPA (Horizontal Pod Autoscaler) strategies.
Storage
Shared-data clusters support building storage volumes on HDFS and object storage.
HDFS
Location
StarRocks supports the following locations for HDFS storage volume:
-
HDFS:
hdfs://<host>:<port>/
NOTE
From v3.2, storage volumes support HDFS clusters with the NameNode HA mode enabled.
-
WebHDFS (Supported from v3.2):
webhdfs://<host>:<http_port>/
-
ViewFS (Supported from v3.2):
viewfs://<ViewFS_cluster>/
Authentication
StarRocks supports the following authentication methods for HDFS storage volume:
-
Basic
-
Username (Supported from v3.2)
-
Kerberos Ticket Cache (Supported from v3.2)
NOTE
StarRocks does not support automatic ticket refresh. You need to set up crontab tasks to refresh the ticket.
Authentication using Kerberos Keytab and Principal ID is not yet supported.