Deploy StarRocks manually
This topic describes how to manually deploy StarRocks. For other modes of installation, see Deployment Overview.
The preparations for manual deployment are outlined in the Deployment prerequisites and Check environment configurations documents. Please start there if you are planning a production deployment. If you are getting started with StarRocks and would like to follow one of the Quick Starts, please refer to Quick Starts.
Step 1: Start the Leader FE node
The following procedures are performed on an FE instance.
-
Create a dedicated directory for metadata storage. We recommend storing metadata in a separate directory from the FE deployment files. Make sure that this directory exists and that you have write access to it.
# Replace <meta_dir> with the metadata directory you want to create.
mkdir -p <meta_dir> -
Navigate to the directory that stores the StarRocks FE deployment files you prepared earlier, and modify the FE configuration file fe/conf/fe.conf.
a. Specify the metadata directory in the configuration item
meta_dir
.# Replace <meta_dir> with the metadata directory you have created.
meta_dir = <meta_dir>b. If any of the FE ports mentioned in the Environment Configuration Checklist are occupied, you must assign valid alternatives in the FE configuration file.
http_port = aaaa # Default: 8030
rpc_port = bbbb # Default: 9020
query_port = cccc # Default: 9030
edit_log_port = dddd # Default: 9010CAUTION
If you want to deploy multiple FE nodes in a cluster, you must assign the same
http_port
to each FE node.c. If you want to enable IP address access for your cluster, you must add the configuration item
priority_networks
in the configuration file and assign a dedicated IP address (in the CIDR format) to the FE node. You can ignore this configuration item if you want to enable FQDN access for your cluster.priority_networks = x.x.x.x/x
NOTE
You can run
ifconfig
in your terminal to view the IP address(es) owned by the instance.d. If you have multiple JDKs installed on the instance, and you want to use a specific JDK that is different from the one specified in the environment variable
JAVA_HOME
, you must specify the path where the chosen JDK is installed by adding the configuration itemJAVA_HOME
in the configuration file.# Replace <path_to_JDK> with the path where the chosen JDK is installed.
JAVA_HOME = <path_to_JDK>f. For information about advanced configuration items, see Parameter Configuration - FE configuration items.
-
Start the FE node.
-
To enable IP address access for your cluster, run the following command to start the FE node:
./fe/bin/start_fe.sh --daemon
-
To enable FQDN access for your cluster, run the following command to start the FE node:
./fe/bin/start_fe.sh --host_type FQDN --daemon
Note that you only need to specify the parameter
--host_type
ONCE when you start the node for the first time.CAUTION
Before starting the FE node with FQDN access enabled, make sure you have assigned hostnames for all instances in /etc/hosts. See Environment Configuration Checklist - Hostnames for more information.
-
-
Check the FE logs to verify if the FE node is started successfully.
cat fe/log/fe.log | grep thrift
A record of log like
2022-08-10 16:12:29,911 INFO (UNKNOWN x.x.x.x_9010_1660119137253(-1)|1) [FeServer.start():52] thrift server started with port 9020.
suggests that the FE node is started properly.
Step 2: Start the BE service
The following procedures are performed on the BE instances.
-
Create a dedicated directory for data storage. We recommend storing data in a separate directory from the BE deployment directory. Make sure that this directory exists and you have write access to it.
# Replace <storage_root_path> with the data storage directory you want to create.
mkdir -p <storage_root_path> -
Navigate to the directory that stores the StarRocks BE deployment files you prepared earlier, and modify the BE configuration file be/conf/be.conf.
a. Specify the data directory in the configuration item
storage_root_path
.# Replace <storage_root_path> with the data directory you have created.
storage_root_path = <storage_root_path>b. If any of the BE ports mentioned in the Environment Configuration Checklist are occupied, you must assign valid alternatives in the BE configuration file.
be_port = vvvv # Default: 9060
webserver_port = xxxx # Default: 8040
heartbeat_service_port = yyyy # Default: 9050
brpc_port = zzzz # Default: 8060c. If you want to enable IP address access for your cluster, you must add the configuration item
priority_networks
in the configuration file and assign a dedicated IP address (in the CIDR format) to the BE node. You can ignore this configuration item if you want to enable FQDN access for your cluster.priority_networks = x.x.x.x/x
NOTE
You can run
ifconfig
in your terminal to view the IP address(es) owned by the instance.d. If you have multiple JDKs installed on the instance, and you want to use a specific JDK that is different from the one specified in the environment variable
JAVA_HOME
, you must specify the path where the chosen JDK is installed by adding the configuration itemJAVA_HOME
in the configuration file.# Replace <path_to_JDK> with the path where the chosen JDK is installed.
JAVA_HOME = <path_to_JDK>For information about advanced configuration items, see Parameter Configuration - BE configuration items.
-
Start the BE node.
./be/bin/start_be.sh --daemon
CAUTION
- Before starting the BE node with FQDN access enabled, make sure you have assigned hostnames for all instances in /etc/hosts. See Environment Configuration Checklist - Hostnames for more information.
- You do not need to specify the parameter
--host_type
when you start BE nodes.
-
Check the BE logs to verify if the BE node is started successfully.
cat be/log/be.INFO | grep heartbeat
A record of log like "I0810 16:18:44.487284 3310141 task_worker_pool.cpp:1387] Waiting to receive first heartbeat from frontend" suggests that the BE node is started properly.
-
You can start new BE nodes by repeating the above procedures on other BE instances.
NOTE
A high-availability cluster of BEs is automatically formed when at least three BE nodes are deployed and added to a StarRocks cluster.
Step 3: (Optional) Start the CN service
A Compute Node (CN) is a stateless computing service that does not maintain data itself. You can optionally add CN nodes to your cluster to provide extra computing resources for queries. You can deploy CN nodes with the BE deployment files. Compute Nodes are supported since v2.4.
-
Navigate to the directory that stores the StarRocks BE deployment files you prepared earlier, and modify the CN configuration file be/conf/cn.conf.
a. If any of the CN ports mentioned in the Environment Configuration Checklist are occupied, you must assign valid alternatives in the CN configuration file.
thrift_port = vvvv # Default: 9060
webserver_port = xxxx # Default: 8040
heartbeat_service_port = yyyy # Default: 9050
brpc_port = zzzz # Default: 8060b. If you want to enable IP address access for your cluster, you must add the configuration item
priority_networks
in the configuration file and assign a dedicated IP address (in the CIDR format) to the CN node. You can ignore this configuration item if you want to enable FQDN access for your cluster.priority_networks = x.x.x.x/x
NOTE
You can run
ifconfig
in your terminal to view the IP address(es) owned by the instance.c. If you have multiple JDKs installed on the instance, and you want to use a specific JDK that is different from the one specified in the environment variable
JAVA_HOME
, you must specify the path where the chosen JDK is installed by adding the configuration itemJAVA_HOME
in the configuration file.# Replace <path_to_JDK> with the path where the chosen JDK is installed.
JAVA_HOME = <path_to_JDK>For information about advanced configuration items, see Parameter Configuration - BE configuration items because most of CN's parameters are inherited from BE.
-
Start the CN node.
./be/bin/start_cn.sh --daemon
CAUTION
- Before starting the CN node with FQDN access enabled, make sure you have assigned hostnames for all instances in /etc/hosts. See Environment Configuration Checklist - Hostnames for more information.
- You do not need to specify the parameter
--host_type
when you start CN nodes.
-
Check the CN logs to verify if the CN node is started successfully.
cat be/log/cn.INFO | grep heartbeat
A record of log like "I0313 15:03:45.820030 412450 thrift_server.cpp:375] heartbeat has started listening port on 9050" suggests that the CN node is started properly.
-
You can start new CN nodes by repeating the above procedures on other instances.
Step 4: Set up the cluster
After all FE, BE nodes, and CN nodes are started properly, you can set up the StarRocks cluster.
The following procedures are performed on a MySQL client. You must have MySQL client 5.5.0 or later installed.
-
Connect to StarRocks via your MySQL client. You need to log in with the initial account
root
, and the password is empty by default.# Replace <fe_address> with the IP address (priority_networks) or FQDN
# of the Leader FE node, and replace <query_port> (Default: 9030)
# with the query_port you specified in fe.conf.
mysql -h <fe_address> -P<query_port> -uroot -
Check the status of the Leader FE node by executing the following SQL.
SHOW PROC '/frontends'\G
Example:
MySQL [(none)]> SHOW PROC '/frontends'\G
*************************** 1. row ***************************
Name: x.x.x.x_9010_1686810741121
IP: x.x.x.x
EditLogPort: 9010
HttpPort: 8030
QueryPort: 9030
RpcPort: 9020
Role: LEADER
ClusterId: 919351034
Join: true
Alive: true
ReplayedJournalId: 1220
LastHeartbeat: 2023-06-15 15:39:04
IsHelper: true
ErrMsg:
StartTime: 2023-06-15 14:32:28
Version: 3.0.0-48f4d81
1 row in set (0.01 sec)- If the field
Alive
istrue
, this FE node is properly started and added to the cluster. - If the field
Role
isFOLLOWER
, this FE node is eligible to be elected as the Leader FE node. - If the field
Role
isLEADER
, this FE node is the Leader FE node.
- If the field
-
Add the BE nodes to the cluster.
-- Replace <be_address> with the IP address (priority_networks)
-- or FQDN of the BE nodes, and replace <heartbeat_service_port>
-- with the heartbeat_service_port (Default: 9050) you specified in be.conf.
ALTER SYSTEM ADD BACKEND "<be_address>:<heartbeat_service_port>";NOTE
You can use the preceding command to add multiple BE nodes at a time. Each
<be_address>:<heartbeat_service_port>
pair represents one BE node. -
Check the status of the BE nodes by executing the following SQL.
SHOW PROC '/backends'\G
Example:
MySQL [(none)]> SHOW PROC '/backends'\G
*************************** 1. row ***************************
BackendId: 10007
IP: 172.26.195.67
HeartbeatPort: 9050
BePort: 9060
HttpPort: 8040
BrpcPort: 8060
LastStartTime: 2023-06-15 15:23:08
LastHeartbeat: 2023-06-15 15:57:30
Alive: true
SystemDecommissioned: false
ClusterDecommissioned: false
TabletNum: 30
DataUsedCapacity: 0.000
AvailCapacity: 341.965 GB
TotalCapacity: 1.968 TB
UsedPct: 83.04 %
MaxDiskUsedPct: 83.04 %
ErrMsg:
Version: 3.0.0-48f4d81
Status: {"lastSuccessReportTabletsTime":"2023-06-15 15:57:08"}
DataTotalCapacity: 341.965 GB
DataUsedPct: 0.00 %
CpuCores: 16
NumRunningQueries: 0
MemUsedPct: 0.01 %
CpuUsedPct: 0.0 %If the field
Alive
istrue
, this BE node is properly started and added to the cluster. -
(Optional) Add a CN node to the cluster.
-- Replace <cn_address> with the IP address (priority_networks)
-- or FQDN of the CN node, and replace <heartbeat_service_port>
-- with the heartbeat_service_port (Default: 9050) you specified in cn.conf.
ALTER SYSTEM ADD COMPUTE NODE "<cn_address>:<heartbeat_service_port>";NOTE
You can add multiple CN nodes with one SQL. Each
<cn_address>:<heartbeat_service_port>
pair represents one CN node. -
(Optional) Check the status of the CN nodes by executing the following SQL.
SHOW PROC '/compute_nodes'\G
Example:
MySQL [(none)]> SHOW PROC '/compute_nodes'\G
*************************** 1. row ***************************
ComputeNodeId: 10003
IP: x.x.x.x
HeartbeatPort: 9050
BePort: 9060
HttpPort: 8040
BrpcPort: 8060
LastStartTime: 2023-03-13 15:11:13
LastHeartbeat: 2023-03-13 15:11:13
Alive: true
SystemDecommissioned: false
ClusterDecommissioned: false
ErrMsg:
Version: 2.5.2-c3772fb
1 row in set (0.00 sec)If the field
Alive
istrue
, this CN node is properly started and added to the cluster.After CNs are properly started and you want to use CNs during queries, set the system variables
SET prefer_compute_node = true;
andSET use_compute_nodes = -1;
. For more information, see System variables.
Step 5: (Optional) Deploy a high-availability FE cluster
A high-availability FE cluster requires at least THREE Follower FE nodes in the StarRocks cluster. After the Leader FE node is started successfully, you can then start two new FE nodes to deploy a high-availability FE cluster.
-
Connect to StarRocks via your MySQL client. You need to log in with the initial account
root
, and the password is empty by default.# Replace <fe_address> with the IP address (priority_networks) or FQDN
# of the Leader FE node, and replace <query_port> (Default: 9030)
# with the query_port you specified in fe.conf.
mysql -h <fe_address> -P<query_port> -uroot -
Add the new Follower FE node to the cluster by executing the following SQL.
-- Replace <fe_address> with the IP address (priority_networks)
-- or FQDN of the new Follower FE node, and replace <edit_log_port>
-- with the edit_log_port (Default: 9010) you specified in fe.conf.
ALTER SYSTEM ADD FOLLOWER "<fe2_address>:<edit_log_port>";NOTE
- You can use the preceding command to add a single Follower FE nodes each time.
- If you want to add Observer FE nodes, execute
ALTER SYSTEM ADD OBSERVER "<fe_address>:<edit_log_port>"=
. For detailed instructions, see ALTER SYSTEM - FE.
-
Launch a terminal on the new FE instance, create a dedicated directory for metadata storage, navigate to the directory that stores the StarRocks FE deployment files, and modify the FE configuration file fe/conf/fe.conf. For more instructions, see Step 1: Start the Leader FE node. Basically, you can repeat the procedures in Step 1 except for the command used to start the FE node.
After configuring the Follower FE node, execute the following SQL to assign a helper node for Follower FE node and start the Follower FE node.
NOTE
When adding new Follower FE node to a cluster, you must assign a helper node (essentially an existing Follower FE node) to the new Follower FE node to synchronize the metadata.
-
To start a new FE node with IP address access, run the following command to start the FE node:
# Replace <helper_fe_ip> with the IP address (priority_networks)
# of the Leader FE node, and replace <helper_edit_log_port> (Default: 9010) with
# the Leader FE node's edit_log_port.
./fe/bin/start_fe.sh --helper <helper_fe_ip>:<helper_edit_log_port> --daemonNote that you only need to specify the parameter
--helper
ONCE when you start the node for the first time. -
To start a new FE node with FQDN access, run the following command to start the FE node:
# Replace <helper_fqdn> with the FQDN of the Leader FE node,
# and replace <helper_edit_log_port> (Default: 9010) with the Leader FE node's edit_log_port.
./fe/bin/start_fe.sh --helper <helper_fqdn>:<helper_edit_log_port> \
--host_type FQDN --daemonNote that you only need to specify the parameters
--helper
and--host_type
ONCE when you start the node for the first time.
-
-
Check the FE logs to verify if the FE node is started successfully.
cat fe/log/fe.log | grep thrift
A record of log like
2022-08-10 16:12:29,911 INFO (UNKNOWN x.x.x.x_9010_1660119137253(-1)|1) [FeServer.start():52] thrift server started with port 9020.
suggests that the FE node is started properly. -
Repeat the preceding procedure 2, 3, and 4 until you have start all the new Follower FE nodes properly, and then check the status of the FE nodes by executing the following SQL from your MySQL client:
SHOW PROC '/frontends'\G
Example:
MySQL [(none)]> SHOW PROC '/frontends'\G
*************************** 1. row ***************************
Name: x.x.x.x_9010_1686810741121
IP: x.x.x.x
EditLogPort: 9010
HttpPort: 8030
QueryPort: 9030
RpcPort: 9020
Role: LEADER
ClusterId: 919351034
Join: true
Alive: true
ReplayedJournalId: 1220
LastHeartbeat: 2023-06-15 15:39:04
IsHelper: true
ErrMsg:
StartTime: 2023-06-15 14:32:28
Version: 3.0.0-48f4d81
*************************** 2. row ***************************
Name: x.x.x.x_9010_1686814080597
IP: x.x.x.x
EditLogPort: 9010
HttpPort: 8030
QueryPort: 9030
RpcPort: 9020
Role: FOLLOWER
ClusterId: 919351034
Join: true
Alive: true
ReplayedJournalId: 1219
LastHeartbeat: 2023-06-15 15:39:04
IsHelper: true
ErrMsg:
StartTime: 2023-06-15 15:38:53
Version: 3.0.0-48f4d81
*************************** 3. row ***************************
Name: x.x.x.x_9010_1686814090833
IP: x.x.x.x
EditLogPort: 9010
HttpPort: 8030
QueryPort: 9030
RpcPort: 9020
Role: FOLLOWER
ClusterId: 919351034
Join: true
Alive: true
ReplayedJournalId: 1219
LastHeartbeat: 2023-06-15 15:39:04
IsHelper: true
ErrMsg:
StartTime: 2023-06-15 15:37:52
Version: 3.0.0-48f4d81
3 rows in set (0.02 sec)- If the field
Alive
istrue
, this FE node is properly started and added to the cluster. - If the field
Role
isFOLLOWER
, this FE node is eligible to be elected as the Leader FE node. - If the field
Role
isLEADER
, this FE node is the Leader FE node.
- If the field
Stop the StarRocks cluster
You can stop the StarRocks cluster by running the following commands on the corresponding instances.
-
Stop an FE node.
./fe/bin/stop_fe.sh --daemon
-
Stop a BE node.
./be/bin/stop_be.sh --daemon
-
Stop a CN node.
./be/bin/stop_cn.sh --daemon
Troubleshooting
Try the following steps to identify the errors that occur when you start the FE, BE, or CN nodes:
-
If an FE node is not started properly, you can identify the problem by checking its log in fe/log/fe.warn.log.
cat fe/log/fe.warn.log
Having identified and resolved the problem, you must first terminate the current FE process, delete the existing meta directory, create a new metadata storage directory, and then restart the FE node with the correct configuration.
-
If a BE node is not started properly, you can identify the problem by checking its log in be/log/be.WARNING.
cat be/log/be.WARNING
Having identified and resolved the problem, you must first terminate the existing BE process, delete the existing storage directory, create a new data storage directory, and then restart the BE node with the correct configuration.
-
If a CN node is not started properly, you can identify the problem by checking its log in be/log/cn.WARNING.
cat be/log/cn.WARNING
Having identified and resolved the problem, you must first terminate the existing CN process, and then restart the CN node with the correct configuration.
What to do next
Having deployed your StarRocks cluster, you can move on to Post-deployment Setup for instructions on initial management measures.