Deploy and manage Broker node
This topic describes how to deploy Broker nodes. With Broker nodes, StarRocks can read data from sources such as HDFS and S3, and pre-process, load, and backup the data with its own computing resources.
We recommend you deploy ONE Broker node on each instance that hosts a BE node, and add all Broker nodes using the same broker_name
. Broker nodes automatically balance the data transmission load when processing tasks.
Broker nodes use the network connection to transmit data to BE nodes. When a Broker node and a BE node are deployed on the same machine, the Broker node transmit the data to the local BE node.
Before you begin
Make sure you have completed the required configurations by following the instructions provided in Deployment prerequisites, Check environment configurations, and Prepare deployment files.
Start Broker service
The following procedures are performed on the BE instances.
-
Navigate to the directory that stores the StarRocks Broker deployment files you prepared earlier, and modify the Broker configuration file apache_hdfs_broker/conf/apache_hdfs_broker.conf.
If the HDFS Thrift RPC port (
broker_ipc_port
, Default:8000
) on the instance is occupied, you must assign a valid alternative in the Broker configuration file.broker_ipc_port = aaaa # Default: 8000
The following table lists the configuration items supported by Broker.
Configuration item Default Unit Description hdfs_read_buffer_size_kb 8192 KB Size of the buffer that is used to read data from HDFS. hdfs_write_buffer_size_kb 1024 KB Size of the buffer that is used to write data into HDFS. client_expire_seconds 300 Second Client sessions will be deleted if they do not receive any ping after the specified time. broker_ipc_port 8000 N/A The HDFS thrift RPC port. disable_broker_client_expiration_checking false N/A Whether to disable the checking and clearing of the expired OSS file descriptors, which, in some cases, causes the broker to stuck when OSS is close. To avoid this situation, you can set this parameter to true
to disable the checking.sys_log_dir ${BROKER_HOME}/log
N/A The directory used to store system logs (including INFO, WARNING, ERROR, and FATAL). sys_log_level INFO N/A The log level. Valid values include INFO, WARNING, ERROR, and FATAL. sys_log_roll_mode SIZE-MB-1024 N/A The mode how system logs are segmented into log rolls. Valid values include TIME-DAY, TIME-HOUR, and SIZE-MB-nnn. The default value indicates that logs are segmented into rolls which are 1 GB each. sys_log_roll_num 30 N/A The number of log rolls to reserve. audit_log_dir ${BROKER_HOME}/log
N/A The directory that stores audit log files. audit_log_modules Empty string N/A The modules for which StarRocks generates audit log entries. By default, StarRocks generates audit logs for the slow_query module and the query module. You can specify multiple modules, whose names must be separated by a comma (,) and a space. audit_log_roll_mode TIME-DAY N/A Valid values include TIME-DAY
,TIME-HOUR
, andSIZE-MB-<size>
.audit_log_roll_num 10 N/A This configuration does not work if the audit_log_roll_mode is set to SIZE-MB-<size>
.sys_log_verbose_modules com.starrocks N/A The modules for which StarRocks generates system logs. Valid values are namespaces in BE, including starrocks
,starrocks::debug
,starrocks::fs
,starrocks::io
,starrocks::lake
,starrocks::pipeline
,starrocks::query_cache
,starrocks::stream
, andstarrocks::workgroup
. -
Start the Broker node.
./apache_hdfs_broker/bin/start_broker.sh --daemon
-
Check the Broker logs to verify if the Broker node is started successfully.
cat apache_hdfs_broker/log/apache_hdfs_broker.log | grep thrift
-
You can start new Broker nodes by repeating the above procedures on other instances.
Add Broker node to cluster
The following procedures are performed on a MySQL client. You must have MySQL client 5.5.0 or later installed.
-
Connect to StarRocks via your MySQL client. You need to log in with the initial account
root
, and the password is empty by default.# Replace <fe_address> with the IP address (priority_networks) or FQDN
# of the FE node you connect to, and replace <query_port> (Default: 9030)
# with the query_port you specified in fe.conf.
mysql -h <fe_address> -P<query_port> -uroot -
Run the following command to add the Broker node to the cluster.
ALTER SYSTEM ADD BROKER <broker_name> "<broker_address>:<broker_ipc_port>";
NOTE
- You can use the preceding command to add multiple Broker nodes at a time. Each
<broker_address>:<broker_ipc_port>
pair represents one Broker node. - You can add multiple Brokers nodes with the same
broker_name
.
- You can use the preceding command to add multiple Broker nodes at a time. Each
-
Verify if the Broker node is properly added to the via MySQL client.
SHOW PROC "/brokers"\G
Example:
MySQL [(none)]> SHOW PROC "/brokers"\G
*************************** 1. row ***************************
Name: broker1
IP: x.x.x.x
Port: 8000
Alive: true
LastStartTime: 2022-05-19 11:21:36
LastUpdateTime: 2022-05-19 11:28:31
ErrMsg:
1 row in set (0.00 sec)
When the field Alive
is true
, this Broker is properly started and added to the cluster.
Stop Broker node
Run the following command to stop the Broker node.
./bin/stop_broker.sh --daemon
Upgrade Broker node
-
Navigate to the working directory of the Broker node and stop the node.
# Replace <broker_dir> with the deployment directory of the Broker node.
cd <broker_dir>/apache_hdfs_broker
sh bin/stop_broker.sh -
Replace the original deployment files under bin and lib with the ones of the new version.
mv lib lib.bak
mv bin bin.bak
cp -r /tmp/StarRocks-x.x.x/apache_hdfs_broker/lib .
cp -r /tmp/StarRocks-x.x.x/apache_hdfs_broker/bin . -
Start the Broker node.
sh bin/start_broker.sh --daemon
-
Check if the Broker node is started successfully.
ps aux | grep broker
-
Repeat the above procedures to upgrade other Broker nodes.
Downgrade Broker node
-
Navigate to the working directory of the Broker node and stop the node.
# Replace <broker_dir> with the deployment directory of the Broker node.
cd <broker_dir>/apache_hdfs_broker
sh bin/stop_broker.sh -
Replace the original deployment files under bin and lib with the ones of the earlier version.
mv lib lib.bak
mv bin bin.bak
cp -r /tmp/StarRocks-x.x.x/apache_hdfs_broker/lib .
cp -r /tmp/StarRocks-x.x.x/apache_hdfs_broker/bin . -
Start the Broker node.
sh bin/start_broker.sh --daemon
-
Check if the Broker node is started successfully.
ps aux | grep broker
-
Repeat the above procedures to downgrade other Broker nodes.