BE Configuration
Some BE configuration items are dynamic parameters which you can set interactively when BE nodes are online. The rest of them are static parameters. You can only set the static parameters of a BE node by changing them in the corresponding configuration file be.conf and restarting the BE node to allow the change to take effect.
View BE configuration items
You can view the BE configuration items using the following command:
curl http://<BE_IP>:<BE_HTTP_PORT>/varz
Configure BE parameters
Configure BE dynamic parameters
You can configure a dynamic parameter of a BE node by using the curl
command.
curl -XPOST http://be_host:http_port/api/update_config?<configuration_item>=<value>
Configure BE static parameters
You can only set the static parameters of a BE by changing them in the corresponding configuration file be.conf, and restarting the BE to allow the changes to take effect.
Understand BE Parameters
Server
priority_networks
- Default: Empty string
- Type: String
- Unit: -
- Is mutable: No
- Description: Declares a selection strategy for servers that have multiple IP addresses. Note that at most one IP address must match the list specified by this parameter. The value of this parameter is a list that consists of entries, which are separated with semicolons (;) in CIDR notation, such as
10.10.10.0/24
. If no IP address matches the entries in this list, an available IP address of the server will be randomly selected. From v3.3.0, StarRocks supports deployment based on IPv6. If the server has both IPv4 and IPv6 addresses, and this parameter is not specified, the system uses an IPv4 address by default. You can change this behavior by settingnet_use_ipv6_when_priority_networks_empty
totrue
. - Introduced in: -
net_use_ipv6_when_priority_networks_empty
- Default: false
- Type: Boolean
- Unit: -
- Is mutable: No
- Description: A boolean value to control whether to use IPv6 addresses preferentially when
priority_networks
is not specified.true
indicates to allow the system to use an IPv6 address preferentially when the server that hosts the node has both IPv4 and IPv6 addresses andpriority_networks
is not specified. - Introduced in: v3.3.0
mem_limit
- Default: 90%
- Type: String
- Unit: -
- Is mutable: No
- Description: BE process memory upper limit. You can set it as a percentage ("80%") or a physical limit ("100G"). The default hard limit is 90% of the server's memory size, and the soft limit is 80%. You need to configure this parameter if you want to deploy StarRocks with other memory-intensive services on a same server.
- Introduced in: -
num_threads_per_core
- Default: 3
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of threads started on each CPU core.
- Introduced in: -
be_http_port
- Default: 8040
- Type: Int
- Unit: -
- Is mutable: No
- Description: The BE HTTP server port.
- Introduced in: -
be_http_num_workers
- Default: 48
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of threads used by the HTTP server.
- Introduced in: -
be_exit_after_disk_write_hang_second
- Default: 60
- Type: Int
- Unit: Seconds
- Is mutable: No
- Description: The length of time that the BE waits to exit after the disk hangs.
- Introduced in: -
compress_rowbatches
- Default: true
- Type: Boolean
- Unit: -
- Is mutable: No
- Description: A boolean value to control whether to compress the row batches in RPCs between BEs.
true
indicates compressing the row batches, andfalse
indicates not compressing them. - Introduced in: -
serialize_batch
- Default: false
- Type: Boolean
- Unit: -
- Is mutable: No
- Description: A boolean value to control whether to serialize the row batches in RPCs between BEs.
true
indicates serializing the row batches, andfalse
indicates not serializing them. - Introduced in: -
Thrift
be_port
- Default: 9060
- Type: Int
- Unit: -
- Is mutable: No
- Description: The BE thrift server port, which is used to receive requests from FEs.
- Introduced in: -
thrift_client_retry_interval_ms
- Default: 100
- Type: Int
- Unit: Milliseconds
- Is mutable: Yes
- Description: The time interval at which a thrift client retries.
- Introduced in: -
thrift_rpc_timeout_ms
- Default: 5000
- Type: Int
- Unit: Milliseconds
- Is mutable: Yes
- Description: The timeout for a thrift RPC.
- Introduced in: -
bRPC
brpc_port
- Default: 8060
- Type: Int
- Unit: -
- Is mutable: No
- Description: The BE bRPC port, which is used to view the network statistics of bRPCs.
- Introduced in: -
brpc_num_threads
- Default: -1
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of bthreads of a bRPC. The value
-1
indicates the same number with the CPU threads. - Introduced in: -
brpc_max_body_size
- Default: 2147483648
- Type: Int
- Unit: Bytes
- Is mutable: No
- Description: The maximum body size of a bRPC.
- Introduced in: -
Heartbeat
heartbeat_service_port
- Default: 9050
- Type: Int
- Unit: -
- Is mutable: No
- Description: The BE heartbeat service port, which is used to receive heartbeats from FEs.
- Introduced in: -
heartbeat_service_thread_count
- Default: 1
- Type: Int
- Unit: -
- Is mutable: No
- Description: The thread count of the BE heartbeat service.
- Introduced in: -
Logging
sys_log_dir
- Default:
${STARROCKS_HOME}/log
- Type: String
- Unit: -
- Is mutable: No
- Description: The directory that stores system logs (including INFO, WARNING, ERROR, and FATAL).
- Introduced in: -
sys_log_level
- Default: INFO
- Type: String
- Unit: -
- Is mutable: Yes (from v3.3.0, v3.2.7, and v3.1.12)
- Description: The severity levels into which system log entries are classified. Valid values: INFO, WARN, ERROR, and FATAL. This item was changed to a dynamic configuration from v3.3.0, v3.2.7, and v3.1.12 onwards.
- Introduced in: -
sys_log_roll_mode
- Default: SIZE-MB-1024
- Type: String
- Unit: -
- Is mutable: No
- Description: The mode in which system logs are segmented into log rolls. Valid values include
TIME-DAY
,TIME-HOUR
, andSIZE-MB-
size. The default value indicates that logs are segmented into rolls, each of which is 1 GB. - Introduced in: -
sys_log_roll_num
- Default: 10
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of log rolls to reserve.
- Introduced in: -
sys_log_verbose_modules
- Default:
- Type: Strings
- Unit: -
- Is mutable: No
- Description: The module of the logs to be printed. For example, if you set this configuration item to OLAP, StarRocks only prints the logs of the OLAP module. Valid values are namespaces in BE, including
starrocks
,starrocks::debug
,starrocks::fs
,starrocks::io
,starrocks::lake
,starrocks::pipeline
,starrocks::query_cache
,starrocks::stream
, andstarrocks::workgroup
. - Introduced in: -
sys_log_verbose_level
- Default: 10
- Type: Int
- Unit: -
- Is mutable: No
- Description: The level of the logs to be printed. This configuration item is used to control the output of logs initiated with VLOG in codes.
- Introduced in: -
log_buffer_level
- Default: Empty string
- Type: String
- Unit: -
- Is mutable: No
- Description: The strategy for flushing logs. The default value indicates that logs are buffered in memory. Valid values are
-1
and0
.-1
indicates that logs are not buffered in memory. - Introduced in: -
Statistic report
report_task_interval_seconds
- Default: 10
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval at which to report the state of a task. A task can be creating a table, dropping a table, loading data, or changing a table schema.
- Introduced in: -
report_disk_state_interval_seconds
- Default: 60
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval at which to report the storage volume state, which includes the size of data within the volume.
- Introduced in: -
report_tablet_interval_seconds
- Default: 60
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval at which to report the most updated version of all tablets.
- Introduced in: -
report_workgroup_interval_seconds
- Default: 5
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval at which to report the most updated version of all workgroups.
- Introduced in: -
status_report_interval
- Default: 5
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval at which a query reports its profile, which can be used for query statistics collection by FE.
- Introduced in: -
periodic_counter_update_period_ms
- Default: 500
- Type: Int
- Unit: Milliseconds
- Is mutable: Yes
- Description: The time interval at which to collect the Counter statistics.
- Introduced in: -
Storage
primary_key_limit_size
- Default: 128
- Type: Int
- Unit: Bytes
- Is mutable: Yes
- Description: The maximum size of a key column in Primary Key tables.
- Introduced in: v2.5
drop_tablet_worker_count
- Default: 3
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The number of threads used to drop a tablet.
- Introduced in: -
alter_tablet_worker_count
- Default: 3
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The number of threads used for Schema Change.
- Introduced in: -
clone_worker_count
- Default: 3
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of threads used for clone.
- Introduced in: -
storage_medium_migrate_count
- Default: 3
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of threads used for storage medium migration (from SATA to SSD).
- Introduced in: -
check_consistency_worker_count
- Default: 1
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of threads used for checking the consistency of tablets.
- Introduced in: -
upload_worker_count
- Default: 0
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of threads for the upload tasks of backup jobs on a BE node.
0
indicates setting the value to the number of CPU cores on the machine where the BE resides. - Introduced in: -
download_worker_count
- Default: 0
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of threads for the download tasks of restore jobs on a BE node.
0
indicates setting the value to the number of CPU cores on the machine where the BE resides. - Introduced in: -
make_snapshot_worker_count
- Default: 5
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of threads for the make snapshot tasks on a BE node.
- Introduced in: -
release_snapshot_worker_count
- Default: 5
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of threads for the release snapshot tasks on a BE node.
- Introduced in: -
max_download_speed_kbps
- Default: 50000
- Type: Int
- Unit: KB/Second
- Is mutable: Yes
- Description: The maximum download speed of each HTTP request. This value affects the performance of data replica synchronization across BE nodes.
- Introduced in: -
download_low_speed_limit_kbps
- Default: 50
- Type: Int
- Unit: KB/Second
- Is mutable: Yes
- Description: The download speed lower limit of each HTTP request. An HTTP request aborts when it constantly runs with a lower speed than this value within the time span specified in the configuration item
download_low_speed_time
. - Introduced in: -
download_low_speed_time
- Default: 300
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The maximum time that an HTTP request can run with a download speed lower than the limit. An HTTP request aborts when it constantly runs with a lower speed than the value of
download_low_speed_limit_kbps
within the time span specified in this configuration item. - Introduced in: -
memory_limitation_per_thread_for_schema_change
- Default: 2
- Type: Int
- Unit: GB
- Is mutable: Yes
- Description: The maximum memory size allowed for each schema change task.
- Introduced in: -
update_cache_expire_sec
- Default: 360
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The expiration time of Update Cache.
- Introduced in: -
file_descriptor_cache_clean_interval
- Default: 3600
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval at which to clean file descriptors that have not been used for a certain period of time.
- Introduced in: -
disk_stat_monitor_interval
- Default: 5
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval at which to monitor health status of disks.
- Introduced in: -
replication_threads
- Default: 0
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of threads used for replication.
0
indicates setting the thread number to four times the BE CPU core count. - Introduced in: v3.3.5
replication_max_speed_limit_kbps
- Default: 50000
- Type: Int
- Unit: KB/s
- Is mutable: Yes
- Description: The maximum speed of each replication thread.
- Introduced in: v3.3.5
replication_min_speed_limit_kbps
- Default: 50
- Type: Int
- Unit: KB/s
- Is mutable: Yes
- Description: The minimum speed of each replication thread.
- Introduced in: v3.3.5
replication_min_speed_time_seconds
- Default: 300
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time duration allowed for a replication thread to be under the minimum speed. Replication will fail if the time when the actual speed is lower than
replication_min_speed_limit_kbps
exceeds this value. - Introduced in: v3.3.5
clear_expired_replication_snapshots_interval_seconds
- Default: 3600
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval at which the system clears the expired snapshots left by abnormal replications.
- Introduced in: v3.3.5
unused_rowset_monitor_interval
- Default: 30
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval at which to clean the expired rowsets.
- Introduced in: -
storage_root_path
- Default:
${STARROCKS_HOME}/storage
- Type: String
- Unit: -
- Is mutable: No
- Description: The directory and medium of the storage volume. Example:
/data1,medium:hdd;/data2,medium:ssd
.- Multiple volumes are separated by semicolons (
;
). - If the storage medium is SSD, add
,medium:ssd
at the end of the directory. - If the storage medium is HDD, add
,medium:hdd
at the end of the directory.
- Multiple volumes are separated by semicolons (
- Introduced in: -
max_percentage_of_error_disk
- Default: 0
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum percentage of error that is tolerable in a storage volume before the corresponding BE node quits.
- Introduced in: -
default_num_rows_per_column_file_block
- Default: 1024
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of rows that can be stored in each row block.
- Introduced in: -
max_tablet_num_per_shard
- Default: 1024
- Type: Int
- Unit: -
- Is mutable: No
- Description: The maximum number of tablets in each shard. This configuration item is used to restrict the number of tablet child directories under each storage directory.
- Introduced in: -
pending_data_expire_time_sec
- Default: 1800
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The expiration time of the pending data in the storage engine.
- Introduced in: -
inc_rowset_expired_sec
- Default: 1800
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The expiration time of the incoming data. This configuration item is used in incremental clone.
- Introduced in: -
tablet_rowset_stale_sweep_time_sec
- Default: 1800
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval at which to sweep the stale rowsets in tablets.
- Introduced in: -
max_garbage_sweep_interval
- Default: 3600
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The maximum time interval for garbage collection on storage volumes. This configuration is changed to dynamic from v3.0 onwards.
- Introduced in: -
min_garbage_sweep_interval
- Default: 180
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The minimum time interval for garbage collection on storage volumes. This configuration is changed to dynamic from v3.0 onwards.
- Introduced in: -
snapshot_expire_time_sec
- Default: 172800
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The expiration time of snapshot files.
- Introduced in: -
trash_file_expire_time_sec
- Default: 86400
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval at which to clean trash files. The default value has been changed from 259,200 to 86,400 since v2.5.17, v3.0.9, and v3.1.6.
- Introduced in: -
compact_threads
- Default: 4
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of threads used for concurrent compaction tasks. This configuration is changed to dynamic from v3.1.7 and v3.2.2 onwards.
- Introduced in: v3.0.0
base_compaction_check_interval_seconds
- Default: 60
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval of thread polling for a Base Compaction.
- Introduced in: -
min_base_compaction_num_singleton_deltas
- Default: 5
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The minimum number of segments that trigger a Base Compaction.
- Introduced in: -
max_base_compaction_num_singleton_deltas
- Default: 100
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of segments that can be compacted in each Base Compaction.
- Introduced in: -
base_compaction_num_threads_per_disk
- Default: 1
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of threads used for Base Compaction on each storage volume.
- Introduced in: -
base_cumulative_delta_ratio
- Default: 0.3
- Type: Double
- Unit: -
- Is mutable: Yes
- Description: The ratio of cumulative file size to base file size. The ratio reaching this value is one of the conditions that trigger the Base Compaction.
- Introduced in: -
base_compaction_interval_seconds_since_last_operation
- Default: 86400
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval since the last Base Compaction. This configuration item is one of the conditions that trigger a Base Compaction.
- Introduced in: -
cumulative_compaction_check_interval_seconds
- Default: 1
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval of thread polling for a Cumulative Compaction.
- Introduced in: -
min_cumulative_compaction_num_singleton_deltas
- Default: 5
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The minimum number of segments to trigger Cumulative Compaction.
- Introduced in: -
max_cumulative_compaction_num_singleton_deltas
- Default: 1000
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of segments that can be merged in a single Cumulative Compaction. You can reduce this value if OOM occurs during compaction.
- Introduced in: -
cumulative_compaction_num_threads_per_disk
- Default: 1
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of Cumulative Compaction threads per disk.
- Introduced in: -
max_compaction_candidate_num
- Default: 40960
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of candidate tablets for compaction. If the value is too large, it will cause high memory usage and high CPU load.
- Introduced in: -
update_compaction_check_interval_seconds
- Default: 10
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval at which to check compaction for Primary Key tables.
- Introduced in: -
update_compaction_num_threads_per_disk
- Default: 1
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The number of Compaction threads per disk for Primary Key tables.
- Introduced in: -
update_compaction_per_tablet_min_interval_seconds
- Default: 120
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The minimum time interval at which compaction is triggered for each tablet in a Primary Key table.
- Introduced in: -
max_update_compaction_num_singleton_deltas
- Default: 1000
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of rowsets that can be merged in a single Compaction for Primary Key tables.
- Introduced in: -
update_compaction_size_threshold
- Default: 268435456
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The Compaction Score of Primary Key tables is calculated based on the file size, which is different from other table types. This parameter can be used to make the Compaction Score of Primary Key tables similar to that of other table types, making it easier for users to understand.
- Introduced in: -
update_compaction_result_bytes
- Default: 1073741824
- Type: Int
- Unit: Bytes
- Is mutable: Yes
- Description: The maximum result size of a single compaction for Primary Key tables.
- Introduced in: -
update_compaction_delvec_file_io_amp_ratio
- Default: 2
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: Used to control the priority of compaction for rowsets that contain Delvec files in Primary Key tables. The larger the value, the higher the priority.
- Introduced in: -
update_compaction_ratio_threshold
- Default: 0.5
- Type: Double
- Unit: -
- Is mutable: Yes
- Description: The maximum proportion of data that a compaction can merge for a Primary Key table in a shared-data cluster. It is recommended to shrink this value if a single tablet becomes excessively large.
- Introduced in: v3.1.5
repair_compaction_interval_seconds
- Default: 600
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval to poll Repair Compaction threads.
- Introduced in: -
manual_compaction_threads
- Default: 4
- Type: Int
- Unit: -
- Is mutable: No
- Description: Number of threads for Manual Compaction.
- Introduced in: -
min_compaction_failure_interval_sec
- Default: 120
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The minimum time interval at which a tablet compaction can be scheduled since the previous compaction failure.
- Introduced in: -
min_cumulative_compaction_failure_interval_sec
- Default: 30
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The minimum time interval at which Cumulative Compaction retries upon failures.
- Introduced in: -
max_compaction_concurrency
- Default: -1
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum concurrency of compactions (including both Base Compaction and Cumulative Compaction). The value
-1
indicates that no limit is imposed on the concurrency.0
indicates disabling compaction. This parameter is mutable when the Event-based Compaction Framework is enabled. - Introduced in: -
compaction_trace_threshold
- Default: 60
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time threshold for each compaction. If a compaction takes more time than the time threshold, StarRocks prints the corresponding trace.
- Introduced in: -
compaction_memory_limit_per_worker
- Default: 2147483648
- Type: Int
- Unit: Bytes
- Is mutable: No
- Description: The maximum memory size allowed for each Compaction thread.
- Introduced in: -
enable_rowset_verify
- Default: false
- Type: Boolean
- Unit: -
- Is mutable: Yes
- Description: Whether to verify the correctness of generated rowsets. When enabled, the correctness of the generated rowsets will be checked after Compaction and Schema Change.
- Introduced in: -
vertical_compaction_max_columns_per_group
- Default: 5
- Type: Int
- Unit: -
- Is mutable: No
- Description: The maximum number of columns per group of Vertical Compactions.
- Introduced in: -
enable_event_based_compaction_framework
- Default: true
- Type: Boolean
- Unit: -
- Is mutable: No
- Description: Whether to enable the Event-based Compaction Framework.
true
indicates Event-based Compaction Framework is enabled, andfalse
indicates it is disabled. Enabling Event-based Compaction Framework can greatly reduce the overhead of compaction in scenarios where there are many tablets or a single tablet has a large amount of data. - Introduced in: -
enable_size_tiered_compaction_strategy
- Default: true
- Type: Boolean
- Unit: -
- Is mutable: No
- Description: Whether to enable the Size-tiered Compaction policy (excluding Primary Key tables).
true
indicates the Size-tiered Compaction strategy is enabled, andfalse
indicates it is disabled. - Introduced in: -
enable_pk_size_tiered_compaction_strategy
- Default: true
- Type: Boolean
- Unit: -
- Is mutable: No
- Description: Whether to enable the Size-tiered Compaction policy for Primary Key tables.
true
indicates the Size-tiered Compaction strategy is enabled, andfalse
indicates it is disabled. - Introduced in: This item takes effect for shared-data clusters from v3.2.4 and v3.1.10 onwards, and for shared-nothing clusters from v3.2.5 and v3.1.10 onwards.
size_tiered_min_level_size
- Default: 131072
- Type: Int
- Unit: Bytes
- Is mutable: Yes
- Description: The data size of the minimum level in the Size-tiered Compaction policy. Rowsets smaller than this value immediately trigger the data compaction.
- Introduced in: -
size_tiered_level_multiple
- Default: 5
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The multiple of data size between two contiguous levels in the Size-tiered Compaction policy.
- Introduced in: -
size_tiered_level_multiple_dupkey
- Default: 10
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: In the Size-tiered Compaction policy, the multiple of the data amount difference between two adjacent levels for Duplicate Key tables.
- Introduced in: -
size_tiered_level_num
- Default: 7
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The number of levels for the Size-tiered Compaction policy. At most one rowset is reserved for each level. Therefore, under a stable condition, there are, at most, as many rowsets as the level number specified in this configuration item.
- Introduced in: -
enable_check_string_lengths
- Default: true
- Type: Boolean
- Unit: -
- Is mutable: No
- Description: Whether to check the data length during loading to solve compaction failures caused by out-of-bound VARCHAR data.
- Introduced in: -
max_row_source_mask_memory_bytes
- Default: 209715200
- Type: Int
- Unit: Bytes
- Is mutable: No
- Description: The maximum memory size of the row source mask buffer. When the buffer is larger than this value, data will be persisted to a temporary file on the disk. This value should be set lower than the value of
compaction_mem_limit
. - Introduced in: -
memory_maintenance_sleep_time_s
- Default: 10
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval at which ColumnPool GC is triggered. StarRocks executes GC periodically and returns the released memory to the operating system.
- Introduced in: -
load_process_max_memory_limit_bytes
- Default: 107374182400
- Type: Int
- Unit: Bytes
- Is mutable: No
- Description: The maximum size limit of memory resources that can be taken up by all load processes on a BE node.
- Introduced in: -
load_process_max_memory_limit_percent
- Default: 30
- Type: Int
- Unit: -
- Is mutable: No
- Description: The soft limit (in percentage) of memory resources that can be taken up by all load processes on a BE node.
- Introduced in: -
load_process_max_memory_hard_limit_ratio
- Default: 2
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The hard limit (ratio) of memory resources that can be taken up by all load processes on a BE node. When
enable_new_load_on_memory_limit_exceeded
is set tofalse
, and the memory consumption of all loading processes exceedsload_process_max_memory_limit_percent * load_process_max_memory_hard_limit_ratio
, new loading processes will be rejected. - Introduced in: v3.3.2
enable_new_load_on_memory_limit_exceeded
- Default: false
- Type: Boolean
- Unit: -
- Is mutable: Yes
- Description: Whether to allow new loading processes when the hard memory resource limit is reached.
true
indicates new loading processes will be allowed, andfalse
indicates they will be rejected. - Introduced in: v3.3.2
sync_tablet_meta
- Default: false
- Type: Boolean
- Unit: -
- Is mutable: Yes
- Description: A boolean value to control whether to enable the synchronization of the tablet metadata.
true
indicates enabling synchronization, andfalse
indicates disabling it. - Introduced in: -
storage_flood_stage_usage_percent
- Default: 95
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: Hard limit of the storage usage percentage in all BE directories. If the storage usage (in percentage) of the BE storage directory exceeds this value and the remaining storage space is less than
storage_flood_stage_left_capacity_bytes
, Load and Restore jobs are rejected. You need to set this item together with the FE configuration itemstorage_usage_hard_limit_percent
to allow the configurations to take effect. - Introduced in: -
storage_flood_stage_left_capacity_bytes
- Default: 107374182400
- Type: Int
- Unit: Bytes
- Is mutable: Yes
- Description: Hard limit of the remaining storage space in all BE directories. If the remaining storage space of the BE storage directory is less than this value and the storage usage (in percentage) exceeds
storage_flood_stage_usage_percent
, Load and Restore jobs are rejected. You need to set this item together with the FE configuration itemstorage_usage_hard_limit_reserve_bytes
to allow the configurations to take effect. - Introduced in: -
tablet_meta_checkpoint_min_new_rowsets_num
- Default: 10
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The minimum number of rowsets to create since the last TabletMeta Checkpoint.
- Introduced in: -
tablet_meta_checkpoint_min_interval_secs
- Default: 600
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The time interval of thread polling for a TabletMeta Checkpoint.
- Introduced in: -
tablet_map_shard_size
- Default: 32
- Type: Int
- Unit: -
- Is mutable: No
- Description: The tablet map shard size. The value must be a power of two.
- Introduced in: -
tablet_max_versions
- Default: 1000
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of versions allowed on a tablet. If the number of versions exceeds this value, new write requests will fail.
- Introduced in: -
tablet_max_pending_versions
- Default: 1000
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of pending versions that are tolerable on a Primary Key tablet. Pending versions refer to versions that are committed but not applied yet.
- Introduced in: -
tablet_stat_cache_update_interval_second
- Default: 300
- Type: Int
- Unit: Seconds
- Is mutable: 是
- Description: The time interval at which Tablet Stat Cache updates.
- Introduced in: -
enable_bitmap_union_disk_format_with_set
- Default: false
- Type: Boolean
- Unit: -
- Is mutable: Yes
- Description: A boolean value to control whether to enable the new storage format of the BITMAP type, which can improve the performance of bitmap_union.
true
indicates enabling the new storage format, andfalse
indicates disabling it. - Introduced in: -
pindex_major_compaction_limit_per_disk
- Default: 1
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum concurrency of compaction on a disk. This addresses the issue of uneven I/O across disks due to compaction. This issue can cause excessively high I/O for certain disks.
- Introduced in: v3.0.9
Loading
push_worker_count_normal_priority
- Default: 3
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of threads used to handle a load task with NORMAL priority.
- Introduced in: -
push_worker_count_high_priority
- Default: 3
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of threads used to handle a load task with HIGH priority.
- Introduced in: -
transaction_publish_version_worker_count
- Default: 0
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of threads used to publish a version. When this value is set to less than or equal to
0
, the system uses the CPU core count as the value, so as to avoid insufficient thread resources when import concurrency is high but only a fixed number of threads are used. From v2.5, the default value has been changed from8
to0
. - Introduced in: -
clear_transaction_task_worker_count
- Default: 1
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of threads used for clearing transaction.
- Introduced in: -
load_data_reserve_hours
- Default: 4
- Type: Int
- Unit: Hours
- Is mutable: No
- Description: The reservation time for the files produced by small-scale loadings.
- Introduced in: -
load_error_log_reserve_hours
- Default: 48
- Type: Int
- Unit: Hours
- Is mutable: Yes
- Description: The time for which data loading logs are reserved.
- Introduced in: -
number_tablet_writer_threads
- Default: 16
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The number of threads used for Stream Load. This configuration is changed to dynamic from v3.1.7 onwards.
- Introduced in: -
streaming_load_max_mb
- Default: 102400
- Type: Int
- Unit: MB
- Is mutable: Yes
- Description: The maximum size of a file that can be streamed into StarRocks. From v3.0, the default value has been changed from
10240
to102400
. - Introduced in: -
streaming_load_max_batch_size_mb
- Default: 100
- Type: Int
- Unit: MB
- Is mutable: Yes
- Description: The maximum size of a JSON file that can be streamed into StarRocks.
- Introduced in: -
streaming_load_rpc_max_alive_time_sec
- Default: 1200
- Type: Int
- Unit: Seconds
- Is mutable: No
- Description: The RPC timeout for Stream Load.
- Introduced in: -
write_buffer_size
- Default: 104857600
- Type: Int
- Unit: Bytes
- Is mutable: Yes
- Description: The buffer size of MemTable in the memory. This configuration item is the threshold to trigger a flush.
- Introduced in: -
load_process_max_memory_limit_bytes
- Default: 107374182400
- Type: Int
- Unit: Bytes
- Is mutable: No
- Description: The maximum size limit of memory resources that can be taken up by all load processes on a BE node.
- Introduced in: -
txn_commit_rpc_timeout_ms (Deprecated)
- Default: 60000
- Type: Int
- Unit: Milliseconds
- Is mutable: Yes
- Description: The timeout for a transaction commit RPC. Since v3.2.0, this parameter is deprecated.
- Introduced in: -
max_consumer_num_per_group
- Default: 3
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of consumers in a consumer group of Routine Load.
- Introduced in: -
flush_thread_num_per_store
- Default: 2
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: Number of threads that are used for flushing MemTable in each store.
- Introduced in: -
lake_flush_thread_num_per_store
- Default: 0
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: Number of threads that are used for flushing MemTable in each store in shared-data mode.
When this value is set to
0
, the system uses twice of the CPU core count as the value. When this value is set to less than0
, the system uses the product of its absolute value and the CPU core count as the value. - Introduced in: 3.1.12, 3.2.7
max_runnings_transactions_per_txn_map
- Default: 100
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of transactions that can run concurrently in each partition.
- Introduced in: -
enable_stream_load_verbose_log
- Default: false
- Type: Boolean
- Unit: -
- Is mutable: Yes
- Description: Specifies whether to log the HTTP requests and responses for Stream Load jobs.
- Introduced in: v2.5.17, v3.0.9, v3.1.6, v3.2.1
Query engine
scanner_thread_pool_thread_num
- Default: 48
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The number of threads which the storage engine used for concurrent storage volume scanning. All threads are managed in the thread pool.
- Introduced in: -
scanner_thread_pool_queue_size
- Default: 102400
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of scan tasks supported by the storage engine.
- Introduced in: -
scanner_row_num
- Default: 16384
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum row count returned by each scan thread in a scan.
- Introduced in: -
max_scan_key_num
- Default: 1024
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of scan keys segmented by each query.
- Introduced in: -
max_pushdown_conditions_per_column
- Default: 1024
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of conditions that allow pushdown in each column. If the number of conditions exceeds this limit, the predicates are not pushed down to the storage layer.
- Introduced in: -
exchg_node_buffer_size_bytes
- Default: 10485760
- Type: Int
- Unit: Bytes
- Is mutable: Yes
- Description: The maximum buffer size on the receiver end of an exchange node for each query. This configuration item is a soft limit. A backpressure is triggered when data is sent to the receiver end with an excessive speed.
- Introduced in: -
file_descriptor_cache_capacity
- Default: 16384
- Type: Int
- Unit: -
- Is mutable: No
- Description: The number of file descriptors that can be cached.
- Introduced in: -
min_file_descriptor_number
- Default: 60000
- Type: Int
- Unit: -
- Is mutable: No
- Description: The minimum number of file descriptors in the BE process.
- Introduced in: -
index_stream_cache_capacity
- Default: 10737418240
- Type: Int
- Unit: Bytes
- Is mutable: No
- Description: The cache capacity for the statistical information of BloomFilter, Min, and Max.
- Introduced in: -
storage_page_cache_limit
- Default: 20%
- Type: String
- Unit: -
- Is mutable: Yes
- Description: The PageCache size. It can be specified as size, for example,
20G
,20,480M
,20,971,520K
, or21,474,836,480B
. It can also be specified as the ratio (percentage) to the memory size, for example,20%
. It takes effect only whendisable_storage_page_cache
is set tofalse
. - Introduced in: -
disable_storage_page_cache
- Default: false
- Type: Boolean
- Unit: -
- Is mutable: Yes
- Description: A boolean value to control whether to disable PageCache.
- When PageCache is enabled, StarRocks caches the recently scanned data.
- PageCache can significantly improve the query performance when similar queries are repeated frequently.
true
indicates disabling PageCache.- The default value of this item has been changed from
true
tofalse
since StarRocks v2.4.
- Introduced in: -
fragment_pool_thread_num_min
- Default: 64
- Type: Int
- Unit: Minutes -
- Is mutable: No
- Description: The minimum number of threads used for query.
- Introduced in: -
fragment_pool_thread_num_max
- Default: 4096
- Type: Int
- Unit: -
- Is mutable: No
- Description: The maximum number of threads used for query.
- Introduced in: -
fragment_pool_queue_size
- Default: 2048
- Type: Int
- Unit: -
- Is mutable: No
- Description: The upper limit of the query number that can be processed on each BE node.
- Introduced in: -
enable_prefetch
- Default: true
- Type: Boolean
- Unit: -
- Is mutable: Yes
- Description: A boolean value to control whether to enable the pre-fetch of the query.
true
indicates enabling pre-fetch, andfalse
indicates disabling it. - Introduced in: -
query_pool_spill_mem_limit_threshold
- Default: 1.0
- Type: Double
- Unit: -
- Is mutable: No
- Description: If automatic spilling is enabled, when the memory usage of all queries exceeds
query_pool memory limit * query_pool_spill_mem_limit_threshold
, intermediate result spilling will be triggered. - Introduced in: v3.2.7
result_buffer_cancelled_interval_time
- Default: 300
- Type: Int
- Unit: Seconds
- Is mutable: Yes
- Description: The wait time before BufferControlBlock releases data.
- Introduced in: -
max_memory_sink_batch_count
- Default: 20
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of Scan Cache batches.
- Introduced in: -
scan_context_gc_interval_min
- Default: 5
- Type: Int
- Unit: Minutes
- Is mutable: Yes
- Description: The time interval at which to clean the Scan Context.
- Introduced in: -
path_gc_check_step
- Default: 1000
- Type: Int
- Unit: -
- Is mutable: Yes
- Description: The maximum number of files that can be scanned continuously each time.
- Introduced in: -
path_gc_check_step_interval_ms
- Default: 10
- Type: Int
- Unit: Milliseconds
- Is mutable: Yes
- Description: The time interval between file scans.
- Introduced in: -