RocketMQ 高可用集群部署教程

RocketMQ 高可用集群部署教程

引用

CSDN

1.

https://blog.csdn.net/radapp/article/details/143032764

Apache RocketMQ 自诞生以来，因其架构简单、业务功能丰富、具备极强可扩展性等特点被众多企业开发者以及云厂商广泛采用。历经十余年的大规模场景打磨，RocketMQ 已经成为业内共识的金融级可靠业务消息首选方案，被广泛应用于互联网、大数据、移动互联网、物联网等领域的业务场景。

本文主要介绍RocketMQ的安装过程。

主机规划

准备 3 台主机，角色和相应端口分配如下。

安装JDK

这个根据大家的喜好来，可以直接通过 yum 之类的工具或者下载相关的 JDK 发行版即可，我前面的文章介绍过几次了，大家可以往前翻一翻（说不定有惊喜哟）。

我这里直接 yum 安装 openjdk 11 了

yum install -y java-11-openjdk

安装RocketMQ

下载解压

下载并解压至 /usr/local/ 目录，我这里以 4.5.1 旧版本为例，大家也可以更换为较新的版本

cd /usr/local/
wget https://archive.apache.org/dist/rocketmq/4.5.1/rocketmq-all-4.5.1-bin-release.zip
# 安装zip解压工具
yum install -y unzip
##解压到指定目录
unzip rocketmq-all-4.5.1-bin-release.zip

配置文件修改

修改 JVM 参数（可选）

##进入脚本目录
cd /usr/local/rocketmq-all-4.5.1-bin-release/bin

##修改启动脚本（可选）
#修改broker
vim runbroker.sh
JAVA_OPT="${JAVA_OPT} -server -Xms8g -Xmx8g -Xmn4g"
#修改nameserver
vim runserver.sh
JAVA_OPT="${JAVA_OPT} -server -Xms4g -Xmx4g -Xmn2g -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"

创建相关的目录（根据自己的规划调整），我们这里准备的是 3 master 架构

##创建所需目录
mkdir /usr/local/rocketmq-all-4.5.1-bin-release/data
mkdir /usr/local/rocketmq-all-4.5.1-bin-release/logs
mkdir -p /data/rocketmq/data
mkdir -p /data/rocketmq/commitlog
mkdir -p /data/rocketmq/consumequeue
mkdir -p /data/rocketmq/index
mkdir -p /data/rocketmq/checkpoint
mkdir -p /data/rocketmq/abort
mkdir -p /usr/local/rocketmq-all-4.5.1-bin-release/conf/3m-noslave

##修改日志打印路径
cd /usr/local/rocketmq-all-4.5.1-bin-release/conf
##替换 ${user.home} 到安装目录
sed -i 's#${user.home}#/usr/local/rocketmq-all-4.5.1-bin-release#g' /usr/local/rocketmq-all-4.5.1-bin-release/conf/*.xml

准备 broker 配置文件，分别在3台机器上准备一个配置文件：
/usr/local/rocketmq-all-4.5.1-bin-release/conf/3m-noslave/broker.properties

注意修改几个配置：

• namesrvAddr, 集群地址，分号分隔多台
• brokerClusterName, 全局唯一，别和其他集群冲突
• brokerName, 按1-3排列
• brokerIP1, 本机IP
• 相关路径配置成我们创建的路径

不同主机的 brokerName, brokerId, brokerIP1 3 处配置项不同，其余是相同的

# 10.70.3.1 broker.properties
namesrvAddr=10.70.3.1:9876;10.70.3.251:9876;10.70.3.252:9876
brokerClusterName=prod-ack-tang-xdf3-cluster-240620
brokerName=broker-1
brokerId=0
deleteWhen=04
fileReservedTime=12
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
maxMessageSize=67108864
brokerIP1=10.70.3.1
storePathRootDir=/data/rocketmq/data
storePathCommitLog=/data/rocketmq/commitlog
storePathConsumeQueue=/data/rocketmq/consumequeue
storePathIndex=/data/rocketmq/index
storeCheckpoint=/data/rocketmq/checkpoint
abortFile=/data/rocketmq/abort
sendMessageThreadPoolNums=24
useReentrantLockWhenPutMessage=true
waitTimeMillsInSendQueue=600
osPageCacheBusyTimeOutMills=5000

# 10.70.3.251 broker.properties
namesrvAddr=10.70.3.1:9876;10.70.3.251:9876;10.70.3.252:9876
brokerClusterName=prod-ack-tang-xdf3-cluster-240620
brokerName=broker-2
brokerId=0
deleteWhen=04
fileReservedTime=12
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
maxMessageSize=67108864
brokerIP1=10.70.3.251
storePathRootDir=/data/rocketmq/data
storePathCommitLog=/data/rocketmq/commitlog
storePathConsumeQueue=/data/rocketmq/consumequeue
storePathIndex=/data/rocketmq/index
storeCheckpoint=/data/rocketmq/checkpoint
abortFile=/data/rocketmq/abort
sendMessageThreadPoolNums=24
useReentrantLockWhenPutMessage=true
waitTimeMillsInSendQueue=600
osPageCacheBusyTimeOutMills=5000

# 10.70.3.252 broker.properties
namesrvAddr=10.70.3.1:9876;10.70.3.251:9876;10.70.3.252:9876
brokerClusterName=prod-ack-tang-xdf3-cluster-240620
brokerName=broker-3
brokerId=0
deleteWhen=04
fileReservedTime=12
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
maxMessageSize=67108864
brokerIP1=10.70.3.252
storePathRootDir=/data/rocketmq/data
storePathCommitLog=/data/rocketmq/commitlog
storePathConsumeQueue=/data/rocketmq/consumequeue
storePathIndex=/data/rocketmq/index
storeCheckpoint=/data/rocketmq/checkpoint
abortFile=/data/rocketmq/abort
sendMessageThreadPoolNums=24
useReentrantLockWhenPutMessage=true
waitTimeMillsInSendQueue=600
osPageCacheBusyTimeOutMills=5000

检查配置

cat /usr/local/rocketmq-all-4.5.1-bin-release/conf/3m-noslave/*.properties | grep -E 'namesrvAddr|brokerClusterName|brokerName|brokerIP1'
# broker-1
namesrvAddr=10.70.3.1:9876;10.70.3.251:9876;10.70.3.252:9876
brokerClusterName=prod-ack-tang-xdf3-cluster-240620
brokerName=broker-1
brokerIP1=10.70.3.1
# broker-2
namesrvAddr=10.70.3.1:9876;10.70.3.251:9876;10.70.3.252:9876
brokerClusterName=prod-ack-tang-xdf3-cluster-240620
brokerName=broker-2
brokerIP1=10.70.3.251
# broker-3
namesrvAddr=10.70.3.1:9876;10.70.3.251:9876;10.70.3.252:9876
brokerClusterName=prod-ack-tang-xdf3-cluster-240620
brokerName=broker-3
brokerIP1=10.70.3.252

启动服务

配置服务自启：

/etc/init.d/rocketmq

#!/bin/sh
#
# rocketmq - this script starts and stops the rocketmq daemon
#
# chkconfig: - 85 15
ROCKETMQ_HOME=/usr/local/rocketmq-all-4.5.1-bin-release
ROCKETMQ_BIN=${ROCKETMQ_HOME}/bin
LOCAL_IP=$(ip a |grep inet |awk '{print $2}'|grep ^1[0,7,9] |awk -F "/" '{print $1}' |head -n 1)
BROKER_IP=${LOCAL_IP}:9876
LOG_DIR=${ROCKETMQ_HOME}/logs
start() {
    if [ ! -d ${LOG_DIR} ];then
     mkdir ${LOG_DIR}
    fi
    cd ${ROCKETMQ_HOME}
    bash -x bin/os.sh
    nohup sh bin/mqnamesrv &
    echo -n "The Name Server boot success..."
    nohup sh bin/mqbroker -c ./conf/3m-noslave/broker.properties &
    echo -n "The broker[%s, ${BROKER_IP}] boot success..."
}
stop() {
    cd ${ROCKETMQ_HOME}
    sh bin/mqshutdown broker
    sleep 3
    sh bin/mqshutdown namesrv
}
restart() {
    stop
    sleep 5
    start
}
case "$1" in
    start)
        start
    ;;
    stop)
        stop
    ;;
    restart)
        restart
    ;;
    *)
    echo $"Usage: $0 {start|stop|restart}"
    exit 2
esac

启动服务

chmod 755 /etc/init.d/rocketmq

/etc/init.d/rocketmq start

# 查看端口
ss -natlp|grep -E "9876|10911"
LISTEN     0      1024             *:9876                     *:*                     users:(("java",pid=14560,fd=61))
LISTEN     0      1024             *:10911                    *:*                     users:(("java",pid=14562,fd=97))

# 查看集群

cd /usr/local/rocketmq-all-4.5.1-bin-release/bin
./mqadmin clusterlist -n 10.70.3.1:9876
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
#Cluster Name     #Broker Name            #BID  #Addr                    #Version                #InTPS(LOAD)       #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE
prod-ack-tang-xdf3-cluster-240620  broker-1                0     10.70.3.1:10911        V4_5_1                  0.00(0,0ms)         0.00(0,0ms)          0 477466.03 0.0000
prod-ack-tang-xdf3-cluster-240620  broker-2                0     10.70.3.251:10911     V4_5_1                  0.00(0,0ms)         0.00(0,0ms)          0 477466.03 0.0000
prod-ack-tang-xdf3-cluster-240620  broker-3                0     10.70.3.252:10911     V4_5_1                  0.00(0,0ms)         0.00(0,0ms)          0 477466.03 0.0000

开机自启

开机启动也可以使用其他方式进行管理，根据个人喜欢

# 增加service服务配置
chkconfig --add rocketmq
chkconfig rocketmq on
chkconfig --list

# 操作命令
service rocketmq status
service rocketmq stop
service rocketmq start

END

好了，今天的分享就到这里，感谢大家的阅读，如果有任何异议的地方，欢迎指正。