rocketmq普通訊息樣例、順序訊息樣例、延時訊息樣例、批次訊息樣例、過濾訊息樣例、事務訊息樣例

2020-08-13 09:35:04

樣例xiao


1 基本樣例

在基本樣例中我們提供如下的功能場景:

  • 使用RocketMQ發送三種類型的訊息:同步訊息、非同步訊息和單向訊息。其中前兩種訊息是可靠的,因爲會有發送是否成功的應答。
  • 使用RocketMQ來消費接收到的訊息。

1.1 加入依賴:

maven:

<dependency>
    <groupId>org.apache.rocketmq</groupId>
    <artifactId>rocketmq-client</artifactId>
    <version>4.3.0</version>
</dependency>

gradle

compile 'org.apache.rocketmq:rocketmq-client:4.3.0'

1.2 訊息發送

1、Producer端發送同步訊息

這種可靠性同步地發送方式使用的比較廣泛,比如:重要的訊息通知,簡訊通知。

public class SyncProducer {
	public static void main(String[] args) throws Exception {
    	// 範例化訊息生產者Producer
        DefaultMQProducer producer = new DefaultMQProducer("please_rename_unique_group_name");
    	// 設定NameServer的地址
    	producer.setNamesrvAddr("localhost:9876");
    	// 啓動Producer範例
        producer.start();
    	for (int i = 0; i < 100; i++) {
    	    // 建立訊息,並指定Topic,Tag和訊息體
    	    Message msg = new Message("TopicTest" /* Topic */,
        	"TagA" /* Tag */,
        	("Hello RocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET) /* Message body */
        	);
        	// 發送訊息到一個Broker
            SendResult sendResult = producer.send(msg);
            // 通過sendResult返回訊息是否成功送達
            System.out.printf("%s%n", sendResult);
    	}
    	// 如果不再發送訊息,關閉Producer範例。
    	producer.shutdown();
    }
}

2、發送非同步訊息

非同步訊息通常用在對響應時間敏感的業務場景,即發送端不能容忍長時間地等待Broker的響應。

public class AsyncProducer {
	public static void main(String[] args) throws Exception {
    	// 範例化訊息生產者Producer
        DefaultMQProducer producer = new DefaultMQProducer("please_rename_unique_group_name");
    	// 設定NameServer的地址
        producer.setNamesrvAddr("localhost:9876");
    	// 啓動Producer範例
        producer.start();
        producer.setRetryTimesWhenSendAsyncFailed(0);
	
	int messageCount = 100;
        // 根據訊息數量範例化倒計時計算器
	final CountDownLatch2 countDownLatch = new CountDownLatch2(messageCount);
    	for (int i = 0; i < messageCount; i++) {
                final int index = i;
            	// 建立訊息,並指定Topic,Tag和訊息體
                Message msg = new Message("TopicTest",
                    "TagA",
                    "OrderID188",
                    "Hello world".getBytes(RemotingHelper.DEFAULT_CHARSET));
                // SendCallback接收非同步返回結果的回撥
                producer.send(msg, new SendCallback() {
                    @Override
                    public void onSuccess(SendResult sendResult) {
                        System.out.printf("%-10d OK %s %n", index,
                            sendResult.getMsgId());
                    }
                    @Override
                    public void onException(Throwable e) {
      	              System.out.printf("%-10d Exception %s %n", index, e);
      	              e.printStackTrace();
                    }
            	});
    	}
	// 等待5s
	countDownLatch.await(5, TimeUnit.SECONDS);
    	// 如果不再發送訊息,關閉Producer範例。
    	producer.shutdown();
    }
}

3、單向發送訊息

這種方式主要用在不特別關心發送結果的場景,例如日誌發送。

public class OnewayProducer {
	public static void main(String[] args) throws Exception{
    	// 範例化訊息生產者Producer
        DefaultMQProducer producer = new DefaultMQProducer("please_rename_unique_group_name");
    	// 設定NameServer的地址
        producer.setNamesrvAddr("localhost:9876");
    	// 啓動Producer範例
        producer.start();
    	for (int i = 0; i < 100; i++) {
        	// 建立訊息,並指定Topic,Tag和訊息體
        	Message msg = new Message("TopicTest" /* Topic */,
                "TagA" /* Tag */,
                ("Hello RocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET) /* Message body */
        	);
        	// 發送單向訊息,沒有任何返回結果
        	producer.sendOneway(msg);

    	}
    	// 如果不再發送訊息,關閉Producer範例。
    	producer.shutdown();
    }
}

1.3 消費訊息

public class Consumer {

	public static void main(String[] args) throws InterruptedException, MQClientException {

    	// 範例化消費者
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("please_rename_unique_group_name");

    	// 設定NameServer的地址
        consumer.setNamesrvAddr("localhost:9876");

    	// 訂閱一個或者多個Topic,以及Tag來過濾需要消費的訊息
        consumer.subscribe("TopicTest", "*");
    	// 註冊回撥實現類來處理從broker拉取回來的訊息
        consumer.registerMessageListener(new MessageListenerConcurrently() {
            @Override
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
                System.out.printf("%s Receive New Messages: %s %n", Thread.currentThread().getName(), msgs);
                // 標記該訊息已經被成功消費
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });
        // 啓動消費者範例
        consumer.start();
        System.out.printf("Consumer Started.%n");
	}
}

2 順序訊息樣例

訊息有序指的是可以按照訊息的發送順序來消費(FIFO)。RocketMQ可以嚴格的保證訊息有序,可以分爲分割區有序或者全域性有序。

順序消費的原理解析,在預設的情況下訊息發送會採取Round Robin輪詢方式把訊息發送到不同的queue(分割區佇列);而消費訊息的時候從多個queue上拉取訊息,這種情況發送和消費是不能保證順序。但是如果控制發送的順序訊息只依次發送到同一個queue中,消費的時候只從這個queue上依次拉取,則就保證了順序。當發送和消費參與的queue只有一個,則是全域性有序;如果多個queue參與,則爲分割區有序,即相對每個queue,訊息都是有序的。

下面 下麪用訂單進行分割區有序的範例。一個訂單的順序流程是:建立、付款、推播、完成。訂單號相同的訊息會被先後 先後發送到同一個佇列中,消費時,同一個OrderId獲取到的肯定是同一個佇列。

2.1 順序訊息生產

package org.apache.rocketmq.example.order2;

import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.MessageQueueSelector;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.common.message.MessageQueue;

import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;

/**
* Producer,發送順序訊息
*/
public class Producer {

   public static void main(String[] args) throws Exception {
       DefaultMQProducer producer = new DefaultMQProducer("please_rename_unique_group_name");

       producer.setNamesrvAddr("127.0.0.1:9876");

       producer.start();

       String[] tags = new String[]{"TagA", "TagC", "TagD"};

       // 訂單列表
       List<OrderStep> orderList = new Producer().buildOrders();

       Date date = new Date();
       SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
       String dateStr = sdf.format(date);
       for (int i = 0; i < 10; i++) {
           // 加個時間字首
           String body = dateStr + " Hello RocketMQ " + orderList.get(i);
           Message msg = new Message("TopicTest", tags[i % tags.length], "KEY" + i, body.getBytes());

           SendResult sendResult = producer.send(msg, new MessageQueueSelector() {
               @Override
               public MessageQueue select(List<MessageQueue> mqs, Message msg, Object arg) {
                   Long id = (Long) arg;  //根據訂單id選擇發送queue
                   long index = id % mqs.size();
                   return mqs.get((int) index);
               }
           }, orderList.get(i).getOrderId());//訂單id

           System.out.println(String.format("SendResult status:%s, queueId:%d, body:%s",
               sendResult.getSendStatus(),
               sendResult.getMessageQueue().getQueueId(),
               body));
       }

       producer.shutdown();
   }

   /**
    * 訂單的步驟
    */
   private static class OrderStep {
       private long orderId;
       private String desc;

       public long getOrderId() {
           return orderId;
       }

       public void setOrderId(long orderId) {
           this.orderId = orderId;
       }

       public String getDesc() {
           return desc;
       }

       public void setDesc(String desc) {
           this.desc = desc;
       }

       @Override
       public String toString() {
           return "OrderStep{" +
               "orderId=" + orderId +
               ", desc='" + desc + '\'' +
               '}';
       }
   }

   /**
    * 生成模擬訂單數據
    */
   private List<OrderStep> buildOrders() {
       List<OrderStep> orderList = new ArrayList<OrderStep>();

       OrderStep orderDemo = new OrderStep();
       orderDemo.setOrderId(15103111039L);
       orderDemo.setDesc("建立");
       orderList.add(orderDemo);

       orderDemo = new OrderStep();
       orderDemo.setOrderId(15103111065L);
       orderDemo.setDesc("建立");
       orderList.add(orderDemo);

       orderDemo = new OrderStep();
       orderDemo.setOrderId(15103111039L);
       orderDemo.setDesc("付款");
       orderList.add(orderDemo);

       orderDemo = new OrderStep();
       orderDemo.setOrderId(15103117235L);
       orderDemo.setDesc("建立");
       orderList.add(orderDemo);

       orderDemo = new OrderStep();
       orderDemo.setOrderId(15103111065L);
       orderDemo.setDesc("付款");
       orderList.add(orderDemo);

       orderDemo = new OrderStep();
       orderDemo.setOrderId(15103117235L);
       orderDemo.setDesc("付款");
       orderList.add(orderDemo);

       orderDemo = new OrderStep();
       orderDemo.setOrderId(15103111065L);
       orderDemo.setDesc("完成");
       orderList.add(orderDemo);

       orderDemo = new OrderStep();
       orderDemo.setOrderId(15103111039L);
       orderDemo.setDesc("推播");
       orderList.add(orderDemo);

       orderDemo = new OrderStep();
       orderDemo.setOrderId(15103117235L);
       orderDemo.setDesc("完成");
       orderList.add(orderDemo);

       orderDemo = new OrderStep();
       orderDemo.setOrderId(15103111039L);
       orderDemo.setDesc("完成");
       orderList.add(orderDemo);

       return orderList;
   }
}

2.2 順序消費訊息

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.common.message.MessageExt;
import java.util.List;

package org.apache.rocketmq.example.order2;

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeOrderlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeOrderlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerOrderly;
import org.apache.rocketmq.common.consumer.ConsumeFromWhere;
import org.apache.rocketmq.common.message.MessageExt;

import java.util.List;
import java.util.Random;
import java.util.concurrent.TimeUnit;

/**
* 順序訊息消費,帶事務方式(應用可控制Offset什麼時候提交)
*/
public class ConsumerInOrder {

   public static void main(String[] args) throws Exception {
       DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("please_rename_unique_group_name_3");
       consumer.setNamesrvAddr("127.0.0.1:9876");
       /**
        * 設定Consumer第一次啓動是從佇列頭部開始消費還是佇列尾部開始消費<br>
        * 如果非第一次啓動,那麼按照上次消費的位置繼續消費
        */
       consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);

       consumer.subscribe("TopicTest", "TagA || TagC || TagD");

       consumer.registerMessageListener(new MessageListenerOrderly() {

           Random random = new Random();

           @Override
           public ConsumeOrderlyStatus consumeMessage(List<MessageExt> msgs, ConsumeOrderlyContext context) {
               context.setAutoCommit(true);
               for (MessageExt msg : msgs) {
                   // 可以看到每個queue有唯一的consume執行緒來消費, 訂單對每個queue(分割區)有序
                   System.out.println("consumeThread=" + Thread.currentThread().getName() + "queueId=" + msg.getQueueId() + ", content:" + new String(msg.getBody()));
               }

               try {
                   //模擬業務邏輯處理中...
                   TimeUnit.SECONDS.sleep(random.nextInt(10));
               } catch (Exception e) {
                   e.printStackTrace();
               }
               return ConsumeOrderlyStatus.SUCCESS;
           }
       });

       consumer.start();

       System.out.println("Consumer Started.");
   }
}

3 延時訊息樣例

3.1 啓動消費者等待傳入訂閱訊息

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.common.message.MessageExt;
import java.util.List;

public class ScheduledMessageConsumer {
   public static void main(String[] args) throws Exception {
      // 範例化消費者
      DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("ExampleConsumer");
      // 訂閱Topics
      consumer.subscribe("TestTopic", "*");
      // 註冊訊息監聽者
      consumer.registerMessageListener(new MessageListenerConcurrently() {
          @Override
          public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> messages, ConsumeConcurrentlyContext context) {
              for (MessageExt message : messages) {
                  // Print approximate delay time period
                  System.out.println("Receive message[msgId=" + message.getMsgId() + "] " + (System.currentTimeMillis() - message.getStoreTimestamp()) + "ms later");
              }
              return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
          }
      });
      // 啓動消費者
      consumer.start();
  }
}

3.2 發送延時訊息

import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.common.message.Message;

public class ScheduledMessageProducer {
   public static void main(String[] args) throws Exception {
      // 範例化一個生產者來產生延時訊息
      DefaultMQProducer producer = new DefaultMQProducer("ExampleProducerGroup");
      // 啓動生產者
      producer.start();
      int totalMessagesToSend = 100;
      for (int i = 0; i < totalMessagesToSend; i++) {
          Message message = new Message("TestTopic", ("Hello scheduled message " + i).getBytes());
          // 設定延時等級3,這個訊息將在10s之後發送(現在只支援固定的幾個時間,詳看delayTimeLevel)
          message.setDelayTimeLevel(3);
          // 發送訊息
          producer.send(message);
      }
       // 關閉生產者
      producer.shutdown();
  }
}

3.3 驗證

您將會看到訊息的消費比儲存時間晚10秒。

3.4 延時訊息的使用場景

比如電商裡,提交了一個訂單就可以發送一個延時訊息,1h後去檢查這個訂單的狀態,如果還是未付款就取消訂單釋放庫存。

3.5 延時訊息的使用限制

// org/apache/rocketmq/store/config/MessageStoreConfig.java

private String messageDelayLevel = "1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h";

現在RocketMq並不支援任意時間的延時,需要設定幾個固定的延時等級,從1s到2h分別對應着等級1到18 訊息消費失敗會進入延時訊息佇列,訊息發送時間與設定的延時等級和重試次數有關,詳見程式碼SendMessageProcessor.java

4 批次訊息樣例

批次發送訊息能顯著提高傳遞小訊息的效能。限制是這些批次訊息應該有相同的topic,相同的waitStoreMsgOK,而且不能是延時訊息。此外,這一批訊息的總大小不應超過4MB。

4.1 發送批次訊息

如果您每次只發送不超過4MB的訊息,則很容易使用批次處理,樣例如下:

String topic = "BatchTest";
List<Message> messages = new ArrayList<>();
messages.add(new Message(topic, "TagA", "OrderID001", "Hello world 0".getBytes()));
messages.add(new Message(topic, "TagA", "OrderID002", "Hello world 1".getBytes()));
messages.add(new Message(topic, "TagA", "OrderID003", "Hello world 2".getBytes()));
try {
   producer.send(messages);
} catch (Exception e) {
   e.printStackTrace();
   //處理error
}

4.2 訊息列表分割

複雜度只有當你發送大批次時纔會增長,你可能不確定它是否超過了大小限制(4MB)。這時候你最好把你的訊息列表分割一下:

public class ListSplitter implements Iterator<List<Message>> { 
    private final int SIZE_LIMIT = 1024 * 1024 * 4;
    private final List<Message> messages;
    private int currIndex;
    public ListSplitter(List<Message> messages) { 
        this.messages = messages;
    }
    @Override public boolean hasNext() {
        return currIndex < messages.size(); 
    }
    @Override public List<Message> next() { 
        int startIndex = getStartIndex();
        int nextIndex = startIndex;
        int totalSize = 0;
        for (; nextIndex < messages.size(); nextIndex++) {
            Message message = messages.get(nextIndex); 
            int tmpSize = calcMessageSize(message);
            if (tmpSize + totalSize > SIZE_LIMIT) {
                break; 
            } else {
                totalSize += tmpSize; 
            }
        }
        List<Message> subList = messages.subList(startIndex, nextIndex); 
        currIndex = nextIndex;
        return subList;
    }
    private int getStartIndex() {
        Message currMessage = messages.get(currIndex); 
        int tmpSize = calcMessageSize(currMessage); 
        while(tmpSize > SIZE_LIMIT) {
            currIndex += 1;
            Message message = messages.get(curIndex); 
            tmpSize = calcMessageSize(message);
        }
        return currIndex; 
    }
    private int calcMessageSize(Message message) {
        int tmpSize = message.getTopic().length() + message.getBody().length(); 
        Map<String, String> properties = message.getProperties();
        for (Map.Entry<String, String> entry : properties.entrySet()) {
            tmpSize += entry.getKey().length() + entry.getValue().length(); 
        }
        tmpSize = tmpSize + 20; // 增加⽇日誌的開銷20位元組
        return tmpSize; 
    }
}
//把大的訊息分裂成若幹個小的訊息
ListSplitter splitter = new ListSplitter(messages);
while (splitter.hasNext()) {
  try {
      List<Message>  listItem = splitter.next();
      producer.send(listItem);
  } catch (Exception e) {
      e.printStackTrace();
      //處理error
  }
}

5 過濾訊息樣例

在大多數情況下,TAG是一個簡單而有用的設計,其可以來選擇您想要的訊息。例如:

DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("CID_EXAMPLE");
consumer.subscribe("TOPIC", "TAGA || TAGB || TAGC");

消費者將接收包含TAGA或TAGB或TAGC的訊息。但是限制是一個訊息只能有一個標籤,這對於複雜的場景可能不起作用。在這種情況下,可以使用SQL表達式篩選訊息。SQL特性可以通過發送訊息時的屬性來進行計算。在RocketMQ定義的語法下,可以實現一些簡單的邏輯。下面 下麪是一個例子:

------------
| message  |
|----------|  a > 5 AND b = 'abc'
| a = 10   |  --------------------> Gotten
| b = 'abc'|
| c = true |
------------
------------
| message  |
|----------|   a > 5 AND b = 'abc'
| a = 1    |  --------------------> Missed
| b = 'abc'|
| c = true |
------------

5.1 基本語法

RocketMQ只定義了一些基本語法來支援這個特性。你也可以很容易地擴充套件它。

  • 數值比較,比如:>,>=,<,<=,BETWEEN,=;
  • 字元比較,比如:=,<>,IN;
  • IS NULL 或者 IS NOT NULL;
  • 邏輯符號 AND,OR,NOT;

常數支援型別爲:

  • 數值,比如:123,3.1415;
  • 字元,比如:'abc',必須用單引號包裹起來;
  • NULL,特殊的常數
  • 布爾值,TRUE 或 FALSE

只有使用push模式的消費者才能 纔能用使用SQL92標準的sql語句,介面如下:

public void subscribe(finalString topic, final MessageSelector messageSelector)

5.2 使用樣例

1、生產者樣例

發送訊息時,你能通過putUserProperty來設定訊息的屬性

DefaultMQProducer producer = new DefaultMQProducer("please_rename_unique_group_name");
producer.start();
Message msg = new Message("TopicTest",
   tag,
   ("Hello RocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET)
);
// 設定一些屬性
msg.putUserProperty("a", String.valueOf(i));
SendResult sendResult = producer.send(msg);

producer.shutdown();

2、消費者樣例

用MessageSelector.bySql來使用sql篩選訊息

DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("please_rename_unique_group_name_4");
// 只有訂閱的訊息有這個屬性a, a >=0 and a <= 3
consumer.subscribe("TopicTest", MessageSelector.bySql("a between 0 and 3");
consumer.registerMessageListener(new MessageListenerConcurrently() {
   @Override
   public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
       return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
   }
});
consumer.start();

6 訊息事務樣例

事務訊息共有三種狀態,提交狀態、回滾狀態、中間狀態:

  • TransactionStatus.CommitTransaction: 提交事務,它允許消費者消費此訊息。
  • TransactionStatus.RollbackTransaction: 回滾事務,它代表該訊息將被刪除,不允許被消費。
  • TransactionStatus.Unknown: 中間狀態,它代表需要檢查訊息佇列來確定狀態。

6.1 發送事務訊息樣例

1、建立事務性生產者

使用 TransactionMQProducer類建立生產者,並指定唯一的 ProducerGroup,就可以設定自定義執行緒池來處理這些檢查請求。執行本地事務後、需要根據執行結果對訊息佇列進行回覆 回復。回傳的事務狀態在請參考前一節。

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.common.message.MessageExt;
import java.util.List;
public class TransactionProducer {
   public static void main(String[] args) throws MQClientException, InterruptedException {
       TransactionListener transactionListener = new TransactionListenerImpl();
       TransactionMQProducer producer = new TransactionMQProducer("please_rename_unique_group_name");
       ExecutorService executorService = new ThreadPoolExecutor(2, 5, 100, TimeUnit.SECONDS, new ArrayBlockingQueue<Runnable>(2000), new ThreadFactory() {
           @Override
           public Thread newThread(Runnable r) {
               Thread thread = new Thread(r);
               thread.setName("client-transaction-msg-check-thread");
               return thread;
           }
       });
       producer.setExecutorService(executorService);
       producer.setTransactionListener(transactionListener);
       producer.start();
       String[] tags = new String[] {"TagA", "TagB", "TagC", "TagD", "TagE"};
       for (int i = 0; i < 10; i++) {
           try {
               Message msg =
                   new Message("TopicTest1234", tags[i % tags.length], "KEY" + i,
                       ("Hello RocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET));
               SendResult sendResult = producer.sendMessageInTransaction(msg, null);
               System.out.printf("%s%n", sendResult);
               Thread.sleep(10);
           } catch (MQClientException | UnsupportedEncodingException e) {
               e.printStackTrace();
           }
       }
       for (int i = 0; i < 100000; i++) {
           Thread.sleep(1000);
       }
       producer.shutdown();
   }
}

2、實現事務的監聽介面

當發送半訊息成功時,我們使用 executeLocalTransaction 方法來執行本地事務。它返回前一節中提到的三個事務狀態之一。checkLocalTransaction 方法用於檢查本地事務狀態,並回應訊息佇列的檢查請求。它也是返回前一節中提到的三個事務狀態之一。

public class TransactionListenerImpl implements TransactionListener {
  private AtomicInteger transactionIndex = new AtomicInteger(0);
  private ConcurrentHashMap<String, Integer> localTrans = new ConcurrentHashMap<>();
  @Override
  public LocalTransactionState executeLocalTransaction(Message msg, Object arg) {
      int value = transactionIndex.getAndIncrement();
      int status = value % 3;
      localTrans.put(msg.getTransactionId(), status);
      return LocalTransactionState.UNKNOW;
  }
  @Override
  public LocalTransactionState checkLocalTransaction(MessageExt msg) {
      Integer status = localTrans.get(msg.getTransactionId());
      if (null != status) {
          switch (status) {
              case 0:
                  return LocalTransactionState.UNKNOW;
              case 1:
                  return LocalTransactionState.COMMIT_MESSAGE;
              case 2:
                  return LocalTransactionState.ROLLBACK_MESSAGE;
          }
      }
      return LocalTransactionState.COMMIT_MESSAGE;
  }
}

6.2 事務訊息使用上的限制

  1. 事務訊息不支援延時訊息和批次訊息。
  2. 爲了避免單個訊息被檢查太多次而導致半佇列訊息累積,我們預設將單個訊息的檢查次數限製爲 15 次,但是使用者可以通過 Broker 組態檔的 transactionCheckMax參數來修改此限制。如果已經檢查某條訊息超過 N 次的話( N = transactionCheckMax ) 則 Broker 將丟棄此訊息,並在預設情況下同時列印錯誤日誌。使用者可以通過重寫 AbstractTransactionalMessageCheckListener 類來修改這個行爲。
  3. 事務訊息將在 Broker 組態檔中的參數 transactionTimeout 這樣的特定時間長度之後被檢查。當發送事務訊息時,使用者還可以通過設定使用者屬性 CHECK_IMMUNITY_TIME_IN_SECONDS 來改變這個限制,該參數優先於 transactionTimeout 參數。
  4. 事務性訊息可能不止一次被檢查或消費。
  5. 提交給使用者的目標主題訊息可能會失敗,目前這依日誌的記錄而定。它的高可用性通過 RocketMQ 本身的高可用性機制 機製來保證,如果希望確保事務訊息不丟失、並且事務完整性得到保證,建議使用同步的雙重寫入機制 機製。
  6. 事務訊息的生產者 ID 不能與其他型別訊息的生產者 ID 共用。與其他型別的訊息不同,事務訊息允許反向查詢、MQ伺服器能通過它們的生產者 ID 查詢到消費者。

7 Logappender樣例

RocketMQ日誌提供log4j、log4j2和logback日誌框架作爲業務應用,下面 下麪是設定樣例

7.1 log4j樣例

按下面 下麪樣例使用log4j屬性設定

log4j.appender.mq=org.apache.rocketmq.logappender.log4j.RocketmqLog4jAppender
log4j.appender.mq.Tag=yourTag
log4j.appender.mq.Topic=yourLogTopic
log4j.appender.mq.ProducerGroup=yourLogGroup
log4j.appender.mq.NameServerAddress=yourRocketmqNameserverAddress
log4j.appender.mq.layout=org.apache.log4j.PatternLayout
log4j.appender.mq.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-4r [%t] (%F:%L) %-5p - %m%n

按下面 下麪樣例使用log4j xml設定來使用非同步新增日誌

<appender name="mqAppender1"class="org.apache.rocketmq.logappender.log4j.RocketmqLog4jAppender">
  <param name="Tag" value="yourTag" />
  <param name="Topic" value="yourLogTopic" />
  <param name="ProducerGroup" value="yourLogGroup" />
  <param name="NameServerAddress" value="yourRocketmqNameserverAddress"/>
  <layout class="org.apache.log4j.PatternLayout">
      <param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss}-%p %t %c - %m%n" />
  </layout>
</appender>
<appender name="mqAsyncAppender1"class="org.apache.log4j.AsyncAppender">
  <param name="BufferSize" value="1024" />
  <param name="Blocking" value="false" />
  <appender-ref ref="mqAppender1"/>
</appender>

7.2 log4j2樣例

用log4j2時,設定如下,如果想要非阻塞,只需要使用非同步新增參照即可

<RocketMQ name="rocketmqAppender" producerGroup="yourLogGroup" nameServerAddress="yourRocketmqNameserverAddress"
   topic="yourLogTopic" tag="yourTag">
  <PatternLayout pattern="%d [%p] hahahah %c %m%n"/>
</RocketMQ>

7.3 logback樣例

<appender name="mqAppender1"class="org.apache.rocketmq.logappender.logback.RocketmqLogbackAppender">
  <tag>yourTag</tag>
  <topic>yourLogTopic</topic>
  <producerGroup>yourLogGroup</producerGroup>
  <nameServerAddress>yourRocketmqNameserverAddress</nameServerAddress>
  <layout>
      <pattern>%date %p %t - %m%n</pattern>
  </layout>
</appender>
<appender name="mqAsyncAppender1"class="ch.qos.logback.classic.AsyncAppender">
  <queueSize>1024</queueSize>
  <discardingThreshold>80</discardingThreshold>
  <maxFlushTime>2000</maxFlushTime>
  <neverBlock>true</neverBlock>
  <appender-ref ref="mqAppender1"/>
</appender>

8 OpenMessaging樣例

OpenMessaging旨在建立訊息和流處理規範,以爲金融、電子商務、物聯網和大數據領域提供通用框架及工業級指導方案。在分佈式異構環境中,設計原則是面向雲、簡單、靈活和獨立於語言。符合這些規範將幫助企業方便的開發跨平臺和操作系統的異構訊息傳遞應用程式。提供了openmessaging-api 0.3.0-alpha的部分實現,下面 下麪的範例演示如何基於OpenMessaging存取RocketMQ。

8.1 OMSProducer樣例

下面 下麪的範例演示如何在同步、非同步或單向傳輸中向RocketMQ代理髮送訊息。

import io.openmessaging.Future;
import io.openmessaging.FutureListener;
import io.openmessaging.Message;
import io.openmessaging.MessagingAccessPoint;
import io.openmessaging.OMS;
import io.openmessaging.producer.Producer;
import io.openmessaging.producer.SendResult;
import java.nio.charset.Charset;
import java.util.concurrent.CountDownLatch;

public class SimpleProducer {
    public static void main(String[] args) {
       final MessagingAccessPoint messagingAccessPoint =
           OMS.getMessagingAccessPoint("oms:rocketmq://localhost:9876/default:default");
       final Producer producer = messagingAccessPoint.createProducer();
       messagingAccessPoint.startup();
       System.out.printf("MessagingAccessPoint startup OK%n");
       producer.startup();
       System.out.printf("Producer startup OK%n");
       {
           Message message = producer.createBytesMessage("OMS_HELLO_TOPIC", "OMS_HELLO_BODY".getBytes(Charset.forName("UTF-8")));
           SendResult sendResult = producer.send(message);
           //final Void aVoid = result.get(3000L);
           System.out.printf("Send async message OK, msgId: %s%n", sendResult.messageId());
       }
       final CountDownLatch countDownLatch = new CountDownLatch(1);
       {
           final Future<SendResult> result = producer.sendAsync(producer.createBytesMessage("OMS_HELLO_TOPIC", "OMS_HELLO_BODY".getBytes(Charset.forName("UTF-8"))));
           result.addListener(new FutureListener<SendResult>() {
               @Override
               public void operationComplete(Future<SendResult> future) {
                   if (future.getThrowable() != null) {
                       System.out.printf("Send async message Failed, error: %s%n", future.getThrowable().getMessage());
                   } else {
                       System.out.printf("Send async message OK, msgId: %s%n", future.get().messageId());
                   }
                   countDownLatch.countDown();
               }
           });
       }
       {
           producer.sendOneway(producer.createBytesMessage("OMS_HELLO_TOPIC", "OMS_HELLO_BODY".getBytes(Charset.forName("UTF-8"))));
           System.out.printf("Send oneway message OK%n");
       }
       try {
           countDownLatch.await();
           Thread.sleep(500); // 等一些時間來發送訊息
       } catch (InterruptedException ignore) {
       }
       producer.shutdown();
   }
}

8.2 OMSPullConsumer

用OMS PullConsumer 來從指定的佇列中拉取訊息

import io.openmessaging.Message;
import io.openmessaging.MessagingAccessPoint;
import io.openmessaging.OMS;
import io.openmessaging.OMSBuiltinKeys;
import io.openmessaging.consumer.PullConsumer;
import io.openmessaging.producer.Producer;
import io.openmessaging.producer.SendResult;

public class SimplePullConsumer {
    public static void main(String[] args) {
       final MessagingAccessPoint messagingAccessPoint =
           OMS.getMessagingAccessPoint("oms:rocketmq://localhost:9876/default:default");
       messagingAccessPoint.startup();
       final Producer producer = messagingAccessPoint.createProducer();
       final PullConsumer consumer = messagingAccessPoint.createPullConsumer(
           OMS.newKeyValue().put(OMSBuiltinKeys.CONSUMER_ID, "OMS_CONSUMER"));
       messagingAccessPoint.startup();
       System.out.printf("MessagingAccessPoint startup OK%n");
       final String queueName = "TopicTest";
       producer.startup();
       Message msg = producer.createBytesMessage(queueName, "Hello Open Messaging".getBytes());
       SendResult sendResult = producer.send(msg);
       System.out.printf("Send Message OK. MsgId: %s%n", sendResult.messageId());
       producer.shutdown();
       consumer.attachQueue(queueName);
       consumer.startup();
       System.out.printf("Consumer startup OK%n");
       // 執行直到發現一個訊息被髮送了
       boolean stop = false;
       while (!stop) {
           Message message = consumer.receive();
           if (message != null) {
               String msgId = message.sysHeaders().getString(Message.BuiltinKeys.MESSAGE_ID);
               System.out.printf("Received one message: %s%n", msgId);
               consumer.ack(msgId);
               if (!stop) {
                   stop = msgId.equalsIgnoreCase(sendResult.messageId());
               }
           } else {
               System.out.printf("Return without any message%n");
           }
       }
       consumer.shutdown();
       messagingAccessPoint.shutdown();
   }
}

8.3 OMSPushConsumer

以下示範如何將 OMS PushConsumer 新增到指定的佇列,並通過 MessageListener 消費這些訊息。

import io.openmessaging.Message;
import io.openmessaging.MessagingAccessPoint;
import io.openmessaging.OMS;
import io.openmessaging.OMSBuiltinKeys;
import io.openmessaging.consumer.MessageListener;
import io.openmessaging.consumer.PushConsumer;

public class SimplePushConsumer {
    public static void main(String[] args) {
       final MessagingAccessPoint messagingAccessPoint = OMS
           .getMessagingAccessPoint("oms:rocketmq://localhost:9876/default:default");
       final PushConsumer consumer = messagingAccessPoint.
           createPushConsumer(OMS.newKeyValue().put(OMSBuiltinKeys.CONSUMER_ID, "OMS_CONSUMER"));
       messagingAccessPoint.startup();
       System.out.printf("MessagingAccessPoint startup OK%n");
       Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
           @Override
           public void run() {
               consumer.shutdown();
               messagingAccessPoint.shutdown();
           }
       }));
       consumer.attachQueue("OMS_HELLO_TOPIC", new MessageListener() {
           @Override
           public void onReceived(Message message, Context context) {
               System.out.printf("Received one message: %s%n", message.sysHeaders().getString(Message.BuiltinKeys.MESSAGE_ID));
               context.ack();
           }
       });
       consumer.startup();
       System.out.printf("Consumer startup OK%n");
   }
}
  • © 2020 GitHub, Inc.