5.2 KahaDB消息存储-The KahaDB message store
The recommended message store for general-purpose messages since ActiveMQ version
5.3 is KahaDB. This is a file-based message store that combines a transactional journal,
for reliable message storage and recovery, with good performance and scalability.
ActiveMQ从5.3版本以后,推荐使用KahaDB作为通用的消息存储方式.KahaDB是一种基于文件的存储,具有
卓越的性能和扩展性,具有事务日志功能,能够可靠的今昔消息的存储和恢复.
The KahaDB store is a file-based, transactional store that’s been tuned and designed
for the fast storage of messages. The aim of the KahaDB store is to be easy to use and as
fast as possible. Its use of a file-based message database means there’s no prerequisite
for a third-party database. This message store enables ActiveMQ to be downloaded and
running in literally minutes. In addition, the structure of the KahaDB store has been
streamlined especially for the requirements of a message broker.
KahaDB存储是一种基于文件,具有事务存储功能,为消息的快速存储而设计并做了优化.
KahaDB存储的目标是易于使用并且做到尽可能的快速.使用基于文件的消息数据库表明
KahaDB不需要使用第三方数据库.这种消息存储使得能在数分钟之内完成ActiveMQ的下载和运行.
并且,为满足消息代理的需求,KahaDB的存储结构已经做了特殊的优化.
The KahaDB message store uses a transactional log for its indexes and only uses
one index file for all its destinations. It’s been used in production environments with
10,000 active connections, each connection having a separate queue. The configurability
of the KahaDB store means that it can be tuned for most usage scenarios, from
high throughput applications (for example, trading platforms), to storing large
amounts of messages (for example, GPS tracking).
KahaDB消息存储使用一个事物日志文件作为索引,并且所有的destination只使用一个索引文件.
KahaDB已可用于这样的生产环境:具有10000个活动连接,并且每一个连接拥有一个独立的队列.
KahaDB的可配置性说明,KahaDB可以针对从要求高吞吐量(比如交易平台)到处理海量消息(比如GPS跟踪)
等不同的应用场景进行调优.
To enable the KahaDB store for ActiveMQ, you need to configure the
<persistenceAdapter> element in the activemq.xml configuration file. Here’s a minimal
configuration for the KahaDB message store:
为了在ActiveMQ启用KahaDB存储,你需要在配置文件activemq.xml中配置<persistenceAdapter>
元素.下面是启用KahaDB存储的一个最小化配置例子:
<broker brokerName=”broker” persistent=”true” useShutdownHook=”false”>
…
<persistenceAdapter>
<kahaDB directory=”activemq-data” journalMaxFileLength=”16mb”/>
</persistenceAdapter>
…
</broker>
If you want to embed an ActiveMQ broker inside an application, the message store can
also be configured programmatically. Here’s an example of a programmatic configuration
for KahaDB:
如果你打算在应用程序内部启动一个嵌入式的ActiveMQ代理,也可以使用程序代码来配置消息存储.
下面是一个使用程序代码配置KahaDB的例子:
public class EmbeddedBrokerUsingAMQStoreExample
{
BrokerService createEmbeddedBroker() throws Exception
{
BrokerService broker = new BrokerService();
File dataFileDir = new File(“target/amq-in-action/kahadb”);
KahaDBStore kaha = new KahaDBStore();
kaha.setDirectory(dataFileDir);
// Using a bigger journal file
kaha.setJournalMaxFileLength(1024*100);
// small batch means more frequent and smaller writes
kaha.setIndexWriteBatchSize(100);
// do the index write in a separate thread
kaha.setEnableIndexWriteAsync(true);
broker.setPersistenceAdapter(kaha);
//create a transport connector
broker.addConnector(“tcp://localhost:61616”);
//start the broker
broker.start();
return broker;
}
}
Although the example seems small, it’s enough to create an ActiveMQ broker using
the KahaDB message store and listen for ActiveMQ clients connecting over TCP. For
more information about embedding ActiveMQ, see chapter 8.
尽管上面的实例代码看起来比较小巧,但是代码已足够创建一个ActiveMQ代理并使用KahaDB作为
消息存储同时监听通过TCP协议连接的ActiveMQ客户端.更多关于嵌入式ActiveMQ的内容请参阅
第8章内容.
In order to better understand its use and configuration, it’s important to examine
the internals of the KahaDB message store.
为了更好的理解和配置KahaDB消息存储,能深入KahaDB消息存储的内部一探究竟变得十分
重要.
5.2.1 The KahaDB message store internals
5.2.1 KahaDB消息存储内部探秘
The KahaDB message store is the fastest of all the provided message store implementations.
Its speed is the result of the combination of a fast transactional journal comprised
of data log files, the highly optimized indexing of message IDs, and inmemory
message caching. Figure 5.3 provides a high-level diagram of the KahaDB
message store.
在所有以提供的消息存储实现中,KahaDB是最快的.KahaDB的高速度源自一种包含数据日志文件的
高速事务日志文件和根据消息ID高度优化的索引以及基于内存的消息缓存.图5.3显示了KahaDB
消息存储概览.
The diagram provides a view of the three distinct parts of the KahaDB message
store including the following:
图5.3展示了KahaDB消息存储的三个独立的构件如下所示:
The data logs act as a message journal, which consists of a rolling
log of messages and commands (such as transactional boundaries and message deletions)
stored in data files of a certain length. When the maximum
length of the currently used data file has been reached,
a new data file is created. All the messages in a data file are reference
counted, so that once every message in that data file is no longer required,
the data file can be removed or archived. In the data logs, messages are only
appended to the end of the current data file, so storage is fast.
数据日志类似消息日记,用固定长度的文件保存滚动消息和命令(比如事务边界和消息删除指令).
当数据日志文件的大小达到最大值时,一个新的数据文件会被自动创建.日志文件中的所有消息
都会被引用计数,这样一旦文件中所有的消息都不在需要了,这个数据文件会被删除或者归档.
在数据日志中,消息只会被附加到当前文件的尾部,所以存储非常快.
The cache holds messages temporarily if there are active consumer(s) for the messages.
If there are active consumers, messages are dispatched at the same time they’re
scheduled to be stored. If messages are acknowledged in time, they don’t need
to be written to disk.
当有活动的消息消费者时,消息会临时放到缓存中.如果有活动的消费者,消息在被存储同时会被分发.
如果消息被处理并及时确认过,就没有必要将消息写入磁盘了.
The BTree indexes hold references to the messages in the data logs that are
indexed by their message ID. The indexes maintain the FIFO data structure for
queues and the durable subscriber pointers to their topic messages. The redo
log is used only if the ActiveMQ broker hasn’t shut down cleanly, and are used
to insure the integrity of the BTree index is maintained.
BTree索引文件保存数据日志文件中的消息引用,这些消息已根据消息ID建立了索引.消息索引
为消息队列维护一个先进先出的数据结构,同时维护由持久的消息订阅者指向其订阅的的主题中消息
的指针.重做日志文件仅用当ActiveMQ没有正常关闭时,保证BTree索引仍然完整.
The KahaDB uses different files on disk for its data logs and indexes, so in the next
section we’ll show a typical KahaDB directory structure.
KahaDB为数据日志和索引使用不同的磁盘文件,所以在下面的章节中我们将看到典型的KahaDB
目录结构.
5.2.2 The KahaDB message store directory structure
5.2.2 KahaDB消息存储的目录结构
When you start an ActiveMQ broker configured to use a KahaDB store, a directory will
automatically be created in which the persistent messages are stored. This directory
structure is shown in figure 5.4.
当你启动了配置KahaDB存储的ActiveMQ代理是,消息存储持久化文件目录会自动创建,该目录结构
如图5.4所示:
Inside of the KahaDB directory, the following directory and file structures can be
found:
KahaDB的目录结构中的目录和文件如下:
db log files—KahaDB stores messages into data log files named db-<Number>.log of
a predefined size. When a data log is full, a new one will be created, and the log
number incremented. When there are no more references to any of the messages
in the data log file, it’ll be deleted or archived.
db log 文件 –KahaDB将消息存储到数据日志文件中,该文件命名方式为db-<编号>.log,文件大小事先已经定义好.
当数据日志文件达到预定的大小,一个新的文件会自动创建,同时文件名中的编号自动加1.如果没有任何引用指向
数据日志文件中的消息,那么该日志文件将被删除或者归档.
archive directory—This exists only if archiving is enabled. The archive is used to store data logs
that are no longer needed by KahaDB, making it possible to replay messages from the archived data
logs at a later point. If archiving isn’t enabled(the default), data logs that are no longer in
use are deleted from the file system.
归档目录 — 归档目录仅在启用归档时才会出现.归档文件用于存储那些不再被KahaDB使用的数据日志文件.
使用归档文件可以保存消息以便以后重新检视消息变为可能.如果没有启用归档功能(默认不启用归档),不在使用的数据
日志文件会从文件系统中删除.
db.data —This file contains the persistent BTree indexes to the messages held in the message data logs.
db.data — 该文件保存了数据日志文件中消息的持久化BTree索引.
db.redo —This is the redo file, used for recovering the BTree indexes if the KahaDB
message store starts after a hard stop.
db.redo — 这是重做日志文件,用于KahaDB从一次非正常关闭后重启时恢复BTree索引.
Now that we’ve covered the basics of the KahaDB store, the next step is to review its
configuration.
现在我们已经了解了KahaDB的基础知识,下一步让我们回顾下KahaDB的配置.
5.2.3 Configuring the KahaDB message store
5.2.3 配置KahaDB消息存储
The KahaDB message store can be configured in the activemq.xml file. Its configuration
options control the different tuning parameters, as described in table 5.1.
可以在activemq.xml文件中配置KahaDB消息存储.不同的配置选择控制控制不同的功能参数,如表
5.1所示:
Table 5.1 Configuration options available for the KahaDB message store
表5.1 KahaDB消息存储的配置选项
Property name Default value Description
属性名 默认值 描述
directory activemq-data Directory path used by KahaDB
directory activemq-data KahaDB使用的目录
indexWriteBatchSize 1000 Number of index pages to write in a batch to disk
indexWriteBatchSize 1000 一次写入磁盘的属性页数量
indexCacheSize 10000 Number of index pages cached in memory
indexCacheSize 10000 内存中缓存的所引用数量
enableIndexWriteAsync false If set, will asynchronously write indexes
enableIndexWriteAsync false 如果设置为true,索引将以异步方式写入
journalMaxFileLength 32mb A hint to set the maximum size of each of the message data logs
journalMaxFileLength 32mb 设置每个消息数据日志文件的最大尺寸
enableJournalDiskSyncs true Ensures every nontransactional journal write is followed
by a disk sync (JMS durability requirement)
enableJournalDiskSyncs true 保证每个非事务日志写操作后进行磁盘同步(JMS持久化要求)
cleanupInterval 30000 Time (ms) before checking for and discarding/moving message
data logs that are no longer used
cleanupInterval 30000 检查 丢弃/移动数据日志文件中是不不再使用的消息 间隔时间(单位 毫秒)
checkpointInterval 5000 Time (ms) before checkpointing the journal
checkpointInterval 5000 执行checkpoint的时间间隔
ignoreMissingJournalfiles false If enabled, will ignore a missing message log file
ignoreMissingJournalfiles false 如果启用,则忽视丢失消息日志(译注:在消息丢失时不记录日志)
checkForCorruptJournalFiles false If enabled, on startup will validate that the message data logs haven’t been corrupted.
checkForCorruptJournalFiles false 如果启用,在启动是会检查消息数据日志文件是否被占用
checksumJournalFiles false If enabled, will provide a checksum foreach message data log
checksumJournalFiles false 如果启用,将为每一个消息数据日志问价开启检查消息总数服务
archiveDataLogs false If enabled, will move a message data log to the archive directory instead of deleting it
archiveDataLogs false 如果启用,不用的消息数据文件会被移动到归档目录 而不是删除掉
directoryArchive null Defines the directory to move data logs to when all the messages they contain have been consumed
directoryArchive null 存放规定文件的目录
databaseLockedWaitDelay 10000 Time (ms) before trying to acquire the database lock (used by shared master/slave)
databaseLockedWaitDelay 10000 尝试获取数据库锁的等待时间(用于共享主/从数据库)
maxAsyncJobs 10000 Maximum number of asynchronous messages that will be queued awaiting
storage (should be the same as the number of concurrent MessageProducers)
maxAsyncJobs 10000 排队等待存储的异步消息的最大数量(应该和并发的消息生产者数量相同)
concurrentStoreAndDispatchTransactions true Enables the dispatching of messages to interested clients to happen concurrently with transaction storage
concurrentStoreAndDispatchTransactions true 允许在存储事务时分发消息到对其感兴趣的客户端
concurrentStoreAndDispatchTopics true Enables the dispatching of topic messages to interested clients to happen concurrently with message storage
concurrentStoreAndDispatchTopics true 允许在存储消息时,分发主题中的消息给对其感兴趣的客户端
concurrentStoreAndDispatchQueues true Enables the dispatching of queue messages to interested clients to happen concurrently with message storage
concurrentStoreAndDispatchQueues true 允许在存储消息是,分发队列中的消息给对其感兴趣的客户端
ActiveMQ provides a pluggable API for message stores, and there are three additional
implementations to KahaDB that are shipped with ActiveMQ:
The AMQ message store—A file-based message store designed for performance
The JDBC message store—A message store based on JDBC
The Memory message store—A memory-based message store
ActiveMQ为消息存储提供了一种可插拔式的API.ActiveMQ版本中包含一下三种KahaDB消息存储实现:
AMQ 消息存储 — 基于文件的高性能消息存储
JDBC 消息存储 — 基于JDBC的消息存储
Memory 消息存储 — 基于内存的消息存储
We’ll look at the use cases and configuration for these additional message stores in the
next three sections. We’ll start with the AMQ message store, which like the KahaDB
message store is a file-based implementation. It predates KahaDB, but because of its
performance characteristics, it can make sense to use the AMQ store instead of
KahaDB, provided the number of persistent destinations is relatively low.
在接下来的三个章节中,我们将看到这三个消息存储的应用实例和配置方法.我们将从AMQ消息存储开始,
AMQ消息存储和KahaDB类似是基于文件的存储实现,但鉴于改存储方式的高性能特性,可以使用AMQ消息存储
替代KahaDB,但AMQ消息存储可提供的持久化目的地的数量比较少.
赞 赏微信赞赏 支付宝赞赏
本文固定链接: https://www.jack-yin.com/coding/translation/activemq-in-action/764.html | 边城网事