5.3 AMQ消息存储-The AMQ message store
The AMQ message store, like KahaDB, is a combination of a transactional journal for
reliable persistence (to survive system crashes) and high-performance indexes, which
makes this store the best option when message throughput is the main requirement
for an application. But because it uses two separate files for every index, and there’s
an index per destination, the AMQ message store shouldn’t be used if you intend to
use thousands of queues per broker. Also, recovery can be slow if the ActiveMQ broker
isn’t shut down cleanly. This is because all the indexes need to be rebuilt, which
requires the broker to traverse all its data logs to accurately build the indexes again.
In the next section, we’ll briefly examine the internals of the AMQ message store,
which are similar to the components of KahaDB.
AMQ消息存储和KahaDB类似,具有保障可靠持久化的事务日志以及搞性能索引,因而当消息吞吐量是主要需求时,
应用程序使用这种消息存储是最好的选择.但是因为AMQ消息存储中每个索引都使用连个文件,而且为每一个
消息目的地都单独使用索引,所以当每个代理中会产生大量的消息队列是不应该使用AMQ消息存储.
同样,使用这种消息存储代理从非正常关闭中恢复会比较慢.原因是代理恢复时,所有的的索引需要重建,这就
要求代理遍历它的数据日志文件以便正确的重建索引.
5.3.1 The AMQ message store internals
5.3.1 深入AMQ消息存储
The main components of the AMQ message store are similar to that of the KahaDB
message store, in that there’s a cache, message data logs, and a reference store for
accessing the data logs in order. Figure 5.5 provides a high-level diagram of the AMQ
message store.
AMQ消息的存储的主要组成部分和KahaDB消息存储类似,包括缓存,以及用于按顺序存取消息数据日志文件
的引用.图5.5是AMQ消息存储机制图.
The diagram provides a view of the three distinct parts of the AMQ message store:
The data logs—These act as a message journal.
The cache —This holds messages for fast retrieval in memory after they’ve been written to the data logs.
The reference store —This holds references to the messages in the journal that are indexed by their message ID.
It’s important to understand the file-based directory structure used by the ActiveMQ
message store. This will help with the configuration and also with problem identification
when using ActiveMQ.
如图所示,AMQ消息存储包含一下三个独立部分:
数据日志文件 — 该文件保存消息引用,这些消息已根据消息ID建立了索引
缓存 — 在消息被写入数据日志文件后,缓存是在内存中保存这些消息以便快速读取这些消息
引用存储 — 引用存储是掉数据日志中消息的引用,这些消息已经根据消息ID建立了索引
5.3.2 The AMQ message store directory structure
5.3.2 AMQ消息存储的目录结构
When you start ActiveMQ with the AMQ message store configured, a directory will
automatically be created in which the persistent messages are held. The AMQ message
store directory contains subdirectories for all the brokers that are running on the
machine. For this reason it’s strongly recommended that each broker use a unique
name. In the default configuration for ActiveMQ, the broker name is localhost, which
needs to changed to something unique. This directory structure is represented in
figure 5.6—the AMQ store directory structure.
当你启动一个配置了AMQ的ActiveMQ代理,一个用于保存持久化消息的目录会被自动创建.
AMQ消息存储目录包含了一些子目录,每一个子目录对应一个在本机上运行的代理.因此,建议
给不同的代理取不同的名称.ActiveMQ的默认配置中,broker的name属性为localhost,这个需要
修改成一个唯一值.图5.6所示为AMQ消息存储的目录结构.
The following directories and files can be found inside the data directory of an
ActiveMQ broker:
下面的目录和文件可以在ActiveMQ代理的data目录中找到:
A lock file— Ensures that only one broker can access this data at any given time.
The lock is commonly used for hot standby purposes where more than one broker
with the same name will exist on the same system.
lock文件 — 保证任何时刻只有一个代理可以存取这个数据.lock通常用于双击人备份环境,此时系统中可能
存在名称相同的代理.
A temp-storage directory— Used for storing nonpersistent messages that can no
longer be stored in broker memory. These messages are typically awaiting delivery
to a slow consumer.
temp文件存储目录 — 用于存储不能在代理的内存中驻留的非持久化信息.这些消正在等待被发送到速度较慢的消息消费者.
The kr-store — The directory structure used by the reference (index) part of the
AMQ message store. It uses the Kaha reference store by default (Kaha is part of
the ActiveMQ core library) to index and store references to messages in the
data logs. There are two distinct parts to the kr-store:
* The data directory—Contains the indexes and collections used to reference
the messages held in the data logs. This data directory is deleted and rebuilt
as part of recovery, if the broker hasn’t shut down cleanly. You can force
recovery by manually deleting this directory before starting the broker.
* The state directory—Holds information about durable topic consumers. The
journal itself doesn’t hold information about consumers, so when it’s
recovered it has to retrieve information about the durable subscribers first
to accurately rebuild its database.
kr-store目录 — 改目录用于AMQ消息存储的引用(索引)部分.默认时使用Kaha引用存储(Kaha是ActiveMQ核心库的一部分),以建立数据
日志文件中消息索引的引用.kr-store包含一下两个独立部分:
The journal directory — Contains the data files for the data logs, and a data-control
file that holds some meta information. The data files are reference counted, so
when all the contained messages are delivered, a data file can be deleted or
archived.
数据日志目录 — 数据文件(保存数据日志)和数据控制文件(保存一些元数据信息).数据文件会被引用计数,因此当文件中包含的所有消息
都被分发了时,改数据文件会被删除或归档.
The archive directory — Exists only if archiving is enabled. Its default location can
be found next to the journal. It makes sense to use a separate partition or disk.
The archive is used to store data logs from the journal directory, which are
moved here instead of being deleted. This makes it possible to replay messages
from the archive at a later point. To replay messages, move the archived data logs
(or a subset) to a new journal directory and start a new broker pointed to the
location of this directory. It’ll automatically replay the data logs in the journal.
归档目录 — 只有启用归档时才产生归档目录.默认情况下,归档目录和数据日志目录同级(即和journal目录在一起).
归档目录应该放到其他系统分区或者磁盘上(不要与journal放在一起).归档文件用来保存journal中不再使用的
数据日志文件(如果不归档的话,这些数据日志文件本来应该会被删除).这样就可以在以后重播已经处理过的消息.
要重播消息,可以移动归档后的数据日志文件(或该文件的一部分)到一个新的journal目录,然后启动一个指向
这个新journal目录的代理,然后这个新启动的代理会自动重播这个journal中的消息.
Now that the basics of the AMQ message store have been covered, the next step is to
review its configuration.
MQ的基础内已经介绍完毕,接下来看看如何配置AMQ消息存储.
5.3.3 Configuring the AMQ message store
5.3.3 配置AMQ消息存储
The AMQ store configuration allows the user to change its basic behaviour around
indexing, checkpoint intervals, and the size of the journal data files. These items and
many more can be customized through the use of properties. The key properties for
the AMQ store are shown in table 5.2.
通过配置AMQ消息存储,用户可以修改它的基本行为,包括索引建立,执行checkpoint的时间间隔和日志数据文件大小.
这些条目还有其他很多条目都可以通过属性自定义.AMQ消息存储的主要属性如表5.2所示:
Table 5.2 Configuration properties for the AMQ message store
表5.2 AMQ消息存储的可配置属性
Property name Default value Description
属性名称 默认值 描述
directory activemq-data The directory path used by the AMQ message store.
directory activemq-data AMQ消息存储使用的目录
useNIO true NIO provides faster write-through to the systems disks.
useNIO true JDK的NIO可以更快的写入文件到系统磁盘
syncOnWrite false Syncs every write to disk.
syncOnWrite false 是否同步所有的磁盘写操作
syncOnTransaction true Syncs every transaction to disk.
syncOnTransaction true 是否同步所有的事务写操作
maxFileLength 32mb The maximum size of the message journal data files before a new one is used.
maxFileLength 32mb 日志数据文件的最大尺寸,超过这个值将创建一个新文件
persistentIndex true Persistent indexes are used. If false, an in-memory HashMap is used.
persistentIndex true 是否使用持久化索引.如果设置为false,将使用内存中的HashMap作为替代.
maxCheckpointMessageAddSize 4kb The maximum memory used for a transaction before writing to disk.
maxCheckpointMessageAddSize 4kb 单个事务可使用的最大内存大小,超过这个值数据会被写入磁盘
cleanupInterval 3000(ms) Time before checking which journal data files are still required.
cleanupInterval 3000(ms) 进行日志数据文件是否需要的检查操作 的时间间隔(不需要的数据日志文件会被删除或归档(配置了归档时))
checkpointInterval 20000(ms) Time before moving cached message IDs to the reference store indexes.
checkpointInterval 20000(ms) checkpoint的时间间隔,缓存的消息ID在内存中存在的时间,超过这个时间缓存被写入索引文件
indexBinSize 1024 The initial number of hash bins to use for indexes.
indexBinSize 1024 用于索引的哈希容器的初始化大小
indexMaxBinSize 16384 The maximum number of hash bins to use.
indexMaxBinSize 16384 哈希容器容量的最大值
directoryArchive archive The directory path used by the AMQ message store to place archived journal files.
directoryArchive archive 归档目录,AMQ消息存储用来存放归档文件的地方
archiveDataLogs false If true, journal files are moved to the archive instead of being deleted.
archiveDataLogs false 是否启用归档.若启用,不用的日志文件会被移动到规定目录,而不是删除掉.
recoverReferenceStore true Recovers the reference store if the broker isn’t shut down cleanly;
this errs on the side of extreme caution.
recoverReferenceStore true 如果代理非正常关闭,是否恢复索引存储. 这个选项需要谨慎配置
forceRecoverReferenceStore false Forces a recovery of the reference store.
Here’s an example of using the properties from table 5.2 in an ActiveMQ XML configuration
file:
下面是使用表5.2中列出的属性,配置ActiveMQ的XML代码示例:
<?xml version=”1.0″ encoding=”UTF-8″?>
<beans>
<broker xmlns=”http://activemq.apache.org/schema/core”>
<persistenceAdapter>
<amqPersistenceAdapter
directory=”target/Broker2-data/activemq-data”
syncOnWrite=”true”
indexPageSize=”16kb”
indexMaxBinSize=”100″
maxFileLength=”10mb” />
</persistenceAdapter>
</broker>
</beans>
This is but a small example of a customized configuration for the AMQ store using the
available properties.
这是使用属性配置AMQ消息存储的小例子.
The AMQ store, like the KahaDB store, enables users to get up and running
quickly, as there are no external dependencies on other databases. But when you want
to run an ActiveMQ broker and use an already established relational database, you
need to use a JDBC message store.
与KahaDB一样,AMQ消息存储也允许用户快速使用,因为这种消息存储方式不需要
依赖外部的其他数据库.但是,假如你想使用已有的关系数据库来运行ActiveMQ代理,你需要
使用JDBC消息存储.
赞 赏微信赞赏 支付宝赞赏
本文固定链接: https://www.jack-yin.com/coding/translation/activemq-in-action/766.html | 边城网事