Redis 5.0.3 配置文件详解(易读-白话翻译)-高级配置与碎片整理

2021年11月21日 阅读数:13
这篇文章主要向大家介绍Redis 5.0.3 配置文件详解(易读-白话翻译)-高级配置与碎片整理,主要内容包括基础应用、实用技巧、原理机制等方面,希望对大家有所帮助。

整个系列的目录: Redis 5.0.3 配置文件详解(易读-白话翻译)-目录  node

############################### ADVANCED CONFIG 高级配置 ###############################
# 下面的配置须要对Redis的原理,特别5中数据类型的底层数据结构有比较清楚的了解才能看得懂,总的来讲就是根据本身的键-值选择5中数据类型在某些条件下使用何种数据结构来存放数据。
# 最多见的高效数据结构就是ziplist、intset,可是他们一般只有元素(条目)较小且元素(条目)较小时才适合
# 要学习这部份内容能够看看redis设计与实现和Redis资深历险两本书,前一本书将原理不少,且深度足够,可是Redis的版本有点太老了,后一步本书能够在原理上对前一本书进行补充,且Redis版本很新,已经到5了。并且它还将了不少实战的知识。
#
# Hashes are encoded using a memory efficient data structure when they have a
# small number of entries, and the biggest entry does not exceed a given
# threshold. These thresholds can be configured using the following directives.
#
# hash数据类型:若是条目数小于512,且条目大小不超过64字节,则使用ziplist做为hash数据类型的底层数据结构
hash-max-ziplist-entries 512
hash-max-ziplist-value 64

# 新版本的Redis针对list数据类型的底层数据结构作了优化采用的是"链表+ziplist",其思想有点像Java HashMap的"数组+链表/红黑树"
# Lists are also encoded in a special way to save a lot of space.
# The number of entries allowed per internal list node can be specified
# as a fixed maximum size or a maximum number of elements.
# 能够经过"list-max-ziplist-size"设置链表中ziplist的条目数量,其值能够是条目数量,也能够最大字节数
# For a fixed maximum size, use -5 through -1, meaning:
# 下面是5个可能取值,建议使用-1 和 -2,其余选项不推荐使用,除非有特殊需求
# -5: max size: 64 Kb  <-- not recommended for normal workloads
# -4: max size: 32 Kb  <-- not recommended
# -3: max size: 16 Kb  <-- probably not recommended
# -2: max size: 8 Kb   <-- good
# -1: max size: 4 Kb   <-- good
#
# Positive numbers mean store up to _exactly_ that number of elements
# per list node.
# 上面的负值就是单个链表节点所包含的条目数
#
# The highest performing option is usually -2 (8 Kb size) or -1 (4 Kb size),
# but if your use case is unique, adjust the settings as necessary.
# 取值为-1 -2 发挥的性能是最好的
list-max-ziplist-size -2

# Lists may also be compressed.
# Compress depth is the number of quicklist ziplist nodes from *each* side of
# the list to *exclude* from compression.  The head and tail of the list
# are always uncompressed for fast push/pop operations.  Settings are:
# 0: disable all list compression
#    表示不压缩任何节点
# 1: depth 1 means "don't start compressing until after 1 node into the list,
#    going from either the head or tail"
#    So: [head]->node->node->...->node->[tail]
#    [head], [tail] will always be uncompressed; inner nodes will compress.
#    表示除链表的头尾之外,其余链表节点都压缩
# 2: [head]->[next]->node->node->...->node->[prev]->[tail]
#    2 here means: don't compress head or head->next or tail->prev or tail,
#    but compress all nodes between them.
#    依次类推,即前两个和后两个之外的都压缩
# 3: [head]->[next]->[next]->node->node->...->node->[prev]->[prev]->[tail]
#    依次类推
# etc.
# 默认压缩深度为0,也就是说不压缩。。。不管如何设置头尾是不会压缩的,好比当list被当作队列使用时,若是压缩了,还须要解压,下降了性能。
list-compress-depth 0

# Sets have a special encoding in just one case: when a set is composed
# of just strings that happen to be integers in radix 10 in the range
# of 64 bit signed integers.
# 当集合(set)存放的值都是64位的无符号10进制整数时,且条目数小于512时会采用intset做为集合的底层数据结构
# The following configuration setting sets the limit in the size of the
# set in order to use this special memory saving encoding.
set-max-intset-entries 512

# 和hash数据类型相似,若是条目数小于128,且条目大小<64会使用ziplist做为有序集合的底层数据结构
# Similarly to hashes and lists, sorted sets are also specially encoded in
# order to save a lot of space. This encoding is only used when the length and
# elements of a sorted set are below the following limits:
zset-max-ziplist-entries 128
zset-max-ziplist-value 64

# HyperLogLog sparse representation bytes limit. The limit includes the
# 16 bytes header. When an HyperLogLog using the sparse representation crosses
# this limit, it is converted into the dense representation.
#
# A value greater than 16000 is totally useless, since at that point the
# dense representation is more memory efficient.
#
# The suggested value is ~ 3000 in order to have the benefits of
# the space efficient encoding without slowing down too much PFADD,
# which is O(N) with the sparse encoding. The value can be raised to
# ~ 10000 when CPU is not a concern, but space is, and the data set is
# composed of many HyperLogLogs with cardinality in the 0 - 15000 range.
# 这个是Redis高级功能,能够用这种数据结构统计网站的UV,可以去重,其准确度接近真实值
# 简单点说:当去重后统计出来的值小于"hll-sparse-max-bytes"指定的值时,Redis会使用稀疏矩阵来存放,一个Key占用的空间比稠密矩阵小,若是统计出来的值大于"hll-sparse-max-bytes"指定的值,那么使用稠密矩阵,此时一个Key占用的空间是12KB
# "hll-sparse-max-bytes"默认为3000,若是设置为16000以上彻底是无用的,由于此时稠密矩阵效果更好
hll-sparse-max-bytes 3000

# Streams macro node max size / items. The stream data structure is a radix
# tree of big nodes that encode multiple items inside. Using this configuration
# it is possible to configure how big a single node can be in bytes, and the
# maximum number of items it may contain before switching to a new node when
# appending new stream entries. If any of the following settings are set to
# zero, the limit is ignored, so for instance it is possible to set just a
# max entires limit by setting max-bytes to 0 and max-entries to the desired
# value.
# 设置Stream的单个节点最大字节数和最多能有多少个条目,若是任何一个条件知足就会新增长一个节点用以保存新的数据
# 若是将任何一个配置项设置为0,表示不限制
stream-node-max-bytes 4096
stream-node-max-entries 100

# Redis数据库存放键值对数据结构是一个类型为字典长度为2的数组,假设这个数组名称为"ht",在rehash的时候就是将其中一个字典(ht[0])中的全部数据搬到另外一个字典(ht[1])中,并且rehash是惰性的(由于redis要高效的响应查询或者写,不可能去一次完成rehash操做,不像Java的HashMap),当方式key时或者CPU比较空闲时会触发,所以也被称之为"渐进式hash"
# Active rehashing uses 1 millisecond every 100 milliseconds of CPU time in
# order to help rehashing the main Redis hash table (the one mapping top-level
# keys to values). The hash table implementation Redis uses (see dict.c)
# performs a lazy rehashing: the more operation you run into a hash table
# that is rehashing, the more rehashing "steps" are performed, so if the
# server is idle the rehashing is never complete and some more memory is used
# by the hash table.
#
# The default is to use this millisecond 10 times every second in order to
# actively rehash the main dictionaries, freeing memory when possible.
# 默认是使用1秒钟的10毫秒进行rehash,在适当的时候回收内存
#
# If unsure:
# use "activerehashing no" if you have hard latency requirements and it is
# not a good thing in your environment that Redis can reply from time to time
# to queries with 2 milliseconds delay.
# 若是系统有严格的延时要求,在2毫秒内不断的查询出结果,能够将"activerehashing"设置no
# 可是这对你的系统并非一个好事情,所以不建议这样设置,因此保持不动吧
#
# use "activerehashing yes" if you don't have such hard requirements but
# want to free memory asap when possible.
# 若是没有很是严格的要求,建议将"activerehashing"设置为yes,这样可让内存尽量快的释放
activerehashing yes

# The client output buffer limits can be used to force disconnection of clients
# that are not reading data from the server fast enough for some reason (a
# common reason is that a Pub/Sub client can't consume messages as fast as the
# publisher can produce them).
# 能够经过设置客户端输出缓冲区大小将待接收数据超过缓冲区大小的客户端断开
# 一般使用pub/sub的时候,客户端没有及时消费而致使超过缓冲区大小
#
# The limit can be set differently for the three different classes of clients:
# 提供三种客户端的设置,分别是普通的、主从复制的、pub/sub的客户端,咱们能够分别对这三种客户端的输出缓冲区设置大小
#
# normal -> normal clients including MONITOR clients
# replica  -> replica clients
# pubsub -> clients subscribed to at least one pubsub channel or pattern
#
# The syntax of every client-output-buffer-limit directive is the following:
# 下面是三种客户端缓冲区大小设置的语法
# 
# client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>
#
# A client is immediately disconnected once the hard limit is reached, or if
# the soft limit is reached and remains reached for the specified number of
# seconds (continuously).
# 若是客户端输出缓冲区的大小达到了"hard limit",服务器会当即断开链接
# 若是客户端输出缓冲区的大小达到了"soft limit",且持续时间达到了"soft seconds",服务器会当即断开链接
#
# So for instance if the hard limit is 32 megabytes and the soft limit is
# 16 megabytes / 10 seconds, the client will get disconnected immediately
# if the size of the output buffers reach 32 megabytes, but will also get
# disconnected if the client reaches 16 megabytes and continuously overcomes
# the limit for 10 seconds.
# 这上面是一个举例,省略。。。。
#
# By default normal clients are not limited because they don't receive data
# without asking (in a push way), but just after a request, so only
# asynchronous clients may create a scenario where data is requested faster
# than it can read.
# 默认状况下普通的client不限制,由于它们都是发起请求后等待接收数据,并不像异步的客户端(好比主从复制客户端和PUB/SUB)会形成数据的挤压,挤压的缘由就是客户端处理速度跟不上数据产生的速度
#
# Instead there is a default limit for pubsub and replica clients, since
# subscribers and replicas receive data in a push fashion.
#
# Both the hard or the soft limit can be disabled by setting them to zero.
# hard or soft limit 均可以经过设置为0而禁止掉
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60

# Client query buffers accumulate new commands. They are limited to a fixed
# amount by default in order to avoid that a protocol desynchronization (for
# instance due to a bug in the client) will lead to unbound memory usage in
# the query buffer. However you can configure it here if you have very special
# needs, such us huge multi/exec requests or alike.
# 客户端查询缓冲区会累加新命令,默认状况下,缓冲区大小是一个固定值以免协议同步失效(如客户端的bug)致使查询缓冲区出现未绑定的内存(即客户端都已经不存在了,可是它发过来的命令还在缓冲区当中)
# 若是有巨大的multi/exec请求,则能够修改这个值以知足咱们的特殊需求
# 
# client-query-buffer-limit 1gb

# In the Redis protocol, bulk requests, that are, elements representing single
# strings, are normally limited ot 512 mb. However you can change this limit
# here.
# 若是一个大容量请求(即客户端单次发送过来的字符串)被限制为512MB,咱们也能够经过修改"proto-max-bulk-len"值
# 不过我可能一生也不会用到
# proto-max-bulk-len 512mb

# Redis calls an internal function to perform many background tasks, like
# closing connections of clients in timeout, purging expired keys that are
# never requested, and so forth.
#
# Not all tasks are performed with the same frequency, but Redis checks for
# tasks to perform according to the specified "hz" value.
#
# By default "hz" is set to 10. Raising the value will use more CPU when
# Redis is idle, but at the same time will make Redis more responsive when
# there are many keys expiring at the same time, and timeouts may be
# handled with more precision.
#
# The range is between 1 and 500, however a value over 100 is usually not
# a good idea. Most users should use the default of 10 and raise this up to
# 100 only in environments where very low latency is required.
# 简单点说:Redis有后台任务,经过设置"hz"可提升或者下降检查这些任务是否应该执行的频率,值越大消耗的CPU越多,反之越少
# 值能够设置在1到500之间,一般不建议将该值设置得比100大,通常都使用10这个默认值,除非咱们的系统有很是严格的延时要求,才会将"hz"设置得等于或者超过100
hz 10

# Normally it is useful to have an HZ value which is proportional to the
# number of clients connected. This is useful in order, for instance, to
# avoid too many clients are processed for each background task invocation
# in order to avoid latency spikes.
#
# Since the default HZ value by default is conservatively set to 10, Redis
# offers, and enables by default, the ability to use an adaptive HZ value
# which will temporary raise when there are many connected clients.
#
# When dynamic HZ is enabled, the actual configured HZ will be used as
# as a baseline, but multiples of the configured HZ value will be actually
# used as needed once more clients are connected. In this way an idle
# instance will use very little CPU time while a busy instance will be
# more responsive.
# 英文有时还真的描述很啰唆,仍是中文编码更高效。。。吐槽一下英文
# 前面的"ht"配置项是固定值,当链接客户端很是多时,若是"ht"仍是10,则可能会致使延迟比较高,所以Redis搞了一个
# "dynamic-hz"配置项,当设置为yes时,能够基于"ht"配置值动态的调整使用的"ht"值,好比链接的客户端不少事,动态将ht调高,能够减小延迟。而当链接客户端比较少,又能够动态下降"ht",这样消耗的CPU会不多
# 默认值是yes,这个根本不须要咱们本身去动,有了它咱们也不须要去动"ht"配置
dynamic-hz yes

# When a child rewrites the AOF file, if the following option is enabled
# the file will be fsync-ed every 32 MB of data generated. This is useful
# in order to commit the file to the disk more incrementally and avoid
# big latency spikes.
# 当子进程在重写AOF文件时,若是将"aof-rewrite-incremental-fsync"设置为yes,那么一旦生成32M数据才会调用一次OS的fsync函数,这样能够下降出现访问峰值时系统的延迟。由于能够减小fsync调用次数和IO请求
aof-rewrite-incremental-fsync yes

# When redis saves RDB file, if the following option is enabled
# the file will be fsync-ed every 32 MB of data generated. This is useful
# in order to commit the file to the disk more incrementally and avoid
# big latency spikes.
# 和"aof-rewrite-incremental-fsync"一个意思,只不过是用在生成RDB文件时用。
# 若是持久化采用的混合方式,即AOF文件是由"RDB部分+AOF部分"组成的话,我想"aof-rewrite-incremental-fsync"和"rdb-save-incremental-fsync"都会使用到
rdb-save-incremental-fsync yes

# Redis LFU eviction (see maxmemory setting) can be tuned. However it is a good
# idea to start with the default settings and only change them after investigating
# how to improve the performances and how the keys LFU change over time, which
# is possible to inspect via the OBJECT FREQ command.
#
# There are two tunable parameters in the Redis LFU implementation: the
# counter logarithm factor and the counter decay time. It is important to
# understand what the two parameters mean before changing them.
#
# Redis的LFU实现有两个可调整的参数:计数器对数因子(couter logarithm factor)和计数器衰退时间(counter decay time)
# 必定要充分理解这两个参数以后才能去修改,若是不懂就不要去瞎搞了,若是非要修改,必定要使用"OBJECT FREQ"命令充分调查并知道如何提高性能的状况下才能进行
# The LFU counter is just 8 bits per key, it's maximum value is 255, so Redis
# uses a probabilistic increment with logarithmic behavior. Given the value
# of the old counter, when a key is accessed, the counter is incremented in
# this way:
# 在介绍maxmemory的时候提到了两个参数做用的原理,这里就不赘述了。
#
# 1. A random number R between 0 and 1 is extracted.
# 2. A probability P is calculated as 1/(old_value*lfu_log_factor+1).
# 3. The counter is incremented only if R < P.
#
# The default lfu-log-factor is 10. This is a table of how the frequency
# counter changes with a different number of accesses with different
# logarithmic factors:
# "lfu-log-factor"的默认值=10,下表是不一样对数因子下计数器的改变频率:
#
# +--------+------------+------------+------------+------------+------------+
# | factor | 100 hits   | 1000 hits  | 100K hits  | 1M hits    | 10M hits   |
# +--------+------------+------------+------------+------------+------------+
# | 0      | 104        | 255        | 255        | 255        | 255        |
# +--------+------------+------------+------------+------------+------------+
# | 1      | 18         | 49         | 255        | 255        | 255        |
# +--------+------------+------------+------------+------------+------------+
# | 10     | 10         | 18         | 142        | 255        | 255        |
# +--------+------------+------------+------------+------------+------------+
# | 100    | 8          | 11         | 49         | 143        | 255        |
# +--------+------------+------------+------------+------------+------------+
#
# NOTE: The above table was obtained by running the following commands:
# 上面的表格能够经过下面的命令获得:
#
#   redis-benchmark -n 1000000 incr foo
#   redis-cli object freq foo
#
# NOTE 2: The counter initial value is 5 in order to give new objects a chance
# to accumulate hits.
# 默认counter的初始值是5,为了让新的对象有机会累加它的命中率
#
# The counter decay time is the time, in minutes, that must elapse in order
# for the key counter to be divided by two (or decremented if it has a value
# less <= 10).
# 计数器衰减时间是key计数器除以2(若是值小于<=10,则递减)所必须通过的时间,单位为分钟。
#
# The default value for the lfu-decay-time is 1. A Special value of 0 means to
# decay the counter every time it happens to be scanned.
# "lfu-decay-time" 的默认值为 1,0 表示每次都对计数器进行衰减
#
# lfu-log-factor 10
# lfu-decay-time 1

########################### ACTIVE DEFRAGMENTATION #######################
########################### 在线碎片整理 #######################
#
# WARNING THIS FEATURE IS EXPERIMENTAL. However it was stress tested
# even in production and manually tested by multiple engineers for some
# time.
# 这还只是一个实验功能,就像Redis Cluster同样,其实已经有不少人在使用了
# What is active defragmentation?
# -------------------------------
#
# Active (online) defragmentation allows a Redis server to compact the
# spaces left between small allocations and deallocations of data in memory,
# thus allowing to reclaim back memory.
#
# Fragmentation is a natural process that happens with every allocator (but
# less so with Jemalloc, fortunately) and certain workloads. Normally a server
# restart is needed in order to lower the fragmentation, or at least to flush
# away all the data and create it again. However thanks to this feature
# implemented by Oran Agra for Redis 4.0 this process can happen at runtime
# in an "hot" way, while the server is running.
# 活动碎片整理容许Redis服务器压缩内存中因为申请和释放数据块致使的碎片,从而回收内存,就好像window的磁盘整理同样
# 碎片是每次申请内存(幸运的是Jemalloc出现碎片的概率小不少)的时候会天然发生的
# 一般来讲,为了下降碎片化程度须要重启服务,或者清除全部的数据而后从新建立。 得益于Oran Agra在Redis 4.0实现的这个特性,进程能够在服务运行时以"热"方式完成
#
# Basically when the fragmentation is over a certain level (see the
# configuration options below) Redis will start to create new copies of the
# values in contiguous memory regions by exploiting certain specific Jemalloc
# features (in order to understand if an allocation is causing fragmentation
# and to allocate it in a better place), and at the same time, will release the
# old copies of the data. This process, repeated incrementally for all the keys
# will cause the fragmentation to drop back to normal values.
# 一般来讲当碎片化达到必定程度(查看下面的配置)Redis 会使用Jemalloc建立连续的内存空间,并在此内存空间对现有的值进行拷贝,拷贝完成后会释放掉旧的数据。
# 这个过程会对全部的致使碎片化的key以增量的形式进行,Redis到处使用渐进式的,真实辛苦设计者了
#
# Important things to understand:
# 要重点理解的三点:
#
# 1. This feature is disabled by default, and only works if you compiled Redis
#    to use the copy of Jemalloc we ship with the source code of Redis.
#    This is the default with Linux builds.
#    默认状况下,该功能是关闭的,而且只有在编译Redis时使用了代码中的Jemalloc才生效(这是 Linux 下的默认行为)
# 2. You never need to enable this feature if you don't have fragmentation
#    issues.
#    若是没有碎片问题,咱们永远也不须要启用该功能
#
# 3. Once you experience fragmentation, you can enable this feature when
#    needed with the command "CONFIG SET activedefrag yes".
#    能够经过命令"CONFIG SET activefrag yes"来启用并试验
#
# The configuration parameters are able to fine tune the behavior of the
# defragmentation process. If you are not sure about what they mean it is
# a good idea to leave the defaults untouched.
# 相关的配置参数能够很好的调整碎片整理过程,若是你不知道这些选项的做用最好使用默认值。

# Enabled active defragmentation
# 开启在线整理
# activedefrag yes

# Minimum amount of fragmentation waste to start active defrag
# 有多少碎片时开始整理
# active-defrag-ignore-bytes 100mb

# Minimum percentage of fragmentation to start active defrag
# 有多少比例的碎片时开始整理
# active-defrag-threshold-lower 10

# Maximum percentage of fragmentation at which we use maximum effort
# 有多少比例的碎片时开始进行整理
# active-defrag-threshold-upper 100

# Minimal effort for defrag in CPU percentage
# 进行碎片整理时使用多少比例的CPU时间
# active-defrag-cycle-min 5

# Maximal effort for defrag in CPU percentage
# 进行整理时使用多少CPU时间
# active-defrag-cycle-max 75

# Maximum number of set/hash/zset/list fields that will be processed from
# the main dictionary scan
# 进行主字典扫描时处理的 set/hash/zset/list 字段的最大数量(就是说在进行主字典扫描时 set/hash/zset/list 的长度小于这个值才会处理,大于这个值的会放在一个列表中延迟处理)
# 由于若是某一个key过大,一次性处理完会很是耗时的
# active-defrag-max-scan-fields 1000