Configuration environments
bsr has no restrictions on the target environment such as physical system and virtual machine (VM), hyperconverged infrastructure (HCI) environment, and can flexibly configure replication in any network environment such as a local network and a remote network environment between WANs.
Specifications
The following are the physical requirements of the platforms and target systems supported by the BSR.
Platforms
Supports Windows 2012 or higher, Linux CentOS 6.4, Ubuntu 16.04 LTS or higher x64 environment.
OS | CPU Architecture |
|---|---|
Windows 2012 or higher | x64 |
RHEL / CentOS 6.4 ~ 8.4 | x64 |
RHEL / Rocky 8.5 or higher | x64 |
Ubuntu 16 LTS or higher | x64 |
ProLinux 7 or higher | x64 |
File systems
Block replication solutions usually work regardless of the type of file system. However, bsr has a specification for supported file systems, as it implements fast synchronization that only synchronizes to the areas used by the file system.
Recently, the capacity of the replication volume has been increasing in capacity (several tens to hundreds of terabytes), and accordingly, the time required to initially synchronize the replication volume is increasing significantly. In many cases, it may take days to dozens of days. This is because the volume is initially synchronized because it targets the entire disk volume. If the volume is 10 terabytes in 1 Gbps network bandwidth, it will take at least 27 hours at least as fast as possible to complete the entire synchronization, and over 10 days if 100 TB.
We have implemented Fast Sync (FastSync) to solve this excessive sync time issue. FastSync is a function that synchronizes only the area used in the file system of the replication volume, dramatically improving the initial synchronization time. For example, if you are using only 10 GB of actual volume at 100 TB capacity, FastSync will complete synchronization within 1 minute in the 1 Gbps band. However, since it is a function developed using the characteristics of a file system, it depends on the file system type and has a support specification for the file system type accordingly.
bsr supports NTFS and ReFS file systems commonly used in Windows, and Ext-like file systems (ext3 or higher) and xfs file systems in Linux. These file systems are the types of file systems most commonly used by general users, and we plan to gradually support other file systems.
For types of filesystems not supported by FastSync, it behaves as a traditional full sync.
CPU
It is recommended to configure at least 2GHz, 4 core or higher, x64 compatible processor. There is no problem in operating in a processor environment with a lower specification, but considering the I / O performance of the system, it is necessary to secure the specification of the configuration machine as high as possible.
Memory
The system typically starts paging virtual memory when memory usage exceeds 70%, depending on kernel settings. Because paging degrades system I/O performance, it is beneficial for replication to be configured to always have at least 30% free physical memory so that paging is suppressed.
The memory used by BSR is primarily required for buffering purposes and is determined by the maximum write request value (max-req-write-count) in the BSR settings and the size of the transmit buffer. Below is an example for a Windows environment.
For synchronous without a send buffer
At the default setting for write requests (10k), use a maximum of 1.5GB per resource.
At the write request maximum setting (100,000), use a maximum of 3 GB per resource.
For asynchronous with a 1GB send buffer setting, use a maximum of 3GB per resource in the
Use a maximum of 2.5 GB per 1 resource at the write request default setting.
Use a maximum of 4 GB per resource at the Write Request Max setting.
For a server with 64 GB of physical memory, approximately 20 GB (30%) of memory free space is required, and of the memory space used, a maximum of 2.5 GB per 1 resource is required for asynchronous by default.
If you don't have that 30% free memory, you'll have to accept a degradation in basic I/O performance due to paging.
The 3 to 4 GB NP memory usage per resource required by replication should be kept free, otherwise you will run out of memory, which can lead to failure.
Transmit buffer size
The size of the transmit (TX) buffer on the local side is ideally set to a number that allows for the free transfer of locally generated I/O data to the remote side. It is found by the following equation
Maximum size of transmit band per time (s) * Transmit timeout time (s)
For example, for a 1 Gbps band, you can get (about 100 MB/s * 5 s) = 500 MB, and you can set a buffer of 500 MB to 1 GB.
If you need to consider the variable bandwidth of the WAN segment, you can add more capacity to the above size plus the time (s) to tolerate the WAN bottleneck delay, or buffer it with a proxy (DRX). Typically, WAN leg buffering is specified at 5 to 10 times the TX buffer.
When paging occurs can vary depending on your system's memory capacity, platform, and OS version. The 70% figure described above is typical and should be understood in the context of your environment.
BSR memory usage on Linux is similar or less than on Windows. It uses a bit more memory on Windows due to some differences in the replication architecture.
Replication configurations with VMs are becoming increasingly common in virtualized environments. A characteristic of these environments is that the CPU or memory resources allocated to individual VMs are not always sufficient. For example, if an individual VM is configured with 2 cores and 2-4 GB of memory, it may not meet the minimum configuration specifications for bsr. When a replication environment is configured with such low-spec VMs, it is more likely to experience frequent performance delays inside the bsr engine and network communication (keep alive) delays between nodes. While there is no problem with the basic behavior, as the I/O load increases or the frequency of interrupts in the HW layer of the system increases, the performance of the VM as a whole will decrease, and bsr will be affected accordingly. The bottom line is that building a replication environment on low-specification VMs has limitations.
There's not much you can do about it except to make sure you have plenty of system configuration resources.
Disk
Basic installation space
It requires about 200MB to install all the binary execution modules of bsr, and 1GB is required to store the log of bsr, which requires about 2GB of disk installation space.
Mirror disks
Theoretically, the capacity of a BSR's mirror disk is unlimited, but the actual disk capacity is limited to 10 TB. At higher capacity, the meta-area corresponding to the mirror disk capacity grows along with it, and the operation (Attach) that needs to process the entire meta-disk becomes time-consuming and impedes operations.
Multivolume
In many cases, storage for a single service task is organized into multiple volumes, and these volumes need to be treated as a single resource. Operating multiple volumes on a single resource is called a multivolume configuration. When operating with multivolumes, the buffer queue for replicated data is operated as a single queue to ensure service-side consistency by serializing the order of read/write requests for service I/O to disk volumes.
The number of multivolumes is theoretically unlimited, but in practice, if the number of volumes becomes too large, the delay in the buffer queue can become severe and difficult to control. On a 1 Gbps network, a configuration of three volumes or less is appropriate.
Meta Disk
Depending on the capacity of the replication volume, you need to estimate the capacity of the metadisk. It requires about 33MB of meta disk space per 1TB of the replication volume, and for more accurate size, see Metadata Size Estimation.
Network
In a recent corporate environment, mirroring on a local network is generally configured with a bandwidth of 1 Gbps to 10 Gbps, and a remote replication environment (Disaster Recovery) for DR (Disaster Recovery) is generally operated with a lower bandwidth. That is, it is applied to a wide range of network environments from very low bandwidth (10 ~ 100Mbps) to high bandwidth. However, because replication in a low-band network environment is bound to affect replication performance due to bandwidth limitations, consider such as linking replication accelerators for performance improvement.
Storage provisioning
It is only supported for the THICK provisioning method of storage provisioning. Thin provisioning environments with disk space reclamation are not compatible with BSR.
Disk space reclamation in THIN provisioning can cause data inconsistency issues in certain situations. To use a THIN environment, you must disable space reclamation.