DRX requires sufficient enough free physical memory for DRX's buffering capability when installed locally in an operating machine environmenton the production server. If you do there is not have enough free physical memory space, you should also consider adding increasing the physical memory on your of the production machineserver.

Also, if you use the compression feature to accelerate replication, you should be aware consider that compression can may cause CPU load on the operating node. Compression can be used locally if production server. If the local I/O load is not heavily burdened and compression heavy and the performance degradation of the operating production node is insignificant. However, if compression loads affect performance across the local system, you should rethink the use of compression. Compressed loads can result in an additional load of approximately 20 ~ 30% on due to compression is minimal, it is not a problem, but if the compression load affects the performance of the local system as a whole, you should reconsider using compression. Compression load can add about 20-30% of the local I/O load . If and can be offloaded by configuring the DRX that performs the performing compression is configured as a dedicated machine and separated separating it from the operating environment, it is possible to distribute the load depending on the compression. These DRX operational policies should be production environment.

The operating policy of such a DRX should be established based on preliminary data on the local I/O load, and should be determined by preliminary investigation of the following items in the configuration environment

2.1. Prior research

2.1.1. Operating System

Windows 2008 or latershould be investigated to determine the appropriate specifications and buffer operating policies for the DRX to run.

Specifications

Operating Systems

Supports Windows 2008 and later 64-bit, Linux CentOS 6.4 or and later, and Ubuntu 12.04 LTS and later 64-bit platforms are supported.

2.1.2. Operating machine minimum specification

...

Minimum Specifications

• Minimum 1 GHz or faster x86/x64 compatible processor (

...

• 2 GHz or

...

• faster recommended), minimum 4

...

• cores recommended
• At least

...

• 4 GB of physical memory

...

• Minimum 10 GB capacity disk

Recommended Memory Specifications

DRX's physical memory specifications are variable depending on the number of resources, maximum I/O, and transfer bandwidth. The following is a formula for DRX's physical memory specifications when the maximum I/O is greater than the transmission bandwidth.

• Number of Resources * (Average I/O (MB/s) - Bandwidth (MB/s)) * I/O duration (seconds) + compression/encryption buffer (2GB)

If the peak I/O is always low compared to the bandwidth, calculate the size of the buffer per resource to be 1 GB to determine the physical memory specification.

• Number of resources * 1 GB + compression/encryption buffer (2 GB)

WAN bandwidth is not really a guaranteed bandwidth, but rather a variable bandwidth that fluctuates based on network conditions. In general, WAN bandwidth is considered to be in the range of 10 to 100 Mbps, and estimating 1 MB/s to 10 MB/s is appropriate from a buffering perspective. Also, it is not advisable to disregard DRX buffering just because the maximum I/O measured within a period of time is lower than the replication bandwidth. Depending on the characteristics of the applications in the operating environment, there may be situations where the maximum I/O spikes at an unspecified time, so it is advisable to configure the DRX buffer with extra space for such situations.

The following is an example of determining the physical memory specification that DRX can buffer for 30 seconds of I/O at an average of 100 MB/s over a 100 Mbps WAN bandwidth.

...

The following is an example of determining the physical memory specification assuming a WAN 100 Mbps bandwidth, a maximum I/O of 500 MB/s, and a maximum I/O duration of 30 seconds. In this example, the required memory for one replication resource operation is 20.7GB (32GB recommended).

Number of resources

You can configure it to any size as long as as many resources as your memory resources allow and limit it to up to 100 channels of replication resources.

...

. The maximum resource is 100 channels, but this can be flexible depending on available memory, and if the number of resources you need to operate is more than a few dozen channels, you may want to consider investing in multiple dedicated DRX nodes.

Investigate preliminary server I/O load

Use the following procedure to measure the I/O load on of the production server.

• Measure the read/write I/O

...

• data of the production server

...

• replication target disk (average I/O, maximum I/O

...

• over a

...

• period of at least 1-4 weeks).

...

• Measurement method
  • Windows:

...

• • Use Performance Monitor tool

...

• • to collect disk I/O

...

• • statistics data

...

• • Linux:

...

• • Use utilities such as iostat

...

• • to collect disk I/O

...

• • statistics data

...

• Consider buffer size, compression, and

...

• buffer operation policies based on the measurement results

...

...

Replication bandwidth

Replication bandwidth requires bands require at least 10 Mbps to 100 Mbps bandwidth.

...

Policy

Determine how you will configure your operating environment your configuration based on the I/O load of your environment and whether you are using compression is enabled. the local configuration is . Local configurations are common, but If the replication load is large and WAN section acceleration is required, a dedicated configuration is recommended.

2.3. Buffer operation policy

...

dedicated configurations are recommended when replication loads are high and acceleration across the WAN is required.

Buffer policy

• Preliminary research on network bands, operating machine I/O load

...

• factors is required to instrument DRX's physical buffer specifications.
• Pre-survey items
  • Average I/O per

...

• • resource on the production machine
  • Maximum I/O

...

• • volume
  • Maximum I/O

...

• • duration
• The average I/O and maximum I/O

...

• figures of the

...

• production machine are the basis for building an appropriate buffering environment.

DRX의 버퍼는 사전 조사한 운영 노드의 I/O 부하 수치가 감당될 수 있도록 적절한 크기로 설정되어야 합니다. 사전 조사를 바탕으로한 I/O 데이터가 확보되지 못했다면 1번 case 기준의 권장 버퍼 사양에 따라 구성 및 시범 운영을 한 후 버퍼 크기에 대한 튜닝 과정을 필요로 합니다.

만일 운영 노드의 I/O 부하가 과도하게 크고 최대 I/O 의 지속시간이 장시간(수분~수십분)에 걸쳐 발생되는 상황이라면 DRX 버퍼링으로도 감당하기 어려울 수 있습니다. 이럴 경우에는 데이터 압축에 대해 고려해야 합니다. 

2.3.1 혼잡 정책

혼잡상태란 복제부하가 가중되어 DRX의 버퍼에 여유 공간이 남아 있지 않아 버퍼링이 불가한 상태를 의미합니다. 이럴 경우 DRX는 어떤 특별한 동작을 수행하지는 않으며 버퍼에 있는 복제 데이터를 원격으로 지속 전달하는 것에 주력합니다. 그리고 혼잡상태에 대한 대응은 DRBD의 혼잡정책에 맡깁니다.

...

DRX's buffer should be set to the appropriate size to handle the I/O load of the production node based on the preliminary survey. If I/O data based on the preliminary survey is not available, the buffer size should be tuned after configuration and piloting based on the recommended buffer specifications in case 1.

If the I/O load on the production node is excessively large and the duration of peak I/O occurs over a long period of time (minutes to tens of minutes), DRX buffering may not be able to handle it. In this case, you should think about data compression.

BSR

Congestion policy

The congestion state means that buffering is impossible because there is no free space in the DRX buffer because the replication load is increased. In this case, DRX does not perform any special action and concentrates on remotely transmitting the replicated data in the buffer. The response to the congestion state is left to the congestion policy of DRBD.

The congestion policy is the corresponding policy in DRBD when the DRX's buffer enters the congestion state. Here's how to set up a congestion policy.

resource r0 {
	proxy {
		memlimit 1G: # DRX 버퍼buffer
	}
	net {
		on-congestion pull-ahead; # 혼잡정책Congestion policy setting 설정(Ahead 모드mode)
		congestion-fill 950M; # Set 혼잡인지congestion awareness 시점point 설정(950Mwhen 950Mbyte 바이트data 데이터가is 버퍼링되는buffered 시점을is 혼잡시점으로congestion 인지point)
	}
}

DRBD의 혼잡정책은 다음의 3가지이며 WAN 구간 비동기 복제 운영 시에는 Ahead 모드의 설정을 권장 합니다.

• block: 버퍼가 비워질 때 까지(버퍼에 큐잉할 수 있을 때까지) I/O 대기 합니다. 혼잡정책을 설정하지 않았을 때의 기본값 입니다.
• disconnect: 복제 연결을 단절하고 StandAlone 상태로 진입합니다. 
• pull-ahead: 지연 복제 모드로 진입합니다. 이럴 경우 복제 연결은 유지하되 복제는 중단되고 로컬 I/O는 out-of-sync로 기록하되 혼잡상황이 해제될 시 기록됬던 out-of-sync 에 대한 재동기화를 수행합니다.

2.3.2. 버퍼 조정 가이드

• DRBD의 Ahead 모드(지연복제) 비동기 복제 구성을 전제로 합니다.
• 복제 연결이 유지되는 구간의 I/O 측정치에 대해서만 계측될 수 있도록 합니다. 복제연결 단절 구간에 대한 I/O 측정치는 고려대상이 아닙니다.
• 다음의 방법을 통해 DRBD가 Ahead 모드로 진입하는 회수(혼잡 진입 회수)를 취합합니다.
  • drbd 로그상의 "Congestion-fill threshold reached" 의 출력 회수 취합
  • drbdsetup events2 명령을 통한 Ahead 진입 회수 확인
• 취합한 혼잡진입 회수를 기준으로 버퍼 크기를 재조정합니다. 혼잡빈도가 잦은 경우 버퍼의 크기는 더욱 증설되어야 합니다.
• 버퍼를 증설 했음에도 혼잡구간이 빈도가 낮아지지 않을 경우 압축에 대해 고려합니다.

2.3.3. 물리 메모리 사양 

DRX의 물리 메모리 사양은 리소스 개수와 최대I/O, 대역폭에 따라 가변적입니다. 다음은 최대I/O 가 대역폭보다 클 경우의 DRX의 물리메모리 사양에 대한 계산식 입니다.

• 리소스 개수 * (최대 I/O(MB/s) - 대역폭(MB/s)) * 최대 I/O 지속시간 (초) + (리소스 개수 * 압축/암호화 버퍼(100MB)) + DRX 기본메모리(100MB)

최대 I/O 가 대역폭에 비해 항상 낮다면 리소스 당 버퍼의 크기를 1GB 수준으로 계산하여 물리 메모리 사양을 결정합니다.

• 리소스 개수 * 1GB + (리소스 개수 * 압축/암호화 버퍼(100MB)) + DRX 기본메모리(100MB)

다음은 WAN 100Mbps 대역폭,  최대 I/O 200MB/s , 최대 I/O 지속시간 10초 일 경우를 상정하여 물리메모리 사양을 결정하는 예시 입니다. 이 예시에서 1개의 복제 리소스 운영에 대해 요구되는 메모리는 2.1GB(4GB 권장)이며 최대 I/O가 10초 이상 지속될 시 혼잡 상태로 진입하게 됩니다. 

...

DRX 기본 메모리

...

WAN 대역폭은 실질적으로는 가변적입니다. WAN 대역폭은 일반적으로 보장 대역폭이 아닌 네트웍 상황에 따라 변동이 심한 가변대역의 특성을 가지기 때문에 통상 WAN 대역폭은 10~100Mbps 정도의 대역으로 간주하여 1MB/s ~10MB/s 로 추산하는 것이 버퍼링 관점에서 적당합니다. 또한 일정 기간 내에 측정한 최대 I/O가 복제 대역폭보다 낮다고 해서 DRX 버퍼링을 고려하지 않는 것은 바람직 하지 않습니다. 운영환경의 애플리케이션의 특성에 따라 최대 I/O가 불특정 시점에 급등하는 상황이 있을 수 있기 때문에 이런 상황에 대비하여 DRX 버퍼에 여유공간을 두고 구성하는것이 바람직 합니다. BSR has the following three congestion policies, and Ahead mode is recommended for asynchronous replication across the WAN.

• block: Wait for I/O until the buffer is empty (until it can be queued into the buffer). This is the default when no congestion policy is set.
• disconnect: Disconnect the replication connection and enter the StandAlone state.
• pull-ahead: Enter lazy replication mode. In this case, the replication connection is maintained, but replication is stopped and local I/O is logged as out-of-sync, but resynchronization of the logged out-of-sync is performed when congestion is released.

Specifying buffer size

• BSR coupling assumes an Ahead mode (lazy replication), asynchronous replication configuration.
• Ensure that measurements can only be taken over measurement segments where the replication connection is maintained; I/O measurements over disconnected segments are not considered.
• Aggregate the number of times BSR enters Ahead mode (congestion entry count) using the following methods
  • Aggregate the number of times the "Congestion-fill threshold reached" output in the BSR log.
  • Check the number of Ahead entries via the bsrsetup events2 command
• Resize the buffer based on the collected congestion entry counts. If the congestion frequency is high, the size of the buffer should be further increased.
• If the congested section does not become less frequent after increasing the buffer, consider compression.

FSR

FSR only needs to specify the size of DRX's buffer. It does not support congestion mode, so it does not need to be set. If the DRX buffer is full while buffering, it will immediately empty the buffer and automatically move to a synchronized state.

When specifying the size of the buffer, you only need to refer to BSR's buffer policy to set the criteria for buffer operation and consider sizing accordingly. You can specify buffers as large as your memory allows, typically at the multi-GB level.