DRX 1.1 User's Guide - ENG

 

1. Overview

1.1. Purpose of use

The Data Replication Xcelerator(DRX) is a DRBD proxy agent used for low-bandwith WAN replication.

DRBD asynchronous (A type) replication basically has better local I/O response performance than synchronous. However, since the size of the socket transmission buffer of the replication data transmission layer (TCP/IP) is limited, when the socket buffer becomes full at the transmission time, it is necessary to wait while the replication data is not transmitted and the I/O of the application program is not completed (This is the architecture of DRBD that asynchronously replicates the completion of data transmission as the completion of replication). In addition, since replicated data is transmitted remotely over a relatively low-bandwidth network, the latency of local I / O is further increased because there is less free buffer space for transmission, and a large amount of local I / O write requests Is encountered and exceeds the bounds of the socket send buffer, degradation of the local I / O response accompanied by replication is unavoidable. These I / O response degradation problems can be overcome using DRX's buffering mechanism.

In addition, DRX provides a replication acceleration (compression) function to increase the availability of remote data, and can greatly improve the target RPO (Recovery Point Objective). Also, by providing data encryption function, It also solves the security problems that may occur via the WAN network.

The following figure shows the DRX-DRBD buffering linkage.

  • DRX buffering is performed from the local DRX prior to going through the WAN interval and does not require buffering on the remote DRX.

 

1.2. Considerations

DRX operates on DRBD-installed system or installed on dedicated server. A DRX instance can also act as a proxy for DRBD devices distributed across multiple nodes. Generally, DRBD activity log must be set in advance with sufficient capacity in case of large amount of data packet being transmitted at the time of transmission. If the primary node is corrupted while the excessive data packet is being processed at the sending time and the primary node is restarted, a large amount of activity log can be set to OOS and resynchronized for a long time. If you are concerned about this long synchronization, we recommend enabling DRBD's csums-alg setting.

1.3. Supported platforms

platform

OS version

platform

OS version

Windows

Windows 2008 or higher (64bit only)

Linux

CentOS 6.4 or higher (64bit only)

2. DRX Network

DRX can be configured in various ways as follows.

 

3. Installation

If the DRX is already installed on the operating machine, you only need to install additional DRX packages. However, if you separately configure DRX server, you need to install additional DRBD utility in addition to DRX package, and install QAT library when using HW compression.

Platform

Local Install

Dedicated Server Install

Remarks

Platform

Local Install

Dedicated Server Install

Remarks

Linux

  • Install drbd-utils(8.9.x)

  • HW compression library installation & build

  • Install DRX: 3.1 Installation on Linux

drbd-utils verson: 8.9.x

Windows

  • Install Windows DRBD

  • Install DRX: 3.2 Installation on Windows

  • Install DRX: 3.2 Installation on Windows

  • Install drbd-uitls(wdrbd-drbd-utils-x64-1_5_2)

    • Copy directly to the DRX installation path (c:\Program Files\drx\bin)

 

3.1. Installation on Linux

This product supports both installation based on RPM and one based on GUI installer. The default installation location is the "/opt" directory.

3.1.1. Installation with Shell Script

The following is the screen output when installing with RPM:

[root@node1opt]# sh DRX_Centos-x64_1_1-A7.sh -c

Unpacking JRE ...
Preparing JRE ...
Starting Installer ...
Please select a language:
1: English [Enter]
2: Korean
[1 - 2]

3.1.2. Installation with GUI Installer

To install the product by using GUI installer, run the "./DRX_unix_1_0.sh" on the command line. After the installation, open a new command line and proceed to the next DRX build processes(This should be done from the command line with the environment variable after installation).

 

3.1.3. Manual building DRX services on Linux

DRX service will be started automatically if the service is set as daemon service and rebooted. When the installation is finished, it is registered with DRX as a service, but it does not start automatically if there is no license file in the installation directory. In this case, you need to configure "drx.conf" and then, place "drx.lic" under the installed directory, then after run the service manually.Things such as these can be coordinated by "systemctl(centos7)" or "chkconfig(centos6)".

 

In the "/opt /DRX/samples" directory" there is an example file "drx.conf.sample.linux" to be referred for making the "drx.conf" file.

[root@node1opt]# cp /opt/DRX/samples/drx.conf.sample.linux /opt/DRX/drx.conf

Run the service with the following command:

# Centos7

[root@node1 ~]# systemctl start drxsvc

# Centos6

[root@node1 ~]# service drxsvc start

 

To stop the service, use the following command:

# Centos7

[root@node1 ~]# systemctl stop drxsvc

# Centos6

[root@node1 ~]# service drxsvc stop

To delete DRX, use the following command:

[root@node1 ~ ]# /opt/DRX/uninstall

Note. The directory may disappear if there is no change in the "/opt/DRX" directory.

3.1.4. DRX Watchdog on Linux

To manage DRX services, run a Linux DRX and a DRX monitor together. However, a DRX watchdog may restart DRX in the following cases:

  • Termination of a DRX process on the unusual or unexpected situation: available to check the log in "/logs/service.log"

  • Termination of DRX by the "drx-ctl -c shutdown" command on Linux: DRX restart occur by considering this case as unusual situation

Therefore, on Linux, you can use the following command to start or stop a DRX watchdog and DRX together:

CentOS6: service drxsvc [start/stop]

CentOS7: systemctl [start/stop] drxsvc

 

3.2. Installation on Windows

An installer for installation on Windows is provided. You can install the product easily through this installer.

 

3.2.1. Automatic building DRX services on Windows

By using the installer, you can register DRX for the Windows service to restart the DRX service automatically whenever system is rebooted.

 

3.2.2. Manual building DRX services on Windows

Use the following commands to Install, start, shut down, or uninstall Windows services: 

  • drxsvc /i: installing a service

  • drxsvc /u: removing a service

  • drxsvc /r: running a service on background

  • drxsvc /k: stopping a service

3.2.3. DRX Watchdog on Windows

To manage DRX services, run a Linux DRX and a DRX watchdog together. Refer to the followings:.

  • On Windows, its recovery policy is registered for the Windows service administrator occurs automatically when the first installation

  • The recovery policy: it forces DRX to restart infinitely if it perceives unusual termination of the DRX process.

  • Termination of DRX by the "drx-ctl -c shutdown" command on Windows: unlike on Linux, this case is considered as normal termination so the Windows service administrator would not run the recovery policy.

4. Uninstall

Log folder and configuration file and license file are not deleted. 

4.1. Windows

There are two ways to uninstall DRX as follows:

  • Deleting DRX on the "Uninstall" program: on Windows, "Control Panel" > "Programs" > "Uninstall"

  • Uninstalling by using the "uninstall.exe" command provided at the installation directory

4.2. Linux

  • Uninstalling by using the "uninstall.exe" command provided at the installation directory

5. License File

License file is required for DRX operation.

  • File name: "drx.lic"

  • Directory path where the file should be installed: the directory where DRX has been installed(necessary authorization of users/groups: administrator or root)

For more information on licenses, contact to the Man Technology.

 

6. DRX Configuration

6.1. Configuration File

 

The DRX configuration is configured in the DRBD's configuration file. Set the buffer size in the "proxy" section and set the DRX ip address in the "proxy on" sub section of the host section.

DRX provides both a local method that is installed on the same system as DRBD and a dedicated machine configuration method that is installed and operated on a separate dedicated machine. By default, it is recommended that each node's configuration file use a "drbd.conf" configuration file with the same contents, so that each node has a consistent configuration. However, in some situations, it is not absolutely necessary to set the DRBD node and DRX configuration file resource names differently. To specify a different configuration file path or name, you can also specify the location of the configuration file using "-c" in the DRX executable option.

DRX works with DRBD by setting Proxy section and Proxy on section in the existing DRBD configuration file as follows. There are some differences in 1:1 replication configuration and 1:N replication configuration, as shown below.

1:1 replication configuration

  • Proxy section

    • memlimit

    • plugin section

  • proxy on [ proxy on hostname ] section in the on [hostname] section

    • inside [ proxy inside ip address]:[ port ]

    • outside [ proxy inside ip address ]:[ port ]

1:N replication configuration

  • Proxy section

    • memlimit

    • plugin section

  • host [ hostname] address [ host ip address ]:[port] via proxy on [proxy on hosname] section in the connection section 

    • inside [ proxy inside ip address]: [ port ]

    • outside [ proxy inside ip address ]: [ port ]

6.1.1. Local method config

  • In case DRX configuration on the same system as node1 and node2, the contents of "drbd.conf" on node1 and node2 are the same on both nodes.

  • The following are examples of a 1:1 replication configuration and a 1:2 replication configuration

  • Note that each port information must be covered by the firewall, and configured so that no ports are used redundantly.

method

Windows config. file

Linux config. file

etc

method

Windows config. file

Linux config. file

etc

1:1 repliction

drbd.conf
resource r0 { device d minor 1; disk d; meta-disk f; net { protocol A; on-congestion pull-ahead; congestion-fill 480M; ping-timeout 30; } proxy { memlimit 500M; # DRX TX Buffer } on node1 { address 10.10.0.250:7789; node-id 0; proxy on node1 { inside 10.10.0.250:7790; outside 10.10.0.250:7791; } } on node2 { address 10.10.0.253:7789; node-id 1; proxy on node2 { inside 10.10.0.253:7790; outside 10.10.0.253:7791; } } }
drbd.conf
resource r0 { device /dev/drbd1; disk /dev/sdb1; meta-disk internal; net { protocol A; on-congestion pull-ahead; congestion-fill 480M; ping-timeout 30; } proxy { memlimit 500M; # DRX TX Buffer } on node1 { address 100.100.10.31:7789; proxy on node1 { inside 100.100.10.31:7790; outside 100.100.10.31:7791; } } on node2 { address 100.100.10.41:7789; proxy on node2 { inside 100.100.100.41:7790; outside 100.100.100.41:7791; } } }

 

 

 

1:2 replication

  • Windows: MDR

  • Linux: stacked 1:2 proxy

drbd.conf
resource r2 { startup { wfc-timeout 1; } proxy { memlimit 500M; } net { protocol A; verify-alg crc32c; on-congestion pull-ahead; sndbuf-size 20M; congestion-fill 480M; ping-timeout 30; } device e minor 2; disk e; meta-disk g "C:/r2_meta.vhd"; on node1 { node-id 1; } on node2 { node-id 2; } on node3 { node-id 3; } connection DR=node1=node3 { # Begin with DR and mark as delimiter between nodes = host node1 address 100.100.100.9:6677 via proxy on node1 { inside 100.100.100.9:6678; outside 100.100.100.9:6679; } host node3 address 100.100.100.11:6677 via proxy on node3 { inside 100.100.100.11:6678; outside 100.100.100.11:6679; } } connection DR=node2=node3 { # Begin with DR and mark as delimiter between nodes = host node2 address 100.100.100.10:6680 via proxy on node2 { inside 100.100.100.10:6681; outside 100.100.100.10:6682; } host node3 address 100.100.100.11:6680 via proxy on node3 { inside 100.100.100.11:6681; outside 100.100.100.11:6682; } } connection { host node1 address 10.20.25.9:6677; host node2 address 10.20.25.10:6677; } }
drbd.conf
global { disable-ip-verification; usage-count no; } common { startup { wfc-timeout 1; } } resource r0{ protocol A; disk { on-io-error detach; } net { ping-timeout 30; } syncer{ rate 100M; verify-alg md5; } on pr1{ device /dev/drbd0; disk /dev/sdb1; address 200.200.23.132:7799; meta-disk internal; } on pr2{ device /dev/drbd0; disk /dev/sdb1; address 200.200.23.137:7799; meta-disk internal; } } resource r0_1{ protocol A; meta-disk internal; device /dev/drbd10; proxy{ memlimit 100M; plugin { lzma level 9; } } net{ on-congestion pull-ahead; congestion-fill 80M; ping-timeout 30; } stacked-on-top-of r0{ address 127.0.0.1:7788; proxy on pr1 pr2{ inside 127.0.0.1:7789; outside 10.30.0.135:7790; # mirror VIP } } on dd1{ address 127.0.0.1:7788; proxy on dd1 { inside 127.0.0.1:7789; outside 10.30.0.199:7790; #DR mirror IP } } }

 

6.1.2. Dedicated server configuration

 

By default, config file shoud be configured both nodes and DRX nodes identically. However, due to the nature of the dedicated server, DRX config needs to be modified to accommodate the characteristics of drbd-utils installed on the dedicated server to accommodate this type of node.

The procedure is as described below and the configuration examples are based on a Linux machine.

  • Windows server

    • Copy the "drbdadm.exe" file to the DRX server

    • Create an etc folder in the parent folder of the folder where the "drbdadm.exe" file is located

    • Configure "drbd.conf" in the "etc" path

  • Linux server

    • Install drbd-utils

    • Configure "drbd.conf" in the etc path

Local nodes


LAN

Local DRX


WAN

Remote DRX


LAN

Remote nodes

Windows node r0
resource r0 { device d minor 1; disk d; meta-disk e; proxy { memlimit 500M; # DRX TX Buffer } disk { resync-rate 100M; } net { protocol A; ping-timeout 30; } on w12std-0 { node-id 0; } on w12std-1 { node-id 1; } connection p-s { host w12std-0 address 10.10.0.77:7789 via proxy on 74proxy1 { inside 10.10.0.115:7790; outside 10.10.0.115:7791; } host w12std-1 address 10.10.0.203:7789 via proxy on 74proxy2 { inside 10.10.0.102:7790; outside 10.10.0.102:7791; } } }
r0
resource r0 { device /dev/drbd001; # dummy disk /dev/sdb1; # dummy meta-disk /dev/sdc1; # dummy disk { resync-rate 100M; } net{ protocol A; ping-timeout 30; } proxy { memlimit 300M; #drx tx buf } on w12std-0 { node-id 0; } on w12std-1 { node-id 1; } connection p-s { host w12std-0 address 10.10.0.77:7789 via proxy on 74proxy1 { inside 10.10.0.115:7790; outside 10.10.0.115:7791; } host w12std-1 address 10.10.0.203:7789 via proxy on 74proxy2 { inside 10.10.0.102:7790; outside 10.10.0.102:7791; } } }
r1
resource r1 { device /dev/drbd001; disk /dev/sdb1; meta-disk /dev/sdc1; disk { resync-rate 100M; } net{ protocol A; ping-timeout 30; } proxy { memlimit 300M; #drx tx bufl } on cen74-1 { node-id 0; } on cen74-2 { node-id 1; } connection DR-p-s { host cen74-1 address 10.10.0.11:7789 via proxy on 74proxy1 { inside 10.10.0.115:7790; outside 10.10.0.115:7791; } host cen74-2 address 10.10.0.105:7789 via proxy on 74proxy2 { inside 10.10.0.102:7790; outside 10.10.0.102:7791; } } }
r0
resource r0 { device /dev/drbd001; # dummy disk /dev/sdb1; # dummy meta-disk /dev/sdc1; # dummy disk { resync-rate 100M; } net{ protocol A; ping-timeout 30; } proxy { memlimit 300M; #drx tx buf } on w12std-0 { node-id 0; } on w12std-1 { node-id 1; } connection p-s { host w12std-0 address 10.10.0.77:7789 via proxy on 74proxy1 { inside 10.10.0.115:7790; outside 10.10.0.115:7791; } host w12std-1 address 10.10.0.203:7789 via proxy on 74proxy2 { inside 10.10.0.102:7790; outside 10.10.0.102:7791; } } }
r1
resource r1 { device /dev/drbd001; disk /dev/sdb1; meta-disk /dev/sdc1; disk { resync-rate 100M; } net{ protocol A; ping-timeout 30; } proxy { memlimit 300M; #drx tx bufl } on cen74-1 { node-id 0; } on cen74-2 { node-id 1; } connection DR-p-s { host cen74-1 address 10.10.0.11:7789 via proxy on 74proxy1 { inside 10.10.0.115:7790; outside 10.10.0.115:7791; } host cen74-2 address 10.10.0.105:7789 via proxy on 74proxy2 { inside 10.10.0.102:7790; outside 10.10.0.102:7791; } } }
Windows node r0
resource r0 { device d minor 1; disk d; meta-disk e; proxy { memlimit 500M; # DRX TX Buffer } disk { resync-rate 100M; } net { protocol A; ping-timeout 30; } on w12std-0 { node-id 0; } on w12std-1 { node-id 1; } connection p-s { host w12std-0 address 10.10.0.77:7789 via proxy on 74proxy1 { inside 10.10.0.115:7790; outside 10.10.0.115:7791; } host w12std-1 address 10.10.0.203:7789 via proxy on 74proxy2 { inside 10.10.0.102:7790; outside 10.10.0.102:7791; } } }
Linux node r1
resource r1 { device /dev/drbd0; disk /dev/sdb1; meta-disk /dev/sdc1; disk { resync-rate 100M; } net{ protocol A; ping-timeout 30; } proxy { memlimit 300M; #drx tx buf } on cen74-1 { node-id 0; } on cen74-2 { node-id 1; } connection DR-p-s { host cen74-1 address 10.10.0.11:7789 via proxy on 74proxy1 { inside 10.10.0.115:7790; outside 10.10.0.115:7791; } host cen74-2 address 10.10.0.105:7789 via proxy on 74proxy2 { inside 10.10.0.102:7790; outside 10.10.0.102:7791; } }
Linux node r1
resource r1 { device /dev/drbd0; disk /dev/sdb1; meta-disk /dev/sdc1; disk { resync-rate 100M; } net{ protocol A; ping-timeout 30; } proxy { memlimit 300M; #drx tx bufl } on cen74-1 { node-id 0; } on cen74-2 { node-id 1; } connection DR-p-s { host cen74-1 address 10.10.0.11:7789 via proxy on 74proxy1 { inside 10.10.0.115:7790; outside 10.10.0.115:7791; } host cen74-2 address 10.10.0.105:7789 via proxy on 74proxy2 { inside 10.10.0.102:7790; outside 10.10.0.102:7791; } } }

 

6.2. DRX configuration procedure

6.2.1. Local Method Configuration

Linux
  1. Creating related settings of a proxy at the "drbd.conf" file

  2. Installing DRX

  3. Copying the license file to the directory(drx.lic) where DRX has been installed

  4. By using one of the following commands, run the DRX service depending on your CentOS version:

    CentOS 6

    CentOS 7

Windows
  1. Createing proxy related settings in "drbd.conf"

  2. Installing DRX

  3. Copying the license file to the drx installation directory(drx.lic)

  4. Shut down and run DRX by using the following commands:

    DRX shutdown

    DRX restart

When reconstructing config
  1. Replacing the new created "drbd.conf" or "proxy config" file with the existing configuration file

  2. Resourcing termination with command "drbdadm down"

  3. Starting resource with command "drbdadm up" 

  4. Exiting DRX by using the following command:

  5. Restarting DRX by using one of the the following commands for each OS:

    Windows

    Linux Centos 6

    Linux CentOS 7

 

6.2.2.  Dedicated Server Configuration

Initial configuration

 

reconfiguration

 

If the above procedure is followed, DRBD and DRX will establish a connection and start replication.

 

6.3. Configuring DRX Properties

With a DRX property file, DRX can define its own properties, Apart from the DRBD configuration. Normally, you do not need to set up a property file because it works with DRX default settings.

This properties is applied when DRX is started. Here is a description of the corresponding DRX property file:

  • file name: "drx.conf"

  • path: The same directory as the executable file

  • etc: You must specify "[global] section" at the beginning of this property file

properties

See the following table for the properties of the DRX property file.

configuration

properties

default value

description

configuration

properties

default value

description

basic

lang

korean

DRX language configuration file

  • Located in the "base-dir/lang" in the preference(It is located under the executable file when it is not specified).

  • Create a file, such as "korean.ini", "english.ini", etc(utf8 file)
    e.g. "korean.ini" settings

base-dir

DRX Install path

Specifies the parent directory where the log storage and language settings files are located(If not specified, it will be DRX installation path).

 

connect-timeout

10

The timeout value used in initialization packet exchange after connection with remote DRX.(The unit is seconds).

(※ It is recommended that you increase the connection time if the connection is not smooth because there may be a connection delay depending on the network environment.)

bwlimit

0 (disabled)

Bandwidth limit value for sending and receiving.

loglevel-file

info

Specifies the log level to be saved as a file.

  • Linux is stored in the "/opt/DRX/logs" directory.

loglevel-system 

none

Specifies the log level stored in the system log.

  • Linux: service.log

compress/crypto


block-size

63K

Compression / encryption unit block size

bab-read-spin-counter

10

The number of retries in case bab is smaller than the block size when read.

bab-read-spin-delay

10

Retry Cycle delay value(ms) for bab read 

min-raw-block-size

1K

Minimum size to send to plain text even when setting compression / encryption

process-thread-count

-1 (auto)

Thread count for compress/encryption process. (0: disable)

※ This value is for tuning and debugging purposes. It is recommended that do not changing it.

 

drx.conf sample

drx.conf

drx.conf

#
# DRX Properties
#

[global]

lang=korean
drx_path=./

#drbd-conf=c:\program files\DRX\drbd.conf

#connect-timeout=10

# log levels: debug < info < warn < err < fatal
loglevel-file=info

block-size=1M # Compression / encryption unit block size (default 63K)
bab-read-spin-counter=10 # The number of retries in case bab is smaller than the block size when read
bab-read-spin-delay=10 # Retry Cycle delay value(ms) for bab read 
min-raw-block-size=1K # Minimum size to send to plain text even when setting compression / encryption

 

 

7. Administrative command

 

Manage DRX with "drx-ctl" command.

  • Provides "show" series commands and "print statistics" series commands of "drbd proxy" compatible command.

  • It does not provide configuration commands such as "add", "del", "set", etc., and can only be configured through the drbd.config configuration file.

 

7.1. Command

 

drx-ctl show 

Print information and status of currently configured resources.

C:\Users\sekim>drx-ctl show Command : show ================================================================================================================================================================================ Name Type Status listen-lan-ip remote-drx-ip local-drx-wan-ip local-drbd-ip Up since ================================================================================================================================================================================ r0-2008r2-x64-pas1-2008r2-x64-act meta established 10.10.0.245:7790 10.10.0.246:7791 10.10.0.245:7791 10.10.0.245:7789 2017-08-17 17:48:40 r0-2008r2-x64-pas1-2008r2-x64-act data established 10.10.0.245:7790 10.10.0.246:7791 10.10.0.245:7791 10.10.0.245:7789 2017-08-21 09:18:58 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- All connections : 2

 

  • Name: channel name

  • type: data or meta

  • Status: connection status

    • stopped: Stopped state

    • waiting: Waiting for connection

    • connecting: Connecting with local drbd

    • bridged: Local DRX and remote DRX connected

    • connected: Local DRX and Local DRBD Connected State

    • establishing: Local DRX and Remote DRX Connections, Local DRBD Connected State

    • established: Local DRX and remote DRX connection, local DRBD Remote DRBD both connected (replication established state)

  • listen-lan-ip

  • remote-drx-ip 

  • local-drx-wan-ip

  • local-drbd-ip

  • up since: drbd/drx Start time.

 

drx-ctl show states

Print detail connection status of currently configured resources.

C:\Users\sekim>drx-ctl show states Command id (4) : show states ========================================================================================================================================================= Name Type local-drbd-ip State local-drx-ip State remote-drx-ip ========================================================================================================================================================= p-s-drxdev1-DESKTOP-L141PH6 data 192.168.35.112:7789 -established-> 192.168.35.1:7790/192.168.35.1:7791 <-bridged----- 192.168.35.101:7791 p-s-drxdev1-DESKTOP-L141PH6 meta 192.168.35.112:7789 <-established- 192.168.35.1:7790/192.168.35.1:7791 -----bridged-> 192.168.35.101:7791 --------------------------------------------------------------------------------------------------------------------------------------------------------- All connections : 2, Established : 2

 

  • Name: channel name

  • Type: data or meta

  • State: connection status (arrow: connection direction)

    • stopped: Stopped state

    • waiting: Waiting for connection

    • connecting: Connecting with local drbd

    • bridged: Local DRX and remote DRX connected

    • connected: Local DRX and Local DRBD Connected State

    • establishing: Local DRX and Remote DRX Connections, Local DRBD Connected State

    • established: Local DRX and remote DRX connection, local DRBD Remote DRBD both connected (replication established state)

  • local-drbd-ip

  • local-drx-ip

  • remote-drx-ip