7-8 SCSI Lock
- 정은진
- 헌규임
- JuneL (Unlicensed)
SCSI reservation agent can manage and protect the shared disks from the H/W perspectives.
If you use a SCSI reservation agent, you can safely protect the data disk against the split brain situations caused by external elements such as OS hanging.
If the shared storage supports SCSI3-PR (Persistent Reservation), you can use it together with SCSI reservation agent to improve the level of security for the shared data.
If you want to access the storage from systems (many or one) through SCSI3-PR, the system must have a storage reserved key. If not, storage access will not be granted.
* Note - If the storage does not support SCSI3-PR (Persistent Reservation), it cannot use a SCSI-reservation agent.
Table of Contents
How to work
Disk resource added through SCSI Lock agent has registration key for each node and it can be checked by it's attribute.
For an online node, set its own registration key as the reservation key.
SCSI reservation agent uses the reservation keys to differentiate nodes that can be accessed and determine whether disk access should be granted.
To be able to safely protect the systems shared data against the Split-Brain situations, you should define the dependency between the SCSI reservation resource and the shared disk resource.
Requirement
External storage must support SCSI-3 PR.
Volume should be configured by LUN(Logical Unit Number) unit. Partitioned volume in a LUN is not supported.
Types of Shared disk resource
To configure HA with shared storage environment, MCCS provides shared disk agent and SCSI Lock agent.
You can use only shared disk resource, or use with SCSI Lock resource. Shared disk resource depends on SCSI Lock resource and multiple shared disk resource can depends on a SCSI Lock resource.
An environment where only shared disk agent is used
- The disk volume is partitioned in a LUN.
- External storage does not support SCSI-3 PR.
Difference between shared disk agent and SCSI Lock agent
1. Shared disk agent manage disk access in local node, but SCSI Lock agent manage node access in storage.
Disk management function in software perspective (shared disk agent)
- Shared disk agent manages access control of shared disk through 'MOUNT/UMOUNT' command.
- Data on the shared disks will be damaged if you try to access to write from multiple nodes.
Thus, to control the situations, a shared disk agent maintains the umount condition, so that they can only mount the file system from an operation node but cannot access to write from a standby node.
Disk management function in hardware perspective (SCSI Lock agent)
- SCSI Lock agent manage Write control and access control of LUN by using SAN protocol.
- Only the node that has imported the disk with this key can access to the disk.
2. Both the shared disk agent and the SCSI reservation agent can read and write data.
However, a SCSI reservation agent intends to protect the shared storage resource, so you are not recommended to use it alone to read and write data.
Configuration
SCSI-Lock resource can be configured in 2 ways with shared disk resource. Configuration process changes according to the attribute value of SCSI Lock resource.
1. When shared disk and SCSI Lock resource is configured with the same LUN
SCSI Lock agent manages DATA-DISK(shared disk resource) using by MOUNT/UMOUNT command and manages LOCK-DISK(SCSI Lock resource) using by SCSI-3 PR as shown below.
By setting up the dependency between SCSI Lock resources and shared disk resources, any damage of data-disk can be prevented which may occur in such case like Split-Brain.
[Figure] Shared disk and SCSI Lock resource configured in same LUN
DATA-DISK
- It is a disk created by using a shared disk agent. It represents a disk where the actual user data is used.
LOCK-DISK
- It is a disk created by using a SCSI reservation agent.
This is the disk that depends on DATA_DISK used in the shared disk when linking a shared disk agent.
Configuration Sample
1. Shared disk and SCSI Lock resource is configured in same LUN
[Figure] Shared disk and SCSI Lock resource configured in same LUN
First, register shared disk agent according to the procedure and after check if it works normally, check disk of SCSI Lock agent.
Registration Procedures
- Install MCCS on both nodes
- Set group
- Add shared disk resource
- Enable shared disk resource and check offline on both nodes
- Add SCSI Lock resource (Select device path that created shared disk resource)
- Enable SCSI Lock resource and check key in resource attribute of MCCS web console.
- Configure dependency of resources (SCSI Lock resource must go online first)
- Test failover from node 2 to 1
- Test failover from node 1 to 2
- Set application that customer uses
Adding
Add the SCSI reservation application resource to a group.
Before configuring the SCSI reservation resource, you must configure the volume in advance by using the SCSI3 supporting storage.
- When add a resource from a group name, select a group → right click → 'Add Resource'.
- Select 'SCSILOCK' from Resource Type lists and click 'Next' button.
- If you select a device path used for the SCSI reservation resource, a name will be automatically recommended.
[Figure] SCSI Reservation Resource Added - Click the 'Finish' button to add the SCSI reservation resource.
You can immediately check the result in the management web console.
Deleting
Select resource type → right click → delete resource.
The deleted resource will immediately disappear from the management web console.
Status
The following table explains the status switching of the MCCS resource caused by a user's command and the status.
The command assumes that it is generated by a user.
SCSI reservation agent: Manages the SCSI reservation disk.
| Status | Agent command | Description | Note |
|---|---|---|---|
Online Disk key is exclusively reserved at an online node. | Offline | It ignores the reservation key stored. It first creates a new reservation key and switches to offline while maintaining the reservation. | |
| Monitoring | Compares the key reserved in the disk and check if there is a reservation made on the disk. | ||
Offline 1. Disk is reserved but the resource is still offline. | Online | Register a key for a disk reservation and wait for a response from the other node. After going online once, the other node reservation is stolen. This is called preempt. | |
| Monitoring | Compares the key reserved in the disk and check if there is a reservation made on the disk. | ||
Trouble It occurs when a reservation is stolen while going online. | Online | Register a key for a disk reservation and wait for a response from the other node. After going online once, the other node reservation is stolen. This is called preempt. | |
| Offline | It ignores the reservation key stored. It first creates a new reservation key and switches to offline while maintaining the reservation. |
Attribute
Display the unique attribute of SCSI reservation resource.
Attribute | Description | Data Type | Default | Edit | Note |
| DevicePath | Define the device path of disk to be used for reservation. When register device path, DUID of connected disk will automatically obtain. | STRING | X | ||
| DiskDUID | DUID(Device Unique ID, 16 byte as Hex value) of disk is used as identifier. Specify DUID of reserve disk. This value is secure when DriveLetter atribute us specified. Try not to insert this value yourself. DUID information can be checked through scsicmd.cmd command and both nodes need to have the same value. | STRING | X | ||
| Key | MAC address is used as Reserved-Key. This key is automatically set by agent. | STRING | X | ||
| PreemptWaitCount | Time waiting for permission of other node before reservation Waiting time is (PreemptWaitCount * MonitorInterval) seconds. | STRING | 2 times | O | |
| PrType | This attribute decides on how to give the permission of reserve disk access as Persist Type specified in SCSI3-PR spec. There are 6 types but MCCS uses 5 (Write Exclusive Registor Only). | STRING | Prytpe = 5 | O |
scsicmd Command
Syntax
scsicmd [option] [devicePath]
Options
This is the utility to use SCSI 3 PR command more easily. Following is the provided options.
Option | Description |
-c devicePath | It deletes Reserved-Key and Registered-Key of local node of disk. |
-cf devicePath | It forcely deletes Reserved-Key and Registered-Key of local node of disk. This is mostly used when access to disk is inavailable. |
| -r devicePath | Registers diskld-specified disk. |
-l devicePath | If node is yet to reserve, close the specified disk and reserve. |
-lf devicePath | It forcelu closes specified disk and reserve, even node has been reserved. Meaning, it takes over other reservation. |
| -h | It shows help of scsicmd |
no option | It shows the state information regarding disk reservation when there is no other option. |
Print Sample
The output of 'scsicmd' command from source node (C29043F13 is MAC Address of NIC from source node.)
| DUID(DeviceUniqueID) | DevicePath | PR | Reserved-Key | Registered-Keys |
|---|---|---|---|---|
| /dev/sda | No | none | none | |
| /dev/sdb | No | none | none | |
| 00c0ffd796550000f38be34f01000000 | /dev/sdc | Yes | none | none |
| 00c0ffd796550000138ce34f01000000 | /dev/sdd | Yes | C29043F13* | C29043F13* |
| 00c0ffd7965500002e8ce34f01000000 | /dev/sde | Yes | none | none |
The output of 'scsicmd' command from target node (C29043F13 is MAC Address of NIC from source node.)
| DUID(DeviceUniqueID) | DevicePath | PR | Reserved-Key | Registered-Keys |
|---|---|---|---|---|
| /dev/sda | No | none | none | |
| /dev/sdb | No | none | none | |
| 00c0ffd796550000f38be34f01000000 | /dev/sdc | Yes | none | none |
| 00c0ffd796550000138ce34f01000000 | /dev/sdd | Yes | C29043F13 | C29043F13 |
| 00c0ffd7965500002e8ce34f01000000 | /dev/sde | Yes | none | none |
Since '/dev/sda' is set as "No"in PR, it shows this does not support SCSI-3 PR. (Normally '/dev/sda' is Booting disk in local node and may dose not support this in many case.)
Also, '/dev/sdb' is set as "No" in PR. It shows this also does not support SCSI-3 PR. (Normally, Internal Disk does not SCSI3-PR.)
'/dev/sdc' and 'dev/sde' are set as "Yes" in PR, they uses storage device that supports SCSI3-PR, but Reserved-Key and Registered-Keys are "none" which means these LUN have never used as SCSI Lock resource in MCCS.
For the "dev/sdd" device, Reserved-Key and Registered-Key at the source node are set to C29043F13* and C29043F13* and at the target node to C29043F13 and C29043F13. This means that a SCSI reservation agent was used to create the resource.
The Reserved-Key and Registered-Keys with (*) marked in the both node indicate that a disk is reserved. In addition, only the * marked node with the Reserved-Key value (ex: C29043F13*) can access the disk.