Storage Spaces Direct is a nice new feature in Server 2016 that lets you do fail-over and redundancy without a SAN. Traditionally your nodes would be connected to a SAN and if one node went down the other node could pick up the VMs that died and carry on. S2D is different, where by there is no SAN, but a bunch of SATA/SAS disks in each server.
Each server holds a copy of its data, and a mirror of the other server's data. If one node goes down then the other node can failover the VMs and carry on with its locally held copy of the VM data. When the downed node comes back up, data is resynced between servers; no SANs are involved!
To do this we must use 10 GbE nics in the servers and in the switches too. I have two Dell T630s hooked up to two Dell S3048 switches and things are not working nicely. I can do live migration between nodes, but when I shut one node down its VMs migrate over to the other node, but then it loses sight of the CSV folders and of course the VMs then stop (enter pause state).
We have to enable the Data Center Bridging role (among others) on the servers and configure the their 10 GbE NICs like so:
# Enable Data Center Bridging (required for RDMA)
Install-WindowsFeature -Name Data-Center-Bridging
# Configure a QoS policy for SMB-Direct
New-NetQosPolicy "SMB" -NetDirectPortMatchCondition 445 -PriorityValue8021Action 3
# Turn on Flow Control for SMB
Enable-NetQosFlowControl -Priority 3
# Make sure flow control is off for other traffic
Disable-NetQosFlowControl -Priority 0,1,2,4,5,6,7
# Apply a Quality of Service (QoS) policy to the target adapters
Enable-NetAdapterQos -Name "Mellanox 1","Mellanox 2"
# Give SMB Direct a minimum bandwidth of 50%
New-NetQosTrafficClass "SMB" -Priority 3 -BandwidthPercentage 50 -Algorithm ETS
# Disable Flow Control on physical adapters
Set-NetAdapterAdvancedProperty -Name "Mellanox 1" -RegistryKeyword "*FlowControl" -RegistryValue 0
Set-NetAdapterAdvancedProperty -Name "Mellanox 2" -RegistryKeyword "*FlowControl" -RegistryValue
On Lenovo switches, we must do a "cee enable" to enable Priority Flow Control
In the "S4810 Switch Configuration Guide for EqualLogic SAN" Dell i see commands like "dcb enable" which my S3048 doesn't have.
So does anyone know if my Dell S3048 switches are suitable for using with Storage Spaces Direct (S2D)?
Thank you for your time.