VMworld 2015: Virtual Volumes Technical Deep Dive

50 %
50 %
Information about VMworld 2015: Virtual Volumes Technical Deep Dive

Published on July 8, 2016

Author: VMworld

Source: slideshare.net

1. Virtual Volumes Technical Deep Dive Ken Werneburg, VMware, Inc Patrick Dirks, VMware, Inc STO4649 #STO4649

2. • This presentation may contain product features that are currently under development. • This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. • Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. • Technical feasibility and market demand will affect final delivery. • Pricing and packaging for any new technologies or features discussed or presented have not been determined. Disclaimer CONFIDENTIAL 2

3. Today’s Challenges in External Storage Architectures External Storage vSphere Array-a Array-b Today 1. Create fixed-size, uniform LUNs 2. Lack of granular control 3. Complex Provisioning cycles 4. LUN centric storage configurations ✖ Extensive manual bookkeeping to match VMs to LUNs ✖ LUN-granularity hinders per-VM SLAs ✖ Overprovisioning (better safe than sorry!) ✖ Wasted resources, wasted time, high costs ✖ Frequent Data Migrations LUN LUN LUN LUN LUN CONFIDENTIAL 3

4. Instead… an App-centric Model Drives Agility and QoS 5 App-centric Automation • Dynamic delivery of storage service levels when needed • Fine control of data services at the VM level • Common management across heterogeneous devices  Rapid provisioning  No overprovisioning of resources  QoS automation  Simple change management Today • Static pre-allocation of shared storage container (LUN) • Data services tied to storage container • Vendor specific management ✖ Long provisioning cycles ✖ Overprovisioning of resources Today’s Infrastructure-centric Model ✖ Management complexity ✖ Frequent data migrations CONFIDENTIAL

5. Replication Snapshots Caching Encryption De-duplication vSphere External Storage Architectures with Virtual Volumes External Storage with VVols Policy based Management Offloaded Data Services Eliminates LUN Management Provides Per-VM Granularity datastore a External Storage without VVols vSphere Array-a Array-b LUN LUN LUN LUN LUN AND CONFIDENTIAL 7

6. Agenda 1 What are VVols? 2 Management Plane 3 Data Plane 4 Consumption Model - SPBM 5 VM Lifecycle with VVols 6 VVol Snapshots 7 Questions and Answers CONFIDENTIAL 9

7. vSphere Virtual Volumes A More Efficient Operational Model For External Storage

8. VMware vSphere Virtual Volumes 13 Integration Framework for VM-Aware Storage Virtual Volumes Overview • Virtual disks are natively represented on arrays • Enables VM granular storage operations using array-based data services • Extends vSphere Storage Policy-Based Management to the storage ecosystem • Supports existing storage I/O protocols (FC, iSCSI, NFS) • Based on T10 industry standards • Industry-wide initiative supported by major storage vendors • Included with vSphere CONFIDENTIAL

9. High Level Architecture Published Capabilities Snapshot Replication Deduplication Encryption Overview • No FileSystem • ESX manages Array through VASA (vSphere APIs for Storage Awareness) APIs. • Arrays are logically partitioned into containers, called Storage Containers • VM disks, called Virtual Volumes, stored natively on the Storage Containers. • IO from ESX to array is addressed through an access point called, Protocol Endpoint (PE) • Data Services are offloaded to the array • Managed through storage policy-based management framework vSphere Storage Policy-Based Mgmt. Virtual Volumes Storage Policy Capacity Availability Performance Data Protection Security PE VASA Provider CONFIDENTIAL 14

10. vSphere Virtual Volumes • Virtual Volumes – Virtual machine objects stored natively on the array – No Filesystem on-disk formatting required • There are five different types of recognized Virtual Volumes: – CONFIG – vmx, logs, nvram, log files, etc – DATA – VMDKs – MEM – Snapshots – SWAP – Swap files – Other – vSphere solution specific type vSphere Web Client View vvol CONFIDENTIAL 16

11. Storage Container Storage Containers • Logical storage constructs for grouping of virtual volumes. • Typically defined and setup by storage administrators on the array in order to define: – Storage capacity allocations and restrictions • Capacity is based on physical storage capacity • Logically partition or isolate VMs with diverse storage needs and requirement – Storage policy settings based on data service capabilities • Minimum one storage container per array • Maximum depends on the array vSphere Virtual Volumes SAN / NAS Storage Containers CONFIDENTIAL 17

12. Differences between Storage Containers and LUNs • Size based on array capacity • Max number of SCs depend only on the array ability • Size of SC can be extended • Can distinguish heterogeneous capabilities for different VMs (Virtual Volumes) provisioned in that SC • Fixed size mandates more number of LUNs • Needs a FileSystem • Can only apply homogeneous capability on all VMs (VMDKs) provisioned in that LUN. • Managed by In-band FileSystem commands Storage Containers LUN CONFIDENTIAL 19

13. Storage Container (SC) vSphere Web Client Storage Management UI Datastore Storage Container • What do the Admins see? • How are the storage containers setup? • What does the vSphere Admins see? • Why are we still creating datastores in this new model? CONFIDENTIAL 20

14. Virtual Volumes VM objects view from a storage container on an arrayVM objects view from a datastore vSphere Web Client Storage Management UI vSphere Admin View Storage Admin View CONFIDENTIAL 22

15. Management Plane vSphere Virtual Volumes

16. VASA Provider (VP) • Software component developed by storage array vendors • ESX and vCenter Server connect to VASA Provider (VP) using standard VASA protocol • VP provides Storage Awareness services and presents array’s SPBM capabilities • Single VP can manage multiple arrays • Interface between VP and array is private and VP can be designed to run anywhere (e.g. within the array’s management server, in firmware, or outside the array) • Responsible for creating and managing Virtual Volumes SAN / NAS Virtual Datastore Standard SAN/NAS Data Path Protocol Endpoint(s) vSphere Virtual Volumes Storage Container VASA Provider (VP) Standard VASA Control Path Private array Control Path CONFIDENTIAL 24 PE PE PE

17. Data Plane vSphere Virtual Volumes

18. Discovery Procedures – Storage Container Virtual Volumes Snapshot Replication vCenter VASA Provider Storage Container Discovery Process • Storage admin sets up Storage Container with desired capacity • Desired Capabilities are configured for the Storage Containers • VASA Provider (VP)’s URL is registered in VC • VASA Provider presents Storage Container and its capabilities to vCenter • Any new VMs that are created will subsequently be provisioned in the Storage Container based on SPBM profile associated with virtual disks CONFIDENTIAL 26

19. Protocol Endpoints Protocol Endpoints • Access points that enables communication between ESXi hosts and storage array systems. – Part of the physical storage fabric – Created by Storage administrators Scope of Protocol Endpoints • Compatible with standard SAN and NAS Protocols: - iSCSI - FC - FCoE - NFS v3 • Existing multi-path policies and NFS topology requirements are applied to the PE Why Protocol Endpoints? SAN / NAS Virtual Datastore Data Path Protocol Endpoint(s) PE vSphere Virtual Volumes Storage Container CONFIDENTIAL 27

20. Discovery Procedures – Protocol Endpoint Protocol Endpoint discovery process • VASA Provider (VP) reports PEs for container to VC at registration • When Virtual Datastore is created on host, VC passes PE information to host • SCSI PEs are discovered during an ESX rescan • NFS PEs are configured and mounted automatically • Hosts report to VP which PEs are accessible SAN / NAS Virtual Datastore Data Path Protocol Endpoint(s) PE vSphere Virtual Volumes Storage Container VASA Provider CONFIDENTIAL 29

21. Binding Operations • Bindings are data paths created by VP, in coordination with the array, on request from ESXi hosts, between hosts and array for accessing virtual volume. • Different Binding Mechanism: • Binding – requests creation of I/O channels for a virtual volume • Unbind – destroys the I/O channel for a virtual volume to a given ESXi host • Rebind – provides the ability to change the I/O channel (PE) for a given virtual volumes run time using events. create bind unbinddelete VM Creation VM power-on open (2) I/O read (2) I/O write (2) VM power-off close (2) VM destroy VP rebalance REBIND I/O Virtual Volume Lifecycle CONFIDENTIAL 30

22. Bind Requests • IO path is established through a VASA Bind request • VASA Provider does two things upon receiving a Bind request • Coordinates with array to select a PE and unique secondary ID for VVol to be bound • Returns a PE ID and secondary ID to which the VVol is bound to host • For SCSI, the secondary ID is the secondary LUN ID • For NAS, the secondary ID is the file path • M:M Relationship between VVol and PE Virtual Volumes VASA Provider VVol PE VVol Bind CONFIDENTIAL 32

23. CONFIDENTIAL Virtual Volumes – Other Bind Scenarios UNBIND Virtual Volumes VASA Provider VVol PE VVol UNBIND • Delete binding of a VVol to the PE • VASA Provider deletes binding on request of host (usualy at VM power-off) REBIND Virtual Volumes VASA Provider VVol PE VVol PE REBIND • Move VVols access to different PE • VASA Provider may choose to issue a rebind for several reasons, such as load balancing or storage migration 33

24. VVol Consumption Model: Ensuring SLOs through SPBM – Storage Policy-Based Management

25. Storage Capabilities and VM Storage Policies • Storage Capabilities – are array based features and data services specifications that capture storage requirements that can be satisfied by a storage arrays advertised as capabilities • Storage capabilities define what an array can offer to storage containers as opposed to what the VM requires • Arrays Storage Capabilities are advertises to vSphere through the Vendor Provider and VASA APIs • In vSphere Storage Capabilities are consumed via VM Storage Policy constructs • VM Storage Policies is a component of the vSphere Storage Policy-based management framework (SPBM) SPBM object manager virtual disk Datastore Profile VM Storage Policy vSphere VM Storage Policy Management Framework Storage Capabilities for Storage Array Access Capacity Published Capabilities Snapshot Replication Deduplication QoS Virtual Datastore CONFIDENTIAL 35

26. Storage Policy Based Management (SPBM) – Array Capabilities Virtual Volumes APIs Storage Policy-Based Mgmt. CV CV CV Storage admin Publish Capabilities • Array based features and data services • Defines what an Array can offer • Advertised to ESX through VASA APIs Sample Default Profile for (6090a058-cd89-ffe3-87763007db37] - capabilities Disk Types Disk Encryption Dedupe Replication Snapshot CONFIDENTIAL 36

27. Storage Policy Based Management (SPBM) – VM Policies CONFIDENTIAL 37

28. Storage Policy Based Management (SPBM) CONFIDENTIAL 38

29. Management Workflow • What do the Admins see? • How are the storage containers setup? • What does the vSphere Admins see? • Why are we still creating datastores in this new model? Storage policies vSphere Web Client Storage Management UI Datastore Storage Container Storage Capabilities virtual volumes virtual machines CONFIDENTIAL 39

30. Provision and Lifecycle Workflows

31. vvol DATA vvol CONF vvol SWAP vvol DATA vvol CONF vvol SWAP Provisioning Workflow storage arrays PE vSphere Admin 1. Create Virtual Machines 2. Assign a VM Storage Policy 3. Choose a suitable Datastore Under the Covers • Provisioning operations are translated into VASA API calls in order to create the individual virtual volumes. Under the Covers • Provisioning operations are offloaded to the array for the creation of virtual volumes on the storage container that match the capabilities defined in the VM Storage Policies offloadtoarray Virtual Datastore Storage Container vSphere Virtual Volumes CONFIDENTIAL 41

32. Snapshots • Snapshots are a point in time copy on write image of a Virtual Volume with a different ID from the original • Virtual Volumes snapshots are useful in the contexts of creating: – a quiesced copy for backup or archival purposes, creating a test and rollback environment for applications, instantly provisioning application images, and so on • Two type of snapshots supported: – Managed Snapshot – Managed by ESX • A maximum of 32 vSphere managed snapshot are supported for linked clones of an individual VM – Unmanaged Snapshot – Manage by the storage array • Maximum snapshot dictated by the storage array Managed Snapshot - vSphere Unmanaged Snapshot - Array CONFIDENTIAL 42

33. Snapshots: Files vs. Vvols flat filemain.vmdk main.vmdk redo logsnap.vmdk snap.vmdk VVol ID 86 VVol ID 42 CONFIDENTIAL 43

34. Revert to Snapshot: Files vs. Vvols flat filemain.vmdk main.vmdk redo logsnap.vmdk snap.vmdk VVol ID 86 VVol ID 42 backout VVol ID 243 Snap Shot data CONFIDENTIAL 44

35. vSphere Virtual Volumes Is An Industry-wide Initiative 46 Multiple Available Now Unique capabilities And Many More… 39Partnersin theProgram CONFIDENTIAL

36. The Benefits of vSphere Virtual Volumes 47 A More Efficient Operational Model For External Storage Improves Resource Utilization • Increase capacity utilization. • Eliminate overprovisioning • Reduce management overhead • Eliminate inefficient handoffs between VI and Storage Admin • Faster storage provisioning through automation • Simplified change management through flexible consumption • Self-service provisioning via cloud automation tools Simplifies Storage Operations • Leverage native array-based capabilities • Fine control at the VM level • Dynamic configuration on the fly • Ensure compliance through policy enforcement using automation Simplifies Delivery of Service Levels CONFIDENTIAL

37. Virtual Volumes Technical Deep Dive Ken Werneburg, VMware, Inc Patrick Dirks, VMware, Inc STO4649 #STO4649

38. Backup

39. Operation Scenarios Offload, Migrations, Snapshots 53

40. Migration Scenario: with VAAI vs Virtual Volumes VASA API Compatible Array 2 vSphere VMFS VVOLs 1 vSphere Admins Migrate VM from VMFS to VVOL datastore software data mover implementation CONFIDENTIAL 54

41. 55 vSphere Virtual Volumes Offload Operations vSphere VMFS VVOLs vSphere Admins offload to array offloadtoarray Virtual Machine Operation Offloaded • Virtual Machine provisioning • Virtual Machine deletes • Virtual Machine full clones • Virtual Machine Linked Clones • Virtual Machine Snapshots • Storage vMotion (Powered off VMs) CONFIDENTIAL

42. VM operation from VVOLs container to different VVOLs container vSphere Admins vSphere 1 2 offloadtoarray 3 Fully VAAI & VASA APIs Compatible Array SC-A SC-B always attempt default operation with VASA API primitives If default operations fails VAAI API primitives are used VM operation from VVOLs container to different VVOLs container vSphere Admins vmkernel data mover uses VAAI primitives for cloning operation vSphere 1 2 offloadtoarray 3 Fully VAAI & VASA APIs Compatible Array SC-A SC-B default1 fail back 2 vendor native clone utilized with VASA primitives cloning operation VAAI vs VVOLs CONFIDENTIAL 58

Add a comment

Related pages

VMworld 2015: STO4649 - Virtual Volumes Technical Deep ...

... STO4649 - Virtual Volumes Technical Deep Dive ... Virtual SAN Architecture Deep Dive ... VMworld 2015: NET5560 - VMware NSX Deep Dive ...
Read more

VMworld 2015: STO4649 – Virtual Volumes Technical Deep ...

via VMware VMworld ... About The Author VMware - VMware is the leader in virtualization and cloud infrastructure solutions that enable businesses to thrive ...
Read more

VMworld 2015: NET4976 - vSphere Distributed Switch 6 ...

VMworld 2015: NET4976 - vSphere Distributed Switch 6 ... VMworld 2015: STO4649 - Virtual Volumes Technical ... Switch Technical Deep Dive: ...
Read more

VMworld 2015: STO4649 – Virtual Volumes Technical Deep ...

via VMware VMworld ... Share this with your friends via: VMworld 2015: STO4649 – Virtual Volumes Technical Deep Dive
Read more

VMworld 2015: STO4649 - Virtual Volumes Technical Deep ...

By accessing and using this Site, you agree to be bound by all Pakistan's applicable laws and regulations.Any user promoting video that contains Religious ...
Read more

Explore VMware Virtual SAN and More at VMworld 2015 ...

Explore VMware Virtual SAN and ... from Virtual SAN to Virtual Volumes. Attend VMworld 2015 to learn how you can ... Virtual Volumes Technical Deep Dive:
Read more

virtualguru.org: #VMworld Virtual Volumes Technical Deep Dive

#VMworld Virtual Volumes Technical Deep Dive ... The secret sauce of Virtual Volumes "VVOLs" is the Storage Policy ... 2015 (37) ...
Read more

VMworld 2015 TAM Event Recommendations - VMware Blogs

VMworld 2015 TAM Event ... , Virtual SAN, Virtual Volumes and end use experience ... We will perform a technical deep dive into subjects such as ...
Read more