Mark Jones, FCIA President, Director of Tech Marketing, Broadcom Limited
In early 2015, the Fibre Channel Industry Association (FCIA) announced the completion of a collection of new INCITS T11 standards referred to as Gen6 Fibre Channel. As the name suggests, Gen6 points to the sixth generation on the Fibre Channel roadmap fundamentally defining the single lane speed of 32GFC. As has been tradition with Fibre Channel every new speed step doubles the link throughput over the previous generation with the current 32GFC capable of 3200MB/s of single lane half-duplex data throughput.
Gen6 Fibre Channel products are now being shipped today by most of the major industry suppliers and we will see many major server and storage vendors shipping solutions with these components in the near future.
Because Gen6 Fibre Channel is backwards compatible at least two generations of speeds, customers have the opportunity to use these products at the full 32GFC speed or connect Gen6 products into previous generation products. Auto-speed negotiation means that the connection will operate at the highest common speed between the two connections.
The connection combination for maximum application performance is going to occur when Gen6 is deployed at 32GFC from server to switch and switch to storage, this will allow for end to end throughput of up to 3200MB/s on a single lane. This same application performance level is possible to be achieved in mixed speed configurations, most storage arrays ship with multiple target ports for redundancy and performance, using multiple lower speed (16GFC or 8GFC) target ports in a load balanced multipath configuration will provide the same throughput level to the application. As a result, there is no reason to delay adopting Gen6 products now as they become available for both maximum future-proofing as well as achieving maximum application performance.
Achieving optimum application performance is a balancing act between server and storage performance. In the last 20 years there has been continuous improvement within the server: CPU speed/cores, memory and IO performance have all improved at staggering rates. Storage array performance has improved at a slower pace until recently; the physical performance limitations of spinning disk drives has meant that delivering performance levels that match the server could only come from expensive solutions with large spindle counts and cache sizes. This imbalance between server and storage has kept the pressure off of the SAN link speed which often has had more than adequate performance to deliver the data at a pace that traditional storage arrays were capable of.
Solid State Drives (SSD), on the other hand, and all flash technologies in storage arrays has brought tremendous performance capabilities to low- and high-cost storage segments alike, has quickly becoming mainstream, and will likely surpass spinning disk arrays during the lifespan of Gen6. Now with mega-performance servers and storage on both ends of the SAN the Fibre Channel link speed is now the critical element in the configuration.
With all elements in place, performance-matched servers, storage and Gen6 Fibre Channel the only question remaining is can common datacenter applications take advantage of the performance that Gen6 Fibre Channel is capable of delivering? The FCIA member companies using Gen6 32GFC HBAs and switches connected to all flash storage arrays conducted a series of application tests to prove this point.
As an aid to product readiness the FCIA hosted it’s 36th Fibre Channel Plugfest at the University of New Hampshire Interoperability lab, eleven FCIA member companies participated in the event to test Gen6 Fibre Channel products. The Plugfest also was the venue for another milestone for the Fibre Channel industry, it was the first industry wide proof-of-concept demonstration of the new NVMe over Fabrics specification as transported over Fibre Channel, part of the FC-NVMe specification being developed within INCITS T11.
One key result from the testing was that it isn’t necessary to have all components in the SAN to be of the same link speed in order to take advantage of the most current Fibre Channel link speed. The configurations of the tests used all-flash storage arrays with Fibre Channel interface speeds of just 8GFC, using four ports of 8GFC the array is capable of equaling the throughput of 32GFC when host multipathing is enabled. Key to the configuration is the Gen6 switch that is capable of 32GFC and is backward compatible of speeds down to 8GFC. In general, three speeds of Fibre Channel are currently dominant in the marketplace, so our application benchmarks compared the performance 8/16/32GFC speeds by using HBA models from each generation.
Database – Data Warehousing
During testing we used a benchmark that closely follows the industry standard – decision support benchmark TPC-H on both Microsoft SQL server 2016 and Oracle 12c database servers. The benchmark uses a series of 22 business oriented queries to a data warehouse instance stored on the SAN attached flash array. The operation must read in the data from the SAN into the database server’s memory in order to perform the queries, with the dataset being many times larger than the server’s memory. Since little opportunity exists to maintain the data in the server’s cache, SAN performance is key to how fast the queries can be completed. Completing the queries in a shorter amount of time means that enterprises can run decision support queries on larger datasets, run more complex queries and deliver greater intelligence to the business.
Virtualization VDI “Bootstorm”
A Virtual Desktop infrastructure (VDI) is a datacenter compute environment that hosts desktop computers within a virtual machine running in a centralized virtual server. This infrastructure often stores the desktop data in a centralized Fibre Channel SAN connected to the VDI virtual server. There are significant organizational benefits to deploying VDI from costs saving and centralized IT efficiencies.
Storage IO performance between the VDI virtual server and storage infrastructure can be a critical factor for user satisfaction and efficiencies. A VDI “Bootstorm” is the event when many VDI users attempt to boot their desktops and begin work at the same time, such as when work begins in the morning, after lunch or recovering from an outage.
For our benchmark we focused on “Heavy” VDI users, which are users that will not only boot their environment but also open numerous large applications and load large amounts of data into their applications to begin work. This particular event will put a heavy load on the Fibre Channel link between the VDI server and storage array, so in our benchmark we compared the time for the boot operations to complete when 8/16/32GFC speeds are used.
As can be seen in the figure below, the results confirm that using Gen6 32GFC can help organizations VDI users become more efficient, potentially support more desktop users per server (VM Density) and ultimately lead to more datacenter cost savings.
Virtualization VM Storage Migration
Live migration is one of the great benefits of server virtualization. It allows for actively running virtual machines to move to another nearby virtual server for resource-balancing and serviceability.
Moving VMs beyond their local resource pools or clusters is the next level of benefit but it involves moving the storage out of the current storage resource and into another, which can affect the storage I/O connections and can cause congestion to other running virtual servers and VMs. Storage migration is a feature offered by all the major server virtualization suppliers and our benchmarking has again shown that Gen6 32G Fibre Channel can offer significant performance improvements.
As indicated in the graphic below, our benchmarking used the storage migration features of these virtualization environments to time how long it takes to migrate a number of virtual machines storage pools from one storage array to another. Again, we changes the HBA to Switch link rate by changing HBA models that support 8, 16 and 32GFC speeds within the virtual servers.
With these benchmark results we have made the case that Gen6 Fibre Channel should be considered for many common high storage throughput applications. Also we conclusively showed that end-to-end speed matching of your SAN is not necessary in order to take advantage of application benefits today with Gen6. Gen6 Fibre Channel products are available today and will be more widely available from major server and storage OEMs in the near future.
