Cisco MDS 9706 Multilayer Director for IBM System Storage
IBM Redbooks Product Guide
Published 23 February 2015, updated 01 June 2018
View online
Authors: Megan Gilge
Abstract
This IBM® Redbooks® Product Guide describes the Cisco MDS 9706 Multilayer Director for IBM System Storage® (9710-E06). The MDS 9706 is a director-class SAN switch that is designed for deployment in small- to medium-sized storage networks that can support enterprise clouds and business transformation. It layers a comprehensive set of intelligent features onto a high-performance, protocol-independent switch fabric.
MDS 9706 addresses the stringent requirements of large virtualized data center storage environments. It delivers uncompromising availability, security, scalability, ease of management, and transparent integration of new technologies for flexible data center SAN solutions. It shares the same operating system and management interface with other Cisco data center switches. By using the MDS 9706, you can transparently deploy unified fabrics with Fibre Channel, IBM Fibre Connection (IBM FICON®), Fibre Channel over Ethernet (FCoE), and Fibre Channel over IP (FCIP) connectivity for low total cost of ownership (TCO).
For mission-critical enterprise storage networks that require secure, robust, cost-effective business-continuance services, the FCIP extension module is designed to deliver outstanding SAN extension performance, reducing latency for disk and tape operations with FCIP acceleration features, including FCIP write acceleration and FCIP tape write and read acceleration.
Contents
This IBM® Redbooks® Product Guide describes the Cisco MDS 9706 Multilayer Director for IBM System Storage® (9710-E06). The MDS 9706 is a director-class SAN switch that is designed for deployment in small- to medium-sized storage networks that can support enterprise clouds and business transformation. It layers a comprehensive set of intelligent features into a high-performance, protocol-independent switch fabric.
MDS 9706 addresses the stringent requirements of large virtualized data center storage environments. It delivers uncompromising availability, security, scalability, ease of management, and transparent integration of new technologies for flexible data center SAN solutions. It shares the same operating system and management interface with other Cisco data center switches. By using the MDS 9706, you can transparently deploy unified fabrics with Fibre Channel, IBM Fibre Connection (IBM FICON®), Fibre Channel over Ethernet (FCoE), and Fibre Channel over IP (FCIP) connectivity for low total cost of ownership (TCO).
For mission-critical enterprise storage networks that require secure, robust, cost-effective business-continuance services, the FCIP extension module is designed to deliver outstanding SAN extension performance, reducing latency for disk and tape operations with FCIP acceleration features, including FCIP write acceleration and FCIP tape write and read acceleration.
Figure 1 shows the MDS 9706.
Figure 1. MDS 9706
Did you know?
- The MDS 9706 is 15.6 inches tall (9RU) and allows up to four MDS 9706 Directors per standard 7-foot rack (42RU).
- You can lower TCO with Fibre Channel, FICON, and FCoE SAN consolidation.
- The MDS 9706 provides up to 12 terabits per second (Tbps) front-panel, Fibre Channel, line-rate, nonblocking system-level switching capacity.
Product highlights
The MDS 9706 offers several important features, which are described in this section.
Lower TCO with SAN consolidation
Organizations need efficient, cost-effective SANs to keep up with today’s exponential data growth. The MDS 9706 lets you easily consolidate data assets into fewer, larger, and more manageable SANs to reduce hardware footprint and the associated capital and operational expenses. It offers industry-leading scalability with the following features:
- Up to 192 32-Gbps, 192 16-Gbps Fibre Channel, or 192 10-Gbps FCoE ports; or up to 96 40-Gbps FCoE ports, per chassis
- Up to 12 Tbps front-panel, Fibre Channel, line-rate, nonblocking system-level switching capacity
- Enable large and scalable deployment of SAN extension solutions with the MDS 9000 24/10-port SAN Extension Module
- Exceptional capabilities with intelligent fabric services
- Virtual SANs (VSANs) for consolidating individual physical SAN islands while maintaining logical boundaries
- Inter-VSAN routing (IVR) for sharing resources across VSANs
For unified fabric deployments with converged LAN and SAN that use lossless Ethernet, the MDS 9706 provides multihop FCoE. Protect your organization’s investments in storage infrastructure with any-to-any connectivity across multiple protocols.
Scalable expansion with outstanding investment protection
The MDS 9706 is designed to make optimal use of valuable data center floor space. It is 15.6 inches tall (9RU) and allows up to four MDS 9706 Directors per standard 7-foot rack (42RU). A smaller footprint makes it an excellent candidate for deployment in smaller storage networks and pod-based converged data center infrastructure solutions for the cloud.
By using Cisco MDS 9700 family switching modules, the MDS 9706 supports up to 192 ports in a 6-slot modular chassis, with up to 768 ports in a single rack. You can configure ports as Fibre Channel (2/4/8-Gbps, 4/8/16-Gbps, 8/16/32-Gbps, or 10-Gbps), FCoE (10-Gbps or 40-Gbps), or a mix of Fibre Channel and FCoE. The MDS 9706 supports the same Fibre Channel and FCoE switching modules as the MDS 9710 director for a high degree of system commonality. Designed to grow with your storage environment, the MDS 9706 provides smooth migration, common sparing, and outstanding investment protection.
The MDS 9000 24/10-Port SAN Extension Module is supported on Cisco MDS 9700 Series Multilayer Directors. With 24 line-rate 2-, 4-, 8-, 10-, and 16-Gbps Fibre Channel ports and eight 1 and 10 Gigabit Ethernet FCIP ports, this module enables large and scalable deployment of SAN extension solutions.
Enterprise-class availability
The MDS 9706 is designed from the beginning for high availability. In addition to meeting the basic requirements of nondisruptive software upgrades and redundancy of all critical hardware components, the MDS 9706 software architecture offers outstanding availability. It provides redundancy on all major hardware components, including the supervisor, fabric modules, and power supplies. The Cisco MDS 9700 Series Supervisor Module automatically restarts failed processes, which makes the MDS 9706 exceptionally robust. In the rare event that a supervisor module is reset, complete synchronization between the active and standby supervisor modules helps ensure stateful failover with no disruption of traffic.
FCIP for remote SAN extension
FCIP for remote SAN extension simplifies data-protection and business-continuance strategies by enabling backup, remote replication, and other disaster-recovery services over WAN distances using open standards FCIP tunneling. It optimizes utilization of WAN resources for backup and replication by enabling hardware-based compression, hardware-based encryption, FCIP write acceleration, and tape read and write acceleration for both FCIP and FICON over IP. The SAN extension module will support four tunnels per interface and can scale up to 32 tunnels (4 tunnels x 8 1/10GE ports).
Business transformation with enterprise cloud deployment
Enterprise clouds provide organizations with elastic computing and network capabilities, which enables IT to scale resources up or down as needed in a quick and cost-efficient manner. MDS 9706 provides industry-leading scalability and the following features for enterprise cloud deployments:
- Pay-as-you-grow flexibility to meet the scalability needs in the cloud
- Multihop FCoE to provision storage in a multiprotocol unified fabric
- Robust security for multitenant cloud applications
- Predictable performance to meet stringent service-level agreements (SLAs)
- Resilient connectivity for an always-on cloud infrastructure
- Advanced traffic management capabilities, such as quality of service (QoS), to allocate network capabilities to cloud applications rapidly and cost-efficiently
Furthermore, Cisco Data Center Network Manager (DCNM) provides resource monitoring and capacity planning on a per-virtual machine basis. You can federate up to 10 Cisco DCNM servers to easily manage large clouds. Resource use information can be delivered through Storage Management Initiative Specification (SMI-S)-based developer APIs to deliver IT as a service.
Convergence with multihop FCoE
FCoE allows an evolutionary approach to network and I/O convergence by preserving all Fibre Channel constructs, maintaining the latency, security, and traffic management attributes of Fibre Channel, and preserving investments in Fibre Channel tools, training, and SANs.
Comprehensive solution for robust security
The MDS 9706 offers an extensive security framework to protect highly sensitive data crossing today's enterprise storage networks. It employs intelligent packet inspection at the port level, including the application of ACLs for hardware enforcement of zones, VSANs, and advanced port-security features. It also uses Fibre Channel Service Protocol (FC-SP) and Cisco TrustSec Fibre Channel link encryption mechanisms to provide comprehensive security for storage networks.
MDS 9000 24/10-Port SAN Extension Module
The MDS 9000 24/10-Port SAN Extension Module is supported on Cisco MDS 9700 Series Multilayer Directors. With 24 line-rate 2-, 4-, 8-, 10-, and 16-Gbps Fibre Channel ports and eight 1 and 10 Gigabit Ethernet FCIP ports, this module enables large and scalable deployment of SAN extension solutions. The SAN extension module has two independent service engines that can each be individually and incrementally enabled to scale as business requirements expand.
The SAN extension module supports the full range of services available on other Cisco MDS 9000 Family Fibre Channel switching modules, including VSAN, security, and traffic management services. The FCIP module uses Cisco expertise and knowledge of IP networks to deliver outstanding SAN extension performance, reducing latency for disk and tape operations with FCIP acceleration features, including FCIP write acceleration and FCIP tape write and read acceleration.
Hardware-based encryption helps secure sensitive traffic with IP Security (IPsec), and hardware-based compression dramatically enhances performance for both high- and low-speed links, enabling immediate cost savings in expensive WAN infrastructure. Multiple FCIP interfaces within a single engine or across service engines can be grouped into a port channel of up to 16 links for high availability and increased aggregate throughput.
Figure 2 shows the MDS 9000 24/10-Port SAN Extension Module.
Figure 2. MDS 9000 24/10-Port SAN Extension Module
Main features and benefits
The MDS 9000 24/10-Port SAN Extension Module is designed for mission-critical enterprise storage networks that require secure, robust, cost-effective business-continuance services. The SAN extension module offers the following main features:
- FCIP for remote SAN extension:
- Simplifies data-protection and business-continuance strategies by enabling backup, remote replication, and other disaster-recovery services over WAN distances using open standards FCIP tunneling.
- Optimizes utilization of WAN resources for backup and replication by enabling hardware-based compression, hardware-based encryption, FCIP write acceleration, and tape read and write acceleration for both FCIP and FICON over IP. The SAN extension module will support four tunnels per interface and can scale up to 32 tunnels (four tunnels x eight 1/10GE ports).
- Preserves MDS 9000 Family enhanced capabilities, including VSANs, advanced traffic management, and security, across remote connections.
- Integrated IP storage services in a high-density form factor: The module supports eight 1 and 10 Gigabit Ethernet ports. Individual ports can be configured with hot-swappable shortwave and longwave Small Form-Factor Pluggable (SFP) connections.
- Integrated hardware-based VSANs and IVR: The module enables deployment of large-scale multisite and heterogeneous SAN topologies. Integration into port-level hardware allows any port in a system or fabric to be partitioned into any VSAN. Integrated hardware-based IVR provides line-rate routing between any ports in a system or fabric without the need for external routing appliances.
- Intelligent network services: The module uses VSAN technology for hardware-enforced, isolated environments in a single physical fabric, access control lists (ACLs) for hardware-based intelligent frame processing, and advanced traffic management features such as fabric-wide QoS to facilitate migration from SAN islands to enterprise wide storage networks.
- Sophisticated diagnostics: The module provides intelligent diagnostics, protocol decoding, and network analysis tools as well as integrated Cisco Call Home capability for greater reliability, faster problem resolution, and reduced service costs.
- Comprehensive network security framework: The module supports RADIUS and TACACS+, FC-SP, Secure File Transfer Protocol (SFTP), Secure Shell (SSH) Protocol, Simple Network Management Protocol Version 3 (SNMPv3) implementing the Advanced Encryption Standard (AES), VSANs, hardware-enforced zoning, ACLs, and per-VSAN role-based access control (RBAC). RBAC provides separate control over management functions and access on a per-VSAN basis, enabling separation of duties among administrators on the same physical switch. Gigabit Ethernet ports support IPsec authentication, data integrity, and hardware-assisted data encryption.
- IP Version 6 (IPv6) support: The module supports IPv6 as mandated by the US Department of Defense (DoD), Japan, and China. IPv6 support is provided for FCIP and for management traffic routed in band and out of band.
Integrated FCIP for remote SAN and mainframe channel extension
Data-distribution, data-protection, and business-continuance services are significant components of today’s information-centered businesses. The capability to efficiently replicate critical data on a global scale helps ensure a higher level of data protection for valuable corporate information, and also increases utilization of backup resources and lowers total cost of storage ownership. The MDS 9000 24/10-Port SAN Extension Module uses the open-standards FCIP protocol to extend the distance of current Fibre Channel and FICON solutions, enabling interconnection of SAN islands over extended distances.
Advanced SAN extension features
The MDS 9000 24/10-Port SAN Extension Module supports hardware-based FCIP compression to increase the effective WAN bandwidth of SAN extension solutions. The module can deliver compression ratios in the range of 4:1 to 5:1 over a wide variety of data sources.
The SAN extension module supports AES 256 IPsec encryption for secure transmission of sensitive data over extended distances. Hardware enablement of IPsec helps ensure line-rate throughput. Together, hardware-based compression and hardware-based encryption provide a high-performance, highly secure SAN extension capability.
Additionally, the SAN extension module supports FCIP write acceleration, a feature that can significantly improve application performance when storage traffic is extended across long distances. When FCIP write acceleration is enabled, WAN throughput is optimized by reducing the latency of command acknowledgments.
VSANs
Well suited for efficient, secure SAN consolidation, ANSI T11-standard VSANs enable more efficient storage network utilization by creating hardware-based isolated environments with a single physical SAN fabric or switch. Each VSAN can be zoned as a typical SAN and maintained with its own fabric services for greater scalability and resilience. VSANs allow the cost of SAN infrastructure to be shared among more users, while helping ensure segregation of traffic and retaining independent control of configuration on a VSAN-by-VSAN basis.
Integrated SAN routing
In another step toward deployment of efficient, cost-effective, consolidated storage networks, the MDS 9000 24/10-Port SAN Extension Module supports IVR, the industry’s first and most efficient routing function for Fibre Channel. IVR allows selective transfer of data between specific initiators and targets on different VSANs while maintaining isolation of control traffic within each VSAN. With IVR, data can transit VSAN boundaries while maintaining control-plane isolation, thereby maintaining fabric stability and availability. IVR eliminates the need for external routing appliances, greatly increasing routing scalability while delivering line-rate routing performance, simplifying management, and eliminating the challenges associated with maintaining separate systems. IVR reduces the total cost of SAN ownership.
Advanced traffic management
The advanced traffic management capabilities integrated into the MDS 9000 24/10-Port SAN Extension Module simplify deployment and optimization of large-scale fabrics:
- Virtual output queuing: Helps ensure line-rate performance on each port, independent of traffic pattern, by eliminating head-of-line blocking.
- Port channels: Allow users to aggregate up to 16 FCIP ISLs into a single logical bundle, providing optimized bandwidth utilization across all links. The bundle can consist of any speed-matched ports from any module in the chassis, helping ensure that the bundle can remain active even in the event of a module failure.
- Fabric Shortest Path First (FSPF)–based multipathing: Provides the intelligence to load-balance traffic across up to 16 equal-cost paths and, in the event of a switch failure, dynamically reroute traffic.
- QoS: Can be used to manage bandwidth and control latency, to prioritize critical traffic.
- Shaper: Rate limits the WAN bandwidth according to the maximum bandwidth configured for the FCIP tunnel.
Advanced diagnostics and troubleshooting tools
Management of large-scale storage networks requires proactive diagnostics, tools to verify connectivity and route latency, and mechanisms for capturing and analyzing traffic. The MDS 9000 Family integrates the industry’s most advanced analysis and diagnostic tools. Power-on self-test (POST) and online diagnostics provide proactive health monitoring. The MDS 9000 24/10-Port SAN Extension Module implements diagnostic capabilities, such as Fibre Channel Tracerout, to detail the exact path and timing of flows, and Cisco Switched Port Analyzer (SPAN) to intelligently capture network traffic.
The module also supports the SAN Extension Tuner (SET) tool, which helps you optimize FCIP performance by generating either direct-access (for magnetic disks) or sequential-access (for magnetic tape) Small Computer System Interface (SCSI) I/O commands, and directing this traffic to a specific virtual target. You can specify the size of the test I/O transfers and the number of concurrent or serial I/O operations to generate while testing. The SET tool reports the resulting I/O operations per second (IOPS) and I/O latency, which helps you determine the number of concurrent I/O operations needed to get the best FCIP throughput.
After traffic has been captured, it can be analyzed with the Cisco Fabric Analyzer, an embedded Fibre Channel analyzer. Comprehensive port-based and flow-based statistics facilitate sophisticated performance analysis and SLA accounting. With the MDS 9000 Family, Cisco delivers a comprehensive tool set for troubleshooting and analyzing storage networks.
Comprehensive solution for robust network security
Addressing the need for fail-proof security in storage networks, the MDS 9000 24/10-Port SAN Extension Module offers an extensive security framework to protect highly sensitive data moving in today’s enterprise networks. The module employs intelligent frame inspection at the port level, including the application of ACLs for hardware enforcement of zones, VSANs, and advanced port security features.
- Extended zoning capabilities restrict broadcasts to only the selected zones (broadcast zones).
- VSANs are used to achieve greater security and stability by providing complete isolation among devices that are connected to the same physical SAN.
- FC-SP provides switch-to-switch and host-to-switch Diffie-Hellman Challenge Handshake Authentication Protocol (DH-CHAP) authentication supporting RADIUS and TACACS+, to help ensure that only authorized devices can access protected storage networks.
Cisco MDS 9700 48-Port 32-Gbps Fibre Channel Switching Module
The Cisco MDS 9700 48-Port 32-Gbps Fibre Channel Switching Module (Figure 3) delivers predictable performance, scalability, and innovative features to support private and virtualized data centers. With industry-leading 768 line-rate 32-Gbps Fibre Channel ports per director, the 32-Gbps 48-port Fibre Channel switching module meets the high-performance needs for flash-memory and Non-Volatile Memory Express (NVMe) over Fibre Channel workloads. It offers innovative services including virtual machine awareness, on-board Fibre Channel analytics engine, E-port and F-port diagnostics, integrated VSANs, IVR, and port channels. It delivers full-duplex aggregate performance of 1536 Gbps, making it well suited for high-speed 32-Gbps storage subsystems, 32-Gbps inter-switch links (ISLs), high-performance virtualized servers, and all-flash and NVMe arrays.
Figure 3. Cisco MDS 9700 48-Port 32-Gbps Fibre Channel Switching Module
The MDS 9700 48-Port 32-Gbps Fibre Channel Switching Module enables administrators to scale and consolidate SAN deployments with fewer hardware components. Your SAN administrators can consolidate workloads from hundreds of high-performance virtual machines and scale them with incremental updates as your SAN grows while protecting your existing investment.
This switching module ships with a built-in analytics engine. The engine can analyze real-time Fibre Channel exchanges and report on various metrics in detail, enabling comprehensive and timely monitoring of any potential performance problems among SAN edge devices. The on-board network processing unit (NPU) in the module allows I/O-level metrics to be computed at every switch. The NPU can monitor all flows on all ports at line rate. The NPU examines every exchange passing through the 32-Gbps application-specific integrated circuit (ASIC) to capture flow metrics such as exchange completion time, maximum number of outstanding exchanges, data access latency, read and write IOPS, throughput, logical unit number (LUN) access pattern (sequential or random), and I/O block sizes.
The switching module is hot swappable and compatible with 4-, 8-, 16-, and 32-Gbps Fibre Channel interfaces. This module also supports hot-swappable Enhanced Small Form-Factor Pluggable (SFP+) transceivers. Individual ports can be configured with Cisco 32-, 16-, and 8-Gbps SFP+ transceivers. Each port supports 500 buffer credits for exceptional extensibility without the need for additional licenses. With the Cisco Enterprise Package license, up to 8191 buffer credits can be allocated to an individual port, enabling full link bandwidth over long distances with no degradation in link utilization.
The 48-port 32-Gbps Fibre Channel switching module also provides existing features such as predictable performance, high availability, advanced traffic management capabilities, integrated VSAN and IVR, resilient high-performance ISLs, hardware-assisted slow-drain support, comprehensive security frameworks, fault detection and isolation of errored packets, and sophisticated diagnostics.
Main features
The 48-port Cisco MDS 9700 48-Port 32-Gbps Fibre Channel Switching Module offers the following main features:
- Outstanding SAN performance: The combination of the 32-Gbps Fibre Channel switching module and Fabric-1 crossbar switching modules enables up to 1.5 Tbps of Fibre Channel throughput between the modules in each direction for each payload slot in the Cisco MDS 9700 Series Multilayer Directors. The MDS 9700 Series architecture, which is based on central arbitration and a crossbar fabric, provides 32-Gbps line-rate, nonblocking, predictable performance across all traffic conditions for every port in the chassis.
- High availability: The MDS 9700 Series directors provide outstanding availability and reliability. They are the industry's first director-class switches considered to support mission-critical workloads because they enable redundancy on all major components, including the fabric card. They also provide grid redundancy on the power supply and 1+1 redundant supervisors. Users can add a fabric card to enable N+1 fabric redundancy at 16-Gbps Fibre Channel speeds. The suggested number of modules per chassis is six if there are 32-Gbps line linecards with 32-Gbps transceivers. If you do not want to upgrade, or you want to partially upgrade, review Table 1.
Table 1. Number of Fabric Modules needed according to ports running at 32 Gbps performance in a 48-port 32-Gbps linecard
# of ports Running at (Performance) # of fabric modules needed Restrictions 24 32 Gbps 3 Other 24 ports shut down 32 32 Gbps 4 Other 16 ports shut down 40 32 Gbps 5 Other 8 ports shut down 48 32 Gbps 6 No restrictions
- Industry-leading scalability: The MDS 9700 Series directors combine industry-leading scalability with performance to meet the needs of even the largest data center storage environments. The Cisco MDS 9718 Multilayer Director supports up to 48 Tbps of Fibre Channel system bandwidth and 768 2-, 4-, 8-, 16-, and 32-Gbps full line-rate autosensing Fibre Channel ports in a single chassis. The Cisco MDS 9710 Multilayer Director supports up to 24 Tbps of Fibre Channel system bandwidth and up to 384 2-, 4-, 8-, 16-, and 32-Gbps full line-rate autosensing Fibre Channel ports in a single chassis. The smaller Cisco MDS 9706 Multilayer Director supports up to 12 Tbps of Fibre Channel system bandwidth and 192 2-, 4-, 8-, 16-, and 32-Gbps full line-rate autosensing Fibre Channel ports in a single chassis. The MDS 9700 Series directors can support both the Cisco MDS 9700 48-Port 16-Gbps Fibre Channel Switching Module and the Cisco MDS 9700 48-Port 32-Gbps Fibre Channel Switching Module in the same chassis, providing compatibility with the customer's existing system.
- Intelligent network services: VSAN technology, ACLs for hardware-based intelligent frame processing, and fabricwide QoS enable migration from SAN islands to enterprise wide storage networks.
- Fibre Channel Redirect (FC-Redirect): FC-Redirect infrastructure provides the capability to redirect a flow to a specific service engine in the fabric to provide intelligent services such as Cisco Data Mobility Manager and Cisco MDS 9000 I/O Accelerator.
- Integrated hardware-based VSANs and IVR: VSANs are integrated into port-level hardware so that any port in a system or fabric can be assigned to any VSAN. The IVR provides line-rate routing between any ports in a system or fabric without the need for external routing appliances.
- Intelligent storage services: The MDS 9700 Series directors support intelligent service capabilities on other Cisco MDS 9000 Family switches to provide services such as acceleration of storage applications for data replication and backup.
- Smart Zoning: When the Smart Zoning feature is enabled, MDS 9700 Series fabrics provision the hardware ACL entries, specified by the zone set, more efficiently. This feature helps prevent superfluous entries that may allow servers (initiators) to talk to other servers or allow storage devices (targets) to talk to other storage devices. This feature makes feasible larger zones with multiple initiators and multiple targets without excessive consumption of hardware resources. Smart Zones can correspond to applications, application clusters, hypervisor clusters, and other data center entities, saving the time that administrators previously spent creating many small zones and enabling automation of zoning tasks.
- Virtual machine transparency: The MDS 9700 Series provides deterministic hardware performance and a comprehensive feature set that allows virtual machines to have the same SAN attributes as a physical server. On a per-virtual machine basis, Cisco NX-OS Software offers VSANs, QoS policies, access control, performance monitoring, and data protection to promote the scalability and mobility of virtual machines. Cisco DCNM enables rapid troubleshooting in mission-critical virtualized environments by providing end-to-end visibility from the virtual machine to storage with resource allocation, performance measurements, and predictions available on a per-virtual machine basis.
- Comprehensive security: The MDS 9700 Series devices supports services including VSANs, hardware-enforced zoning, ACLs, per-VSAN RBAC, and Cisco TrustSec Fibre Channel link encryption. The comprehensive security framework consists of RADIUS and TACACS+, FC-SP, SFTP, SSH Protocol, and SNMPv3. The Cisco TrustSec Fibre Channel link encryption delivers transparent, hardware-based 32-Gbps line-rate AES 128-bit encryption of Fibre Channel data on 32-Gbps Fibre Channel switching modules.
- Resilient high-performance ISLs: The MDS 9700 Series 48-port 32-Gbps Fibre Channel switching modules support high-performance ISLs consisting of 32-Gbps secure Fibre Channel. Fibre Channel switching modules also support port-channel technology for additional scalability and resilience. Administrators can group up to 16 links spanning any port on any module in a chassis into a logical link. Administrators can also allocate up to 8191 buffer-to-buffer credits to a single Fibre Channel port, providing industry-leading extension of storage networks to up to 4000 kilometers at 4 Gbps, 2000 kilometers at 8 Gbps, 1000 kilometers at 16 Gbps, and 500 kilometers at 32 Gbps, while maintaining full link utilization. The MDS 9700 Series switch architecture helps ensure that frames are never reordered within a switch, leading to optimal application performance because end devices never have to use resources to reorder the frames.
- Sophisticated diagnostics: The MDS 9700 48-Port 32-Gbps Fibre Channel Switching Module provides intelligent diagnostics, protocol decoding, E-port diagnostics, and network analysis tools along with an integrated Cisco Call Home capability for greater reliability, faster problem resolution, and reduced service costs.
- Compatibility: The MDS 9700 48-Port 32-Gbps Fibre Channel Switching Module can co-exist with previous-generation supported modules on the MDS 9700 Series director.
- Investment protection: The MDS 9700 48-Port 32-Gbps Fibre Channel Switching Module can easily be inserted into existing MDS 9700 Series directors without any impact on operations. Existing 16-Gbps customers can transparently move to 32-Gbps speeds without the need for a major equipment upgrade. Cisco continues its tradition of providing multiple generations of speeds (16Gbps FC, 32Gbps FC, 10GbE FCoE, and 40GbE) on the same chassis, providing customers with exceptional investment protection for their existing systems.
Main benefits
The Cisco MDS 9700 48-Port 32-Gbps Fibre Channel Switching Module offers the following main benefits:
- Lower TCO with SAN consolidation: With the exponential growth of data in today's business environment, organizations need to deploy large-scale SANs in the most efficient and cost-effective ways. To meet scalability requirements while managing TCO, the MDS 9700 Series directors offer the following features:
- Industry-leading port density of line-rate 32-Gbps Fibre Channel ports per chassis
- 1.5-Tbps Fibre Channel performance per slot
- Up to 48-Tbps front-panel Fibre Channel line-rate nonblocking system-level switching
- Exceptional capabilities with intelligent fabric services
- VSANs for consolidating individual physical SAN islands while maintaining logical boundaries
- IVR for sharing resources across VSANs
- Built-in network processing unit for inline analytics
These features enable the consolidation of an organization's data assets into fewer, larger, and more manageable SANs, thus reducing the hardware footprint and associated capital and operating expenses:
- Enterprise-class availability: The MDS 9700 Series was specifically designed from the beginning for high-availability and mission-critical environments. Beyond meeting the basic requirements of nondisruptive software upgrades and redundancy of all critical hardware components, the MDS 9700 Series software architecture offers outstanding greater than five-nines availability.
- Virtual machine-aware SAN deployment: The growing adoption of server virtualization in data centers increases the number of hosts attached to the SAN, places higher workloads on the SAN, and requires more storage, thereby increasing the need for SAN services. The Cisco VMpath technology, part of Cisco DCNM, provides end-to-end visibility from the virtual machines to the storage devices. Resource allocation, performance measurements, and predictions are available on a per-virtual machine basis to enable rapid troubleshooting in mission-critical virtualized environments.
- Advanced traffic management: The following advanced traffic management capabilities, integrated into every MDS 9700 48-Port 32-Gbps Fibre Channel Switching Module, simplify deployment and optimization of large-scale fabrics:
- The virtual output queue (VOQ) helps ensure line-rate performance on each port, independent of traffic pattern, by eliminating head-of-line blocking.
- Up to 8191 buffer-to-buffer credits can be assigned to any individual port for optimal bandwidth utilization across distances.
- Port channels allow up to 16 physical ISLs to be aggregated into a single logical bundle, providing optimized bandwidth utilization across all links. The bundle can be a mix of any port from any module in the chassis. This approach helps ensures that the bundle can remain active even if a module fails.
- Fabric Shortest Path First (FSPF)-based multipathing provides the intelligence to load-balance across up to 16 equal-cost paths and dynamically reroute traffic if a switch fails.
- QoS helps manage bandwidth and control latency to prioritize critical traffic and is available on every port.
- The lossless network-wide in-order delivery guarantee helps ensure that frames are never reordered within a switch. This guarantee extends across the entire multiswitch fabric, assuming that the fabric is stable and no topology changes are made.
- Advanced diagnostics and troubleshooting tools: The MDS 9700 Series integrates proactive diagnostics, tools to verify connectivity and route latency, and to capture and analyze traffic, thereby simplifying the management of large-scale storage networks. The POST and online diagnostics provide proactive health monitoring. Starting with Cisco MDS 9000 NX-OS Software Release 6.2, the powerful Cisco Generic Online Diagnostics (GOLD) framework replaces the Cisco Online Health Management System (OHMS) diagnostic framework on the new MDS 9700 Series Multilayer Director chassis. GOLD is a suite of diagnostic facilities that verify whether the hardware and internal data paths are operating as designed. Boot-time diagnostics, continuous monitoring, standby fabric loopback tests, and on-demand and scheduled tests are part of the GOLD feature set. This industry-leading diagnostics subsystem enables rapid fault isolation and continuous system monitoring, which are critical features in today's continuously operating environments.
- Integrated hardware functions support diagnostic capabilities such as Fibre Channel traceroute to identify the exact path and timing of flows, and Cisco Switched Port Analyzer (SPAN) and Remote SPAN (RSPAN) to intelligently capture network traffic. The captured Fibre Channel traffic can be analyzed with the embedded Cisco Fabric Analyzer. Comprehensive port-based and flow-based statistics enable sophisticated performance analysis and SLA accounting.
- Comprehensive solution for robust security: Addressing the need for stringent security in storage networks, the MDS 9700 Series 32-Gbps Fibre Channel line card offers an extensive security framework to protect the highly sensitive data crossing today's enterprise networks. The MDS 9700 Series employs intelligent packet inspection at the port level, including the application of ACLs for hardware enforcement of zones, VSANs, and advanced port security features. VSANs are used to achieve greater security and stability by providing complete isolation of devices that are connected to the same physical SAN. IVR enables controlled sharing of resources between VSANs.
- In addition, FC-SP1 provides switch-to-switch and host-to-switch DH CHAP authentication supporting RADIUS or TACACS+ to help ensure that only authorized devices access protected storage networks. Cisco TrustSec1 Fibre Channel link encryption, available on the MDS 9700 Series 32-Gbps modules, allows you to transparently encrypt ISLs at up to line-rate speeds, providing an additional layer of protection for traffic within and between data centers. The MDS 9700 Series supports a fabric binding feature that helps ensure that ISLs are enabled only between specified switches in the fabric binding configuration.
- Integrated mainframe support: The MDS 9700 Series is mainframe ready, with full support for FICON for IBM z Systems and Linux environments. The MDS 9700 Series supports transport of the FICON protocol in both cascaded and noncascaded fabrics. It also supports a mix of FICON and open-systems Fibre Channel Protocol (FCP) traffic on the same switch.
Key features
Table 2 summarizes the main features and benefits of the MDS 9706.
| Feature | Benefit |
| Performance and scalability | |
| Outstanding SAN performance | The combination of the 32 Gbps Fibre Channel switch module, 16 Gbps Fibre Channel switching module, or both; and the Fabric-1 crossbar switching modules enables up to 1.5 Tbps of front-panel Fibre Channel throughput between modules in each direction for each of the four MDS 9706 payload slots. This per-slot bandwidth is twice the bandwidth that is needed to support a 48-port 16 Gbps Fibre Channel module at full line rate. At 32 Gbps, all linecards operate at full line rate with six fabric modules installed. Based on central arbitration and crossbar fabric, the MDS 9706 architecture provides 32-Gbps line-rate,16-Gbps line-rate, or both nonblocking, predictable performance across all traffic conditions for every port in the chassis. |
| Industry-leading scalability | The MDS 9706 provides up to 12 Tbps of Fibre Channel system bandwidth and 192 full line-rate autosensing Fibre Channel (2/4/8 Gbps, 4/8/16 Gbps, 8/16/32 Gbps, or 10 Gbps) or 10 Gbps FCoE ports or 96 40 Gbps FCoE ports in a single chassis. It is designed to meet the requirements of large data center storage environments. |
| Reliability and availability | |
| High availability | The MDS 9706 combines nondisruptive software upgrades, stateful process restart and failover, and full redundancy of all major components for best-in-class availability. Redundancy is enabled on all major components, including the fabric card. It provides grid redundancy on the power supply and 1+1 redundant supervisors. Users can add fabric cards to enable N+1 fabric redundancy redundancy when using 16 Gbps FC switching modules. When you are using 32 Gbps transceivers in the 32 Gbps switching modules, see Table 1 on page 8 to determine the number of the fabric modules that are necessary to support availability. |
| Multiprotocol connectivity | |
| Multiprotocol architecture | The multilayer architecture of the Cisco MDS 9700 Series enables a consistent feature set over a protocol-independent switch fabric. The MDS 9706 transparently integrates Fibre Channel, FCoE, and FICON. Consider the following points:
|
| Integrated mainframe support | The Cisco MDS 9706 supports FICON protocol in cascaded and noncascaded fabrics, and an intermix of FICON and open systems Fibre Channel Protocol traffic on the same switch. IBM Control Unit Port (CUP) support enables in-band management of Cisco MDS 9000 Family switches from mainframe management applications. It also supports a fabric-binding feature that helps ensure that inter-switch links (ISLs) are enabled only between specified switches in the fabric-binding configuration. |
| Software features | |
| Advanced traffic management | The following advanced traffic management capabilities in the MDS 9706 simplify deployment and optimization of large-scale fabrics:
|
| Intelligent network services | VSAN technology, ACLs for hardware-based intelligent frame processing, and fabric-wide QoS enable migration from SAN islands to enterprise-wide storage networks and includes the following features:
|
| Virtual machine transparency | The Cisco MDS 9700 Series provides deterministic hardware performance and a comprehensive feature set that allows virtual machines to have the same SAN attributes as a physical server. On a per-virtual machine basis, Cisco NX-OS Software offers VSANs, QoS policies, access control, performance monitoring, and data protection to promote virtual machine scalability and mobility. Cisco DCNM provides end-to-end visibility from the virtual machine to storage, with resource allocation, performance measurements, and predictions available on a per-virtual machine basis. |
| Security | |
| Comprehensive security | Comprehensive security services include VSANs, hardware-enforced zoning, ACLs, per-VSAN role-based access control (RBAC), and Cisco TrustSec Fibre Channel link encryption. The Cisco MDS 9700 Series also supports a comprehensive security framework with RADIUS and TACACS+, FC-SP, Secure File Transfer Protocol (SFTP), Secure Shell (SSH) Protocol, and Simple Network Management Protocol Version 3 (SNMPv3). FC-SP provides switch-to-switch and host-to-switch Diffie-Hellman Challenge Handshake Authentication Protocol (DH-CHAP) supporting RADIUS or TACACS+ to help ensure that only authorized devices access protected storage networks. Cisco TrustSec Fibre Channel link encryption, which is available in the Cisco MDS 9700 Series 16-Gbps modules, allows you to transparently encrypt ISLs at up to line-rate speeds, which providing another layer of protection for traffic within and between data centers. |
| Diagnostics and troubleshooting | |
| Sophisticated diagnostic tests | The MDS 9706 provides intelligent diagnostic tests, protocol decoding, and network analysis tools; and integrated Cisco Call Home capability for added reliability, faster problem resolution, and reduced service costs. Consider the following points:
|
| Management | |
| Ease of management | The Cisco MDS 9700 Series includes built-in storage network management with all features available through a command-line interface (CLI) or Cisco DCNM, which is a centralized management tool that simplifies the management of unified fabrics. DCNM supports integration with third-party storage management applications to allow transparent interaction with existing management tools.
Adhering to the syntax of the widely known Cisco IOS Software CLI, the Cisco MDS 9000 Family CLI is easy to learn and delivers broad management capabilities. This highly efficient direct interface optimizes management. You can enable debugging modes for each switch feature and view a real-time updated activity log of control protocol exchanges. Each log entry is timestamped and listed in chronological order. Through the use of the intuitive GUI, DCNM simplifies day-to-day operations of unified fabrics in highly virtualized data center environments. DCNM SAN Advanced Edition supports the following features:
DCNM can federate up to 10 DCNM servers to manage up to 150,000 devices by using a single management window. The solution can scale to large enterprise deployments through a scale-out server architecture with automated failover capability. You gain a resilient management system that centralizes infrastructure and path monitoring across geographically dispersed data centers. The DCNM base management function is available at no charge, and advanced features are unlocked with a license. DCNM can be installed on Linux and Microsoft Windows operating systems and supports PostgreSQL and Oracle databases. |
Licensing
Table 3 summarizes the optional licenses that can be purchased to enable the extra features and capabilities of the Cisco MDS 9706.
Table 3. Optional licenses
| License | Description |
| MDS 9700 Enterprise Package (FC 7610) | Includes advanced traffic-engineering and network security features, such as IVR, QoS, and zone-based QoS, FC-SP, port security, VSAN-based access control, and fabric binding for open systems. Licensed per switch for all the ports on the switch. |
| MDS 9700 DCNM SAN Advanced Edition (FC 7611) | Includes advanced management capabilities, such as VMware vCenter integration, performance trending, advanced provisioning, backup, and dashboards. Licensed per switch for all the ports on the switch. |
| 9710 Mainframe Package (FC 7612) | Includes FICON protocol support and allows IBM CUP management for in-band management from IBM S/390® and z/900 processors. Licensed per switch for all the ports on the switch. |
Specifications
This section lists the specifications and supported protocols for the MDS 9706.
Table 4 summarizes the MDS 9706 specifications.
Table 4. Product specifications
| Feature | Description |
| Product compatibility | Cisco MDS 9000 Family |
| Software compatibility | Cisco MDS NX-OS Software Release 6.2.11c or later |
| Operating systems | For the most current and complete information, see the IBM System Storage Interoperation Center (SSIC): http://ibm.co/1Pmc6de |
| Optional features |
|
| Indicators |
|
| Fibre Channel standards (protocols) |
|
 |
|
| Chassis slot configuration |
|
| Performance and scalability |
|
| Features and functions | |
| Fabric services |
|
| Advanced functions |
|
| Diagnostic and troubleshooting tools |
|
| Network security |
|
| IBM FICON |
|
| Serviceability |
|
| Reliability and availability |
|
| Network management |
|
| Programming interface |
|
| Approvals and compliance |
|
Table 5 lists the switching capability per fabric.
Table 5. Switching capability per fabric
| Number of fabric cards | Front panel Fibre Channel bandwidth per slot | FCoE bandwidth per slot |
| 1 | 256 Gbps | 220 Gbps |
| 2 | 512 Gbps | 440 Gbps |
| 3 | 768 Gbps | 660 Gbps |
| 4 | 1024 Gbps | 880 Gbps |
| 5 | 1280 Gbps | 1100 Gbps |
| 6 | 1536 Gbps | 1320 Gbps |
Physical specifications
Table 6 lists the physical, electrical, and environmental specifications.
Table 6. Physical, electrical, and environmental specifications
| Item | Description | |
| Power and cooling |
| |
| Power consumption (typical) | MDS 9706 with 3 fabrics (Watts [W]) | |
| Ports | Watts | |
| 96 | 1465 | |
| 192 | 2425 | |
| Environmental |
| |
| Physical dimensions (H x W x D) |
| |
| Weight |
| |
MDS 9000 24/10-Port SAN Extension Module specifications
Table 7 lists the specifications for the MDS 9000 24/10-Port SAN Extension Module.
Table 7. Product specifications for the MDS 9000 24/10-Port SAN Extension Module
| Feature | Description |
| Product compatibility | Cisco MDS 9000 Family |
| Software compatibility | Cisco MDS 9000 NX-OS Software 7.3(0)DY(1) |
| Protocols | Fibre Channel standards:
|
| Feature | |
| Protocols (cont.) |
IP standards:
Ethernet standards:
Internet Key Exchange (IKE) standards:
|
| Cards, ports, and slots | 24 x fixed 2/4/8/10/16-Gbps Fibre Channel ports, 8 x 1/10 Gigabit Ethernet ports |
| Features and Functions | |
| Fabric services |
|
| Feature | |
| Advanced functions |
|
| Diagnostics and troubleshooting tools |
|
| Network security |
|
| Serviceability |
|
| Performance |
|
| Feature | Description |
| Reliability and availability |
|
| Network management | Access methods through Cisco MDS 9700 Series Supervisor Module:
Access protocols:
Network security:
Management applications:
|
| Programming interfaces |
|
| Environmental |
|
| Physical dimensions |
|
| Feature | Description |
| Approvals and compliance | Safety compliance:
EMC compliance:
|
Specifications for the MDS 9700 48-Port 32-Gbps Fibre Channel Switching Module
Table 8 lists the specifications for the MDS 9700 48-Port 32-Gbps Fibre Channel Switching Modules.
Table 8. Specifications for the MDS 9700 48-Port 32-Gbps Fibre Channel Switching Module
| Item | Description |
| Product compatibility | Cisco MDS 9700 Series Multilayer Directors |
| Software compatibility | Cisco MDS 9000 NX-OS Software Release NX-OS 8.1(1) |
| Protocols |
|
| Item | Description |
| Protocols (continued) |
|
| Cards, ports, and slots |
|
| Features and functions | |
| Fabric services | Name server
|
| Item | Description |
| Advanced functions |
|
| Diagnostics and troubleshooting tools |
|
| Network security |
|
| Serviceability |
|
| Performance |
|
| Reliability and availability |
|
| Network management |
|
| Programming interfaces |
|
| Environmental |
|
| Physical dimensions |
|
| Approvals and compliance |
|
Table 9 lists supported Cisco optics, media, and transmission distances.
Table 9. Supported Cisco optics, media, and transmission distances.
| Speed | Media | Distance |
|
|
|
Cisco MDS 9700 Series 48-port 16Gbps Fibre Channel Switching Module
Table 10 lists the product specifications for the Cisco MDS 9700 Series 48-port 16-Gbps Fibre Channel Switching Module.
Table 10. Product specifications for the Cisco MDS 9700 Series 48-port 16-Gbps Fibre Channel Switching Module
| Feature | Description |
| Product compatibility | Cisco MDS 9700 Series |
| Software compatibility | Cisco MDS NX-OS Software Release 6.2.3 or later. NX-OS 6.2.5 is recommended. |
| Cards, ports, and slots |
|
| Feature | Description |
| Fabric services |
|
| Advanced functions |
|
| Diagnostic and troubleshooting tools |
|
| Network security |
|
| Serviceability |
|
| Feature | Description |
| Performance |
|
| Reliability and availability |
|
| Network management |
|
| Programming interfaces |
|
| Physical dimensions |
|
Table 11 lists the product specifications for the Cisco MDS 9700 Series 48-port 10-Gbps FCoE Switching Module.
Table 11. Product specifications for the Cisco MDS 9700 Series 48-port 10-Gbps FCoE Switching Module
| Feature | Description |
| Product compatibility | Cisco MDS 9700 Series Multilayer Directors |
| Software compatibility | Requires Cisco MDS NX-OS Release 6.2(7) or later |
| Feature | Description |
| Ports | 48 fixed autosensing 10 Gbps FCoE ports |
| Slots | Can be used on any payload slot of the MDS 9700 Series directors |
| Fabric services |
|
| Advanced capabilities |
|
| Diagnostic and troubleshooting tools |
|
| Security |
|
| Serviceability |
|
| Performance |
|
| Feature | Description |
| Reliability and availability |
|
| Network management |
|
| Programming interfaces |
|
| Environmental |
|
| Feature | Description |
| Physical dimensions |
|
| Power | Typical: 400 watts (W) |
| Approvals and compliance |
|
Table 12 lists the product specifications for the Cisco MDS 9700 Series 24-port 40-Gbps FCoE Switching Module.
Table 12. Product specifications for the Cisco MDS 9700 Series 24-port 40-Gbps FCoE Switching Module
| Feature | Description |
| Product compatibility | Cisco MDS 9700 Series Multilayer Directors |
| Software compatibility | Requires Cisco MDS 9000 NX-OS Software Release 7.3.0.D1.1 or later |
| Fibre Channel and FCoE protocols |
|
| Feature | Description |
| Fibre Channel and FCoE protocols (cont.) |
|
| Ethernet protocols |
|
| Ports | 24 fixed autosensing 40 Gbps FCoE ports |
| Slots | Can be used on any payload slot of the MDS 9700 Series directors |
| Feature | Description |
| Fabric services |
|
| Advanced capabilities |
|
| Diagnostics and troubleshooting tools |
|
| Security |
|
| Serviceability |
|
| Performance |
|
| Reliability and availability |
|
| Feature | Description |
| Network management |
|
| Programming interfaces |
|
| Environmental |
|
| Physical dimensions |
|
| Power | Typical: 680 watts (W) |
| Feature | Description |
| Approvals and compliance |
|
Product specifications for the Cisco MDS 9700 Series Supervisor-1 Module
Table 13 lists the specifications for the Cisco MDS 9700 Series Supervisor-1 Module.
Table 13. Specifications for the Cisco MDS 9700 Series Supervisor-1 Module
| Feature | Description |
| Product compatibility | Cisco MDS 9700 Series |
| Software compatibility | Cisco MDS NX-OS Software Release 6.2.3 or later. NX-OS 6.2.5 is recommended. |
| Interfaces |
|
| Indicators |
|
| Backplane bandwidth |
|
| Chassis slot configuration | Two Cisco MDS 9700 Series Supervisor-1 modules are required per system. |
| Feature | Description |
| Fabric services |
|
| Advanced functions |
|
| Diagnostic and troubleshooting tools |
|
| Network security |
|
| Serviceability |
|
| Feature | Description |
| Reliability and availability |
|
| Network management |
|
| Programming interface |
|
| Physical dimensions |
|
Related publications and links
For more information, see the following documents:
- Cisco MDS 9148S 16G Multilayer Fabric Switch for IBM System Storage, TIPS1255
http://www.redbooks.ibm.com/abstracts/tips1255.html
- Cisco MDS 9250i Multiservice Fabric Switch for IBM System Storage, TIPS1167
http://www.redbooks.ibm.com/abstracts/tips1167.html
- Cisco MDS 9396S 16G Multilayer Fabric Switch for IBM System Storage, REDP5274
http://www.redbooks.ibm.com/abstracts/redp5274.html
- Cisco MDS 9710 Multilayer Director for IBM System Networking, TIPS1046
http://www.redbooks.ibm.com/abstracts/tips1046.html
- Cisco MDS 9718 Multilayer Director for IBM Storage Networking, REDP5355
http://www.redbooks.ibm.com/abstracts/redp5355.html
- IBM Support Fix Central support site (select a product from drop-down menus)
http://www.ibm.com/support/fixcentral/
- IBM System Storage Interoperation Center (SSIC)
http://www.ibm.com/systems/support/storage/ssic/interoperability.wss
- Cisco MDS 9706 Multilayer Director for IBM System Storage
http://www.ibm.com/systems/storage/san/ctype/9706
Special Notices
This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment.