Careful with that dual-hop FCoE setup with HP Virtual Connect and Nexus 5500/5000

HP just recently launched its new Firmware release, the 4.01 – for its Blade Interconnect darling, the Virtual Connect. You can find the release notes here. Also noteworthy are the cookbooks, and HP just released its “Dual-Hop FCoE with HP Virtual Connect modules Cookbook” .

One of the new features is dual-hop FCoE. What this means is that until so far you had only one hop FCoE from Virtual Connect to the server Converged Network Adaptor (CNA). This facilitates quite a lot FCoE implementation, since you don’t need any of the DCB protocols – its only one hop. Starting from version 4.01 you can now have an uplink connection where you pass FCoE traffic.


The VC serves thus as a FIP-snooping device and the upstream switch as Fiber Channel Forwarder (FCF), in case you want to have retrocompatibility with former existing FCP environments. Please note that only a dual-hop topology is supported. You cannot continue hopping FCoE traffic accross your Network. Well you can, but its just not supported.Dual-hp-FIP-snooping

Some important considerations:

  • On HP side, only VC FlexFabric (FF) and Flex10/10D are able to do dual-hop FCoE. On Cisco’s side, only Nexus 5500 and 5000 (NX-OS version 5.2(1)N1(3)) are supported.
  • Only Nexus FCF mode is supported (NPV mode is not supported), and Nexus ports must be configured as trunk ports and STP edge ports;
  • VC-FF, only ports X1 to X4 support FCoE; VC Flex10/10D supports using all its Uplinks;
  • Only VC uplink Active/Active design is supported; Active/Standby design will not be supported due to potential extra-hop on VC stacking links due to misconfiguration on Server profile;
  • Cross-Connect links between VC uplinks to Nexus are not supported. Even if you’re using vPC, it will be a design violation of traffic mix in SAN A (say connected to Nexus 01) and SAN B (say connected to Nexus 02).

VCFF - Dual-hp-constraints

  • Moreover, if you want to converge all traffic on the same uplink LAG ports, you will have to have a single Shared Uplink Set (SUS) per VC. Note that this SUS can only have one FCoE Network.

Dual-hop SUS

  • However, if you have more uplink portson your VCs available, then consider seperating Ethernet traffic from Ethernet traffic, using SUS for Ethernet, and a vNet for the FCoE traffic. So in this scenario, and when using vPC on the Nexus it is still advisable to cross-connect VC SUS Uplinks from the same VC Module to different Nexus for pure Ethernet traffic.


  • Note: Remember VC rules. You assign uplinks from one VC module to one SUS and uplinks from anotherdifferent VC module to the other SUS. It is very important that you get this right, as VC modules are not capable of doing IRF withemselves (yet). More on how setup VC with IRF here.


  • So when using port-channels on VC FCoE ports, you will need to configure ports on the Nexus to belong to the same Unified Port Controller (UPC) ASIC. Nexus 5010/5020 support a maximum of 4 ports when carrying FCoE traffic, 5548/5596 a maximum of 8.


More on the VC setup and installation “HP Virtual Connect for c-Class BladeSystem Version 4.01 User Guide” here.

Disclamer: note that these are my own notes. HP is not responsable for any of the content here provided.

Setup for VMware, HP Virtual Connect, and HP IRF all together

Trying to setup your ESXi hosts networking with HP Virtual Connect (VC) to work with your HP’s IRF-Clustered ToR switchs in Active-Active mode?


This setup improves performance, as you are able to load balance traffic from several VMs across both NICs, across both Virtual Connect modules, and across both ToR switchs, while in the meanwhile gaining more resilience. Convergence on your network will be much faster, either if a VC uplink fails, a whole VC module fails, or a whole ToR fails.

Here’s a couple of docs that might help you get started. First take a look on the IRF config manual of your corresponding HP’s ToR switches. Just as an example, here’s the config guide for HP’s 58Xo series. Here are some important things you should consider when setting up an IRF cluster:

  1. If you have more than two switches, choose the cluster topology.  HP IRF allows you to setup a daisy-chained cluster, or in ring-topology. Naturally the second one is the recommended for HA reasons.
  2. Choose which ports to assign to the cluster. IRF does not need specific Stacking modules and interfaces to build the cluster. For the majority of HP switchs, the only requirement is to use 10GbE ports, so you can flexibly choose whether to use cheaper CX4 connections, or native SFP+ ports (or modules) with copper Direct Attach Cables or optical transcevers. Your choice. You can assign one or more physical ports into one single IRF port. Note that the maximum ports you can assign to an IRF port varies according to the switch model.
  3. Specify IRF unique Domain for that cluster, and take into consideration that nodes will reboot in the meanwhile during the process, so this should be an offline intervention.
  4. Prevent from Split-brain Condition potential events. IRF functions by electing a single switch in the cluster as the master node (which controls Management and Control plane, while data plain works Active-Active on all nodes for performance). So if by any reason stacking links were to fail, slave nodes by default initiate new master election, which could lead into a split-brain condition. There are several mechanisms to prevent this front happening (called Multi-Active Detection (MAD) protection), such as LACP MAD or BFD MAD.

Next hop is your VC. Whether you have a Flex-Fabric or a Flex-10, in terms of Ethernet configuration everything’s pretty much the same. You setup two logically identical Shared Uplink Sets (SUS) – note that VLAN naming needs to be different though inside the VC. Next you assign uplinks from one VC module to one SUS and uplinks from another different VC module to the other SUS. It is very important that you get this right, as VC modules are not capable of doing IRF withemselves (yet).

If you have doubts on how VC works, well nothing like the VC Cookbook. As with any other documentation that I provide with direct links, I strongly recommend that you google them up just to make sure you use the latest version available online.

If you click on the figure above with the physical diagram you will be redirected to HP’s Virtual Connect and IRF Integration guide (where I got the figure in the first place).

Last hop is your VMware hosts. Though kind of outdated document (as Vmware already launched new ESX releases after 4.0), for configuration purposes the “Deploying a VMware vSphere HA Cluster with HP Virtual Connect FlexFabric” guide already allows you to get started. What you should take into consideration is that you will not be able to use the Distributed vSwitch environment with new vmware’s LACP capabilities, which I’ll cover in just a while. This guide provides indications on how to configure in a standard vSwitch or distributed setup on your ESX boxes (without using VMware’s new dynamic LAG feature).

Note that I am skipping some steps on the actual configuration, such as the Profile configuration on the VC that you need to apply to the ESX hosts. Manuals are always your best friends in such cases, and the provided ones should provide for the majority of steps.

You should make sure that you use redundant Flex-NICs if you want to configure your vSwitchs with redundant uplinks for HA reasons. Flex-NICs are present themselves to the OS (in this case ESXi kernel) as several distinct NICs, when in reallity these are simply Virtual NICs of a single physical port. Usually each half-height HP Blade comes with a single card (the LOM), whcih has two physical ports, each of which can virtualize (up to date) four virtual NICs. So just make sure that you don’t choose randomly which Flex-NIC is used for each vSwitch.

Now comes the tricky part. So you should not that you are configuring Active-Active NICs for one vSwitch.


However, in reality these NICs are not touching the same device. Each of these NICs is mapped statically to a different Virtual Connect module (port 01 to VC module 01, and port 02 to VC module 02). Therefore in a Virtual Connect environment you should note that you cannot choose the load balancing algorithm on the NICs based on IP-hash. Though you can choose any other algorithmn, HP’s recommendation is using Virtual Port ID, as you can see on the guide (this is the little piece of information missing on VC Cookbook on Active-Active config with Vmware).

Finally, so why the heck can’t you use IP-hash, nor VMware’s new LACP capabilities? Well  IP-Hash is intended when using link aggregation mechanisms to a single switch (or Switch cluster like IRF, that behaves and is seen as a single switch). As a matter a fact, IP-SRC-DST is the only algorithm supported for NIC teaming on the ESX box as well for the Switch when using port trunks/EtherChannels:

” […]

  • Supported switch Aggregation algorithm: IP-SRC-DST (short for IP-Source-Destination)
  • Supported Virtual Switch NIC Teaming mode: IP HASH
  • Note: The only load balancing option for vSwitches or vDistributed Switches that can be used with EtherChannel is IP HASH:
    • Do not use beacon probing with IP HASH load balancing.
    • Do not configure standby or unused uplinks with IP HASH load balancing.
    • VMware only supports one EtherChannel per vSwitch or vNetwork Distributed Switch (vDS).

Moreover, VMware explains in another article: ” What does this IP hash configuration has to do in this Link aggregation setup? IP hash algorithm is used to decide which packets get sent over which physical NICs of the link aggregation group (LAG). For example, if you have two NICs in the LAG then the IP hash output of Source/Destination IP and TCP port fields (in the packet) will decide if NIC1 or NIC2 will be used to send the traffic. Thus, we are relying on the variations in the source/destination IP address field of the packets and the hashing algorithm to provide us better distribution of the packets and hence better utilization of the links.

It is clear that if you don’t have any variations in the packet headers you won’t get better distribution. An example of this would be storage access to an nfs server from the virtual machines. On the other hand if you have web servers as virtual machines, you might get better distribution across the LAG. Understanding the workloads or type of traffic in your environment is an important factor to consider while you use link aggregation approach either through static or through LACP feature.

So given the fact that you are connecting your NICs to different VC modules (in a HP c7000 Enclosure), the only way you will be able to have active-active teaming is with static etherchannel, and without IP-Hash.

None-the-less please note that if you were configuring for instance a non-blade server, such as a HP Dl360, and connecting it to an IRF ToR cluster, then in that case you would be able to use vDS with LACP feature, and thus use IP-Hashing. Or if you were using a HP c3000 Enclosure, which is usually positioned for smaller enviroments. So it does not make that much sense to be using a c3000 and in the meanwhile having the required vSphere Enterprise Plus licensing.

That’s it, I suppose. Hopefully this overview should already get your ESX farm up-and-running.


Disclamer: note that these are my own notes. HP is not responsable for any of the content here provided.

Architectural overview of HP’s Vertica DB

If you start digging for more info on HP Vertica, chances are high that you stumble uppon this image, which summarizes it’s key selling points:Vertica

So the first thing one should start talking about should always be the origin of Vertica’s name. Instead of horizontal data population, Vertica stores in a verticaL disposition – a columnar storage of data (imagine a 90 degree turn on you DB, where the properties of each row are now the main agents to consider). The goal is to benefit both in querry as in storage capabilities. Allocation in columnar fashion allows to sort data (no need for indexing, since data is already sorted), which is then followed by an encoding process that allows to store only one time each unique value, as well as how many times that value repeats itself.

Vertica prefers enconding instead of compressing data. You can still querry data while it is on its enconded state. Vertica applies different encoding mechanisms in the same table to different Columns, accordingly . You can also apply both encoding and compression. to the type of data stored. Besides using encoding to provide additional query speed, storage reduction is also in stake. Vertica claims a 4:1 reduction of storage footprint. 

Another important differentiator is Vertica’s distributed grid-like architecture. Likewise Big Data architectures, instead of having hercule muscled machine, you are able to gain the advantages of multiple smaller and, most importantly, less expensive computing nodes working in parralel. In other words, you use a Massively-parallel processing (MPP) system -grid-base architecture clustering a bunch of x86-64 servers. So it is a shared-nothing architecture – each node functions as if it were the only node, and uses its own processing capabilities to return part of the answer of a query that involves that data that it owns.

How to design such a system? Each node has a minimum hardware recommendation of two quad-core CPU, and 16GB of RAM, 1TB of disk space and two VLANs assigned – one for client connection, another for private inter-node communication. In the private inter-node communication VLAN is where nodes distribute queries along them, obtaining parallel processing.

You might be wondering by now, what would happen if you were to loose one or more of those commondity hardware servers. The Db provides all the HA mechanisms you need by spreading data repeatadidly across several nodes. We are talking about a RAID-like storage of the same data at least twice in different nodes, thus eliminating SPOFs and tolerating node/s failure. To do so, Vertica does table segmentation, distributing tables evenly across different nodes. The result: you get a clean HA, without any need for manual log-based recovery.

Queries are indeed run on all nodes, so if one node is done, you will suffer on performance (comparing to multi-node performance, not other DBs!), since at least one node will have to do double work on failure. If you lose enough nodes to loose at least one segment of data, Vertica goes automatically down so not to corrupt the DB.

Also quite neat is the auto-recover process. When a failing node comes back online, other nodes automatically sync data.

As you can see, the answer to the following question is yes, ou can forget about your centrallized SAN environment, and using your big-ass Storage Array. Each node uses local disk which also allows better performance, as redundancy is guaranteed at the DB-level.

Another important aspect is that you can load your DB schema into Vertica, and it uses what they call “Automatic DB designer” to facilitate the whole process of DB design. You can you the designer repeatadily in different points of time, as you evolve to adjust.

Finally Vertica has a standard SQL interface, and supports SQL, ODBC, JDBC and the majority of ETL and BI reporting products.

Disclamer: note that these are my own notes. HP is not responsable for any of the content here provided.

HP’s new Switching platform 11900

HP showcased one of its soon to release new big&mean switching platform in InterOp 2013, the 11900, and today you can already get prices on this peace. Although having been considered hottest products in InterOp 2013, if you ask me, this might not be a game changer for HP’s portfolio as I believe the 12900 might be when it releases. But it is interesting enough to be noted.

This box is positioned for aggregation layer in the Datacenter (HP’s FlexFabric Portfolio), right next to today’s main Datacenter core platform, the 12500. Although probably a lot of work like me in countries qith “slightly” different companies as the US does. In countries where big companies considering substituting their Cisco 6500 with a 11900 might be more than enough, and therefore surely suffice as a core.

So we are talking about another platform with CLOS architecture designed to provide a non-blocking architecture with 4 Fabric Modules (each with 3x 40Gbps per I/O line card), with performance figures of 7,7Tbps, 5.7 Mpps, and achieving latency levels of 3 μs.

As for scalability, we are talking about being able to scale up to 64x 40GbE ports (non-blocking), and 384x 10GbE (also non-blocking) now-a-days. As a matter a fact, technically each I/O line card has 480Gbps (4x3x40) of total backend bandwidth,

Anyway, as you may already suspect, it does indeed run HP’s new version of Network OS, the Comware 7 (Cw7), and will therefore also contain almost all of the cool “Next-Generation” Datacenter features, namely:

  • DCB and FCoE
  • OpenFlow 1.3 (thus standards-based SDN-ready)
  • Though not available in this first release, MDC (context/ Virtual Switchs inside your chassis box) and SPB (sort of alternative to TRILL) will also become available

You might be wondering why the hell have a new Box, when HP has just released new management modules for the 10500, which also bring Comware 7 to it, and some of its features. Note that the 10500 is positioned for Campus-Core, not DC. As such, it will not support features as DCD/FCoE and EVB/VEPA, and those the major differences in the mid-term that you can count on.

As for it being positioned as a Core platform, please note that when it comes to scalability numbers in terms table size of FIB, ARP, MAC, etc 11900 platform does not even scale as near as HP’s 12500. Along with high buffering capability, these might be important features in DC environments with high density and number of VMs.

Networktest also publiced its test results on the 11900. The tests consist essentially of a Spirent TestCenter hitting a L2&3 bombardment simultaneously on all 384 10Gbps ports (all 8 I/O slots filled with 48 10GbE line cards), and another bombardment simultaneously on all 64 40Gbps ports (all 8 I/O slots filled with 8 40GbE line cards).

Disclamer: note that these are my own notes. HP is not responsable for any of the content here provided.

HP 10500 Campus-Core switch now supports new OS

HP just released a new hardware model of Management Processing Units (MPUs) for their  10500 Core switch platforms. (For those of you more Cisco-oriented, we are talking about sort of the equivalent of the Supervisors on a Nexus 7k platform, since the 10500 has a CLOS architecture, with dedicated Switching Fabric modules that offload all the switching between the line cards.) Here are some specs of the new MPU on the right side, along with a (for example) 10508 mapping where they fit:

10500 MPU v2

Now though the hardware specs show a some hardware enhancements (comparing with the previous one, which had a 1GHz single-core CPU, and 128MB of Flash), the most interesting part is definitely the support for the new version of OS: Comware 7.

Until so far the only platforms that supported this OS were the muscled ToR 5900 (yes 5920 as well, but for the sake of simplicity I will try not making it complicated) and the 12500 high-end Switching Core platform. Here are some of the enhanced and exclusive features that the boxes running CW7 were blessed with:

  • ISSU (In-Service Software Update, which means online Software update without forwarding interruption)
  • RBAC
  • DCB Protocols (PFC, DCBX, and ETS) + FCoE (exclusive to the 5900 on this first release)
  • EVB (naturally exclusive to the 5900, as this is a feature intended for ToR)
  • MDC (Multi-Device Context – supports completely isolated Contexts/vSwitches, exclusive to 12500)
  • TRILL (until so far was exclusive to 5900)

So what this means is that HP is bringing some of this juice to its platform positioned for Campus-Core. So if you upgrade your existing MPUs with the new ones, you will be able to add ISSU (without needing a second box clustered with IRF), as well as TRILL

And, as you can imagine, there might be some updates in the near future on which Comware 7 features are supported on this box. Just be aware that 10500 is and will remain (at least for quite a while) a platform positioned for Campus-core, and as such, it does not make sense to add Datacenter funcionalities, such as EVB and DCB/FCoE.


HP SDN beta-testing

HP is beta-testing its Security SDN-based solution: Sentinel.

The article describes a School implementing HP still to come SDN Security solution. The school implemented a hybrid OpenFlow solution – the most likely usual implementation in this initial SDN phase – where “intelligent” switchs are used running all usual old-school networking protocols simultaneously with OpenFlow enabled. OpenFlow is used to forward all DNS request to HP Security Controller – Sentinel. Sentinel uses HP’s IPS DB – Tipping Point’s Reputation DB – which is updated every 2 hours with spoted Internet suspicious threats.  Sentinel accepts or rejects traffic forwarding to a certain IP, based on what network administrator choose to do. The network admin can configure Sentinel to follow all Tipping Point recommendations, or rather specify his prefered alternatives. Thus when an OpenFlow switch requests what to do with a certain DNS querry, the controller simply “tells” what to do with related packets by populating its OpenFlow table.

This might be a very simplistic security implementation. However the most interesting is the promising margin for development. As this solution gains increasing intelligent, this may well start suiting as low-cost IPS/firewall solutions, using a distributed computing model with already existing OpenFlow switchs. I find this model very appealing for instance for ROBO sites.

Another alternative use-case is HP’s Beta-testing example in the article: BYOD. Securing devices at the edge perimeter greatly simplifies network security.

SDN might be a simple reengineering of the way things are done. Still, it’s a cool one in deed…

Disclamer: note that these are my own notes. HP is not responsable for any of the content here provided.

Wondering about the differences between HP AP’s MSM460 and MSM466?


  • Both radios in MSM466 can support simultaneously 5GHz;
  • MSM466 Antennas are external, MSM460 are internal.

Key features in common:

  • Dual 802.11n radios;
  • 450 Mbps Max performance per radio;
  • Number of transmitters/receivers: 3×3
  • 3 Spatial Streams;
  • Lifetime warranty.


Disclamer: note that these are my own notes. HP is not responsable for any of the content here provided.