Browse Author

Erik Hinderer

VMware NSX-T 2.4.1 Upgrade Live Demo

VMware NSX-T was recently announced just a bit ago, so I thought it would be helpful to do a live demo of an upgrade to 2.4.1. In short, there’s good news that all but a few of the upgrades I’ve seen have been successful on the initial attempt thanks to the Pre-Upgrade / Post-Upgrade Checks that are built into each section, but any that weren’t successful on the initial upgrade were due to hitting timeouts waiting for a manager node to respond post upgrade (due to really old, under-performing hardware), which were easily remedied with a check of the error and then retry the Manager Node upgrade.

The 2.4.1 update is a core maintenance release, with a new enhancement for VMware HCX, adding functionality for virtual machine migration to on-premises NSX-T based deployments. I have a client doing this at the moment and can attest that it’s greatly welcome functionality for companies acquiring other companies with overlapping networks. VMware HCX provides capabilities to migrate from disparate versions as well, so migrating workloads from an acquired company with many different ESXi versions, is a real plus.

To begin the upgrade of NSX-T 2.4.1, we download the .mub upgrade file from and log into NSX manager. Once we’re logged into NSX manager, we navigate to System on the top toolbar and then Upgrades on the left navigation pane. In the Upgrade window, choose the location of the NSX .mub upgrade bundle from local disk or URL. After we’ve chosen our upgrade bundle, we click upgrade and the upgrade bundle is validated by NSX Manager and then staged for upgrades. Once the upload status changes to Upgrade Bundle retrieved successfully, we click Begin Upgrade and the VMware NSX Upgrade Coordinator starts.

There are five steps in the upgrade, separated by a clickable top toolbar. Bundle and Status – Hosts – Edges – Controller Nodes – Management Nodes. After accepting the End User Agreement, we run the Host Upgrade on our compute workload clusters that have NSX installed and then we click on Run Post Checks to ensure they’re operable. Edge upgrades are next with the same process as well as Controller Nodes.

The final step in the NSX-T 2.4.1 upgrade process is Management Nodes, which have an option to return the NSX management cluster into service after a single or a 3-node cluster is formed. As a bit of guidance, it’s always a good idea to wait for the 3-node cluster to be operational before a return to service. However, if you’ve got a short outage window or allowance, you can return to service with a single NSX manager node, but be advised, performance usage will increase greatly as other NSX manager nodes rejoin the cluster and sync.

Check out my YouTube video of the VMware NSX-T 2.4.1 Upgrade for a preview of what to expect:

VMware NSX-v Identity Firewall Configuration Overview and How-To Create Active Directory User Access Rules for Applications (with or without NSX networking deployed)

One of the quickest and easiest returns on investment in the VMware product stack is VMware NSX Identity Firewall. What other product, with an hour of configuration and initial policy creation, returns such business outcomes? I’d hesitate to say there’s any such solution out there that can provide such an immediate return on investment. VMware NSX IDFW provides an incredibly valuable and easy to use solution for VDI and data center “jumpboxes” alike. With that said, I’m going to demonstrate how you can deploy and configure VMware NSX Identity Firewall in under an hour and have an identity-based security solution that can be easily inserted into any existing vSphere environment. …with or without deploying VMware NSX networking.

There are two methods IDFW uses for logon detection: Guest Introspection and/or the Active Directory Event Log Scraper. Guest Introspection is deployed on ESXi clusters where IDFW virtual machines are running. When network events are generated by a user, a guest agent installed on the VM forwards the information through the Guest Introspection framework to the NSX Manager. The second option is the Active Directory event log scraper. Configure the Active Directory event log scraper in the NSX Manager to point at an instance of your Active Directory domain controller. NSX Manager will then pull events from the AD security event log. You can use both in your environment, or one or the other. Note that if both the AD event log scraper and Guest Introspection are used, the two are mutually exclusive: if one of these stops working, the other does not begin to work as a back up.

Before we get started, let’s talk reality. While security postures such as Micro-segmentation and Zero-Trust may indeed be your desired end-state, they’re a much longer journey than Macro-segmentation or Application Fencing. With that said, you can imagine how quickly you could create Application Fencing security policies for application servers or groups in your environment and start by simply controlling user access to them. Now, with the understanding that you can accelerate your NSX Identity Firewall implementation by leveraging macro-segmentation strategies in the initial phase, you’ll quickly realize that it will allow you to create more granular micro-segmentation policies in a secondary phase.

My lab scenario will demonstrate how NSX-v Identity Firewall can quickly secure an HR, Finance and CRM application, based on the users Active Directory group. The data center consists of three clusters, one for management and two for compute resources. RegionA01-COMP02 serves the hr-web-01a, fin-web-01a and web-04a VMs that serve each HR, Finance and CRM application respectively. The web VMs are running on a stereotypical “server VLAN”, on one subnet, as commonly seen in many enterprise environments. The jumpbox or VDI desktop, win-12-jump VM, is on a “user VLAN”, in another subnet.

NSX Identity Firewall configuration requires that NSX Manager be deployed and registered with vCenter. The NSX Manager appliance is deployed from OVA via vCenter and takes about 30 minutes to complete deployment and registration to vCenter. For details on installing NSX Manager, read Install the NSX Manager Virtual Appliance.

Requirements for VMware NSX Identity Firewall are:

  • NSX Manager 6.2 or greater (the latest release is recommended)
  • VMware Virtual Distributed Switch or NSX N-VDS
  • FQDN for NSX Manager
  • NTP configured in NSX Manager to the same source as vCenter, vSphere hosts and Active Directory domain controllers
  • AD account to query the domain (this user must be able to access the entire directory tree structure)
  • AD read-only account with permissions to read Windows Event Log locally or via Group Policy (see MS KB)

**In NSX-v, controllers and networking components are not required

After NSX Manager has been deployed and registered to vCenter, we begin configuring IDFW Event Log Scraping by setting the LDAP and CIFs properties for directory queries and event log reading. After setting the LDAP and CIFS properties, we validate that the directory has performed a sync and that the AD event servers have populated in the Event Server fields. Guest Introspection is also deployed, to explain that configuration.

This demonstration video that I created, will guide you step-by-step through the process of configuring VMware NSX Manager to enable Identity Firewall:

Some simple best-practices for leveraging Microsoft Active Directory user groups are:

  • Create new user-groups in a top level OU when possible and then nest existing groups which may be deeper in the forest.
  • Limit the nesting of Active Directory user groups to three(3) deep for best performance.
  • When leveraging a large enterprise forest, configure the LDAP and CIFs properties for directory queries to a smaller child domain that the user groups are in, instead of the top level forest domain.

Once you’ve finished with the installation and configuration of VMware NSX-v Identity Firewall, it’s time to map Active Directory user groups to NSX security groups and create security objects for enforcement. There are static and dynamic objects that can be leveraged. Dynamic objects yield a more simplified security policy, reducing the number of overall rules and security objects needed. Thus, dynamic object types should be used whenever possible. Static NSX object types are IP Sets, vNICs and Virtual Machine names, where as dynamic security object types contain a set of resources grouped by another construct, such as a vCenter Cluster or Datacenter, Distributed Port Group, Legacy Port Group, Logical Switch, Resource Pool or vApp.

The Object Types for an NSX-v firewall rule are: Security Group, IP Sets, Cluster, Datacenter, Distributed Port Group, Legacy Port Group, Logical Switch, Resource Pool, Virtual Machine, vApp and vNIC. With such a wide array of selection criteria, there are many object types that can be leveraged to create a strategic advantage in your security policy.


Now that we’ve configured NSX Identity Firewall, mapped Active Directory user groups to NSX security groups, we’ll create some Active Directory based rules to test access to our applications.

If you want to test blocking without changing the default Layer 3 rule, simply create a rule blocking what you need in a user defined firewall rule section above the rule you want to test. We’ll use this in in the live demo with a firewall rule called VDI to APP that blocks the VDI desktop network, leveraging an IP SET from the Internal Services security group that contains our protected web app servers. See the image below.

Now, let’s login as both users and test the access of the NetAdmin and HRAdmin users to see if they have the appropriate access.

Hope you enjoyed this post and feel free to hit me up on Twitter, LinkedIn and subscribe to my YouTube Channel with any requests for content that you’d like to see. Until next time, as the sun sets slowly in the west, I bid you a fond farewell. Adios amigos!

NSX Design Sessions – Lessons Learned – Two Separate Design Groups – Functional before Physical

This may not be for everyone, but if you’re facilitating NSX design sessions and you haven’t solved these situations by other means, it’s been very evident to me that having separate design sessions may be more effective than “let’s get everyone together” strategies.

I do a fairly large number of NSX designs a year, for a pretty wide client base and regardless of the industry, no two organizations are alike. The vast majority of clients have similar infrastructure, but often times very different functional and operational requirements, both in current and future design goals.

After experiencing this for a number of years and at times, allowing the client to drive who the stakeholders are and who would be contributing, I came to the realization that there really needs to be two key design stakeholders and two separate design groups. Each of the design sessions needs to be performed in order to facilitate the actual business outcomes and to ensure that design features and functional requirements are met.

The infrastructure teams of compute, networking and storage, need physical design meetings for connectivity, capacity and resiliency planning, while the application stakeholders need functional use-case design sessions to determine NSX feature use, options and functional requirements.

Performing functional before physical design is probably the single most important factor in determining the success of your client design and how long the design process will take.

It’s extremely important to have the client spell out any current and future functional use-cases. Each use-case should be documented, with dependencies and requirements, including the outcomes desired. Going through each scenario in detail with each dependency will likely add requirements that were unknown or undiscovered.

As to who should be in each group, the answer is simple, but against many organization’s natural instincts to include everyone in each. Requirements gathering needs to be done without interjection and competition between groups. The vast majority of enterprises have histories of clashes and competing agendas between infrastructure (compute, network, storage) and application owners, developers, and even business units in larger organizations. To avoid these unproductive scenarios, we remove the infrastructure “power players” from the Functional Design Group and then the inverse from the Physical Design Group.The Functional Design Group should include developer system architects, at least one hands-on technical developer, the application owner or owners, a security architect and security owner, and any business sponsors who are responsible for the intended business outcomes. To support the Functional Design Group, infrastructure should provide a technical engineer from compute, networking and storage, with senior level knowledge of current operations and architecture for their respective domains to answer technical questions on current capabilities and operations only. The business sponsors are the key stakeholders for the Functional Design Group.

The Physical Design Group attendees are the infrastructure architects, a technical engineer from compute, networking and storage, with senior level knowledge of current operations and architecture for their respective domains and the stakeholders for overall infrastructure. The key stakeholders for infrastructure are preferably not the managers from compute, network and storage, but rather an overarching director, technical directors or even a vice-president of infrastructure, if that role exists. In support of the infrastructure team, the application team should provide a senior devops member, a security architect and a technical representative from the application team who are to provide answers on current and future design operations only. Again, we don’t want application “power players” in this group to avoid any historically combative scenarios.

Stakeholders need only attend the first and last of their respective design sessions and workshops to ensure that they’re getting what they asked for in the business outcomes they owe back to the business. Exposing stakeholders to “how the sausage is made” or any design challenges, can affect their confidence in the overall initiative and be counter productive.

Again, these are recommendations only and different engagements and clients will at times call for adjustments to group membership, but again, the idea is to keep the infrastructure teams from telling the application teams “they can’t do that” and the like for the infrastructure groups.

Lesson 1: Keep each design group’s membership weighted so they can express their desires and concerns without conflict. The majority of infrastructure and app teams enjoy defeating each other too much and already have a taste for blood. Be the peacekeeper, the broker, the communicator for the groups and ensure any supporting staff from other groups understand their roles in only providing answers, without adding push-back.

Lesson 2: Don’t try to hold a physical design session first. Even though most infrastructure teams are anxious for their connectivity, capacity and redundancy requirements and options, explain the need for an NSX functional use-case design first and how those NSX functional use-cases will supply the physical design requirements for it. If you start with physical design, you may very likely (WILL) end up redesigning (and rebuilding) it again after the functional use-cases are determined.

It’s not easy to tell infrastruture teams that they can’t start with connectivity and physical design, but everyone will be better off, in the end, once you know what the functional use-case requirements are.

Lastly, always listen to exactly what the client is telling you and don’t make assumptions, ask questions. I had this beaten into my head by Paul Mancuso at VMware, a few years back in a mock NSX design defense and I’ve never forgotten it since. There is probably not more sage advice in regards to requirements gathering, than that one point alone.