Sunday, October 2, 2011

Single ( Same ) Domain for Active Directory and Company Website

Almost every organization (except one) that I have worked at or done consulting complained about this issue. They want to use the same domain for Active Directory as for company's website but doing so does not allow internal users (behind the firewall) to get to the website using the top level domain format (i.e. which is hosted externally or in DMZ. This happens because AD member computers must have DNS servers set to those that  reslove AD domain to domain controllers for purposes of authentication, LDAP queries, etc. Since DC doesn't have a listener for port 80, it doesn't repsond to HTTP requests.

As a workaround these organizations' IT department has to educate their users to use www or another A (HOST) record when going to company's website when sitting in the office behind the firewall.  However, they don't have to do this when they are outside. While it works for majority, some users get annoyed as they are used to typing top level domain format becuase it's quick and so on and so forth.

There are two solutions to this problem:

1. Well known DSN Rewrite - Unless you have Cisco firewall on your edge or another device that support this feature you are pretty much out of luck.

2. Installing Web Server on Domain Controller - Using this option will require installing a web server on your domain controllers and perform a permanant redirect  to a fully qualified domain (i.e. to This option is not a recommended one but works just fine.
It's not recommended because Micorsoft does not recommend installing mulitple roles on your domain controllers as a best practice for load, security, etc. However, since you will only be doing a redirect which doesn't cause server to consume many resources but you need to keep an eye on the servers to watch out for any unusual spikes after the permanent redirect. You also have be to be careful as to allow anyone from outside to connect to web servers on your DC as this may result into a security breach. Only internal users should be allowed.

Wednesday, September 28, 2011

Workaround for VMDKs Larger than 2 TB (2048 GB)

Largest virtual disk (VMDK) that VMware supports inside a virtual machine is 2TB - 512 KB unless you want to do RDM - Raw Device Mapping. However, you can get around this by spanning multiple disks inside Guest (Windows).

To do this, you need to do the following:

1. Create and attach 2 or more VMDKs of various sizes you want.
2. Start VM and go into Disk Management Utility (diskmgmt.msc)
3. Bring disks online and initialize.
4. Convert Basic disks to dynamic or the next step will do it for you.
4. Right click on one of the disks and click "New Spanned Volume"
5. Follow the wizard and assign drive letter.

You are done.

Thursday, September 15, 2011

Internet Edge Design: SSL VPN Placement?

Internet Edge Design: SSL VPN Placement?

This is another question that arose today. Where do you want place your VPN gateway (SSL or other VPN concentrators) in you internet edge for the network? Should they be placed next to firewalls on the edge, routers, or behind the firewall?

In my opinion, VPN concentrators should be behind a firewall with OOB - Out of Band Management - capabilities, especially if it's a SSL or Web VPN device. Because, HTTP/HTTPs are proned to web based attacks like ssl stripping and SSL VPN has two parts (Web Server and VPN Server).

However, I would like to know what others have to say about it in the community and if there is a better approach.

Please feel free to leave you suggestions and thoughts in the comment section.


How to add static ARP on a Nortel switch and other ARP operations?

Configuring ARP operations on Nortel Passport 8600 or any other Nortel switch on your network should be very easy thing but if you don't have much Nortel background, it can be a tedious task as there isn't much documentation out there for Nortel as it is for Cisco. So I thought it would be nice to post it in case someone is looking to add static ARP on their core or other Nortel switches on the Network.

Show ARP table
8600#show ip arp info

Clear ARP (Port or VLAN)
8600#clear ip arp vlan 444
8600#clear ip arp port 1/4

Configure Static ARP (Port or VLAN)
8600#config ip arp add ports 1/4 ip mac 00:00:00:44:44:44
8600#config ip arp add ports 1/4 ip mac 00:00:00:44:44:44 vlan 444

Delete an ARP Entry
8600#config ip arp delete

Set ARP Age Time
8600#config ip arp aging 4

Wednesday, September 14, 2011

Internet Edge Design: Single Firewall or Layered Firewalls?

Internet Edge Design: Single Firewall or Layered Firewalls?

While redesigning Internet Edge of our network many ideas came to the table. One was to use single firewall vs. layered (or dual) firewall design.

In my opinion and experience - if your company have the budget - opting for a layered firewall approach is a better and more secure design than just having a standalone firewall on your edge. In addition, I like to have at least different vendors for each firewall at different layers. Each firewall should be running in different mode (i.e. transparent vs routed).

In this post, I would like to know what others think? Please leave your opinions and ideas in the comment section.


Saturday, September 10, 2011

Internet Edge Design: Secure Web Gateway - Proxy or Not To Proxy?

Internet Edge Design: Proxy or Not To Proxy?We are building a new Enterprise Internet Edge for our organization and are debating whether to use a web proxy for Internet traffic or not. Here are some of the requirements we see fitting for proxy solution:

So far we have looked at following vendors which offer both appliances and cloud-based solutions. We like the cloud based solution because it's simple. But, we have concerns about performance during peak Internet hours for our network.

-Bluecoat (Expensive + Too many appliances)
-Websense (Very Expensive but well known and single clustered appliance)
-Zscalar (Cloud based + Cheaper than previous two + No appliance required)

In this post I would like to see what others are doing and thinking for a web proxy solution for their network.  Do you think a web proxy is still needed for a network where advanced firewalls on the edge have the capabilities to fight worms and viruses that can make their way into a network on HTTP/HTTPs ports?

Please leave your thoughts in the comment area.


Thursday, September 8, 2011

VMware ( vSphere and ESX ) VMDK Size Limit for NFS

There is a lot of confusion on the web about what is the largest size of a VMDK file on a NFS mounted datastore for VMware vSphere and ESX. Today, I was able to confirm that the largest vmdk you can create on an NFS mounted datastore is 2TB minus 512KB. Yes that's the same as for a VMFS formatted datastore.

There was lot confusion because I thought the size of a file on an NFS mounted export is restricted by the underlying storage device. I called both VMware and NetApp about this issue. VMware support told me what I already thought the answer was (Storage vendor restriction) and NetApp support said they are not sure and will research.

So I decided to try creating a 1.99, 2.0, 2.1, 3,4, and 5 TB vmdks on both NetApp and Windows 2008 NFS storage. I got the same error message stating

DiskCapControl: ... Out of Range ... ( )

for all sizes except 1.99 which is less than 2TB-512KB.
Hopefully someone out there will find this info helpful.

Monday, September 5, 2011

KDC Event ID 11 - Solved

If you are creating SPN records in AD and don't take pre-cautions, you may end with duplicate SPN records. This will prevent AD users from logging on to the machine.  Following event id is logged in event log.

Event Type: Error

Event Source: KDC
Event Category: None
Event ID: 11
Date: 4/11/2011
Time: 3:57:13 PM
User: N/A
Computer: DC00
There are multiple accounts with name cifs/ of type DS_SERVICE_PRINCIPAL_NAME.
For more information, see Help and Support Center at

Event Type: Error
Event Source: KDC
Event Category: None
Event ID: 11
Date: 4/11/2011
Time: 3:31:53 PM
User: N/A
Computer: DC00
There are multiple accounts with name host/ of type DS_SERVICE_PRINCIPAL_NAME.
For more information, see Help and Support Center at

Refer to this Micrsoft KB article to reslove this issue.

Ad tools required are ldp.exe and adsi.msc.

Friday, September 2, 2011

NetApp Virtual Storage Console Plugin and Windows 7 64-bit

NetApp Virtual Storage Console - VSC - in vCenter would not load unless you install 32-bit version of JRE for Java. As it turns out, Virtual Infrastructure Client uses 32-bit Internet Explorer to display pages inside Windows frames. So if you have installed 64-bit of Java, you would also need to download and install 32-bit of Java. Latest version 1.7 as of the time this post was written works with NetApp VSC version 2.0.1.

Hope this helps someone out there struggling to make VSC work on a 64-bit Windows 7 machine.


Monday, August 1, 2011

Change TCP Port Range for DCOM with WMI

WMI - Windows Management Instrumentation - is used by many different type of applications for different purposes but most recently our team used it for an application to monitor file integrity on servers for PCI compliance requirement. The software we used for file integrity monitoring is called nCircle File Integrity Monitor (nFIM) from nCircle. As a security measure we decided to use non-default range of ports for WMI and DCOM.  I was tasked to make WMI work with dynamic ports since nFIM utilizes DCOM to connect to a range of TCP ports. Here is how we made it work for each server.

1. Log on to the server in question
2. Go to Administrative Tools
3. Click on Component Services
4. Expand Component Services tree until My Computer
5. Right click my computer and go the tab shown in the image below
Click on image for full view

You can further narrow down the security for DCOM by specifying trusted endpoints as shown in the image below.
Click on image for full view

Thursday, July 28, 2011

Quick Nortel MLT (Link Aggregation) Reference

These excerpts and images were taken from Nortel Reference Manual for Passport 8600. These notes are very helpful for those who are configuring a LAG (MLT) on a Nortel Switch for the first time.

Click image for full view

IEEE 802.3ad-based link aggregation, through the Link Aggregation Control Protocol (LACP), supports a dynamic link aggregation function as they become available to a trunk group. LACP dynamically detects when links can be aggregated into a link aggregation group (LAG) and does so as links become available. LACP also provides link integrity checking at Layer 2 for all links within the LAG.

Supported Link Aggregation Types:
- MLT with LACP
- MLT : statically configured link bundling

Virtual LACP (VLACP) is a Nortel modification that provides end-to-end
failure detection. VLACP is not a link aggregation protocol; You can run VLACP on single ports or on ports that are part of a MLT.

MLT provides module redundancy via Distributed MultiLink Trunking (DMLT). DMLT allows you to aggregate similar ports from different modules.
- Nortel recommends that LACP not be configured on the IST MLT.
- Nortel recommends that you do not configure VLACP on LACP-enabled ports. VLACP does not operate properly with LACP. You can configure VLACP with any SMLT configuration.
- Nortel recommends always using DMLT when possible.

MLT and MLT with LACP configuration rules:
• MLT is supported on 10BASE-T, 100BASE-TX, 100Base-FX, Gigabit Ethernet, and 10 Gigabit Ethernet module ports.
• All MultiLink trunk ports must have the same speed and duplex settings, even when auto-negotiation is set.
• The media type of MLT ports can be different; a mix of copper and fiber are allowed.
• All MultiLink trunk ports must be in the same STG unless the port is tagged. Tagging allows ports to belong to multiple STGs, as well as multiple VLANs.
• MLT is compatible with Spanning Tree Protocol (STP), Multiple Spanning Tree Protocol (MSTP) (IEEE 802.1s), and Rapid Spanning Tree Protocol (RSTP) (IEEE 802.1w).
• Tagging (IEEE 802.1Q) is supported on a MultiLink trunk.
• MLT ports can span modules, providing module redundancy.
• Apply filters individually to each port in a MultiLink trunk.
If identical BPDUs are received on all ports, the MultiLink trunk mode is forwarding. You can disable the Nortel STP (ntstg ) if you do not want to receive BPDUs on all ports.

LAG rules:
• All LAG ports operate in full-duplex mode.
• All LAG ports operate at the same data rate.
• Assign all LAG ports in the same VLANs.
• Link aggregation is compatible with the STP, MSTP, and RSTP.
• Assign all ports in an LAG to the same STP groups.
• Ports in an LAG can exist on different modules.
• For Gigabit and 10 Gigabit ports, you can use link aggregation groups 1 to 31.
• For Fast Ethernet ports, you can use link aggregation groups 1 to 7 only.
• Each LAG supports a maximum of eight active links.
• Each LAG supports a maximum of eight standby links.
• After a MultiLink trunk is configured with LACP, you cannot add or delete ports or VLANs manually without first disabling LACP.

SMLT is an option that improves Layer 2 and Layer 3 resiliency. These SMLT switches form a Switch Cluster and are referred to as an IST Core Switch pair
Switch Clusters are always formed as a pair, but pairs of clusters can be combined in either a square of full-mesh fashion to increase the size and port density of the Switch Cluster.
When configured in a Layer 3 or routed topology, the configuration is referenced as Routed SMLT (RSMLT).

  • Before you reboot a switch that is the LACP master, you must configure the LACP system ID globally to prevent an RSMLT failure.
  • A properly designed SMLT network inherently does not have any logical loops.
  • SMLT solves the spanning tree problem by combining two aggregation switches into one logical MLT entity, thus making it transparent to any type of edge switch. In the process, it provides quick convergence, while load sharing across all available trunks.
Single Port SMLT rules:
• The dual-homed device that connects to the aggregation switches must support MLT.
• Single-port SMLT is supported on Ethernet ports.
• Each single-port SMLT is assigned an SMLT ID from 1 to 512.
• You can designate Single Port SMLT ports as Access or Trunk (IEEE 802.1Q tagged or not); changing the type does not affect behavior.
• You cannot change a Single Port split MultiLink trunk to an MLT-based split MultiLink trunk by adding additional ports. You must delete the single port split MultiLink trunk and reconfigure the port as SMLT/MLT.
• You cannot change an MLT-based split MultiLink trunk into a single port split MultiLink trunk by deleting all ports except one. You must remove the SMLT/MLT and reconfigure the port as Single Port SMLT.
• You cannot configure a port as an MLT-based SMLT and as single-port SMLT at the same time.
• Two or more aggregation switches can have single port Split MultiLink trunk with the same IDs. You can have as many single port Split MultiLink trunk as there are a available ports on the switch.
• LACP is supported on single port SMLT.

Simple Loop Prevention Protocol (SLPP) is used to prevent loops in a SMLT network. SLPP is focused on SMLT networks but works with other configurations. Nortel recommends that you always use SLPP in any SMLT environment. SLPP requires the use of 4.0.x code or higher.

MLT with LACP configuration considerations
When you configure standard-based link aggregation, you must enable the aggregation parameter. After you enable the aggregation parameter, the LACP aggregator is mapped one-to-one to the specified MultiLink trunk.

Perform the following steps to configure an LAG:
1. Assign a numeric key to the ports you want to include in the LAG.
2. Configure port aggregation to true.
3. Enable LACP on the port.
4. Create an MultiLink trunk and assign the same key as in step 1 to it. The MultiLink trunk/LAG only aggregates ports whose key matches its own. The newly created MultiLink trunk or LAG adopts the VLAN membership of its member ports when the first port is attached to the aggregator associated with this LAG. When a port detaches from an aggregator, the associated LAG port deletes the member from its list. After a MultiLink trunk is configured with LACP, you cannot add or delete ports or VLANs manually without first disabling LACP. To enable tagging on ports belonging to a LAG, disable LACP on the port and then enable tagging and LACP on the port.

MLT with LACP and SMLT configuration considerations
Split MultiLinkTrunks (SMLT) can be configured with MLT or MLT with LACP. Follow these guidelines when you configure SMLT with LACP:
• When you set the LACP system ID for SMLT, configure the same LACP SMLT system ID on both aggregation switches to avoid the loss of data. Nortel recommends that you configure the LACP SMLT system ID to be the base MAC address of one of the aggregate switches, and that you include the SMLT-ID. Ensure that the same System ID is configured on both of the SMLT core aggregation switches.
• If you use LACP in an SMLT square configuration, the LACP ports must have the same keys for that SMLT LAG; otherwise, the aggregation can fail if a switch fails.
• If an SMLT aggregation switch has LACP enabled on some of its MultiLink trunks, do not change the LACP system priority. If some ports do not enter the desired MultiLink trunk after a dynamic configuration change, enter the following CLI command:
config mlt lacp clear-link-aggrgate
• When you configure SMLT links, Nortel recommends that you set the multicast packets-per-second value to 6000 pps.
• Nortel recommends that you do not enable LACP on interswitch trunks to avoid unnecessary processing. Use VLACP if a failure detection mechanism is required when there is an optical network between the SMLT core switches.

Click image for full view

Click image for full view

Sunday, July 24, 2011

Cisco Nexus 1000v Gotchas!

Have you deployed the new Cisco N1KV yet or thinking about doing it?

There are many online tutorials and of course Cisco documentation is great and very useful as always for installing and setting up Nexus 1000v distributed virtual switch. However, nothing beats first hand experience of testing, playing and implementing a new technology in production environment. That's why I though I should share some of the important points and guidelines from my experience for installing N1KV in our VMWare environment. It was a hassle to get it right but once done it turned out to be the next greatest thing in virtual Networking for us.

1. VLANs: As you probably read it, you need several new private VLANS (Control, Packet, and Management) for N1KV and these have to exists on the system uplink. However, you also need to put vCenter and vMotion VLANs on the system uplink port-profile as well. To do so, do the following:

N1KV84(config)#port-profile type ethernet system-uplink
N1KV84(config)#switchport mode trunk

N1KV84(config)#switchport trunk allowed vlan 111,113,249,261-262


VLAN-111 is vCenter Management VLAN
VLAN-113 is vMotion VLAN
VLAN-249 is for Nexus 1000v Management IP (This VLAN can be same as for vCenter but we chose it to be different since we have dedicated management VLAN in our environment.)
VLAN-261 Control
VLAN-262 Packet
2. How many N1KV dvSwitches do I need for my VMware environment?
First of all you should know that you need 2 VSM - Virtual Supervisor Module - virtual machines (VMs) per N1KV.  This will vary from environment to environment but for our environment, we created 2 N1KV swtiches across two datacenters with each datacenter hosting 3-4 clusters. 
Also, our datacenters are separated at physical boundaries so it made more sense for us to have 2 dvSwitches. Otherwise if we chose 1 per cluster, we will be creating 16 VSM - Virtual Supervisor Engine - VMs which I think is an overkill.

3. NLB - Network Load Balancing:
If you currently have Windows Network Load Balancing in your VMware environment, you will have to disable IGMP snooping in Nexus 1000v on VLANs to which NLB VIP (Virtual IP) is bound or to which vNIC (port group) NLB enabled VMs are connected. Further, remember that only multicast and IGMP-mulitcast are supported on Nexus 1000v distributed virtual switch. Unicast is not supported on Nexus 1000v.

4. LACP (No static LAGs): You can't create static LAG- Link Aggregation - between a physical switch (or stack) and Nexus 1000v to achieve higher bandwidth and port redundancy. To achieve more than 1 Gbps speed, you must enable LACP feature in Nexus using following command:

 N1KV84(config)# feature lacp

Then configure / activate LACP on ethernet port-profile like this:

 N1KV84(config)#port-profile type ethernet system-uplink N1KV84(config)#vmware port-group
 N1KV84(config)#channel-group auto mode active

Make sure LACP is also enabled on physical switch / stack as passive on ports connected to Nexus 1000v.
5. Persistent Connections across Host (Server) reboots:
To make sure that upstream connectivity stay intact during normal reboots or server failures, you need to define certain VLANs as system VLANs for uplinks configured as trunks. These include:
  A. vMotion, vCenter, Control, Packet, and Management for system uplink
        N1KV84(config)#port-profile type ethernet system-uplink    N1KV84(config)#system vlan 111,113,249,261-262
   N1KV84(config)#vmware port-group

 B. Storage VLAN on storage uplink(s) for iSCSI or NFS
   N1KV84(config)#port-profile type ethernet storage-uplink-iscsi  
   N1KV84(config)#vmware port-group
   N1KV84(config)#switchport mode access
   N1KV84(config)#switchport access vlan 321
   N1KV84(config)#mtu 9000
   N1KV84(config)#no shutdown
   N1KV84(config)#system vlan 321
   N1KV84(config)#port-profile type ethernet storage-uplink-nfs
   N1KV84(config)#vmware port-group
   N1KV84(config)#switchport mode access
   N1KV84(config)#switchport access vlan 320
   N1KV84(config)#mtu 9000
   N1KV84(config)#channel-group auto mode active
   N1KV84(config)#no shutdown
   N1KV84(config)#system vlan 320
   N1KV84(config)#max-ports 32
   N1KV84(config)#state enabled
  C. Any VLAN(s) used for for data uplinks
  D. You don't need to define system vlans for access ports.
6. VSM Management IP Address - Make sure you assign the same management IP address during the installation of both VSMs.
7. L2/L3 (Layer 2 or Layer 3) - During the setup you will be asked to configure N1KV for L2 or L3 mode. If your upstream physical switch to which Nexus 1000v will directly connect is running as L2 mode (no routing) then you should configure N1KV in L2 mode otherwise if upstream switch is running in Layer 3 mode (switching as well as routing) then configure Nexus in L3 mode. Our was L2.

8. Finally, start with latest version of Nexus 1000v because earlier releases are buggy. If because of you ESX/ESXi version you can't install latest version then read the release notes and install any patches available.

Hope after reading this post your experience with N1KV won't be as rocky as mine. :-)

Saturday, July 23, 2011

How to disable IGMP snooping on Nexus 1000v when using Microsoft NLB in IGMP multicast mode?

If you have Cisco Nexus 1000v distributed virtual switch in your VMware virtual environment and you have virtual machines (VMs) running Microsoft NLB (Network Load Balancing) in IGMP multicast mode, then you will need to disable IGMP snooping on N1KV to allow multicast traffic to pass through on the VLAN for those NLB VMs. By default IGMP snooping is enabled on all VLANs in Cisco Nexus 1000v, which is a good thing.

Here is the command to disable IGMP snooping on N1KV on let's say VLAN 100.

N1KV#config t
N1KV(config)#vlan 100
N1KV84(config-vlan)# no ip igmp snooping
N1KV84(config-vlan)#copy run start

Note: Nexus 1000v doesn't support MS NLB in Unicast mode.

Friday, July 22, 2011

Dell OpenManage 6.3 on ESXi 4.1

You will need to set following advance option and reboot ESXi host after installing OpenManage 6.3.

UserVars.CIMoemProviderEnabled to 1

click image for full view

Friday, July 15, 2011

NetApp 3140 FilerView Not Working with Java 1.6 and SSL (HTTPS) - Workaround

Recently got my new NetApp 3140 FAS series unit running Data ONTAP 8.0.1. I wanted to see different ways to manage it (i.e. CLI, NSM, and Browser). It turned out everything worked with web management except any Java applets with SSL on HTTPs. It turned out that Java 6 JRE didn't like the code since FilerView applets are written using Java 1.4 JRE classes. After some trial and error with trying JRE 1.6.x, 1.5.x  I finally got it working with Java 1.4 JRE. Here is the exception stack trace that JRE 1.6 kept throwing because it couldn't load the right library due to not able to process SSL library correctly.

Java Plug-in 1.6.0_07
Using JRE version 1.6.0_07 Java HotSpot(TM) Client VM
User home directory = C:\Documents and Settings\
c: clear console window
f: finalize objects on finalization queue
g: garbage collect
h: display this help message
l: dump classloader list
m: print memory usage
o: trigger logging
p: reload proxy configuration
q: hide console
r: reload policy configuration
s: dump system and deployment properties
t: dump thread list
v: dump thread stack
x: clear classloader cache
0-5: set trace level to
load: class com.netapp.meter.HealthMonitor not found.
java.lang.ClassNotFoundException: com.netapp.meter.HealthMonitor
at sun.applet.AppletClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.applet.AppletClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.applet.AppletClassLoader.loadCode(Unknown Source)
at sun.applet.AppletPanel.createApplet(Unknown Source)
at sun.plugin.AppletViewer.createApplet(Unknown Source)
at sun.applet.AppletPanel.runLoader(Unknown Source)
at Source)
at Source)
Here is the screenshot of error message in Internet Explorer.

Click on image for full size view
So in my opinion CLI is the best way to manage all your NetApp filers but if you are not command-line savvy and want to use browser, make sure you install Java 1.4 on your system where you will be accessing NetApp FilerView. I hope NetApp will fix this in later releases of Data OnTAP.

Thursday, July 14, 2011

Suppress Redundant NIC Warning for ESX Service Consoles

After you add a second service console on an ESX box you may receive a warning in vCenter about not having redundant NIC connections on your Service Console. This can happen when you want to use two service consoles for redundant access to your ESX box but only want to dedicate single NIC for each. Because using 4 NICs for service consoles is wasting those precious NICs which we all know how valuable they become in virtual world.  Here is how you can suppress dual NIC warning for service console port group.

1. Go to VMware Cluster in virtual center
2. Click edit settings and go to VMware HA
3. Click on Advanced Options
4. In the box, type das.ignoreRedundantNetWarning = true such that das.ignoreRedundantNetWarning is under Option column and true is under Value column.

Hope this will help those who needs two service consoles in their environment.

Saturday, April 9, 2011

Default vNIC Type in vSphere 4.1

vSphere/vCenter 4.1 automatically sets vNIC (vritual NIC) type to be Flexible when a new VM is created, one is resulted after a P2V operation using VMware converter, or when one is upgraded to Virtual Hardware Version 7. 

This may have adverse effects on a VM if drivers are not loaded correctly during or after booting up a VM. We noticed that a couple of our VMs were timing out in our environment and this was fixed by changing the adapter type e1000.

You can find more about VMware vNIC types here,

For production environments, I recommend using e1000 even though your VM may eat up a few CPU cycles.

Friday, March 11, 2011

How to find dead iSCSI/FC paths in vSphere ESX 4.x ?

Have you been receiving Storage Disconnects alarms from your vCenter quite often? Here are some quick commands to see if there are any dead paths.

esxcfg-mpath -l | grep -i dead

Following command does the same thing but lists extra information

esxcfg-mpath -b | grep -i dead

If resulting output contains dead paths and you want to be sure that it's not from some earlier known issues in your environment, you can run the following command to rescan all your targets

esxcfg-rescan vmhbaXX , where XX is the last two digits your HBA under Storage Adapters in vCenter.

Run esxcfg-mpath -l | grep -i dead or esxcfg-mpath -b | grep -i dead again and see if it results in any dead paths, if it does then there may be some serious connectivity or storage issues with your SAN. 

Use vmkernel ping to verify you can ping all your iSCSI targets.

vmkping 192.168.x.x, where 192.168.x.x is the IP of target IP of your SAN.

For Fiber Channel, make sure your SAN software sees all FC HBAs as registered and that are no zoning issues.

Wednesday, March 9, 2011

Free VMware Utilities

This site has some pretty cool VMware tools that come in very handy for daily VMware operations.

Check it out!

Tuesday, March 8, 2011

Daily Exchange Health Checklist

Here a few daily tasks that I perform each morning after a cup of coffee to ensure that my Exchange environment is running smoothly. This has proven very helpful in preventing many issues with my Exchange servers. I came up with the list while managing Exchange 2000/2003 environments but it still helps me with Exchange 2007 as well. Hope it will help others out there.
  • Check event viewer for warnings/errors on all Exchange Servers
  • Check for database fragmentation
  • Check postmaster mailbox for NDRs
  • Check exchange statistics
  • Check bad mail folder for trends
  • Check OS status on Exchange boxes
  • Check if plenty of disk space is available on all BE & FE servers
  • Check for cluster failover in cluster admin
  • Check message load in the queue viewer
  • Check OS/Exchange Services
  • Check results of real-time performance monitoring for all servers
  • Review event, performance vs. anti-virus logs
  • Track message for security project
  • Verify integrity of Exchange Store

Monday, March 7, 2011

Fiber Connector Types

Large portion of my datacenter infrastructure currently consist of clusters using fiber storage on EMC SANs and Brocade fiber switches. There are also uplinks to data switches from core routers and all these different fiber connections use many different types of fiber connectors that at times confuse the hell out me. I am no cabling expert so I thought it would be nice if I can put it all together on piece of paper as a reference. Finally, I found this image in a document and thought would be nice to share it with all. It explains all major types of fiber connector types used in today's datacenters.  Hope this will help someone else out there trying to go nuts between MT-RJ and LC-LC or ST-LC fiber connections.