iSCSI Best Practices - VMware, iSCSI, Microsoft

Recently, we came across the need to re-implement iSCSI for Microsoft, VMware and EMC environments. The goal was to reconfigure iSCSI using all best practices that are necessary to successfully design and implement iSCSI storage for an enterprise. We went through many different documents from VMware, Microsoft and iSCSI and squeezed out the best information we could find. Here is summary of what we discovered and hopefully it will help others out there trying to implement iSCSI SANs.

You can download the complete documents here:

HTML - iSCSI-Best-Practices-VMware-EMC-Microsoft

Word - iSCSI-Best-Practices-VMware-EMC-Microsoft

PDF - iSCSI-Best-Practices-VMware-EMC-Microsoft

VMware Virtual Network Concepts - Summarized (VI3)

Virtual Networking is new and confusing to many of us used to traditional networking concepts. We recently found ourselves struggling to resolve issues involving VMware environment and Link Aggregation. We thought it would be a good idea to put a document summarizing all the concepts of VMware Networking since it's new and huge. So here is the document that describes VMware networking in a nutshell (well, not really!).

HTML - VMware Virtual Network Concepts Summarized
Word - VMware Virtual Network Concepts Summarized
PDF - VMware Virtual Network Concepts Summarized

A Quick PowerShell Reference Pocketbook

As most of us already know that PowerShell is a new scripting language from Microsoft based on .NET framework. Microsoft and other vendors are heavily placing the scripting needs around PowerShell for administrative use. Specially starting Windows 2008 PowerShell has become an integral part of the Windows environment. Here is a quick reference from my studies of PowerShell language. Since I am already familiar with other programming and scripting languages, I used Dr. Tobias PowerShell E-book for quickly learning the syntax and semantics of PowerShell.

Here is a quick reference guide that I created for my self for future reference which I thought will also be helpful for others. Please remember this is just a quick reference for those who have some programming or scripting back ground.

You can download MS Word and PF versions from here:
Word - A Quick PowerShell Reference Pocketbook
PDF - A Quick PowerShell Reference Pocketbook

Statement Evaluation Order:

Parenthesis ( ) are always evaluated first

Automatic Unit Recognition:

Units like KB, MB, and GB can be used with numbers like 1MB + 2GB. Results are in bytes by default.

Windows Batch & Other Commands:

cmd /c command-name

use & to execute commands in quotes, i.e. & "c:\autoexec.bat"

To execute a program in current directory, prefix program name with \ i.e. .\editplus.exe . Unless the folder is on

the windows path $env:path

Add folder to path: $env:path += ";C:\programs\Windows NT\accessories"

Cmdlet Convention: Use the forms of verbs and nouns i.e. Get-Help

Add, Clear, Compare, Convert, Copy, Export, Format, Get, Group, Import, Measure, Move, New, Out, Read, Remove, Rename, Resolve, Restart, Resume, Select, Set, Sort, Split, Start, Stop, Suspend, Tee (Split up), Test, Trace, Update, Write.

Cmdlet Parameters:

Common Parameters i.e. ErrorAction, WhatIf

Named Parameters i.e. string, string[]

Switch Parameter i.e. boolean ($true,$false)

Positional Parameters i.e. Get-ChildItem -recurse, -name, c:\windows *.exe

Disable Parameters: Use -- or "-Parameter" (quotes) to disable parameter recognition

Variables:

Definition:

$syntax i.e. $company = “Axiom Dynamics Corp.”

Typeless i.e. $one = 1

${special variable} i.e. ${I am special} = “Special”

$assignment = value or “cmdlet Result”

Multiple inline assignments are supported i.e. $ten = $10 = 10

separate statements by ; i.e. $one=1;$two=2

$_ - this reference

Swap variable values:

$Value1 = 10; $Value2 = 20;

$Value1, $Value2 = $Value2, $Value1

$Value1

$Value2

Virtual drive is a variable

Write-Protected variables

"backtick" character ’

"." before the script file path to turn off restrictions on visibility

Scope modifiers are private, local, script, and global

Type: $variable.GetType()

Type safety: [oneoftypes] $variable where,

[oneoftypes]: [array], [bool], [byte],[char],[datetime], [decimal], [double], [guid], [hashtable], [int16], [int32], [int], [int64],, [long], [nullable], [psobject], [regex], [sbyte], [scriptblock], [single], [float], [string], [switch], [timespan], [type], [uint16], [uint32], [uint64], [xml]

Variable Options: "None","ReadOnly","Constant","Private","AllScope"

Variable Validation Classes: ValidateNotNullOrEmptyAttribute,ValidatePatternAttribute,ValidateRangeAttribute,ValidateSetAttribute

Operators:

Arithmetic:

Hex are prefixed with 0x

Operators: +, -, *, /, %

Conditional: -eq, -ceq, -ieq, -ne, -cne, -ine, -gt, -cgt, -igt, -ge, -cge, -ige, -lt, -clt, -ilt, -le, -cle, -ile, -contains, -ccontains, icontains, -notcontains, -cnotcontains, -inotcontains, -not/!, -and, -or, -xor

If/ElseIf/Else

Switch w/ default

strings (case sensitive)

wildcards

regex

expressions,

Arrays:

The comma always creates an array.

Verify e.g. $a is [Array]

Index starts at 0 and -1 is last element

Force to return array by using @()

Create New Array i.e.

$arr = 1..4

$arr = 1,2,3,4

$arr = "1","2"

$arr = 1

$arr = @()

Reverse Arrays

New elements Can be added by using += operator

Strongly Typed Arrays: [[int] ]$arr = 1,2,3

Arrays (returns matching / unlatching members)

Hash Tables:

Key-Value pair sep by ;

Access/Modify using either [] or .dot

Note: Arrays and HT are references to data

Pipeline:

Cmdlets To Pipe: Compare-Object, ConvertTo-Html, Export-Clixml,Export-Csv, ForEach-Object, Format-List, Format-Table, Format-Wide, Get-Unique, Group-Object, Import-Clixml, Measure-Object, more, Out-File, Out-Host, Out-Host, -paging, Out-Null (suppress output), Out-Printer, Out-String, Select-Object, Sort-Object, Tee-Object, Where-Object (?), Out-Default, Out-Host, * (wild card), Scriptblocks,

Modes: Sequential/Streaming Mode

ETS - Extended Type System: Emergency mode,

$$ - last token of pipeline

Loops:

ForEach-Object ("{indexes}") == %

ForEach - works with collection

Do/While, While, For, break, continue, :label

Aliases:

Example: PowerShell Virtual Drives

$alias: Get-PSDrive

dir alias:

- Group-Object definition

Create new aliases using Set-Alias, i.e. Set-Alias edit notepad.exe

Save Aliases using profiles or Export-Alias/Import-Alias cmdlets.

Life span of alias and function are only when PS console is open.

Functions:

Syntax: myFunc { commands/code $args }, multi line (enter) / single line (;

Calling: $function:tabexpansion | Out-File myscript.ps1/>file.ps1

Write-protected functions: Set-Item function:{//code} -option constant

Delete function: Del function:test,

Arguments: Args: $args - spaces ""

Parameters: function myFunc([type] $Param1=110, $Param2=$(Get-Date), [Switch] ParamN)

Returns are array or return keyword or omit returns

Write-Debug ,$ErrorActionPreference,

Stop-Process -nameofFunction

Dir function:

$function: prompt

Function {begin{}process{}end{}}

Object:

New-Object: Creates new object

Add-Member (NoteProperty, ScriptMethod, CodeMethod, Method)

Properties and Methods: $objectName.PropertyName or $objectName.MethodName()

Versioning: Major, Minor, Build and Revision

-static, constructors

Can load assemblies and COM.

Scripts & Other Language Integration:

Windows Batch Files

VBScript (Wscript //H:CScript - console, WScript //H:WScript – Windows)

.\filename.ext

Own: ps1.

Before executing scripted created on other machines you must Set-ExecutionPolicy to RemoteSigned

.\myscript.ps1

Get-ExecutionPolicy, Set-ExecutionPolicy RemoteSigned

md $env:appdata\PSScripts

\myscript.ps1 Tobias,

$($args[0])

param ($path, $name), param ([string] $Name=$( ` Throw "Parameter missing: -name Name")

[int]$age=$( `,. .\calcfunctions.ps1,

. $env:appdata\PSLib\calcfunctions.ps1,

{begin{} process{} end{}}

Four Different Profile Scripts: All users[private],Current user[private]

Debugging:

What-if

Confirm:Low, Medium, and High, $ConfirmPreference = "Low"

ErrorAction/$ErrorActionPreference:SilentlyContinue,Continue,Stop,Inquire

$?

Trap {}

$_.Exception.Message

ErrorVariable myError

$Error

Throw

Write-Debug, $DebugPreference:SilentlyContinue,Continue,Stop,Inquire, Set-PSDebug –step

@"

{index[,alignment][:format]}

Fix: Windows Update Failing in Windows Server 2008 and Vista

If you are having issues with Windows Updates in Windows 2008 or Vista and Updates fail often. You can resolve this by deleting the DataStore.edb file and restarting your system.

The file is located at %systemroot%\SoftwareDistribution\DataStore.

You may also need to delete Log folder if Updates don't work after removing DataStore.edb file.

Windows 2008 needs NetBIOS to join Windows domain

Today, I was installing Windows 2008 Enterprise on a 64-bit machine. I decided to use this installation to write up a new server build document for the organization. As part of the process, I decided to disable any unnecessary services in Windows 2008 that were not of use. Later on, I noticed that I could not join the newly created server to our Domain. As it turned out, I had disabled an important service, TCP/IP Helper for NetBIOS. After enabling this service, I was able to join the Windows 2008 server to the domain without any problem. The error message that I was getting stated that 'Network path cannot be found'.

Come on, I thought MS was doing away with WINS and NETBIOS. What's this?

What's wrong when viewing Exchange 2007 quota settings for all users?

Recently I was asked by management to provide them with the list of users who do not have default mailbox quotas restrictions on an Exchange 2007 CCR server. I told myself, well that's pretty straight forward. I can use Get-Mailbox cmdlet in Exchange Management Shell (EMS) and send the output to a text file, nicely format it in excel and I am done. So, I went ahead and ran the following command with the output show below.

>Get-Mailbox -ResultSize "Unlimited" | more

I noticed in the output that last column (ProhibitSendQuota) contained a lot of "unlimited" values. That meant that I had 99% percent users that had unlimited quotas on their mailbox for sending email. But that could not be. Because during the CCR setup, every database had quotas defined and all mailboxes (new and migrated) by default were set to use database defaults. Yes, there were some exceptions but these mailboxes were given unlimited on a purpose. Anyways, I wanted to confirm this so I looked at two mailboxes above in Exchange Management Console (EMC) and noticed that both are set to use database default quota settings as expected.

So, why does the command Get-Mailbox show these values incorrectly (marked in Red). Well that's a question to be answered by Microsoft Exchange Team. To me, this is a minor bug in the Exchange 2007 that should be fixed.

When using Get-Mailbox to get mailbox settings I would expect to see correct values as defined at the mailbox database level and shown below.

So, to answer my original question and provide management with the mailbox quota report, I run the following cmdlet.Get-Mailbox --ResultSize Unlimited | Where {$_.UseDatabaseQuotaDefaults -eq $false} | Select Name, IssueWarningQuota, ProhibitSendQuota, ProhibitSendReceiveQuota, ExchangeVersion | FT

This will give you list of all users who are not using the default database settings for mailbox quotas on Exchange 2007.

How to display Desktop alerts from Outlook for new email when using IMAP?

So you have configured your Outlook for receiving email using an IMAP server but you don't see the ususal deskotp popup on the lower right corner of your Windows machine when new email arrives.

Well that's what happened to me when I switched to Google Apps from my Godaddy account. To fix this issue, I created a new Outlook rule as shown in the screenshot bleow.

Manually Remove Applications & Software from Windows XP or 2003

There are many times when you try to uninstall a software using Add/Remove snap-in under Windows control panel that won't remove itself very easily using the uninstaller program. During such cases you have to search through vendor's support pages to find out a way to manually remove the product but you are not always successful finding the right instructions. It may also be the case that software you installed was a Shareware or Open Source.

A few months back I came across such issue where I was unable to uninstall a software using vendor provided uninstaller. So I had to go through the procedure of removing the software manually off of my machine. I thought I would share this procedure with others so I wrote down the instructions and here they are.

Warning: The steps below make changes to your Windows Registry and may cause damaged to your OS installation so please use these instructions at your own risk.
1. Find the directory for the application and delete all the files in the directory. Delete the directory.

2. Open regedit and navigate to HKEY_LOCAL_MACHINE\SOFTWARE and find the folder for the application. Delete the folder.

3. Open regedit and navigate to HKEY_CURRENT_USER\SOFTWARE and find the folder for the application. Delete the folder.

4. To remove the application entry from Add/Remove Programs (if present) open regedit and navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall and find the folder for the application. Delete the folder.

5. Some applications have Services attached to them. If this is the case, navigate to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services, locate and delete the service.

6. In Windows Explorer, navigate to the individual user settings and delete program references. Common places to check would be:

[Do this for each User ID listed]
- C:\Documents and Settings\All Users\Start Menu\Programs and delete relevant entries.
- C:\Documents and Settings\All Users\Start Menu\Programs\Startup and delete relevant entries.
- C:\Documents and Settings\%YourUserID%\Start Menu\Programs and delete relevant entries.

[Do this for each User ID listed]
- C:\Documents and Settings\%YourUserID%\Start Menu\Programs\Startup and delete relevant entries.

7. If no entries were found in the previous step and the application launches automatically, navigate to
HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Windows
and delete the entry.

8. Look for any references to the application under these folders in registry. Delete any you find.
HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID
HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Interface
HKEY_LOCAL_MACHINE\SOFTWARE\Classes\TypeLib

How to find an IP address of a device connected to a Switch port?

As IT professionals, we have come across this problem at least once in our careers specially if you are a Network Engineer or Administrator. You need to find an IP address of a certain device connected to a physical port of a switch on the network. But you don't have any information about that host.

Well, not really! You can easily find out the MAC address of the host from the forwarding ARP or CAM table on the switch and the physical port through which this MAC address was learned. However, how do you find an IP address of the host (or one of the NICs on that host) to whom this MAC belongs? Switches don't know anything about IPs. They are layer 2 devices.
No, PING -a 255.255.255.255 or other subnet broadcast addresses won't work.

I have come across such a problem several times and this time I decided I would put this procedure on my blog so next time there is a place I can find these instructions or anyone else who is interested can also look it up here. After all such situations don't arise very everyday.

So I thought there has to be a tool or command that would give me the IP address of the device connected to a port. At the time, I was working with Cisco 3750 and Windows 2003. I searched all over the web only to find out that there is no tool or command that could help me with this problem.
Of course, if you know the IP it is very easy to find the MAC but not as easy the other way around.

After scratching my head for several minutes I realized that IP to MAC entries are stored in ARP cache when 2 hosts communicate with each other and I already have the MAC address of the host that was connected to the port on the switch from CAM table. So why don't I log into one of the machines that I know the IP address of and sweep the segment (or VLAN) using an ICMP sweeper. This will build up IP to MAC mappings in the ARP cache on the host I remotely connected to. Then all I have to do is simply find the MAC from the switch's forwarding CAM table and compare it against ARP cache entries. Bingo! there was the IP address of the host connected to the switch port.

Don't worry. I am going to explain the whole process step by step and how I came to use this useful trick to solve my problem. While in my example I am using Windows 2003 and Cisco 3750 Catalyst Switch, this procedure should work with any OS and switch vendor assuming you know the necessary commands and functions to achieve the results.

Here is what you will need: Windows 20003, Cisco Switch, ICMP (PING) Sweeper program

1) Run the following command on the switch to which the host is connected: sh mac address-table

2) Find the interface or physical port (i.e. FastEthernet 1/0/34) and the corresponding MAC address that was learned via this interface.
Remember if there is NIC teaming or some sort of Load Balancing (Clustering) or VMware on the host then you may see multiple MAC addresses that were learned via this interface.

3 Connect to one of the hosts on the same VLAN or segment for which you know the IP and install a PING Sweeper program like Solarwinds PING Sweep from Engineer's Toolset suite of network tools.

4) Scan the whole segment (i.e. From 1.1.1.1 To 1.1.1.254 if netmask is /24). This will build the ARP cache on the host

5) Immediately display the arp cache in the console by running command such as, arp -a on Windows 2003 / XP.

6) Now find the MAC address from step 2 above and you should have the IP address listed.

There, you have it. You now know the IP address of the unknown device on your LAN.

Show Hidden(Removed) Hardware in Windows 2008 / Vista

Today I came across an interstging delima when trying to install Exchange 2007 SP1 passive node on a Windows 2008 machine (actually VMware VM). I had mistakenly added vmxnet vNICs to the vm instead of adding old Intel e1000 vNICs thinking that these would perform better and faster as VMware KB suggested. Anyway, so I went through the normal process of removing the vmxnet vNICs from control panel in W2k8 and added my normal Intel e1000 assigning them IP addresses and other normal NIC settings.

However, when I tried to rename the Cluster Replication vNIC, it won't let me and would compalin that another vNIC adapter already exists with that name. I realized that I had renamed the original vmxnet card to the same name and wanted to be consistent on both nodes of the cluster. So, I decided to remvoe the old vNICs. I though it would be a simple straight forward process as I have done this many times when doing P2V and went into "Device Manager" and clicked on "Show Hidden Devices". But I couldn't see my old vmxnet cards anywhere.

Hold on, I had removed them earlier so they should not be there in the first place, right?

Well, that wasn't the case and OS was defintely seeing these while I could not.

Since, I didn't use the normal Hardware Removal wizard of Windows 2008 and just powered off the VM and removed the vNICs, they were marked as hidden in the OS. After searching for over an hour on Google, I finally came across Windows 2008 video which showed how to display hidden hardware in Windows 2008/Vista. This is pretty trikcy because in Windows 2003/XP this was very straight forward.

Since I didn't find this on any blog but instead in a video, I thought I would share this with others. You need to create a system variable in order to see the removed (hidden) devices in Windows 2008 / Vista.

Here are the steps you would perform in order to display hidden (removed) devices in Window 2008 / Vista.

1) Logon with an account that has local admin rights on the machine
2) Go to Control Panel --> System -->Advance System Settings --> Environment Variables
3) Under System Variables panel, click New
4) For Variable Name, enter devmgr_show_nonpresent_devices, and for Variable Value, enter 1
5) Click ok, ok and close out all windows.

You may need to restart the machine after creating this system variable. I didn't have to. Go to device manager and click on show hidden devices. Now you will see all devices that were removed from Windows 2008 are dimmed. You can select the device and remove it.

This time, it will be gone for good!