This is a step-by-step guide to installing VMWare ESXi 4.1 and vSphere server, including vlan configuration and Dell Equallogic MEM Extensions
Getting Started — what you’ll need You will need:
- a copy of VMWare ESXi 4.1
- a copy of VSphere Server 4.1
- at least one physical managed switch (Cisco preferred)
- at least two ESXi capable hosts (x64, with VT enabled)
- at least one storage device capable of iSCSI (Synology or Equallogic preferred)
- a copy of the Dell Equallogic MEM plugin (if using Equallogic iSCSI)
Create the VLAN setup on a Cisco switch
First, console into the Cisco switch, go into global configuration mode, and give it a hostname:
- Commands: ‘configure terminal’ and ‘hostname vmwareswitch’
Next, create the vlans required for vmware. It is considered best practices to place vmotion, iscsi, management, fault tolerance, and server traffic into their own vlans so that’s what will be done here. Create the required five vlans and give them IP address ranges:
Commands, in order:
- ‘interface vlan 501’
- ‘description iSCSI traffic’
- ‘ip address 10.114.112.1 255.255.255.192’
- ‘interface vlan 510’
- ‘description vMotion traffic’
- ‘ip address 10.114.112.65 255.255.255.192’
- ‘interface vlan 520’
- ‘description Fault tolerance traffic’
- ‘ip address 10.114.112.129 255.255.255.192’
- ‘interface vlan 530’
- ‘description Management traffic’
- ‘ip address 10.114.112.193 255.255.255.192’
- ‘interface vlan 540’
- ‘description Virtual Server traffic’
- ‘ip address 10.114.113.1 255.255.255.0’
After all of the vlans have been created, they need to be assigned interfaces, you can do this by entering the following commands (for a 48 port switch)
- ‘interface range gig0/1-12’
- ‘description iSCSI and iLo’
- ‘switchport access vlan 501’
- ‘switchport mode access’
- ‘interface range gig0/13-18’
- ‘description Server Traffic’
- ‘switchport access vlan 540’
- ‘switchport mode access’
- ‘interface range gig0/19-29’
- ‘description Management and Fault Tolerance traffic’
- ‘switchport trunk encap dot1q’
- ‘switchport trunk allowed vlan 520,530
- ‘interface range gig0/30-35’
- ‘description vMotion’
- ‘switchport access vlan 510’
- ‘switchport mode access’
Next, create the ports that will allow your switch to connect to another switch on the network. In this configuration, we’ll be using port-channel to allow multiple interfaces to pass traffic as if they were one large port:
- !!!WARNING!!!: Port-channeling must be enabled on the receiving end switch as well or traffic will not be passed.
Commands:
- ‘interface port-channel 1’
- ‘switchport access vlan 56’
- ‘switchport trunk encap dot1q’
- ‘switchport trunk allowed vlan 56’
- ‘switchport mode trunk’
- ‘switchport nonegotiate’
- ‘interface range gig0/44-48’
- ‘description to CORE switch’
- ‘switchport access vlan 56’
- ‘switchport trunk encap dot1q’
- ‘swithcport trunk allowed vlan 56’
- ‘switchport mode trunk’
- ‘switchport nonegotiate’
- ‘channel-group 1 mode on’
- ‘interface vlan 56’
- ‘ip address 10.237.48.231 255.255.255.0’
- ‘description CORE switch traffic’
Finally, we’ll have to add a gateway of last resort so the traffic from the vmware servers can reach the rest of the network. This ip address should be the address of the next hop for IP traffic, aka, the switch/router your connecting to on your port-channel interface:
- Note: After your done entering all of the commands above, don’t forget to ‘copy run start’ to save your work!
commands:
- ‘ip route 0.0.0.0 0.0.0.0 10.237.48.1’
- ‘copy run start’
Install ESXi
Install ESXi on all host servers… Place the ESXi setup cd in the servers cd tray, boot from cd and wait for the setup program to boot up:
After the setup application boots, press ‘enter’ to start the install process:
Next, press F11 to accept the license agreement:
Select the disk drive you’d like to install ESXi on and press ‘enter’:
Press ‘enter’ again at the confirmation dialog box:
And again, Press ‘F11’ to begin installation:
Wait for install to complete…..
Then, when the machine finishes rebooting and comes back up, press ‘F2’ to go into ‘customize system’, when you see a login prompt, just hit ‘enter’ accepting the defaults. Go to ‘Configure Management Network’
And click on ‘network adapters’, at the network adapters screen, select the port(s) that you have your management network on with ‘space’ and then press ‘enter’
Next, go to the ‘VLAN’ option and set it to the VLAN for your management interface:
Finally, go to ‘IP Configuration’, check ‘set static IP address’, and set the correct information:
When you ‘esc’ out, you’ll see a prompt about restarting the management network, just click ‘Y’ and continue:
And your all set, ESXi has been installed successfully! You can now reach the ESXi host from the management IP address you’ve set earlier
Configure the network on the ESXi hosts
Create an iSCSI connection
Use the vmware vSphere Client to connect to the ESXi host you want to set iSCSI up on using ‘root’ as the login username. Then, click on the ‘configuration’ tab and then click on ‘storage adapters’ on the left hand side:
Next, click on the ‘iSCSI Software Adapter’ and then click ‘Properties’ on the right hand side:
Then, on the iSCSI properties window, click ‘configure’
And then check the ‘enabled’ checkbox and next click ‘ok’
Then, go back into ‘configure’ and enter the iSCSI name and alias. These should be the same thing and can be found from your iSCSI machines configuration:
Click ‘ok’ after entering the IQN and then click on the ‘Dynamic Discovery’ tab, then click ‘add’ at the bottom:
On the ‘Add’ window, enter the IP address of your iSCSI server and then click ‘ok’ and ‘close’. If you see a prompt asking about rescanning drives, click ‘yes’:
You should now see 1 connected target available, if so, click on ‘storage’ on the left hand side:
- NOTE: This part is to only be completed once, after this has been done on the first host machine, it will not need to be performed again
Under ‘Storage’, click ‘Add Storage…’ on the right hand side:
On the ‘Add Storage’ window, verify ‘Disk/LUN’ is selected and click ‘next’:
Select the iSCSI device, and click ‘next’
Accept all defaults for each screen, and then click finish. Once the disk has been created, it will become available on any ESXi host that has the iSCSI adapter enabled.
Install VSphere
First, create a new virtual machine and install Windows Server 2008 R2. After Windows setup completes, install vmware tools, run windows update, and then install windows SQL server 2008 if you would like to use a seperate SQL server for vsphere.
After SQL Server 2008 is installed, or if you choose to allow vmware to create the SQL instance, mount the vSphere ISO to the Windows 2008 server and then run ‘setup.exe’. Click the ‘vCenter Server’ link in the install window:
Accept the default language and then when the installer launches click ‘next’ on the welcome prompt. Click ‘next’ again on the patents screen and then accept the license agreement and click ‘next’ a third time.
Enter the relevant username/company name and key if applicable and click ‘next’:
Select the radial button for ‘Use an existing supported database’ and select ‘vsphere’ from the dropdown menu if you installed SQL Server 2008. If you did not install SQL Server 2008 (RECOMMENDED!!!) then select the Microsoft SQL 2005 instance and then click ‘next’:
Enter your SQL Server username and password and then click ‘next’:
Click ‘OK’ if you see the following warning:
Accept defaults on the next screen and click ‘next’:
Accept the default install directory and click ‘next’. On the next screen, make sure the ‘Create a standalone VMWare vCenter Server instance’ option is selected and then click ‘next’:
Accept all defaults on the configure ports window and click ‘next’. Then click ‘next’ again on the vCenter Server JVM Memory window accepting all defaults:
Finally, click ‘install’ and install vSphere Server:
Installing Microsoft SQL Server 2008 for vSphere
- Note: This is optional and is not required for vSphere functionality, but is recommended
Console or RDP into the vSphere server and then mount the SQL Server 2008 .ISO into the virtual drive. When ‘AutoPlay’ appears, click ‘Run SETUP.EXE’:
At the .Net install prompt, click ‘OK’:
Go to ‘start -> my computer’ and launch the setup again from the cd-rom drive. Then click on ‘installation’ on the left and then ‘new installation or add features to an existing installation’ at the top:
Click ‘OK’ at the checks approved dialog screen and then click ‘next’ at the cd-key window:
Accept the license terms, click ‘next’ and then click ‘finish’ at the install setup support files screen.
Click ‘Next’ through the setup support rules window screen, and then verify ‘SQL Server Feature Installation’ is selected and click ‘next’:
Check the boxes for ‘Database Engine Services’ ‘Business Intelligence Development Studio’ and ‘Management Tools – Basic’ and then click ‘next’:
Click ‘next’ through install rules and then accept all defaults for the ‘Instance Configuration’ screen:
Click ‘next’ through the disk usage option. Then, under the ‘Account Name’ options, use the dropdown arrow and select ‘NT AUTHORITY\NETWORK SERVICE’ for both the ‘SQL Server agent’ and ‘SQL Server Database Engine’ options. Also, change the startup type for the ‘SQL Server Agent’ to ‘automatic’ and then click ‘next’:
Select ‘Mixed mode’ and enter a password for the ‘sa’ account. Also, click the ‘Add Current User’ button and then click ‘next’:
Click ‘next’ through the error reporting screen and then ‘next’ again through installation configuration rules. Finally, click ‘Install’ and install SQL Server 2008:
When the installation completes, launch ‘SQL Server Management Studio’ and login using a server name of “.” without the “s and windows authentication :
Right click on the ‘Databases’ folder and then select ‘New Database…’:
In the new database window, name the new database “vSphere” and click ‘ok’
Close management studio and then under ‘start -> administrative tools’ click on ‘Data Sources (ODBC)’:
Click on the ‘System DSN’ tab and then click ‘Add…’:
Select ‘SQL Server Native client 10.0’ and click ‘finish’:
under the ‘create new datasource’ window, use the name ‘vsphere’ and use the server ‘.’ then click ‘next’
On the next window, select the ‘With SQL Server authentication’ radial button and enter your ‘sa’ account information, then click ‘next’
Check the box labeled ‘Change the default database to’ and then, from the dropdown, select ‘vSphere’ and click ‘next’:
Accept defaults on the next screen and click ‘Finish’:
Microsoft SQL Server 2008 is now ready for a vSphere installation.
Create a new datacenter in VSphere
After your VSphere server has been installed, connect to it via a vsphere client:
And Click on the ‘Create a datacenter’ task:
Create a host cluster in VSphere
After you have created your new datacenter, you’ll need to create a cluster so you can start adding hosts. First, click on your newly created datacenter and click on the ‘Create a cluster’ task:
Then, at the ‘New Cluster Wizard’, give your cluster a name and check both the ‘Turn on VMWare HA’ and ‘Turn On VMWare DRS’ checkboxes, and then click next:
On the next screen, make sure fully automated is enabled and the migration threshold is at the middle marker and click ‘next’:
Click the ‘Off’ radial button for power management and click ‘next’
Make sure ‘Enable Host Monitoring’ is checked, and verify ‘Disable: power on VMs that violate availability constraints’ is selected and click ‘next’:
Change ‘Host Isolation response’ to ‘Leave powered on’ and leave restart priority at ‘Medium’ and click next:
Use the dropdown to make sure ‘VM Monitoring’ is disabled, and click ‘next’:
On the EVC screen, make sure ‘Disable EVC’ is selected and click ‘next’:
Accept the default of ‘Store the swapfile in the same directory as the virtual machine’ and click ‘next’:
Verify all settings are correct and click ‘Finish’:
After the process completes, you’ll have a newly created cluster. As you add hosts to the new cluster, they will be able to make use of things like VMotion and High Availability (HA). The next step is to start adding the hosts.
Add hosts to a VMWare VSphere cluster
First, click on your VSphere cluster and click on the task ‘Add a host’:
Next, enter the IP address of the host you’ll be adding, along with the ESXi login information for that host and click ‘next’:
You may see a warning similar to the one shown here, if so, just click ‘Yes’ and continue:
Verify the information is correct and click ‘next’:
Assign a license if available or leave it as is and click ‘next’:
Leave ‘Enable Lockdown Mode’ unchecked and click ‘next’:
Select your cluster and click ‘next’
Verify the add information is correct and click ‘Finish’:
Enabling Jumbo packets for faster file transfers
To enable jumbo packets, first make sure they are supported on the physical switch that will be used between the VMWare ESXi hosts and the iSCSI devices as well as the devices themselves.
Next, enable jumbo packets on the switch. In the case of a Cisco 3750 switch use the following command: system mtu jumbo 9000 then save the running config and restart the switch.
- For other Cisco devices, see: this Cisco link
After jumbo frames have been enabled on the switch, enable them on the iSCSI device. For a Synology iSCSI system, go to ‘Control Panel -> Networking -> Network Interface’ and select jumbo frames from the drop-down.
Finally, set jumbo frames on the VMWare ESXi host. Either console into the host directly or use something like iLo to access the console remotely. Then, go into ‘troubleshooting options’:
Enable the ‘Enable Local Tech Support’ option and then press ‘Alt + F2’ to access the technician console.
Once at the console, enter the following command: esxcfg-vswitch -m 9000 vswitch1 Where ‘vswitch1’ is the name of the vswitch you previously set up for iSCSI traffic.
Create a Host Profile
Right click on your VSphere cluster and select ‘Host Profile -> Manage Profile’. You will see a prompt similar to the one below, click ‘Yes’:
Make sure the ‘Create Profile from existing host’ option is selected and click ‘next’:
Next, select the first host you’ve added to your cluster. This host should be the one that is running VSphere and has all networking vswitch options set correctly:
Name your profile and click ‘next’:
Verify all information is correct and click ‘Finish’:
Host Profile Compliance
Go to the ‘profile compliance’ tab in your cluster and click the ‘check compliance now’ option at the top. you should see all hosts added after your first host showing as ‘noncompliant’:
Put each of the non-compliant hosts into ‘maintenance mode’ and if you see a prompt similar to the one below just click yes:
Finally, right click on each of the non-compliant hosts and click ‘Apply’. Enter any required information and once complete, click ‘Finish’:
You can click the ‘Check compliance now’ button at the top again after all hosts have had the profile applied to verify the profiles match, then bring each of the hosts back out of maintenance mode.
Install vSphere Update Manager
RDP or console into the vSphere server, navigate to the setup folder for vSphere, and run ‘VMware-UpdateManager.exe’ from the updateManager folder, clicking ‘next’ at the welcome window:
Click ‘next’ through the patents agreement screen:
Accept the license agreement and click ‘next’:
Enter the information for the vSphere server and then click ‘next’:
Select the ‘Use an existing supported database’ option if applicable (x86 SQL server only), if not, just select ‘Install a Microsoft SQL Server 2005 instance’
Accept all defaults for Update Manager ports and click ‘next’
Accept the defaults for install locations and click ‘next’
click ‘finish’
Finally, in vSphere Client, click on the ‘Plug-ins’ dropdown at the top and then select ‘Manage Plug-ins….’. Next, click on the status for VMware vCenterUpdateManager where it will have an option similar to ‘install plug-in’
vSphere Update manager is now installed and ready for use.
Install Dell Equallogic MEM plugin
- Reference: http://www.thatcouldbeaproblem.com/?p=137
First, click on ‘Home’ at the top bar and then click on ‘Update Manager’ at the bottom, under ‘solutions and applications’:
Next, click on the ‘Patch Repository’ tab and then click on the ‘Import Patches’ button at the top right.
Browse to the location of the Dell Equallogic MEM plugin zip file and then click ‘next’
If you receive a warning like the one below, click ignore and continue:
Verify the Dell Equallogic plugin is to be added and then click ‘Finish’
Next, go back to ‘Hosts and Clusters’ view in vSphere and then click on the ‘update manager’ tab. Next, right click on any blank space and click ‘attach…’ or click ‘attach…’ in the top right:
On the next window, click ‘Create Baseline…’ at the top right:
Name the baseline something similar to “EquallogicStoragePlugin”, select the ‘Host Extension’ radial button, and then click ‘next’
Scroll to the bottom of the extension list, highlight the ‘Dell Equallogic iSCSI plugin’ and click the down arrow to add it to the ‘Extensions to Add’ list. Click ‘next’
click ‘finish’
Back at the ‘Attach baseline’ window, check the box for the newly created ‘EquallogicPlugin’ baseline extension and click ‘Attach’
Highlight all of the hosts in the ‘Host Name’ list at the bottom and then click ‘Stage…’ and then ‘Remediate…’
- !!!!WARNING!!!!: This will put the selected Hosts into maintenance mode, install the plugin, and then restart the Hosts. DO NOT do this if you have mission critical VMs currently running on one of the highlighted hosts.
On the ‘Remediate’ window, accept all defaults and click ‘next’
verify the Dell Equallogic plugin is selected and click ‘next’
Accept all defaults on the next screen and click ‘next’
Accept all defaults again and click ‘next’
Verify the information is correct and click ‘Finish’ to start the Remediation process:
If you receive the below error, you will have to perform additional step as outlined below before Remediation can occur.
- ADDITIONAL STEP: only to be used if you receive the vSphere error code 10 while attempting Remediation:
Console into the affected ESXi host, ‘alt + F1’ into local technician console, and then run the following command: ‘mkdir -p /tmp/scratch/var/tmp’
To install the MEM plugin on the ESXi host currently running vSphere, you will have to perform the following steps:
- First, stage the dell plugin using the update manager
- Next, shut down any currently running VMs on that host, including the vSphere server
- Then console into the host and enter maintenance mode with command ‘vim-cmd /hostsvc/maintenance_mode_enter
Next, copy the Dell .vib files from ‘/tmp/scratch/var/tmp/cache’ to a more permanent location
After that, run the following commands to install the plugin: ‘esxupdate -b cross_dell-eql-host-connection-mgr_410.1.0.0.130413-214324.vib –nosigcheck –nodeps update’ ‘esxupdate -b cross_dell-routed-psp_410.1.0.0.130413-214324.vib –nosigcheck –nodeps update’
Finally, reboot the host ESXi server, exit maintenance mode via ‘vim-cmd /hostsvc/maintenance_mode_exit’ and bring back up the vSphere server