Article ESX under $900 Cheap ESX Server: Hardware Recommendations Author: Alexey Vasilyev Applies to: ESX Server Date: February 21, 2007
	Download PDF Comment on this article

To be better prepared for VMWare VI3 certification, I decided to build a couple of ESX servers at home. VMWare offers free 30 days evaluation licenses for VI3, and therefore the software licenses were not a problem. After the evaluation license expires, upgrading to the commercial one is as easy as registering the new license file. There is no need to rebuild your servers or reinstall virtual machines created during the evaluation. On the other hand, the hardware officially supported by VMWare is pretty expensive while my budget was limited.
After looking through a couple of forums and trying out various configurations, I was able to build servers that are reasonably fast and, the most important, reasonably priced - under $900 each. The configuration I finally came up with is not officially supported by VMWare, but it really works. I wouldn’t risk putting these servers into production, but for the certification purpose they are great. I hope my experience will be useful for those who is in the certification exam preparation and for software developers who is writing for ESX.

The table below describes the configuration I finally came up with:

I decided to use a brand-new motherboard from ASUS P5M2/SAS. This MB

• supports any dual core Intel processor
• has two Broadcom network Gigabit Ethernet adapters (supported by ESX)
• is ATX form factor
• has SAS interface based on supported by ESX LSI Logic 1068 chip.

The last point is the most interesting one. As you probably know, SAS has a native support for SATA by design.
The motherboard fits into an ordinary desktop case. I didn’t have to adjust any BIOS settings. During installation, ESX 3.0.1 detected everything correctly with the default settings. It looked like everything was working, except for one thing.
The Virtual Infrastructure Client couldn’t find any local VMFS storage. Having spent a few hours recreating partitions by fdisk and vmkfstools and reinstalling the system, I decided to compare vmkernel logs with the logs from our production server.

As you can see from the log, the ID looks pretty unusual. These 0x20 codes made me think that the ID was just an encoded text string. Having decoded the data, I discovered the product ID and the serial number of my disk to be “DATA ST380211AS 5PS02ET5”. I saw the ID was wrong and couldn’t be correctly interpreted by ESX drivers.
I made an assumption that if I integrated two SATA disks into RAID (it makes no difference what type of RAID you use; as for me, I chose RAID 0 to make full use of the disk’s capacity), the system will detect this storage. I was right. The logs looked pretty normal, as you can see below.

I didn’t investigate any further, but the conclusion I came up with was: everything works as soon as we are using RAID.

The table below shows approximate prices for the components I used to build the system:

As you can see, the total price is about $875.

Using the above-said hardware and a software based iSCSI SAN on the Linux machine, I was able to set up ESX cluster and try all the VI3 features, including VMotion. Having the cluster at home helped me a lot with the certification. So far everything runs smoothly and I found no issues related to the hardware. The servers are easy to assemble as ASUS P5M2/SAS has everything I needed on board and I had no extra devices to buy/install and configure.

Another thing I like about the hardware is the fact that it’s up to date and after I finish playing with ESX, I’ll reuse the servers and will replace my and my wife’s three-year-old desktops with it.

In the end, I’d like to ask you to share your experience in building inexpensive ESX servers and provide comments to the article. I will do my best to update the article based on your feedback and my own findings.

39 Comments
	Walter Ring (May 14, 2007) I have the same problem as Ryan. Bought me a P5M2/SAS Board, and it always defaults to Intel Raid which makes it useless for ESX. Jumper RAID_SEL1 is not even there, just 3 soldered pins on the motherboard. Tried to short two of the pins with a scissors, but did not help. RAID always defaults to Intel. Any ideas how I can force the thing to LSI RAID?
	Alexey Vasilyev (May 15, 2007) The trick is that you should plug your SATA disks into SAS controller and use LSI Logic MPT Setup Utility (User Manual p.3-35) to configure RAID. LSI Logic MPT Setup Utility configures SAS RAID on LSI 1068 controller which is natively supported by ESX3. To enter to LSI Logic MPT Setup Utility you should press Ctrl-C when it is offered to you. LSI RAID manager and Intel Matrix Storage Manager configure SATA RAID built on Intel ICH7R controller that is not supported by ESX3.
	Walter Ring (May 15, 2007) Alexey, you made my day. (That rhymes :-) Thanks for the hint, everything works fine now. This is my config: Motherboard ASUS P5M2/SAS € 300.- CPU Core 2 Duo E4300 € 107,50 RAM 4 x 1 GB DDR2 Corsair € 159,80 HDD 2 x SATA II Seagate Barracuda 320GB € 142,40 DVD LG GDR-8164B € 15,70 Case Standard Midi Tower € 26,90 which sums up to € 752,30 this is an unbeatable price for an ESX Server, and it is running really quiet! The only strange thing is that my keyboard is not working, when I'm in the start screen of ESX boot or ESX CD boot, but as soon as ESX is loaded, the keyboard works fine.
	Alexey Vasilyev (May 17, 2007) Probably you are using USB keyboard. At least I have the same issue with mine even on Windows. Try to use a PS/2. It works fine with my config.
	Frank Walles (May 20, 2007) Regarding the SAS: This is not absolutely necessary as one can build a free NAS (including iSCSI) on any old hardware using openfiler (http://www.openfiler.com).
	Simon Smith (June 14, 2007) Great articel, thanks. I'm confused about this comment though - "Using the above-said hardware and a software based iSCSI SAN on the Linux machine"...which is the Linux machine? And if you're using iSCSI, why did you need to go to the trouble of getting the SAS devices to appear as local VMFS storage? Thanks!
	Alexey Vasilyev (June 15, 2007) Dear Simon, the ESX compatibility guide provides two options to have ESX installed: first – machine has to have SCSI controller (for both purposes – system installation and VMFS), second – machine should have IDE disk to install system on and remote SAN to VMFS partitions. However, with the second scenario your machine must have either iSCSI, Fiber channel or RAID controller (of course ESX compliant only) to get connected to remote SAN. I have chosen the first scenario as the least expensive one, but to play with VMotion HA & DRS functionality I used emulated on Linux iSCSI LUN (you can even install this Linux as a VM on one of your ESX servers :) ) You can find this package on http://iscsitarget.sourceforge.net/
	Jim Nickel (June 21, 2007) Great information! Can this board support 4 hard drives in the configuration for ESX 3?
	Alexey Vasilyev (June 22, 2007) This board has 2 SAS channels with 4 SATA/SAS connectors each. Thought, you can connect up to 8 drives.
	dc (June 26, 2007) Hi, Don't you need 2 esx hosts to vmotion? Thanks.
	Alexey Vasilyev (June 27, 2007) Well you definitely need more than one. That's why I started my articles with "I decided to build a couple of ESX servers"
	Simon Merel (June 29, 2007) Can anyone recommend a web site where I can buy this ASUS P5M2/SAS motherboard? Either a UK site, or someone that ships to the UK? I have only managed to find uk.insight.com but they are quoting me a 2 weeks+ lead time on my order.
	Walter Ring (July 10, 2007) Alexey, I have a PS/2 keyboard, no USB. So this seems to be a general issue with the board/Bios and how it handles keyboard input.
	Joe (July 16, 2007) Hi all, Attempting to build my first home/test ESX Server (newbie) and would appreciate some feedback; CPU-Intel P D820 $105.00 MB-Intel D945GNT $120.00, Storage Controller- Adaptec 2610SA SATA (not on HCL) mfg by HP- $385 or LSISAS304IE (on HCL) mfg by HP $177, Seagate SATA II 320Gb-$100, 2 x Intel PRO/1000 GT Nics (on HCL)-$60, 2 x DDR2 RAM-$120 --Cannot find the ASUS MB anywhere:-( Just seeking advice from the experienced ones. Thanks for your time
	Red- (July 22, 2007) 2 Quick Questions- What Software based iSCSI SAN do you recommend? Does ESX require a SCSI HD or SAS Controller utilizing SATA Dive... Will it just work with a standard SATA controller / SATA HD in a Raid configuration? Attempting to get ESX to work on a ASUS M2A-VM, AMD X2 4800 CPU, 4 GB Ram, Gigabit LAN
	Charles Windom (August 09, 2007) Great Article Alexey. How did you get the SATA drives connected to the LSI 1068 to show up as local VMFS storage? Configuration: ASUS P5M2/SAS Intel Quad Core 2 Duo Q6600 2 - Seagate SATA/3 500GB drives 8 GB of DDR2-800 memory DVD Qlogic QLA2342 not connected to MSA1000 yet
	Alexey Vasilyev (August 10, 2007) Charles, SAS interface supports SATA drives “by design”. So you need to connect all of your SATA disks to SAS controller (there were two 4-channel connectors with my mother board). This controller is based on LSI 1068 chip. But… ESX module loader cannot detect correctly these drives. To fix this issue you should join disks (at least 2 of them) into RAID. Please, for this use LSI Logic MPT Setup Utility (ASUS p5m2/SAS User Manual p.3-35) BTW, I found your configuration very interesting and strong :)Please, when you get it working could you post performance results (number of VMs etc...).
	Charles Windom (August 17, 2007) When I get everything up and running, I will let you know how it is performing. I also have two AMD x2 6000+ 6GB memory, 2 - 400GB Seagate SATA II harddrives, DVD Burner servers running using OpenFiler v2.2 iSCSI and NFS for VMFS. great performance. Hosting Exchange Server 2003, Active Directory, WSUS, SQL Server 2005 Backup done with VCB and Symantec BackupEXEC version 11D
	toan trac (August 18, 2007) Guys, Would this cpu work? Intel Core 2 Duo E6320 Conroe 1.86GHz 4M shared L2 Cache LGA 775 Processor Also, does the raid of the motheroard work in Redhat Linux ES 4? I am planning for future with Redhat too. Thanks, Tony
	Delaure Hage (September 14, 2007) I need a similar AMD Opteron based solution. Any recommendations??? Thanks
	Edward Grigson (September 17, 2007) I've just bought the P5M2/SAS motherboard but having lots of problems. Firstly I bought the Intel 6750 dual core, which doesn't work with this MB. Swapped that out for a 6600 which works fine. BUT, after installing ESX 3.01 I get PSOD at intermittant times. I'm using two SAS disks rather than SATA, and ESX finds them fine, lets me create VMFS etc. The ESX install completes succesfully, boots etc, but will typically die with a PSOD within five minutes of starting. I've not added anything else to the system (bar an IDE CD-ROM) so not sure what's causing the issue - this seems to be the same setup others have working fine. Firmware is probably my next stop.
	David T. (September 24, 2007) Hey guys, Please help. I have Intel DP35DP Core 2Duo Q6600 2.4gz. 2 500gb Western Digital Sata. 8gb ram. Trying to Load ESX3.02 and it does not see the Hard Drives. I Raided the SATA drives w/Intel Storage Mgr built on Raid S/W. If i play around w/bios, switch from Raid to AHCI, it does not work. If I switch to IDE and use Legacy mode istead of Native, it sees the drive but then gives me error during initial install or partitioning. Your expertise in installing ESX would be GRATELY appreciated. By the way, I have 2 configured like this. I spent boo-coo bucks and hope I can figure this out so I can play w/my lab for the VI3 cert. Thank you much!
	Shan N (September 27, 2007) How can I connect SATA drives to the SAS controller on the Asus P5M2/SAS MB???
	Greg C (October 11, 2007) Hello, Which Intel Quad Core CPU would you recommend for this articles motherboard? I am about to purchase the MB but noticed a Dual Core was used, however, this MB can handle a quad core. Would appreciate any advice! Thanks!
	Edney François (October 16, 2007) Hello. The processor Intel E6600 Core 2 Duo 2.4 4Mb 1066 works fine with ESX 3, or just the Pentium D ? Thanks any help.
	Jason B (October 16, 2007) Anyone have any luck with quad core processors supporting the VT bit? Looking to run 64bit guests...
	Jody N (October 17, 2007) Hi All, I would like to use a higher end (but not too high end) SAS Raid controller for this mb for better performance. Would you folks have any recommendations? I see these as contenders Dell Perc 5/i HP P400 256/512 w batter backup? LSI? Unsure of models. Thanks all!
	DJ (October 23, 2007) Since the following M/B uses the LSI 1068 controller, should it also work? ASUS DSBF-D/SAS M/board
	scotte1976 (November 14, 2007) Hi, I am about to purchase the P5M2/SAS motherboard. What is the best CPU to buy for this? How many VM's can I install with the best CPU available ? Thanks
	Maxim (November 15, 2007) I believe this one will work well for you: Quad-Core Duo Xeon 3220 (2.40GHz, 1066FSB, L2:8MB) http://support.asus.com/cpusupport/cpusupport.aspx?SLanguage=en-us You can type P5M2/SAS in the CPU support field to view all the supported options. Choose among dual cores.
	osmethod (November 23, 2007) Thank you Alexey for sharing this method with us. FYI, Intel's Core 2 Quad Q6600 G0 processor works with this motherboard. ESX Server 3.0.2. sees 4 CPU x 2.4GHz.
	Nick Gapshin (November 27, 2007) Hi! Is there any ESX supported Zero Channel RAID adapter for this motherboard to build RAID5 ?
	Robert (December 02, 2007) Hi, Excellent tip! I have bought the P5M2/SAS and installed VMWare 3.0.2 on it (build 61618) and all is working well, except the performance of the network from a normal workstation to a win2003 R2 standard virtual machine using a windows share is fairly slow.. i'm getting about 6 megabyte/sec max on it, but everything is switched 1Gb. Also, i think the onboard videocard is stealing 256mb from my 1GB RAM.
	Robert (December 04, 2007) As it turns out, it is write-cache related (there is none). I have ordered the MegaRAID SAS 8300LXP, which is the Zero-Channel Raid adapter for the LSI1068 (in the ASUS manual) and it has 128mb of RAM so i'm assuming i can then set it to 50% read and 50% write or 75/25 perhaps.
	Nick Gapshin (December 12, 2007) Robert, if you install ESX 3 on MegaRAID SAS 8300XLP + P5M2/SAS, please inform on it. I wish to make it as described here: http: // communities.vmware.com/message/695520*695520 but I'm not assured of compatibility of this controller and the motherboard.
	Stefan Nguyen (April 23, 2008) Can someone show me how to access RAID manager on this P5M2/SAS board. I have 4 (500 WD SATA2) drives and want to configure RAID 0 for all drives but having prolem accessing RAID per instructedo n the board guide. If I have to connect my SATA drives to SAS connections, than it only allow 2 connections correct? So what should I do with other 2 SATA drives?
	Abhilash (July 23, 2008) I am currently trying to learn ESX 3.5 by installing on intel D915GEV desktop board with P4 CPU and 4GB DDR2 RAM. It is working flawlessly on IDE and SATA disks without needing any additional storage/network controller!
	CaptainChaos (August 08, 2008) Abhilash, please post with an update on how you are getting on. Thanks.
	Michael Tucker (September 02, 2008) I have ESX3i running on the following, works great, The anboard SATA and NIC don't work. And I needed to have the CD plugged in the onboard IDE controller. ASUS M2A-VM AMD AM2 4800+ CPU 2ea 2gig RAM Intel GB PCIEX GT NIC Promise 300 TX2 SATA Controller 160 GB SATA II Drive