Operton vs. Nehalem-EP at AnandTech

AnandTech’s Johan DeGelas has an interesting article on what he calls “real world virtualization” using a benchmark process his team calls “vApus Mk I” and runs it on ESX 3.5 Update 4. Essentially, it is a suite of Web 2.0 flavored apps running entirely on Windows in a mixed 32/64 structure. We’re cautiously encouraged by this effort as it opens the field of potential reviewers wide open.

Additionally, he finally comes to the same conclusion we’ve presented (in an economic impact context) about Shanghai’s virtualization value proposition. While his results are consistent with what we have been describing – that Shanghai has a good price-performance position against Nehalem-EP – there are some elements about his process that need further refinement.

Our biggest issue comes with his handling of 32-bit virtual machines (VM) and disclosure of using AMD’s Rapid Virtualization Indexing (RVI) with 32-bit VMs. In the DeGalas post, he points out some well known “table thrashing” consequences of TLB misses:

“However, the web portal (MCS eFMS) will give the hypervisor a lot of work if Hardware Assisted Paging (RVI, NPT, EPT) is not available. If EPT or RVI is available, the TLBs (Translation Lookaside Buffer) of the CPUs will be stressed quite a bit, and TLB misses will be costly.”

However, the MCS eFMS web portal (2 VMs) is running in a 32-bit OS. What makes this problematic is VMware’s default handling of page tables in 32-bit VM’s is “shadow page table” using VMware’s binary translation engine (BT). In otherwords, RVI is not enabled by default for ESX 3.5.x:

“By default, ESX automatically runs 32bit VMs (Mail, File, and Standby) with BT, and runs 64bit VMS (Database, Web, and Java) with AMD-V + RVI.”

–    VROOM! Blog, 3/2009

This same guidance is delivered on page 18 of VMware’s current “VI3.5 Performance Guide” (available in PDF format here):

“RVI is supported beginning with ESX 3.5 Update 1. By default, on AMD processors that support it ESX Update 1 uses RVI for virtual machines running 64-bit guest operating systems and does not use RVI for virtual machines running 32-bit guest operating systems.

Although RVI is disabled by default for virtual machines running 32-bit guest operating systems, enabling it for certain 32-bit operating systems may achieve performance benefits similar to those achieved for 64-bit operating systems. These 32-bit operating systems include Windows 2003 SP2, Windows Vista, and Linux.

When RVI is enabled for a virtual machine we recommend you also, when possible, configure that virtual machine’s guest operating system and applications to make use of large memory pages.”

The 32-bit + RVI confusion is depened by the chart on page 9 of DeGelas’ article with indicates “SVM + RVI” for 32-bit hosts. While he goes into great detail about “disabling” RVI for 64-bit tests to show the value of RVI acceleration in the 64-bit workloads there is no mention of steps taken to enable RVI for 32-bit workloads.

The oversight does not invalidate the results of the test, it simply makes them more difficult to interpret. Additionally, it does bring into question some issues of process and how thoroughly that process was reviewed. Still, public benchmarks move the technology in a good direction and underscores a need to develope additional benchmarking of virtual workloads in addition to VMmark.

Enabling RVI for 32-bit Workloads in ESX 3.5

Using the VI Client:

1. Select the virtual machine to be configured,
2. Edit virtual machine settings,
   1. Click the Options tab,
   2. Select Virtualized MMU from the “Settings” column,
      1. Select the desired radio button:
         • Enable RVI – Select “Force use of these features where available”,
         • Disable RVI – Select “Forbid use of these features”, and
         • Default – “Allow the host to determine automatically”