Virtual lab limitations
Still, virtual test lab management tools won’t be enough to stop the environments from going awry, some say. IT
organizations need to define what can be tested, approach it with best practices and ensure anything tested on
the VMs also is run in tests on physical machines before hitting production.
With that in mind, Sister’s of Mercy Healthcare uses a virtual environment for one of three stages of tests and
always completes a test run on physical machines before going live.
“We deploy an application package to production workstations in a pilot stage prior to going live. This way we
make sure the software meets all the requirements and doesn’t hit any snags specific to a physical machine and
performs as expected,” Neubauer says.
Edward Christensen, director of technical operations at Cars.com in Chicago, say he refrains from load or
performance testing in the virtual test environment. “We limit our virtualization to functional and integration
testing only. And unless your production environment is also virtualized, it shouldn’t be used in performance
testing,” he says.
Others agree that performance testing, such as that for application load and availability, is not suited for virtual
“You can’t ramp up the number of concurrent users to see how well the application will perform under of a load of
say 10,000 concurrent users. Virtual machines do share some resources with the physical machine, no matter
how few, that would cause those types of performance tests to be in accurate,” Forrester’s Schwaber says.
Gary Chen, senior analyst at Yankee Group, says he encourages clients to adopt virtualization for test
environments because “if they do, their lives will get much easier and they will get more testing for less money.”
But he also warns IT professionals not to fall victim to the promise of virtualization without heeding some simple
“No one should rely fully on a virtual environment for testing. Physical testing is still a must,” Chen says.
Learn more about this topic