Performance Testing in the Cloud

What is Cloud?

A cloud consists of three components, one or multiple datacenters, a network and a “zillion” number of devices. That’s what it’s all about, the components and their interaction. Ultimately the user is interested in the end-to-end experience, regardless of the components.

A cloud is grouped into private or public cloud based on the location of the data center where the services are being virtualized. In general, a public cloud is an environment that exists outside the purview of company firewall, could be a service/technology offered by a third party vendor while private cloud acts behind the firewall for the exclusive benefit of an organization and its customers.


Source : HP

Testing and the Cloud
While many companies are approaching cloud computing with cautious optimism, testing appears to be one area where they are willing to be more adventurous. There are several factors that account for this openness toward testing in the cloud:

• Testing is a periodic activity and requires new environments to be set up for each project
  Test labs in companies typically sit idle for longer periods, consuming capital, power and space.   Approximately 50% to 70% of the technology infrastructure earmarked for testing is underutilized, according  to both anecdotal and published reports.

• Testing is considered an important but non-business-critical activity. Moving testing to the cloud is seen as a safe bet because it doesn’t include sensitive corporate data and has minimal impact on the organization’s business-as-usual activities.

• Applications are increasingly becoming dynamic, complex, distributed and component- based, creating a multiplicity of new challenges for testing teams. 
For instance,mobile and Web applications must be tested for multiple operating systems and updates,multiple browser platforms and versions, different types of hardware and a large number of concurrent users to understand their performance in real-time. The conventional approach of manually creating in-house testing environments that fully mirror these complexities and multiplicities consumes huge capital and resources.

Why Opting for Cloud Computing as source of Performance testing
Many companies use their Web sites for sales and marketing purposes. In fact, a company can spend millions of pounds creating engaging content and running promotional campaigns to draw users to its site. Unfortunately, if the site crashes or response time crawls all that time, energy and money could be wasted.
A first case is when demand for a service varies with time. Provisioning a data center for the peak load it must sustain a few days per month leads to underutilization at other times, for example. Instead, Cloud Computing lets an organization pay by the hour for computing resources, potentially leading to cost savings even if the hourly rate to rent a machine from a cloud provider is higher than the rate to own one. A second case is when demand is unknown in advance.

For example, a web startup will need to support a spike in demand when it becomes popular, followed potentially by a reduction once some of the visitors turn away. Finally, organizations that perform batch analytics can use the ”cost associativity” of cloud computing to finish computations faster: using 1000 EC2 machines for 1 hour costs the same as using 1 machine for 1000 hours. For the first case of a web business with varying demand over time and revenue proportional to user hours, we have captured the tradeoff in the equation below.The left-hand side multiplies the net revenue per user-hour by the number of user-hours, giving the expected profit from using Cloud Computing. The right-hand side performs the same calculation for a fixed-capacity datacenter by factoring in the average utilization, including nonpeak workloads, of the datacenter. Whichever side is greater represents the opportunity for higher profit.

Cloud Areas and their Differentiators:

• Private cloud is on premise, dedicated to one organization, often located in the same datacenter(s) as the legacy applications; address the needs of mission critical applications and applications having access to large amount of confidential data.

• Hosted private cloud is really a variant of the previous category, located either in the client’s or the hoster datacenter, but managed by a hosting provider. It addresses the same needs as the private cloud.

• Virtual private cloud is a single or multi-tenant environment, located in a clearly defined geography, with documented and auditable security processes & procedures, clear SLA’s, addressing the needs of business critical applications. Formal contracts are established, payment is by consumption, but with regular invoices. Access to the environment can be over the internet, VPN or leased lines. HP calls this an Enterprise Class Cloud.

• Public Cloud is a multi-tenant environment providing IaaS, PaaS and/or SaaS services on a pay- per-use basis without formal contracts. Payment is typically via credit card. Such environments address the needs of web applications, test & development, large scale processing, (if not too much data needs to be transferred).

Source: HP

The first two categories are asset intensive and typically do not lend themselves to a pay-per-use model; the latter two are pay-per-use. The first two are typically single tenant, this means they are used by one organization; the latter two are most often multi-tenant environments where multiple companies share the same assets.

Performance Testing in Cloud – Benefits & Limitations:
• Fast provisioning using preconfigured images. You can set up the infrastructure you need in minutes.

• Simplified security. All required protections are set up by default, including firewall, certificates, and encryption.

• Improved scalability. Leading load testing solution providers have negotiated with cloud providers to allow users of their software to employ more virtual machines (for the purpose of load testing) than are allowed by default.

• A unified interface for multiple cloud providers. Load testing solutions can hide provisioning and billing details, so you can take maximum advantage of the cloud in a minimum of time.

• Advanced test launching. You can save time and effort by defining and launching load generators in the cloud directly from the load testing interface.

• Advanced results reporting. Distinct results from each geographic region involved in the test are available for analysis.

Limitations:
Although testing from the cloud is, in many cases, more realistic than testing in the lab, simply moving to the cloud is not enough to ensure the most realistic tests. Real users often have access to less bandwidth than a load generator in a cloud data center. With a slower connection, the real user will have to wait longer than the load generator to download all the data needed for a web page or application. This has two major implications:

• Response times measured as-is from the cloud with virtually unlimited bandwidth are better than for real users. This can lead test engineers to draw the wrong conclusions, thinking that users will see an acceptable response time when in reality they will not.

• The total number of connections established with the server will increase, because on average, connections for real users will be open longer than connections for the load generator. This can lead to a situation in which the server unexpectedly refuses additional connections under load.

Conclusion: The benefits of Cloud Computing Solution cannot be ignored by companies running Performance tests striving to overcome the constraints of their current IT hardware to simulate the realistic environment whilst struggling to justify the cost of investing in major upgrades.

Visit Performance Testing In Cloud For Know More.