Automation Tool Selection Recommendation

• Overview
• Information Gathering
• Tools and Vendors
• Evaluation Criteria
• Tools Evaluation
• Matrix
• Conclusion
• Overview

“Automated Testing” means automating the manual testing process currently in use. This requires that a formalized “manual testing process” currently exists in the company or organization. Minimally, such a process includes:

–        Detailed test cases, including predictable “expected results”, which have been developed from Business Functional Specifications and Design documentation.

–        A standalone Test Environment, including a Test Database that is restorable to a known constant, such that the test cases are able to be repeated each time there are modifications made to the application.

Information Gathering

Following are sample questions asked to tester who have been using some the testing tools:

How long have you been using this tool and are you basically happy with it?

How many copies/licenses do you have and what hardware and software platforms are you using?

How did you evaluate and decide on this tool and which other tools did you consider before purchasing this tool?

How does the tool perform and are there any bottlenecks?

What is your impression of the vendor (commercial professionalism, on-going level of support, documentation and training)?

Tools and Vendors

• Robot – Rational Software
• WinRunner 7 – Mercury
• QA Run 4.7 – Compuware
• Visual Test – Rational Software
• Silk Test – Segue
• QA Wizard – Seapine Software

Tools Overview

Robot – Rational Software

–        IBM Rational Robot v2003 automates regression, functional and configuration testing for e-commerce, client/server and ERP Applications. It’s used to test applications constructed in a wide variety of IDEs and languages, and ships with IBM Rational TestManager. Rational TestManager provides desktop management of all testing activities for all types of testing.

WinRunner 7 – Mercury

–        Mercury WinRunner is a powerful tool for enterprise wide functional and regression testing.

–        WinRunner captures, verifies, and replays user interactions automatically to identify defects and ensure that business processes work flawlessly upon deployment and remain reliable.

–        WinRunner allows you to reduce testing time by automating repetitive tasks and optimize testing efforts by covering diverse environments with a single testing tool.

QA Run 4.7 – Compuware

–        With QA Run, programmers get the automation capabilities they need to quickly and productively create and execute test scripts, verify tests and analyze test results.

–        Uses an object-oriented approach to automate test script generation, which can significantly increase the accuracy of testing in the time you have available.

Visual Test 6.5 – Rational Software

–        Based on the BASIC language and used to simulate user actions on a User Interface.

–        Is a powerful language providing support for pointers, remote procedure calls, working with advanced data types such as linked lists, open-ended hash tables, callback functions, and much more.

–        Is a host of utilities for querying an application to determine how to access it with Visual Test, screen capture/comparison, script executor, and scenario recorder.

Silk Test – Segue

–        Is an automated tool for testing the functionality of enterprise applications in any environment.

–        Designed for ease of use, Silk Test includes a host of productivity-boosting features that let both novice and expert users create functional tests quickly, execute them automatically and analyze results accurately.

–        In addition to validating the full functionality of an application prior to its initial release, users can easily evaluate the impact of new enhancements on existing functionality by simply reusing existing test casts.

QA Wizard – Seapine Software

–        Completely automates the functional regression testing of your applications and Web sites.

–        It’s an intelligent object-based solution that provides data-driven testing support for multiple data sources.

–        Uses scripting language that includes all of the features of a modern structured language, including flow control, subroutines, constants, conditionals, variables, assignment statements, functions, and more.

Evaluation Criteria

Record and Playback         Object Mapping
Web Testing Object              Identity Tool
Environment Support        Extensible Language
Cost                                            Integration
Ease of Use                             Image Testing
Database Tests                     Test/Error Recovery
Data Functions                    Object Tests
Support

3 = Basic  2 = Good  1 = Excellent

Tool Selection Recommendation

Tool evaluation and selection is a project in its own right.

It can take between 2 and 6 weeks. It will need team members, a budget, goals and timescales.
There will also be people issues i.e. “politics”.

Start by looking at your current situation
– Identify your problems
– Explore alternative solutions
– Realistic expectations from tool solutions
– Are you ready for tools?

Make a business case for the tool

–What are your current and future manual testing costs?
–What are initial and future automated testing costs?
–What return will you get on investment and when?

Identify candidate tools

– Identify constraints (economic, environmental, commercial, quality, political)
– Classify tool features into mandatory & desirable
– Evaluate features by asking questions to tool vendors
– Investigate tool experience by asking questions to other tool users Plan and schedule in-house demonstration by vendors
– Make the decision

Choose a test tool that best fits the testing requirements of your organization or company.

An “Automated Testing Handbook” is available from the Software Testing Institute (www.ondaweb.com/sti), which covers all of the major considerations involved in choosing the right test tool for your purposes.

Thanks For Reading This Blog. You Know More About: Automation Tool

Automation Tool Selection Recommendation

“Automated Testing” means automating the manual testing process currently in use. This requires that a formalized “manual testing process” currently exists in the company or organization. Minimally, such a process includes:
–        Detailed test cases, including predictable “expected results”, which have been developed from Business Functional Specifications and Design documentation.
–        A standalone Test Environment, including a Test Database that is restorable to a known constant, such that the test cases are able to be repeated each time there are modifications made to the application.
Information Gathering
Following are sample questions asked to tester who have been using some the testing tools:
How long have you been using this tool and are you basically happy with it?
How many copies/licenses do you have and what hardware and software platforms are you using?
How did you evaluate and decide on this tool and which other tools did you consider before purchasing this tool?
How does the tool perform and are there any bottlenecks?
What is your impression of the vendor (commercial professionalism, on-going level of support, documentation and training)?
Tools and Vendors

• Robot – Rational Software
• WinRunner 7 – Mercury
• QA Run 4.7 – Compuware
• Visual Test – Rational Software
• Silk Test – Segue
• QA Wizard – Seapine Software

Tools Overview
Robot – Rational Software
–        IBM Rational Robot v2003 automates regression, functional and configuration testing for e-commerce, client/server and ERP applications. It’s used to test applications constructed in a wide variety of IDEs and languages, and ships with IBM Rational TestManager. Rational TestManager provides desktop management of all testing activities for all types of testing.
WinRunner 7 – Mercury
–        Mercury WinRunner is a powerful tool for enterprise wide functional and regression testing.
–        WinRunner captures, verifies, and replays user interactions automatically to identify defects and ensure that business processes work flawlessly upon deployment and remain reliable.
–        WinRunner allows you to reduce testing time by automating repetitive tasks and optimize testing efforts by covering diverse environments with a single testing tool.
QA Run 4.7 – Compuware
–        With QA Run, programmers get the automation capabilities they need to quickly and productively create and execute test scripts, verify tests and analyze test results.
–        Uses an object-oriented approach to automate test script generation, which can significantly increase the accuracy of testing in the time you have available.
Visual Test 6.5 – Rational Software
–        Based on the BASIC language and used to simulate user actions on a User Interface.
–        Is a powerful language providing support for pointers, remote procedure calls, working with advanced data types such as linked lists, open-ended hash tables, callback functions, and much more.
–        Is a host of utilities for querying an application to determine how to access it with Visual Test, screen capture/comparison, script executor, and scenario recorder.
Silk Test – Segue
–        Is an automated tool for testing the functionality of enterprise applications in any environment.
–        Designed for ease of use, Silk Test includes a host of productivity-boosting features that let both novice and expert users create functional tests quickly, execute them automatically and analyze results accurately.
–        In addition to validating the full functionality of an application prior to its initial release, users can easily evaluate the impact of new enhancements on existing functionality by simply reusing existing test casts.
QA Wizard – Seapine Software
–        Completely automates the functional regression testing of your applications and Web sites.
–        It’s an intelligent object-based solution that provides data-driven testing support for multiple data sources.
–        Uses scripting language that includes all of the features of a modern structured language, including flow control, subroutines, constants, conditionals, variables, assignment statements, functions, and more.
Evaluation Criteria
Record and Playback         Object Mapping
Web Testing Object              Identity Tool
Environment Support        Extensible Language
Cost                                            Integration
Ease of Use                             Image Testing
Database Tests                     Test/Error Recovery
Data Functions                    Object Tests
Support

3 = Basic  2 = Good  1 = Excellent

Tool Selection Recommendation

Tool evaluation and selection is a project in its own right.
It can take between 2 and 6 weeks. It will need team members, a budget, goals and timescales.
There will also be people issues i.e. “politics”.
Start by looking at your current situation
– Identify your problems
– Explore alternative solutions
– Realistic expectations from tool solutions
– Are you ready for tools?

Make a business case for the tool

–What are your current and future manual testing costs?
–What are initial and future automated testing costs?
–What return will you get on investment and when?

Identify candidate tools

– Identify constraints (economic, environmental, commercial, quality, political)
– Classify tool features into mandatory & desirable
– Evaluate features by asking questions to tool vendors
– Investigate tool experience by asking questions to other tool users Plan and schedule in-house demonstration by vendors
– Make the decision

Choose a test tool that best fits the testing requirements of your organization or company.

An “Automated Testing Handbook” is available from the Software Testing Institute (www.ondaweb.com/sti), which covers all of the major considerations involved in choosing the right test tool for your purposes.

Read More about Automation Tool

AUTOMATION OF BW QUERIES USING WEB BROWSER

In the fiercely increasing competition amongst corporations it has become mandatory to make quick and sound crucial business decisions based on analysis of business critical data. This is the point where SAP BW comes into play. BW Queries are the core of the reporting and analysis functionality in SAP BW. They provide a flexible and intuitive platform for data analysis that can be developed using the SAP Business Explorer (BEx) Query Designer.


AUTOMATION OF BW QUERIES – OVERVIEW


The Automation of queries in SAP BW is require where the data from BW is extracted by reporting analyst using the BW queries and send to business frequently on a regular basis for their analysis for example monthly or weekly.


Queries using web browser can be configured to be run:
Only Once: We can schedule the BW queries to be run on a specific day or time to be refreshed and sent to the email id of the recipients.
Scheduled Basis: We can schedule the BW queries to be run on a regular basis i.e. every day, specific day of a week or monthly basis.


This document outlines a set of guidelines to automate a query for a given scenario.


PROCEDURE TO AUTOMATE THE QUERIES USING WEB BROWSER:


Step 1: Log in SAP BW using the User Name and Password. Enter the RRMX as transaction code into the transaction window and hit the Enter. New Excel window will open with the additional BEx (Business Explorer) tool bar.

Step 2: Open a query which you want to automate through BEx and refresh the same.

Step 3: Launch the query in web browser.

Tool Button –> Launch in Web Browser –> Query (Default View)

When you click on the Query (Default View) it will pop up the message box asking the user name and password.

While you enter the user name and password it will populate the query in web browser.

Step 4: Modify the output structure of the query according to your requirement using the Variable screen at the left side of the window

Applying filters


If you want to apply any filter on the query then you can do that using filter icon against the characteristics. (Ex. If you have to produce the report for particular business unit only then click on the filter icon in front of Master Cost Centers it will pop up another window. Select the Business unit as per your requirement and click on transfer at the bottom left.)

If you want to set the properties of any characteristic then right click on that it will pop up window and select the properties. It will pop up another properties window that will allow you to change the properties of the characteristics.

Step 5: Information Broadcasting: When you are done with the formatting of the query click on the Information Broadcasting and mention the email addresses of the recipients whom the report needs to be sent then click on Create new Setting with the Wizard.

Step 6: Select on the check box as Zip File, it will create the Zip file for the report then click on continue at the bottom.

Step 7: Enter the subject line for the report and enter the E-mail content in the content window and continue at the bottom.

Step 8: Insert the technical name and description for the automated report.

Step 9: Schedule the report: Select the Create New Scheduling and Periodic All and enter the date which you want to schedule the report for and the schedule as weekly or monthly. Mention the date and time on which you want the report to be sent in front of Next Start at and click on the transfer below.

If you want to execute the report at the time of creation then click on the Execute button at the bottom otherwise close that. Now the report is scheduled for the period you mentioned this will automatically refresh the query in the same format you created on the scheduled date and E-mail this report to the recipients with the report attached in Zip format.
MODIFICATION IN SCHEDULE OF AUTOMATED QUERIES:-
If you want to make changes in the schedule then select the query and click on the Schedule button at the bottom it will pop up the window just create the new schedule and delete the old as well and click on transfer.

The BEx Web query automation is a standalone, comfortable Web application for data analysis that the user can call up using an URL or as an iView in the Enterprise Portal. The Open a query which you want to automate through BEx and offers a wide range of functions for data analysis via various tab pages and the associated view-specific toolbars.
SAP query automation is an information modeling tool which can be used to analyze business data. To help analyze data in a more user friendly manner, SAP BW used to describe any kind of numeric information from a business process level. There are various ways to describe key figures in SAP BW.
Essentially, a query is a database research action with interesting additional functionality like currency scenarios, complex calculation options, and analysis functions. However, analyses born out of queries can be applied flexibly to a multitude of areas in a multidimensional dataset of an SAP BW Info Provider. This is enabled by combining analytical functionality with the provided drilldowns and filter options. Therefore, one query or few queries can often map an entire analytical application.

Test Consulting: How to Improve a QA Area

Is your client having difficulty to measure QA performance? Is your client undecided on what test strategy to follow for new application implementation? Is your client looking for testing tool recommendations or need to improve the usage of the existing ones?
Hexaware has a dedicated Test Consulting practice where testing experts are involved in providing test consulting services to clients. During the consulting engagements, Hexaware assesses and evaluates the Approach, People and Technology and fill in the gaps by bringing in our domain experience, best practices, frameworks, tools experience, etc. The consulting services also include tools selection, tools optimization and TCoE Creation and/or optimization.
If you want to learn more about test strategy, the next information will help you to execute a test strategy engagement.
Test Strategy Approach
The first step is to identify the problem(s) that the client is facing and define the strategy objectives. After that, I recommend to follow this approach to execute a test strategy project:
Assessment Areas
As part of the information gathering phase, we leverage its proprietary ATPTM (Approach, People, and Technology) methodology to focus on the right areas and meet the strategy objectives. Hexaware’s APT™ Methodology is the foundation of all of our QA service offerings, below is the description of each component:
• The Approach component is designed to lay the foundation for the processes that each client use as part of testing
• People is the component of the IT organization with focus on testing, Hexaware analyze the groups, roles and responsibilities involved in QA and testing
• The Technology component includes the use of QA and automation testing tools for efficiency to optimize technology and lower costs.

At the end of the project, we provide to the client the following component:
•Current State: An analysis of the current state with regard to the testing objectives
•Gaps: The gaps found between the current state and the best practices and the desired state
•Recommendations: Our recommendations to close the gaps and meet the organization objectives
•Implementation Road Map: A recommended path to follow in order to implement the recommendations.
As a result of the analysis phase, we show to our clients the current state in a quantitative graph. This graph evaluates all relevant aspects of an IT organization and prioritizes each category according to the testing objectives. One example of this graph is showed below.

This was an assessment provided as part of a test strategy we created for a leading bank in Mexico for a T24 product implementation. The benefits showed in this strategy was to reduce testing cycles by 30%, automate at least 50% of the manual test cases and have a defect free implementation using robust and repeatable testing processes.
Other examples of metrics commonly used as part of the strategy objectives are the increment of automation coverage by 30%, increase productivity by 25%, reduce overall testing cost by 15%, etc.
A test consulting practice is an area full of innovation, industry best practices and shared experiences. Now with Hexaware blogs all of us will be able to formally share our experience and our colleagues can leverage them for future assignments.

Read More about QA

Test Consulting: How to Improve a Quality Assurance Area

Is your client having difficulty to measure QA performance? Is your client undecided on what test strategy to follow for new application implementation? Is your client looking for testing tool recommendations or need to improve the usage of the existing ones?

Hexaware has a dedicated Strategic Consulting practice where testing experts are involved in providing test consulting services to clients. During the consulting engagements, Hexaware assesses and evaluates the Approach, People and Technology and fill in the gaps by bringing in our domain experience, best practices, frameworks, tools experience, etc. The consulting services also include tools selection, tools optimization and TCoE Creation and/or optimization.

If you want to learn more about test strategy, the next information will help you to execute a test strategy engagement.

Test Strategy Approach
The first step is to identify the problem(s) that the client is facing and define the strategy objectives. After that, I recommend to follow this approach to execute a test strategy project:

Assessment Areas
As part of the information gathering phase, we leverage its proprietary ATPTM (Approach, People, and Technology) methodology to focus on the right areas and meet the strategy objectives. Hexaware’s APT™ Methodology is the foundation of all of our QA service offerings, below is the description of each component:
• The Approach component is designed to lay the foundation for the processes that each client use as part of testing
• People is the component of the IT organization with focus on testing, Hexaware analyze the groups, roles and responsibilities involved in QA and testing
• The Technology component includes the use of QA and automation testing tools for efficiency to optimize technology and lower costs.


At the end of the project, we provide to the client the following component:
•Current State: An analysis of the current state with regard to the testing objectives
•Gaps: The gaps found between the current state and the best practices and the desired state
•Recommendations: Our recommendations to close the gaps and meet the organization objectives
•Implementation Road Map: A recommended path to follow in order to implement the recommendations.
As a result of the analysis phase, we show to our clients the current state in a quantitative graph. This graph evaluates all relevant aspects of an IT organization and prioritizes each category according to the testing objectives. One example of this graph is showed below.



This was an assessment provided as part of a test strategy we created for a leading bank in Mexico for a T24 product implementation. The benefits showed in this strategy was to reduce testing cycles by 30%, automate at least 50% of the manual test cases and have a defect free implementation using robust and repeatable testing processes.
Other examples of metrics commonly used as part of the strategy objectives are the increment of automation coverage by 30%, increase productivity by 25%, reduce overall testing cost by 15%, etc.

A test consulting practice is an area full of innovation, industry best practices and shared experiences. Now with Hexaware blogs all of us will be able to formally share our experience and our colleagues can leverage them for future assignments.

To know More: Visit Quality Assurance Area 

Manual Testing (Vs) Automation Testing

As the world is moving toward a new era in Technology and business with respect to Testing, we will just have a walkthrough on Manual Testing and Automation testing.

1. Manual Testing is Boring
• No one wants to keep filling the same forms
• There is nothing new to learn when one tests manually
• People tend to neglect running manual tests
• No one maintains a list of the tests required to be run if they are manual tests

Automated tests on the other hand are Code
• They are fun and challenging to write
• One has to carefully think of design for reusability and coverage
• They require analytical and reasoning skills
• They represent contribution that is usable in the future

2. Manual Testing is not reusable
• The effort required is the same each time
• One cannot reuse a Manual Test

Automated Test are completely reusable
• Once written the Automated Tests form a part of the codebase
• They can be reused without any additional effort for the lifetime of the Project

3. Manual Testing has a high risk of missing out on something
• Each time a developer runs manual tests it is likely he will miss out on an important test case
• New developers may have no clue about the battery of tests to be run
Automated Tests have zero risk of missing out a pre-decided test
• Once a Test becomes a part of Continuous Integration – it will run without someone having to remember to run it

4. Manual Tests do not provide a safety-net
• Manual tests are run post-facto and hence only drive bug-patching
Automated Tests provide a safety-net for refactoring / additions
• Even New developers who have never touched the code can be confident about making changes

5. Manual Tests have no training value
• Manual Test are a mechanical process and does not have defined training values
Automated Tests act as documentation
• Reading a set of Unit Tests clarifies the purpose of a codebase
• They provide a clear contract and define the requirement
• They provide visibility into different use cases and expected results
• A new developer can understand a piece of code much more by looking at Unit Tests than by looking at the code
• Unit Tests define the expected behavior of the code

Read More: About Manual & Automation Testing

MoSCoW Principle – Key for success of any project

Problems hindering a successful project:

To make any project successful, requirements and project objectives should be prioritised properly. Though few methods were followed for the prioritization, but still the results were observed to be flawed.

Problem 1: Prioritizing the Project Requirements

An important factor for any project to be successful is to ensure that the requirements are prioritized. Sometimes it is the customers’ fault who wants the entire system to be delivered now or it is the project manager’s fault because they do not discuss the project with the customer.

However prioritising is not an easy process, and on using number system people found it troublesome to prioritize requirements by 1,2,3 etc. However who wants a requirement to be a “2″ or even a “3″? As a result all requirements become a “1″, which is useless. So effective prioritisation is important but how can it be done if number systems are not effective?

Problem 2: Prioritizing the Project Objectives

When a set of requirements has been prioritized then key project objectives of the project such as scope, quality, timescale and resources should be prioritized.

If nearly all requirements are prioritised as “Must”, then there is not much flexibility in the scope of a project. However many studies have shown that it is better if a project is delivered on time, even if it has few features, than if a project is delivered late, but with a full set of features.

If quality is sacrificed then faults will occur in the software. One way around this is to train the users in the use of a new system, so that they only use it in proper fashion, and know how to get around any bugs that are discovered.

Finally all systems must be produced to a budget, and a business does not have unlimited resources to put into a project. Moreover the business case normally assumes a rate of return, which will be considerably reduced if the resources are increased significantly on a project. Therefore resources have a strong case for being the most important factor.

Regardless you cannot “have it all and have it now”, and a balanced and planned prioritisation of the factors must take place if a project is to have a chance of delivering business value. If it is not then the fifth factor of risk goes sky high, and ceases to be risk and become inevitable.

MoSCoW Principle

To overcome the above problems we have a useful method MoSCoW which was first developed by Dai Clegg of Oracle UK Consulting.

This stands for

M – “MUST” be implemented. Requirements falling under this category must be implemented and should not be left off for any reason. If they are not delivered then the project is a failure.

S – “SHOULD” be implemented. This represents requirement should be implemented in the project if it is possible. If it is found to be difficult to implement or not possible, then an alternate solution is also acceptable.

C – “COULD” be implemented. These requirements are considered as nice to have in other ways not a mandatory one. Requirements mapped under this category are desirable ones if time and resources permit.

W – “WON’T” be implemented. This requirement won’t be implemented at this time and likely to be implemented in future.

The two lower case “o” were added to pronounce the word easier.

Conclusion

To deliver business value and be successful a project requires prioritisation of:

• The requirements.
• The main project objectives: scope, quality, timescale and resources.

To Know More About: MOSCOW

When to automate?

• Business scenario that will run with each new version of your application
• Business scenario that uses multiple data values for the same operation
• Business scenario that create data for additional business scenario
• Business scenario that require low end-user decision making
• Complex or lengthy business scenario that are often run during business day.

Read More about  Automate