I would recommend downloading both products and make sure that your WSDL load up properly first. Most products choke on sophisticated WSDLs so trying that out should be your first step. Next see how easy is it to build simple test cases. Subsequently you should try more complex standards such as WS-Security, SAML,WS-Policy, etc and see which product meet your needs.Also keep all aspects of testing functional, regression, security

DCOM and CORBA traditionally achieved what web services are now offering but with an exception to interoperability which they later provide in true sense.In addition ,COM+,DCOM (Distributed component object model) implementation from Microsoft were resource intensive and were native to specific Microsoft OS flavor.In other words,consumer and provider were mandated to be on the same version of OS.

Testing Methodology

At very basic level, one need to consider how the remote calls take place via SOAP, how the data is passed, how to test the transparent layer,discovery methods,response data,scalability,performance,integrity. Test Case execution shall be top down and bottom up respectively. Documenting test cases for the individual services can be tedious task and subject matter experts can take part in unit testing the individual service so that they are error free in down stream process. This approach facilitates meeting the high level business requirements and can also be used to create data contents for the exchange message. This will avoid the major pitfall of delivering a technically acceptable solution which fails to deliver business value. Once this is done, the data must be formatted to messages (XML) that express service service request and replies from consumer and provider respectively.

Test the Transport Layer

Due to problems introduced by latency and unreliability of the underlying transport, testing the transport layer (HTTP/HTTPS/MQ/JMS) – protocols which form the back bone  in shipping the messages around . This testing can be done in isolation initially wherein the transport is tested  with the messages sent over the transport as data files instead of actual data and consequently integrate with either side once the application is complete.

Test in isolation

Isolation can be achieved by storing message in data files . For instance , a message received  by a consumer can be saved to data file or a file. The emphasizes should be more on test the web service as an independent entity regardless  of its dependency on other web services The messages must be based on specific standard and some of the industries have already started  these activities, namely Association for Corporate Operation Research and Development, regulatory standards such as health insurance Portability and accountability Act (HIPPA) for health care related transections. Interoperability can be verified using tools available such as WSS (web service studio) for a MS.NET Implementation. The new web services studio can be used to automatically generate a compile SOAP proxy classes

Verify Complex Undo Activity

Testing the fragility  of distributed systems if incompatible updates are introduced to any participant may involve validating a two phase commit if transaction has not been completed using the entire web services involved in unit of work.XLANG verifies two phases of commit for distributed transactions wherein if transaction is incomplete, express it explicitly . Additional benefits are not limited to:

• Sequential and parallel control flow constructs
• Long running transactions with compensation.
• custom correlation of  messages.
• Flexible handling of internal and external exceptions.

Functional Integration/Testing Tool Selection

Verify the flow of application where the remote method call to the SOAP server(s), is followed in a specific order as per business rule which additionally involve verifying the nested functionality from WSDL. This may include testing for interoperability as sanity tests which validate the agreed upon semantics between the services.We ‘ve experienced that all good web services test tools share the certain features (1)Scan WSDL (file or URL) to create request and response structure. (2) Contain user interfaces to XML for request and response editing. (2.1) Response is stored as checkpoint.(2.2) Regression test compares check points to run time value.(3) Allow parametrized values for request and correspondence response (data driven) After determining the requirements of functional testing it is necessary  to examine some of readily available testing tools. we are having good expertise of using SOAPSonar for testing web services. There are following SoapSonar Setup Steps

• Download SoapSonar.
• Load x.509 & WSDL.
• Setup SOAP Authentication.
• Call EC2 Soap API

Picking a Project Management Methodology:

A short study from Vertebrae on choosing between an agile or waterfall approach for their latest software development project. We were having an internal meeting to pick a project management methodology for a web project we are working on for a new clients. As developers of commercial software, our instinct was to lean towards an agile based approach.

Metrics that Matter in Agile Projects

Agile methods need only the most important metrics: the ones that tell the whole story about the project. Metrics measure the health of a project and are by far the most objective ways by which a project manager enables all project sponsors and delivery teams to see where resources are needed or spent, or which areas of a project need more focus. So how do Agile teams determine the most important metrics?Like we ‘re collecting different metrics on different stages of our SDLC.

Agile Through the Waterfall

Many organizations have adopted Agile practices into their development methodologies and they have proved to be successful for the organization as a whole. There also are many organizations that have pockets of people who wish to be Agile, but can’t get traction within to make it a widely accepted practice throughout the enterprise. I recently had an opportunity to participate in an Open Space session where we explored how organizations that are mainly guided by Waterfall methodologies, unwittingly also employed Agile practices.

CMMI and Agile: Opposites Attract

The myths surrounding the compatibility of CMMI and Agile have recently been debunked by SEI. Learn how these seemingly opposing strategies can be paired to foster dramatic improvements in business performance! Despite the perception that CMMI best practices and Agile development methods are at odds with each other, new research suggests just the opposite train of thought. In fact, CMMI and Agile champions can benefit from using both methods within organizations, with the potential to dramatically improve business performance.

What Agile Methods Mean to Your Process, People and Products

Studies show that most successful projects were those that followed agile principles, proving that model-driven methods are not always the best when it came to managing changes, fastpaced project implementation, or even meeting market demands. The concept of agile development is not new. However, many technologists still stick to the age-old notion that software development can be easily designed and the outputs predicted without giving much
thought to the more dynamic factors of projects, such as communication lines, people, and change.

Which Life Cycle Is Best for Your Project?

When choosing a development life cycle, don’t just trust your feelings. Decide based on factors that really matter. Which life cycle will work best for your project? This is an important strategic question because making the wrong choice could lead to disastrous results of catastrophic proportions. Think about delayed deliveries, unhappy clients, project overruns, and cancelled projects.

Can We Combine Agile and Waterfall Development Strategies?

While there are likely as many unique Project Management approaches as there are Project Managers, there are two well-know production cycle methodologies that have been the topic of much discussion in PM circles – agile and waterfall methodologies. As I evolve in my own area of expertise, I am constantly reinventing small aspects of what I consider best practice. Most recently, to address the incredibly complex requirements of a large client imitative, I
challenged myself to come up with a “super” Project Management process that would not only improve the way in which we deliver, but what we deliver at the end of the engagement.  I determined there was a way to combine the best features of waterfall development disciplines with agile principles for superior results.

The Blending of Traditional and Agile Project Management

Traditional project management involves very disciplined and deliberate planning and control methods. With this approach, distinct project life cycle phases are easily recognizable. Tasks are completed one after another in an orderly sequence, requiring a significant part of the project to be planned up front. For example, in a construction project, the team needs to determine requirements, design and plan for the entire building, and not just incremental
components, in order to understand the full scope of the effort.

As the project takes shape our testing team aims to explore and learn more about it. At the end of each iteration we look for bugs, missing features and also suggest improvements to our clients. We believe that a piece of software cannot be properly understood by us until we have thoroughly used it.During the course of the entire project our software testing cycle goes through the following steps:

Step 1
This includes a formal and informal inspection of all the documents required for software development and entails the following activities:
• All documents related to the customer’s requirements and business rules for software design and development are handed over to QA.
• QA reviews these documents to locate any discrepancy and raises it in review meetings with the developers, designers and the customer.
• Once the document is finalized QA prepares a Test Plan and shares with the relevant stakeholders.
• QA also writes test cases based on the requirements specified in the documents and then generates test data.
• The QA team Lead prepares traceability matrices and verifies test cases with actual requirements using these matrices.
• The team then prepares acceptance tests and shares with the developers, designers and the customer.
Step 2
At Mindblaze Technologies we emphasize test driven development. Our development team writes unit test cases first and then starts the actual programming. The QA team assures that all development protocols are followed and code is properly commented.
Step 3
If automated unit tests are successfully passed the development team then deploys the source code on the test servers for QA members. The development team clearly explains the testing environment to QA. This usually includes the following information:
• Tools that will be used to benchmark the testing server.
• How to take updates and install the latest builds on the test server.
QA then starts requirements mapping on the actual application by using traceability matrices, test scenario and work flows documents. The test duration is usually decided by the customer, QA Manager and the Project Manager based on scope of work.
Step 4
Depending on the needs of the project the Quality Assurance team may perform all or a certain subset
of the tests given below:

Step 5
A comprehensive Bugs Report is prepared by the Testers and review/verification is performed by the QA team lead before handing over this report to Development Team. In case any clarification is required on the bugs submitted, the QA Team lead discusses the bugs with the assigned developers.The development team responds by providing time estimates for fixing the bugs and apprises the QAteam of any changes that will be made in the software during the course of these fixes.The testing team then retests or verifies the bugs fixed by the development team and also performs a ripple analysis and runs planned regression cycles for assurance. During this entire process test reports
are submitted to the QA Manager and Development Manager. The criterion for ending the regression cycles is defined by the relevant stakeholders including the customer and the QA team lead.
Step 6
Once the software has stabilized and the customer is happy with its performance QA prepares an application stability report and shares it with everyone in the team. At this point the entire team also carries out a lessons learned exercise to understand any problems faced during the project and how to improve the entire process to better serve our customers.

18.Combination of supported development environments (like Java version).

19.OS specific cases.
20.Test with multiple user accounts and login as different users at different times.
21.Concurrent usage scenarios if applicable.
22.Performance boundaries -> Which variable effect performance of the product.
23.How can we verify accuracy or consistency of the system? (Is client and server
compatible, is repository data consistent etc.).
24.Are there explicit date conditions? Current date, future date, invalid dates, range of dates, expiry Dates.
25.What can cause corrupt inputs and how does system respond

Using Amazon EC2 and S3 WSDL interface is easy. We ‘re impressed with the interoperability displayed between a web services client tool, Soap Sonar and Linux based products such as Amazon EC2 and S3. The interoperability is evident across a couple of aspects.

1: Class platform WSDL generation and consumption

2: WS-Security x.509 profile generation by .NET and consumption by the Amazon Platform.
The WSDL based interfaces provided by both EC2 and S3 are powerful for quick integration within existing web services aware management and provision products. We would like to see these APIs aggressively enhanced to include performance related operation calls, listing multiple instances much like the command line interface, and better authentication characteristics. At a minimum, the WSDL based API should be always kept par with command line interface.

Highlights of the QC Role during the Testing Phase:

1.    Executes testing strategy described in the test plan.
2.    Executes end to end testing cycles known as regressions.
3.    Performs acceptance testing.
4.    Performs black box, white box, integration and system testing.
5.    Performs load and stress testing.
6.    Performs usability testing.
7.    Performs concurrency testing.
8.    Performs security testing.
9.    Performs Alpha and Beta Testing.
10.    Performs Ad Hoc testing.
11.    Performs scalability testing.
12.    Performs usability testing.
13.    Performs sanity and smoke testing.
14.    Performs browser compatibility testing for web applications.
15.    Creates and presents test log reports and test incident reports.
16.    Logs defects in bug tracking tools.
17.    Assign bugs to relevant stakeholders.
18.    Performs regression testing.
19.    Maintains bug reports and categorizes them into in various ways. Category examples include:

• logic level issue
• GUI issue
• Database level issue.

20.    Performs ripple analysis when changes are incorporated into the code.
21.    Prepares ripple analysis report.
22.    Calculates DRE (Defect Removal Efficiency).
23.    Prepares bug sheet.

Highlights of QA/QC Role in Deployment Phase:

1.    QA prepares CRF (Change request form) template.
2.    Prepares installation/deployment guide.

Alpha Release

The first semi-functional version of the application is available for testing, mostly done at the unit level by engineers. Additional QA Resources are acquired to help with the testing cycles and the application is deployed in a separate QA environment for testing.

Beta Release

Software is considered beta quality when the features identified in its scope are complete and it has stabilized. At this point, development is exclusively targeted at fixing defects. Fixed bugs should outnumber new and open bugs. Defects continue to be found & reported and fixes are verified or rejected.

GMC (Golden Master Candidate)

This is usually the final build candidate and all severity level one and two defects are retested and verified. Severity level three and four defects are also tested to make sure the extent of the problem is known.

GM (Golden Master)

This is the final build where all severity level one and two defects have been fixed and all severity level three and level four defects are understood. QA signs off on the product and confirms that it meets all performance and functionality requirements.

A Tester’s role on Agile Projects:

First let’s take a look at testers in the traditional software development life cycle. Some examples of what we usually find are:

1. A separate QA group.
2. The tests are derived from detailed requirements and specifications.
3. QA may or may not participate in planning sessions, but is not usually privy to design considerations until after they have been finalized

Now, if we observe a tester in an agile project, we find that the tester:

1. is part of the team and attends all team sessions.
2. is an integral part of the planning game.
3. practices pair testing, i.e. collaborates with the developers to get good tests

Agile Testing Values

The four values that are part of the XP philosophy can be applied to testing as well. We will discuss what this means, and how they are applied to the tester role.

• Communication
• Simplicity
• Feed Back
• Courage

Understanding the tools

What tools does an agile tester need? Many of them are the same that a traditional tester would use, but applied differently. Some of the questions are listed below. I can explain them in comprehensive and precise way based on the demand of the customer:
1.    Can the same skills be applied that were learned in the traditional QA role?
2.    What is the difference between unit tests and acceptance tests?
3.    What is testing Life Cycle.
4.    What are different stages of testing cycle.
5.    How QA pro actively participate in requirements gathering phase?.
6.    How do you write an automated acceptance test?
7.    When would you do manual testing?
8.    Why use a risk-based approach to testing?
9.    What do we mean by Context Driven testing?

The Role of QA in Agile Projects

An interesting question in Agile Development is “how do you incorporate QA?” I guess this depends on what you see as the role of QA. For instance, if you have your QA people involved from start to finish, then you really wouldn’t have to change a thing. For instance, a QA person can:
•    Help to determine if your stories are well defined
•    Suggest adding stories related to testability
•    Make sure that people are creating good unit tests and/or write unit tests
•    Increase your code coverage by adding more automated tests
•    Write tests for stories for test first development.
•    Organize usability testing
•    Add stories for improving usability
•    Do exploratory testing on early builds
•    Verify the completion of stories as they are completed

Highlights of QA Role in Requirements Analysis Phase:
1. Pro actively participate with Business Analyst in requirements gathering.
2. Help the Business Analyst in preparing requirements specification document.
3. Help the Business Analyst in preparing Functional specification document.
4. Plan and organize requirements document and conduct formal and informal reviews.
5. Pinpoint ambiguous statements in the requirements document and discuss them with relevant stakeholders.
6. Help the Business Analyst in documenting Business Rules.
7. Plan and organize the formal and informal reviews of the Business Rules document.
8. Plan and organize the inspections of all requirements documents.
9. Prepare minutes of the meeting of every formal review and assign action items to relevant stakeholders.
10. Plan and organize formal walk through of all the requirements documents.
11. Separate functional and non functional requirements.
12. Plan and organize database architecture walk through on white board and write minutes of meetings.
13. Help the Business Analyst to prepare UML diagrams such as use case diagrams.
14. Discuss assumptions for non functional requirements with all relevant stake holders.
15. Make sure that all preventive measures are taken.
16. Prepare test strategies and test plans when requirements have matured.
17. QA scope identification and scheduling coordinated by the development manager. QA’s timelines are added in the master project schedule.
18. QA team plans internal audits.
19. Plan the number of testing cycles required.
20. Prepare application acceptance criteria.
21. Test Case preparation.
22. Test Scenario identification.
23. Test Data Preparation.
24. Test Bed preparation.
25. Prepare test data scripts.
26. Ensure test first development in Agile projects.
27. Prepare testing life cycle.

QA Role in Requirements Analysis Phase for Agile Projects
i. Help to communicate if user stories are not well defined.
ii. Suggest adding stories related to testability.
iii. make sure that people are creating good unit tests and/or write unit tests
iv. write tests for stories for test first development.
v. add stories for improving usability.
vi. verify the completion of stories as they are completed.
vii. Actively Participate in life cycle of AMDD(Agile Model Driven Development).
viii. Give suggestions to Business Analyst for improvements in Requirement Engineering protocols and practices.

QA Envisioning in the Analysis Phase:
1. What will be built?
2. When it should be built?
3. In what order should we build it?

Highlights of the QA Role in the Design Phase:
1. Plans and organizes design review meetings
i. Primary design review
ii. Critical design review.
2. Makes sure that process and protocols are followed.
3. For web applications, ensures that the design is w3c compliant and works fine in all web browsers.
4. Discuss application flows with Design Architect and Business Analyst and suggest them user friendly flows.
5. If issues are found in design review meetings, QA logs those defects in a bug tracking tool.
6. Assigns action items to relevant stakeholders and follows up with them.
7. Identifies possible concurrency issues for web applications.
8. Identifies possible deadlocks for web applications.
9. Updates design defects statuses and minutes of meetings.
10. Adds or updates test scenarios.
11. Assists design architects in design web applications with respect to web 2.0 standards.
12. Helps development teams to write/update unit test cases.
13. Plans and organizes unit test review meetings by coordinating with the development team lead.
14. Ensures company processes and protocols are being followed in every level of the Software Development Life Cycle.
15. Documents all the work being done by the developers in the form of data flow diagrams.
16. Helps in preparing the Design Specification document.
17. Makes sure that all preventive measures are taken.
18. Plans and organizes inspection of design document.
19. Prepares minutes of the meeting and logs them into the organization’s bug tracking tool.
20. Assigns action items to relevant stakeholders and follow up with them.
21. Make sure that all corrective actions are taken on design level issues before hand over to development team.

Highlights of QA Role in Coding Phase:
1. Prepares checklists for developers that contain standards and guidelines for them.
2. Ensures test driven development protocols are being followed and suggests unit tests to write.
3. Prepares documents containing coding practices and standards for the development team lead.
4. Ensures that defined coding standards, procedures and protocols are followed.
5. Ensures that code is properly commented.
6. Prepares conventions document for variable declaration, file names, file type, database table names and database field names.

1. Acceptance testing:
This type of testing is normally performed to validate the software and ensure that it meets a set of agreed acceptance criteria.

2. Accessibility testing:
This testing is conducted to verify that the software in question is accessible to the intended target audience. This kind of testing is especially useful when the target audience includes people that have disabilities.

3. Ad Hoc Testing:
This is a testing phase where the tester tries to break the system by randomly trying the system’s functionality. It can include negative testing as well.

4. Agile Testing:
We apply agile testing practices for projects using agile methodologies; we treat development as a customer of testing and emphasize a test first design paradigm.

5. Basis Path Testing:
This is a white box test case design technique that uses algorithmic flow of the program to design tests.

6. Beta Testing:
This is a test of the software that is conducted by its actual customers. At the end of this testing their feedback is logged and relevant actions are taken by all the stakeholders involved.

7. Binary Portability Testing:
Our QA department performs this testing on executable applications for portability across system platforms and environments, usually for conformation to an application binary interface specification.

8. Black Box Testing:
Testing based on analysis of the specification of a piece of software without reference to its internal workings. The goal is to test how well the component conforms to the published requirements for the component.

9. Bottom Up Testing:
This is an approach for conducting integration testing where the lowest level components are tested first, then used to facilitate the testing of higher level components. The process is repeated until the component at the top of the hierarchy is tested.

10. Boundary testing:
These are tests that focus on the boundary or limit conditions of the software being tested, some of these tests can also be considered as stress tests.

11. Branch testing:
This is a form of testing in which all branches in the program source code are tested at least once.

12. Breadth Testing:
This consists of a test suite that spans the entire functionality of the software but does not test each feature in detail.

13. Compatibility Testing:
This is testing to ascertain a software’s ability to run with other elements of a system with which it should operate e.g. Browsers, operating systems or hardware.

14. Component Testing:
Testing of individual software components (Unit testing).

15. Concurrency Testing:
This is multiuser testing geared towards determining the effects of accessing the same application code, module or database records. It identifies and measures the level of locking, deadlocking, and use of single threaded code and locking semaphores.

16. Conformance Testing:
The process of testing that an implementation conforms to the specification on which it is based. Usually applied to the tested conformance to formal standards.

17. Context Driven Testing:
The context-driven school of software testing is a flavor of agile testing that advocates continuous and creative evaluation of testing opportunities in light of the potential information revealed, and the value of that information to the organization right now.

18. Conversion Testing:
Testing of programs or procedures used to convert data from the existing system for use in replacement systems.

19. Data Driven Testing:
Testing in which the action of test case is parameterized by externally defined data values, maintained as a file or spreadsheet. It’s a common technique in automated testing.

20. Dependency Testing:
Examine an application’s requirements for pre-existing software, initial states and configuration in order to maintain proper functionality.

21. Depth Testing:
Testing that exercises a feature of a software in full detail.

22. Dynamic Testing:
Testing software by actually using it.

23. End to End Testing:
Testing a complete application environment in a situation that mimics real word use, such as interacting with a database, using network communications, or interaction with other hardware, applications or systems if appropriate.

24. Exhaustive Testing:
Testing which covers all combination of input values and preconditions for an element of the software under test.

25. Gorilla Testing:
Testing a particular module’s functionality intensively.

26. Gray Box Testing:
A combination of black box and white box testing methodologies; testing a piece of software against its specification but using some knowledge of its internal working.

27. Integration Testing:
Testing of combined parts of an application to determine if they function together correctly. Usually performed after unit and functional testing. This type of testing is especially relevant to client/server and distributed systems.

28. Installation Testing:
Testing that confirms that the application under test recovers from expected or unexpected without loss of data and functionality. Events can include shortage of disk space, unexpected loss of communication, or power out conditions.

29. Localization Testing:
This term refers to making software especially designed for a specific locality.

30. Loop Testing:
A White box testing technique that exercises program loops.

31. Monkey Testing:
Testing a system or an application on the fly, i.e. just a few tests here and there to ensure that a system or an application does not crash out.

32. Negative Testing:
Testing aimed at showing that the software does not work, also known as ‘test to fail’.

33. Performance Testing:
Testing conducted to evaluate the compliance of a system or component with specified performance requirements. Often this is performed using automation testing tools to simulate large number of users and checking the result.

34. Ramp Testing:
Continuously raising input signals until the system breaks down.

35. Recovery Testing:
This confirms that the program recovers from expected or unexpected events without loss of data or functionality. Such events can include shortage of disk space, unexpected loss of communication etc.

36. Sanity Testing:
Brief testing of the major functional elements of a piece of software to determine if it’s operational.

37. Scalability Testing:
Performance testing focused on ensuring that the application under test gracefully handles increase in the work load.

38. Security Testing:
Testing which confirms that the program can restrict access to authorized personnel and that the authorized personnel can access the functions available to their security level.

39. Smoke Testing:
A quick and dirty test to check if the major functions of a piece of software work. Originated in hardware testing practice when turning on a new piece of hardware for the first time and considering it a success if it does not catch fire.

40. Soak Testing:
Running a system at high load for a prolonged time.

41. Static Testing:
Analysis of program carried out without executing the program.

42. Unit Testing:
Test of individual software components.

43. Validation Testing:
The process of evaluating the software at the end of the software development process to ensure compliance with the software requirements. The techniques of validation are testing, inspection and reviewing.

44. Verification Testing:
Testing done by walk through, inspections, meetings, requirement checklists, and traceability that whether it is going to meet the requirements or not.

45. Work Flow Testing:
Scripted end to end testing which duplicates specific work flows which are expected to be utilized by the end user.