Tools
- SoapUI Pro and/or open source (http://www.soapui.org/) for development/or RFS
- Jenkins to run SoapUI/RF tests
Test Goal
- Cover all URL endpoints for Health Test and Bad Request
- Cover all HTTP statuses served by the service (positive and negative)
- Cover basic request response requirements.
- Cover incremental URL endpoints along with its variations for rest of test types.
- Cover data variations that results response variations.
- Cover use cases that simulates user behavior from an application.
- Cover performance test using various strategies
Test Types
1. Sanity Test
- Health Test (Checks for service does not return HTTP status 500 and 503)
- Bad Request Test (Checks for service handles negative requests with service defined appropriate HTTP status codes)
- Service Sanity Test (Checks for required responses exists and not null, data types where appropriate, requests node data matches response node data if exists)
2. Functional Test (User scenarios on client side that can be achieved by series of service requests - Helpful for isolation test)
3. Regression Test (All Service Sanity and Functional tests)
4. Smoke Test (Health and bad request test types and used for post deployment)
5. Data Driven Test (Cover service sanity test with data variations)
6. Performance Test
- Simple Strategy
- Fixed Rate Strategy
- Variance Strategy
Target Environment To Run Test & Schedule
All Environments
- Smoke test must run after every deployment
Development
- All tests run on demand
QA
- Regular scheduled test (All types)
Staging
- All test run on demand
Production
- Health, Bad Request and Sanity test types should run against production. This comprehensive check should be scheduled 3 or 4 times a day to run against each production server. Scheduled time must be agreed upon QA, product owners.
Test Data
- Test data running in QA, Staging, and Production environment should be controlled data set. These controlled data set should be agreed by QA and Product owners.
- Test data for Production should also be configurable so that when needed for quick test, data could be passed as a parameter.