About ten years ago I was working on a testing programme back in the UK and they were undergoing a CMMi certification, during this certificate I found out that there was a specialist certificate for testing, this I came to know as TMM. Skip a few years and this is now the TMMi and so I did some investigation into this and uncovered arguably one of the best certification and measurement models that I have come across that defines the maturity of testing in an organisation.
This is a summary of who and what the TMMi Foundation and TMMi is.
What is the TMMi Foundation?
The TMMi Foundation has developed and owns the TMMi model. It is a non-profit organisation which focuses its activities on maintaining the only independent test maturity assessment model and accreditation for individuals and organisations to deliver professional TMMI assessment – see TMMi.org.
What is TMMi?
The TMM (Testing Maturity Model) structure is based on the Capability Maturity Model (CMM). The concept was originally postured by the Illinois Institute of Technology. In 2004 a group of individual practitioners got together and generated the TMMi model. This group later became the original TMMi Foundation members.
TMMi is a test process improvement and accreditation model which can be used to complete formal and informal assessments, both of which use the same maturity requirements (as per the levels below) however the informal assessment does not have a certification award associated with it.
The assessment consist of five levels, it is these levels that show the testing maturity of an organisation. Each maturity level is split in to Process Areas for an Assessor to review. Within each process areas there are Specific Goals (SG) and Generic Goals (GG) that must be achieved for an organisation.
The test maturity levels start at Level 1 – Initial and go through to Level 5 – Optimisation. In pictorial form these are the levels and the process areas that are attributed to each level.
The TMMi model can been applied across many testing domains world-wide. The main differences in TMMi and other test improvement models is its independence and its adherence with the most common international testing standards, this has made it the standard for test improvement and assessment.
Level 1 - Initial Definition
At TMMi Level 1, testing has no defined processes and is mainly completed by developers. Tests are developed in an ad-hoc way after coding is completed. The objective of testing at this level is to show that the software has no major failures however a side effect of this is that a software application may not meet the needs of the customer or is unstable.
Level 2 - Managed Definition
At TMMi Level 2, testing is more of a managed process consisting of a test team or test resource to complete formal testing. One of the main objectives of maturity Level 2 is that there is the ability for repeatable testing. Test plans are also developed and as part of this the approach, these test plans should include details of when testing will take place, how testing will be completed and the testing resources. Included at this level is also a certain amount of reporting for management which is to ensure that testing is going to plan and is tracked. There are still defects and project related issues at this level as testing is still later in the development lifecycle.
The process areas at TMMi Level 2 are:
- 2.1 Test Policy and Strategy
- 2.2 Test Planning
- 2.3 Test Monitoring and Control
- 2.4 Test Design and Execution
- 2.5 Test Environment
Level 3 – Defined Definition
At TMMi Level 3, testing is part of the SDLC and has associated milestones. Test planning is still done in this stage of the SDLC although is completed earlier. The development of a master test plan builds on the test planning skills and commitments acquired at TMMi Level 2. The organisation now has a set of standard test processes and a specific test training program exist. Organisations at level 3 now understand the importance of reviews in quality control and a formal review process has been implemented. At this level the organisation has its own set of standard processes and is able to modify these to each project without impacting the testing processes and procedures of the organisation.
The process areas at TMMi Level 3 are:
- 3.1 Test Organisation
- 3.2 Test Training Program
- 3.3 Test Lifecycle and Integration
- 3.4 Non-functional Testing
- 3.5 Peer Reviews
Level 4 – Measured Definition
TMMi Level 4; testing at this level builds upon that which has been completed in level 2 and 3. The view of an organisation is that testing is now part of life in a programme or project and is self-sustaining and evolving where possible. It can also be measured through its processes and accomplishments. Measurements are also stored in an organisation’s testing repository to support the decision making that is required It will also be possible to support some predictions relating to test performance and the cost of testing in programme and projects. Reviews of artefacts are part of the test process and it will be possible to measure quality earlier in the lifecycle.
The process areas at TMMi Level 4 are:
- 4.1 Test Measurement
- 4.2 Product Quality Evaluation
- 4.3 Advanced Peer Reviews
Level 5 – Optimisation
TMMi Level 5, the organisation has achieved all the previous levels of maturity and it will be possible for the organisation to be capable of self-improvement based on its processes and procedures. The testing methods and testing techniques are optimised and there is a continuous focus on fine tuning and process improvement.
The process areas at TMMi Level 5 are:
- 5.1 Defect Prevention
- 5.2 Quality Control
- 5.3 Test Process Optimisation
As per TMMi, an optimised test process is one that is:
- managed, defined, measured, efficient and effective
- statistically controlled and predictable
- focused on defect prevention
- supported by automation as much is deemed an effective use of resources
- able to support technology transfer from the industry to the organisation
- able to support re-use of test assets
- focused on process change to achieve continuous improvement
In summary, an accredited TMMi assessment is one of the most comprehensive assessments for test process improvement covering all aspects of an organisations testing capability. For all organisations, even if they went through an informal assessment, this would give a detail examination of the testing capability and it would be possible to identify the areas for improvement far more easily than a TPI assessment can.
If you are in an organisation and can see the testing team struggling and can’t put your finger on the exact issue then it may be a good idea to get an assessment done, it would be far more cost effective in the long run.