Skip to main content
  • The Wellington City Council (WCC) wanted to deliver quality outcomes without breaking the bank. Find out how Planit’s fast and flexible resources helped WCC achieve this goal.

this is a test Who We Are Landing Page

INSIGHTS / Articles

TMMi in the Agile World – PART I: Test Planning, Monitoring and Beyond

 1 Aug 2018 
INSIGHTS / Articles

TMMi in the Agile World – PART I: Test Planning, Monitoring and Beyond

 1 Aug 2018 

In the last two TMMi in the Agile World articles (PART I: Test Policy and Strategy, PART II: Test Planning, Monitoring and Beyond) we looked at how TMMi Level 2 assessment can be gained in agile projects. This third paper looks at how we get to Level 3 competency for the capabilities listed in the title. The remaining capabilities for Level 3 will form a separate article.

As a contributor to TMMI in the Agile World, I have taken some extracts from the whole paper for this article to help you understand how it can be used to assess your organisation.

How to assess your organisation

3 TMMi Level 3 Defined
3.1 Process Area 3.1 Test Organisation

The purpose of the Test Organisation process area is to identify and organise a group of highly skilled people that is responsible for testing.

3.1.1 SG 1 Establish a Test Organisation

Test Organisation is an often misunderstood process area. Many read this as the TMMi requires an independent test group that does independent testing. A tester on a day-to-day basis is typically part of the Agile team, but on top of that is it common for a tester to belong to a test organisation, often known as a test guild.

A test guild is typically an informal, self-managing group of testers, who are member of different Agile teams. Important activities of a test guild can be knowledge sharing, group wise learning, trend watching, etc. Membership of a guild is on a voluntarily basis.

A test guild can also be assigned more formal tasks like establishing test career paths, organise training, and determine, plan and implement test process improvements. An important constraint for a test guild to be successful is that its members are assigned time to spend on the various test guild activities to be performed.

Another option is to have an independent, separate test organisation where testers are assigned to Agile teams at the beginning of the project on a long-term basis, allowing them to maintain their independence while gaining a good understanding of the product and build strong relationships with other team members. In addition, the independent test organisation can have specialised testers outside of the Agile teams to work on long-term and/or iteration-independent activities, such as developing automated test tools, carrying out non-functional testing, and creating and supporting test environments and data.

Note that there are also possibilities to practice independent testing within an Agile team. As long as someone else other than the creator of assets or code also tests the product then this is still independence.

Therefore, developers can test each other’s code via code reviews and/or unit test, or business analysts that contribute to testing from a user’s perspective using for example different personas. Last but not least, independence can be organised through an organisational and responsibility structure, but above all it is about having the right critical mindset.

3.1.2 SG 2 Establish Test Functions for Test Specialists

Testing is regarded as a valued profession and discipline. Test knowledge and skills are needed by the Agile team. The test specialist will provide these, and also coach other team members while performing test activities. Test functions and test career paths are defined and supported by a test training program.

Since an Agile tester is working in a team, soft skills are equally important as increasing technical skills.

3.1.3 SG 3 Establish Test Career Paths

Test career paths are established to allow test professionals to improve their knowledge, skills, status and rewards, and thus allow them to advance their careers. With the Agile emphasis on the people aspect, this is clearly an important specific goal. The Agile team needs testers that understand their challenging job, can coach other team members, and are motivated.

However, the actual definition of test career paths will typically be somewhat different in an Agile organisation. Test career paths in an Agile organisation tend to focus on a horizontal growth. There is typically less need for various hierarchical functions and roles, such as test management, and a higher need to grow along the roles of a Product Owner or Scrum master.

3.1.4 SG 4 Determine, Plan and Implement Test Process Improvements

The improvement process as described by this specific TMMi goal and its specific practices are largely covered by the retrospective meeting performed by an Agile team. The iteration retrospective is an opportunity for the Agile team to inspect itself and define actions for improvements to be enacted during the next iteration. By the end of the retrospective, the Agile team should have identified (test) improvements that it will implement in the next iteration.

The focus of these improvements is often not on cross-project learning and institutionalisation of improvements. Looking at how (test) process improvement is organised and managed, there is likely to be less focus on a (test) process group at an organisational level, and more emphasis on the self-management of teams within the project.

While this is not a bad thing, it is important to also create a mechanism whereby local test improvements that are successful are shared with the rest of the organisation, and those improvement areas that are beyond the power of the Agile team can also be addressed. A possible solution could be to have a representative of the test organisation attend local retrospective meetings to ensure that improvements are shared and institutionalised across the organisation when appropriate.

3.1.5 SG 5 Deploy the Organisational Test Process and Incorporate Lessons Learned

Also for an Agile project, it’s important to have access and re-use the organisational standard test process and test process assets that add value as much as needed. The implementation of the organisation’s standard test process, and the use of test process assets on Agile projects, can be monitored by the self-organising teams themselves, e.g. by addressing this in retrospectives. The lessons learned and test improvement can subsequently be submitted as test process improvement proposals and managed accordingly.

The lessons learned from testing with the Agile team are thus, if appropriate, incorporated into the organisational standard (Agile) (test) process and test process assets.

3.2 Process Area 3.2 Test Training Program

The purpose of the Test Training Program process area is to develop a training program which facilitates the development of the knowledge and skills of people so test tasks and roles can be performed effectively and efficiently. The main objective of the Test Training Program is to provide necessary (test) training to the testers and other team members.

As a result of the Agile manifesto statement of “individuals and interactions over processes and tools”, processes in an Agile environment will typically be documented in lightweight fashion and not address all possible scenarios. Training of people, however, then becomes a critical success factor that needs focus and is almost like a countermeasure for the lightweight processes.

A quality training program ensures that those involved in testing continue to improve their testing knowledge and skills, and acquire the necessary domain knowledge. Since quality and testing now are team responsibilities, it is expected that test related training is not provided to test professionals only, but rather to the whole Agile team.

3.2.1 SG1 Establish an Organisational Test Training Capability

The process starts with identifying the strategic test training needs of the organisation. In addition to the traditional training needs for testers, e.g. on test design techniques and coverage measures, there is now also a need to acquire knowledge and skills on the specifics of Agile testing. Examples include the Agile manifesto and its principles, frameworks, Agile testing concepts (e.g. testing pyramid and testing quadrants), Test Driven Development (TDD), and user-story development, including supporting the definition of acceptance criteria and reviewing them for testability, test automation and exploratory testing.

Also, test management knowledge and skills areas, such as planning, estimation (e.g. planning poker), monitoring and risk analysis, will now become areas that need to be part of the knowledge and skill set for most testers working in an Agile team, and not just for the test manager. The identification of strategic test training needs is not limited to just the testers. It is also of utmost importance to also specify the test related training needs for other members of Agile teams.

Just like in any other organisation, the test training needs will be translated into a (lightweight) training plan. The training plan needs to be reviewed and commitments need to be established. Note that especially in Agile, some skills are effectively and efficiently imparted through informal vehicles (e.g. training-on-the-job, lunch training sessions, coaching and mentoring), whereas other skills require formal training.

3.2.2 SG2 Provide Necessary Test Training

Largely speaking, the specific practices for SG2 are the same in a traditional and Agile organisation. Based on the organisational training plan, team members that need to receive training necessary to be able to perform their test role effectively are identified. The training is subsequently scheduled, including required resources, and conducted.

This seems like an obvious statement, but some Agile organisations struggle to find time for training. Agile teams are always fully engaged and trying to meet the end-of-iteration deadline. Of course, time for training needs to be addressed in iteration planning whereby the attendee will be less available for the next iteration.

Working in pairs is another commonly used vehicle by Agile teams for up-skilling and knowledge transfer. When using Scrum, it’s the Scrum masters responsibility to coach the team in Scrum practices.

Records of the test training conducted, either informal or formal, will be created and the effectiveness of the test training is assessed. The (test) training attendance can also be evaluated as part of a retrospective meeting, whereby a discussion takes place whether the knowledge and skills of the attendees are now more adequate for performing the test tasks.

3.3 Process Area 3.3 Test Life Cycle and Integration

The purpose of Test Lifecycle and Integration is to establish and maintain a usable set of organisational test process assets (e.g. a standard test lifecycle) and work environment standards, and to integrate and synchronise the test lifecycle with the development lifecycle. The integrated lifecycle ensures early involvement of testing in a project.

The purpose of Test Lifecycle and Integration is also to define a coherent test approach across multiple test levels, based on the identified risks and the defined test strategy, and to provide an overall test plan, based on the defined test lifecycle.

3.3.1 SG 1 Establish Organisational Test Process Assets

An important responsibility of the test organisation is to define, document and maintain a standard test process, in line with the organisation’s test policy and goals. It, amongst others, contains a description of the various test types and levels to be executed.

While TMMi does not mandate full blown detailed, defined and documented test processes, often focused templates, elaborated with examples, are a great approach of establishing a common way of working. Refer also to Bob Galen’s three pillars of Agile quality and testing, where he identifies standards being checklists, templates and repositories as one of the pillars of Agile software testing [Galen].

A process does not need to be a strict sequence of activities. As an Agile organisation, if you have effective processes you may just need to document them in a light way manner. While the choice of process assets is up to each organisation, most Agile organisations found that two levels of process assets is sufficient.

This is accomplished by consolidating a policy statement with the associated process description that encapsulates “what must be done”, carrying out the process. The second level contains “how to” guidelines carrying out the process and tailoring it. This level can be viewed as aids for tailoring the process, giving autonomy to the team to decide how exactly to work within the defined framework and usually includes supporting templates.

In most Agile organisations, step-by-step procedures are replaced by tool guides and training/mentoring. A template, such as a test charter, test session sheet or test plan template, can serve as a process with the required process activities implied with the template. The organisation’s set of standard test processes can be tailored by projects to create their specific defined processes.

In the context of SP 1.3 Establish tailoring criteria and guidelines, most organisations today use a scaled tailor down approach. However, if you start at a minimum set, where the minimum set is what everyone must do, you eliminate the risk of tailoring out a “must do”.

A tailoring up approach is much more consistent with Agile approaches, and completely TMMi compliant. To support the deployment of the standard test process, a test process asset library will be created and maintained, often on a wiki, in which testers can find, for example, templates, best practices and checklists, that will assist them in their daily work.

3.3.2 SG 2 Integrate the Test Lifecycle Models with the Development Models

The standard test lifecycle model defines the main activities and deliverables for the various test levels. In Agile, development and testing should be totally integrated within the lifecycle, and testing should already be taking place at early as possible.

Many organisations struggle with this and testing only starts towards the end of the iteration, is often performed under time pressure and sometimes even cut short. Of course, this is not an implementation of an integrated Agile development and testing life cycle. It does, however, show that for some organisations that are “Agile-like” this specific goal is an area for improvement.

This specific goal is thus also applicable to an organisation doing Agile software development. If done appropriately, development and testing should be fully integrated, and there is a common understanding on how this is done.

3.3.3 SG 3 Establish a Master Test Plan

At TMMi level 3, testing is concerned with master test planning, which addresses the coordination of testing tasks, responsibilities and test approach across all test levels. This prevents unnecessary redundancy or omissions of tests between the various test levels, and can significantly increase the efficiency and quality of the overall test process.

The master test plan describes the application of the test strategy for a particular project. A TMMi level 2, (test) planning is already addressed both at release and at iteration level as part of the Test Planning process area.

However, some projects that are working according to an Agile life cycle model have adopted a somewhat hybrid approach, in that in addition to testing within an Agile team, some testing is organised outside the team. In such circumstances, a master test plan will define and manage the relationship from Agile teams to those test levels that are performed outside their scope. e.g., specific non-functional testing (security, performance), hardware/software integration testing, system integration testing or beta testing. This is particularly true where an organisation is using scaled Agile frameworks.

Where all testing takes place with the Agile teams, a specific master test plan has no added value, and thus this TMMi goal can in such cases be rated as Not Applicable. Here planning at release and iteration level is expected to cover all necessary testing aspects.

Reflect and Make Improvements

So, we can see that for the Test Organisation, Testing Training and Testing Lifecyle and Integration TMMi assessments can be run for agile projects and in fact supports the Agile principles of reflection and continuous improvement.

In part two of this article, we’ll have a look at the role of Non-Functional Testing and Peer Reviews in terms of TMMi.

If you want help in optimising your organisation and projects, assessing their maturity against industry standards or get a baseline then roadmap to improve, then contact us. Successful organisations are continuously improving and are customer centric focused.


TMMi in the Agile World

Leanne Howard

Business Agility Practice Director

Elevate Your Testing Maturity

TMMi is an internationally recognised model designed to help organisations optimise their testing. Our expert insights will enable you to achieve better testing outcomes with less effort.
Find out how we can help you build your understanding of the TMMi model, identify your testing maturity and systematically implement improvements and attain TMMi certification.


Find out more

Get updates

Get the latest articles, reports, and job alerts.