Site Tools


participating:maturity

Component Maturity

4S manages open source software components at any level of maturity, but the requirements and quality assurance processes of a mature component must be strict, while a research prototype component may not need any requirements at all. So in order to indicate for each software component, which processes are mandatory or recommended, a concise method of stating the maturity of a component is necessary.

Technology Readiness Level (TRL)

The concept of a Technology Readiness Level scale was originally developed by NASA, but is now used by many different organisations to identify the maturity of a product, by assigning it a TRL value on a scale between 1 (vision or vague idea) and 9 (mature product). More background on the use and history of this concept can be found in the Wikipedia article on TRL.

Different organisations may use the TRL concept in slightly different ways and for different purposes. Below, this page explains how 4S uses this concept to describe the maturity of software components.

Why do we need TRLs / what is the scale used for?

The first question, one may ask, is why 4S needs this scale, as this is key to understanding the way this scale is adapted and used in a 4S context.

As stated in the introduction of this page, 4S first of all needs a way to distinguish – and clearly label – the maturity level of individual software components, as the quality assurance processes differ, depending on the maturity. So in order to indicate for each software component, which processes are mandatory or recommended, a TRL value will be assigned to all independent components. Furthermore, a component may at any time co-exist in different versions of different TRLs (e.g. an experimental feature branch of an otherwise mature component), therefore the TRL value must be stored along with the source code and documentation of the component (i.e. in the repository).

A secondary use for the TRLs is in the planning and roadmap activities. For instance, if a system depends on a number of components, those components probably need to be at least at the same TRL as the system. So if the system should be raised to a certain TRL, the individual components would typically first be lifted to this level, in order to raise the entire system to the new level. And if this is not done, the TRLs will act as a warning sign of “premature” use of components, which may be used as an input for the project's risk management.

Challenges of the TRL concept

The TRL concept, although widely used, has a few challenges one has to be cautious about:

  • Setbacks. Suppose the development of a product, which is already at a high TRL level (say, 7), runs into serious problems which forces the team back to the drawing board and prototyping (TRL 3-5). An example could be discovering that a technology does not scale well when adding many users.
  • Component vs. systemic view. The lower TRL levels (1-5/6) are focused on a single component or technology, whereas the higher levels (6/7-9) are focused on the entire system or product. A system, however, is usually composed of multiple components/technologies – and those could potentially be at different maturity levels.
  • Rocket science. The TRL concept was originally developed for the needs of NASA and has been adapted for many different purposes. In general, one has to adapt the concept to the problem at hand.

The TRL scale in a 4S context

As mentioned above, 4S needs the TRL as an annotation for each individual software component in order to keep track of the different maturity levels of different components – or even different maturity levels of different concurrent branches of the same component (e.g. adding an experimental feature to an otherwise mature component on a separate branch).

As a system is developed in 4S (as e.g. the OpenTele server), the architecture will organise the logical structure and create different components, each of which will get its own repository and be individually tracked with respect to TRL value. The system itself will of course also have a TRL value, and it is up to the politics of the governance of this system, whether it is allowed to include a software component of a lower TRL value (e.g. under which conditions the OpenTele server at TRL 9 may include an experimental TRL 6 component).

With respect to the challenges listed in the previous section, the 4S use of TRLs does not worry about setbacks. As each version of every component have individual TRL values, it is really not a problem, if the TRL value of a component is decreased. Furthermore, huge TRL reductions at a component level will be rare, as it is more likely that a component is abandoned in favour of a new component rather than re-developing the component from an earlier stage. Finally, the component vs. system challenge is a non-issue, as the TRL of every component is tracked independently of the system(s) the component is used in.

4S TRL scale defined

The 9 “classical” TRLs have been grouped 4 phases in order to simplify the 4S process requirements. Thus 4S may give recommendations to the 9 individual TRLs, but the mandatory requirements apply to the entire phase. The requirements of a phase are developed in the previous phase, so in order to “graduate” from one phase to the next; the requirements of the following phase must be produced. Correspondingly, the description and artefacts of a TRL is developed during the previous TRL, and are ready when ascending to the level.

The four phases are:

  • Idea & concept (TRL 0-3)
  • Prototyping (TRL 3-5)
  • Demonstration (TRL 5-7)
  • Production (TRL 7-9)

Idea & Concept phase (TRL 0-3)

Requirements: None (as there is no previous phase)

Description: In this phase a concept is formulated, elaborated, and defined as a 4S project. There are no rules imposed by 4S in this phase.

Recommendations: For systems and components interacting with users, the methodology of Participatory Design (PD) is strongly recommended.

TRL 0

Description: Not really a “TRL” as such – there is nothing yet.

TRL 1

Description: Basic principles observed. A new component or system has been planned, and the basic properties of this new “thing” have been sketched.

Typical artefacts: Initial concept/use-case description, email discussions, meeting minutes, whiteboard notes/drawings etc.

Recommendations going from TRL 0 to 1: 4S should be involved in these discussions, and the concept should be included in 4S roadmaps and planning activities at the earliest possible stage (in order to avoid parallel work).

TRL 2

Description: Technology/application concept formulated.

Typical artefacts: Use case descriptions, requirements specification, system description etc.

Recommendations going from TRL 1 to 2: For any component or system interacting with users, early user involvement should be considered. This is a good time to start involving users in a Participatory Design process. Also, this is a good time to start describing the project on the 4S wiki and perhaps to start using the 4S issue tracker for further planning.

TRL 3

Description: Analytical/experimental Proof-of-Concept demonstrated.

Typical artefacts: For a component interacting with users, a lo-fi mockup which has been successfully tested with a small group of users. For components of a technical nature, the PoC would typically be a minimal implementation addressing the key challenges without too much “noise”.

Recommendations going from TRL 2 to 3: A lo-fi mockup of a user interface could be paper-based as this Youtube video illustrates, and paper-based mockups have the additional advantage that the users themselves can modify them during a user test session.

Prototyping phase (TRL 3-5)

Requirements: To qualify as a 4S project (single component or system of interacting components) in the prototyping and later phases (i.e. TRL >= 3) the following is mandatory:

  • The project has a clearly formulated description of its purpose and the proposed solution.
  • The project has a validated Proof-of-Concept, which demonstrates the proposed solution. If the project interacts directly with users, this PoC must include a lo-fi mockup, which has been tested on a few representative users. If the project is purely “technical” (i.e. not user related), the PoC could be a validated demo implementation or a related work study.
  • The project has a space on the 4S wiki and this space gives an introduction to the project. The space must be reasonably kept up-to-date (mild requirement).
  • The project has a named co-ordinator and preferably a project group of named individuals who are actively engaged.
  • The project is actively using the 4S tool for issue tracking and the 4S source code and documentation repository.

Description: In this phase the initial prototype is constructed and tested in a realistic/relevant environment involving end-users.

Recommendations: For systems and components interacting with users, the methodology of Participatory Design (PD) is strongly recommended. At this point it is recommended to plan whether a throwaway or evolutionary prototyping strategy is used (Wikipedia on Types of Prototyping). If the strategy is evolutionary, consider adopting the requirements and recommendations from the demonstration phase in this phase as well.

TRL 4

Description: Prototype implemented and validated in “lab”.

Typical artefacts:

Recommendations going from TRL 3 to 4:

TRL 5

Description: Prototype validated in “relevant environment”.

Typical artefacts:

Recommendations going from TRL 4 to 5:

Demonstration phase (TRL 5-7)

Requirements: In addition to the prototyping phase requirements, all 4S projects in the demonstration phase or later (TRL >= 5) will require:

  • The project space on the 4S wiki must be maintained and kept up to date.
  • The project has a validated prototype, which demonstrates the proposed solution in a relevant environment (e.g. a user interface component tested out-of-lab in work-like conditions with end-users).
  • Interface, architecture
  • should have tests

Note: Pointen er at hvis prototypen skal smides væk, skal der være nok materiale tilbage til at beskrive det kommende system (arkitektur interfacebeskrivelser og tests)

Description:

Recommendations:

TRL 6

Description:

Typical artefacts:

Recommendations going from TRL 5 to 6:

TRL 7

Description:

Typical artefacts:

Recommendations going from TRL 6 to 7:

Production phase (TRL 7-9)

Requirements: All 4S requirements apply in this phase

Description:

Recommendations:

TRL 8

Description:

Typical artefacts:

Recommendations going from TRL 7 to 8:

TRL 9

Description:

Typical artefacts:

Recommendations going from TRL 8 to 9:

Learn More

participating/maturity.txt · Last modified: 2018/12/12 13:27 (external edit)