4S manages open source software components at any level of maturity, but the requirements and quality assurance processes of a mature component must be strict, while a research prototype component may not need any requirements at all. So in order to indicate for each software component, which processes are mandatory or recommended, a concise method of stating the maturity of a component is necessary.
The concept of a Technology Readiness Level scale was originally developed by NASA, but is now used by many different organisations to identify the maturity of a product, by assigning it a TRL value on a scale between 1 (vision or vague idea) and 9 (mature product). More background on the use and history of this concept can be found in the Wikipedia article on TRL.
Different organisations may use the TRL concept in slightly different ways and for different purposes. Below, this page explains how 4S uses this concept to describe the maturity of software components.
The first question, one may ask, is why 4S needs this scale, as this is key to understanding the way this scale is adapted and used in a 4S context.
As stated in the introduction of this page, 4S first of all needs a way to distinguish – and clearly label – the maturity level of individual software components, as the quality assurance processes differ, depending on the maturity. So in order to indicate for each software component, which processes are mandatory or recommended, a TRL value will be assigned to all independent components. Furthermore, a component may at any time co-exist in different versions of different TRLs (e.g. an experimental feature branch of an otherwise mature component), therefore the TRL value must be stored along with the source code and documentation of the component (i.e. in the repository).
A secondary use for the TRLs is in the planning and roadmap activities. For instance, if a system depends on a number of components, those components probably need to be at least at the same TRL as the system. So if the system should be raised to a certain TRL, the individual components would typically first be lifted to this level, in order to raise the entire system to the new level. And if this is not done, the TRLs will act as a warning sign of “premature” use of components, which may be used as an input for the project's risk management.
The TRL concept, although widely used, has a few challenges one has to be cautious about:
As mentioned above, 4S needs the TRL as an annotation for each individual software component in order to keep track of the different maturity levels of different components – or even different maturity levels of different concurrent branches of the same component (e.g. adding an experimental feature to an otherwise mature component on a separate branch).
As a system is developed in 4S (as e.g. the OpenTele server), the architecture will organise the logical structure and create different components, each of which will get its own repository and be individually tracked with respect to TRL value. The system itself will of course also have a TRL value, and it is up to the politics of the governance of this system, whether it is allowed to include a software component of a lower TRL value (e.g. under which conditions the OpenTele server at TRL 9 may include an experimental TRL 6 component).
With respect to the challenges listed in the previous section, the 4S use of TRLs does not worry about setbacks. As each version of every component have individual TRL values, it is really not a problem, if the TRL value of a component is decreased. Furthermore, huge TRL reductions at a component level will be rare, as it is more likely that a component is abandoned in favour of a new component rather than re-developing the component from an earlier stage. Finally, the component vs. system challenge is a non-issue, as the TRL of every component is tracked independently of the system(s) the component is used in.
The 9 “classical” TRLs have been grouped 4 phases in order to simplify the 4S process requirements. Thus 4S may give recommendations to the 9 individual TRLs, but the mandatory requirements apply to the entire phase. The requirements of a phase are developed in the previous phase, so in order to “graduate” from one phase to the next; the requirements of the following phase must be produced. Correspondingly, the description and artefacts of a TRL is developed during the previous TRL, and are ready when ascending to the level.
The four phases are:
Requirements: None (as there is no previous phase)
Description: In this phase a concept is formulated, elaborated, and defined as a 4S project. There are no rules imposed by 4S in this phase.
Recommendations: For systems and components interacting with users, the methodology of Participatory Design (PD) is strongly recommended.
Description: Not really a “TRL” as such – there is nothing yet.
Description: Basic principles observed. A new component or system has been planned, and the basic properties of this new “thing” have been sketched.
Typical artefacts: Initial concept/use-case description, email discussions, meeting minutes, whiteboard notes/drawings etc.
Recommendations going from TRL 0 to 1: 4S should be involved in these discussions, and the concept should be included in 4S roadmaps and planning activities at the earliest possible stage (in order to avoid parallel work).
Description: Technology/application concept formulated.
Typical artefacts: Use case descriptions, requirements specification, system description etc.
Recommendations going from TRL 1 to 2: For any component or system interacting with users, early user involvement should be considered. This is a good time to start involving users in a Participatory Design process. Also, this is a good time to start describing the project on the 4S wiki and perhaps to start using the 4S issue tracker for further planning.
Description: Analytical/experimental Proof-of-Concept demonstrated.
Typical artefacts: For a component interacting with users, a lo-fi mockup which has been successfully tested with a small group of users. For components of a technical nature, the PoC would typically be a minimal implementation addressing the key challenges without too much “noise”.
Recommendations going from TRL 2 to 3: A lo-fi mockup of a user interface could be paper-based as this Youtube video illustrates, and paper-based mockups have the additional advantage that the users themselves can modify them during a user test session.
Requirements: To qualify as a 4S project (single component or system of interacting components) in the prototyping and later phases (i.e. TRL >= 3) the following is mandatory:
Description: In this phase the initial prototype is constructed and tested in a realistic/relevant environment involving end-users.
Recommendations: For systems and components interacting with users, the methodology of Participatory Design (PD) is strongly recommended. At this point it is recommended to plan whether a throwaway or evolutionary prototyping strategy is used (Wikipedia on Types of Prototyping). If the strategy is evolutionary, consider adopting the requirements and recommendations from the demonstration phase in this phase as well.
Description: Prototype implemented and validated in “lab”.
Recommendations going from TRL 3 to 4:
Description: Prototype validated in “relevant environment”.
Recommendations going from TRL 4 to 5:
Requirements: In addition to the prototyping phase requirements, all 4S projects in the demonstration phase or later (TRL >= 5) will require:
Note: Pointen er at hvis prototypen skal smides væk, skal der være nok materiale tilbage til at beskrive det kommende system (arkitektur interfacebeskrivelser og tests)
Recommendations going from TRL 5 to 6:
Recommendations going from TRL 6 to 7:
Requirements: All 4S requirements apply in this phase
Recommendations going from TRL 7 to 8:
Recommendations going from TRL 8 to 9: