Above we have presented integration dimensions that are in need of further development. We believe that addressing the dimensions marked with ‘high priority’ in Table 2 constitutes the greatest progress beyond the state-of-the-art in integrated EST assessment. In the project it was assumed that such integration could be strengthened with a defined approach, assisting practitioners in carrying out such integrated assessments. The so-called TranSTEP approach was thus developed. This integration approach involves organising assessment dialogues across institutional and disciplinary domains; transparent, collaborative situation analysis, problem framing and method reflection; and continual process reflection to adapt to the situation under scrutiny (for details see the webpage http://transtepapproach.wordpress.com/). This includes the previously described key elements, but also includes additional elements considered useful for the approach. We will here spell out in more detail the main elements of TranSTEPFootnote 7 (see Fig. 1).
Using the TranSTEP approach involves initiating and facilitating an assessment group composed of people from different advisory domains, as well as problem owners and other stakeholders, to integrate assessment perspectives on complex technology issues. A TranSTEP group will be convened when a problem owner identifies a specific, complex problem that needs resolution or action. This problem must be given a preliminary definition by the problem owner, allowing for establishing a TranSTEP secretariat that will assist to initially select relevant participants to the trans-domain, or TranSTEP, group. Participants in such processes can be assessment practitioners from domains such as economics, risk assessment, ethics, foresight, impact assessment or technology assessment, or from outside these domains. What domains should be involved will vary from problem to problem. In order to ensure that all relevant perspectives are brought in, problem owners and other actors should also be involved, such as representatives from industry and public research, private sector stakeholders, public sector decision-makers or administrators, NGOs or, if appropriate, the public.
The TranSTEP group will have a preliminary mandate or initial problem formulation from the initiator of the process (problem owner). Situation analysis builds on this initial formulation and it is the role of the TranSTEP group to challenge it and/or elaborate on it in close dialogue with the problem owner. Situation analysis is the first phase of any assessment (even if it is sometimes implicit) and ends up in a framing of the problem that is to be tackled by the assessment. In a trans-domain assessment process, it is particularly necessary to explicate assumptions, purposes and values and place the integrated assessment among them. This includes clearly stating the purpose to be achieved by carrying out an integrated assessment and which role the assessment aims to play. The problem will be further defined, relevant actors will be identified, perspectives and interests explored and the social and political contexts described in detail.
Where participants in a group have to explicate their assumptions, learning about themselves and about their own assumptions related to others’ is inevitable. Moreover, coming to agree on a common assessment framing necessarily involves what van de Poel and Doorn (2013, p. 123) calls ‘reflective learning’. This work is challenging and requires specific process management competencies. Bringing together such a wide range of individuals in a meta-assessment process means that it is likely that they will bring a multitude of implicit situation analyses to the table. The first task in the TranSTEP group is therefore to bring out the assumptions about the situation, critically reflect on them and agree on a common situation analysis and problem understanding that allows the group to work together.
Situation analysis should also include a preliminary reflection on what methods would be appropriate for addressing the problem framed within the group. This is necessary firstly for searching for current and available evidence that may help to address the problem, and secondly if the TranSTEP group decides that new assessments or dialogical activities are needed, since then they will also have to design such new actions. In a TranSTEP group there will be no agreed routine method to be used; the different participants may have different views on what methods would be appropriate. This is a benefit, as it allows for a transparent and reflective method discussion in the group, which in the end may yield more robust judgements than method choices based on implicit conventions or institutional traditions.
Though deliberative situation analysis, problem framing and method choice may sound like a very challenging task for a trans-domain group our experiences from organising four ‘testing workshops’ in the EST-Frame project shows that it is indeed possible. Generally, the participants in these workshops found this work hard, but fruitful.Footnote 8
Once there is a common understanding about what characterises the issue to be assessed and there is agreement upon the problem formulation and upon suitable methods to provide knowledge on the problem, the TranSTEP group, with the assistance of the secretariat, can assess whether existing evidence (previous assessments including deliberative initiatives) can provide the foundation the group needs to conclude on the issue.Footnote 9 The review will end up with a judgement on whether there is sufficient evidence for integrating existing knowledge into a conclusion on the problem. If the group believes that it is not, then new assessment activities will, if possible, be initiated by the group.
The group may have resources to undertake such actions themselves (such as organising a citizen’s panel) or the group may encourage relevant problem owners or stakeholders to organise such an assessment (for instance an impact assessment). In any case, the group must engage in detailed reflection on the required methods. As noted above, a wide range of methods can be used in assessments and reflecting on the strengths and weaknesses of this broad range of methods for tackling the agreed assessment problem is therefore necessary. Several former and ongoing European projects, such as DoingForesight, Sustainability A Test and Engage 2020, provide overviews of tools to make up a comprehensive tool box. These can be used to raise awareness of the wide range of method available to provide evidence for the problem solution, so that such choices are made in a reflected and transparent manner and not simply in an intuitive, implicit way resorting to default methods that might not fit the sophisticated situation analysis developed in the TranSTEP group.
Note that potential new assessment activities do not necessarily need to apply inter- or trans-disciplinary methods. What is needed may be (for instance) a traditional risk assessment, ethical assessment or foresight, if this is the knowledge lacking for an integrated conclusion in the TranSTEP group. New assessment activities can, but do not necessarily have to, be undertaken by the TranSTEP group (assisted by the secretariat). However, if outsourced, the TranSTEP group should be involved in or regularly informed about the new assessment activities and outcomes.
By drawing lessons from earlier assessments and initiating new assessments/deliberative events to fill knowledge gaps (including clarifying the extent of uncertainties that will have to be addressed by decision makers), the TranSTEP group will produce integrated conclusions to support the creation of responsible policies for research and innovation. If the group cannot initiate new assessment activities it will integrate the review into a statement of the current knowledge status, with recommendations for further assessment activities to be initiated by other relevant actors.
The TranSTEP group will decide to end the process when a) they believe there is sufficient evidence (on facts, values, perceptions or alternatives) for concluding on the issue they have defined; or b) when practical constraints (such as available funding) make it impossible to continue. At this point a report will be written integrating the results and deliberations of the process. Results integration is a matter of collective judgement in the TranSTEP group. No algorithm can be provided, only argumentation based on the preceding steps. Integration of the results will take the lessons from previous and, potentially, new assessments and apply them to the problem formulation, allowing a judgement to be taken on each aspect of the problem formulation. As such the integrated assessment will be both a meta-assessment, in the sense that it integrates the current assessment knowledge base, and a new transdisciplinary assessment. Depending on the reviewed evidence and the problem formulation the group may end up with a consensus on recommendations regarding specific decisions or policies or mapping of points of consensus and dissent. Even if the group does not end up with an agreement, reporting the situation analysis, assessment design deliberations and the type of dissent will still be of great value to policy makers and other decision makers.
It should be noted that even if Fig. 1 indicates a procedure with a clear direction, there might in reality be a need for revisiting previous stages as the group’s understanding of the issues develops. New insights might reveal the need for adjusting the initial situation analysis and framing of the issues. The procedure depicted in Fig. 1 is not intended to limit such reflective iteration. Being open to adjusting the process to new circumstances or new perspectives is an important condition for the assessment of emerging technologies in situations of complexity and uncertainty.
Moreover, it should be clear that the robustness of the conclusions of the process will depend upon the quality of the deliberations. The conclusions will mirror the process of deliberation and will be a contribution to the knowledge on the issue at hand that reflects the knowledge status at the time of integration and the composition of the TranSTEP group.Footnote 10
Finally, it should be mentioned that transparency is a fundamental condition for the work in an integrated assessment process. Transparency involves being open about all issues of public interest: the situation analysis (including the problem framing), the justification of the method choices, the assessment reviews and the contested versus undisputed points of the dialogue process. Transparency is crucial in assessments that aim to give substantial advice and concrete recommendations but also in assessments that aim to explore issues in a more open fashion (see Stirling 2008) and is of particular importance of integrated assessments where the procedures by definition extends beyond the established, and often documented, conventions in the individuals domains. Revealing thoroughly the limitations and assumptions of the integrated assessment means to reveal fully the assessment as an act carried out in a specific time and place and to allow recipients to take this into account in their own reflections. By revealing the limitations of the assessment, the nature of the subsequent use of the results by others can become transparent in turn. But even if transparency is important for the legitimacy of the integrated assessment, it needs to be balanced with the need for a protected space for open dialogue.Footnote 11