Introduction

TransformationsFootnote 1 as fundamental changes of societal systems are increasingly advocated for to address great societal challenges and moving towards a sustainable future (WBGU 2011; United Nations General Assembly 2015; Köhler et al. 2019). Societal challenges are related to persistent and complex problems, such as climate change and biodiversity loss. New forms of science-society collaboration play an important role in research on and for sustainability transformations. Here, science aims to more actively address pressing societal issues (Fazey et al. 2018) and to take on its societal responsibility (Cornell et al. 2013).

Making use of the transdisciplinary research mode scholars aims to contribute to understanding and solving complex real-world problems (Jahn et al. 2012; Klein 2004; Lang et al. 2012). Therefore, transdisciplinary processes strive for both scientific and societal contributions. They allow to look at problems from many angles by integrating knowledge from multiple disciplines and societal practice. Current research confirms that the participation of actors outside academia has the potential to augment the societal impact of research (Newig et al. 2019). Thereby, co-designs of the research aims, and method-driven, integrative collaboration are key for scientifically and societally impactful research.

The complex nature of problems of unsustainability makes linear and technocratic solution approaches often insufficient. This has led to an experimental turn in sustainability-related social science (Overdevest et al. 2010). Adaptive and reflexive approaches emerged (e.g. Loorbach 2007; Voss and Bournemann 2011; Olsson and Folke 2004). These center around processes of experimentation, evaluation, learning and innovation to facilitate societal transformation (Loorbach et al. 2017).

New research settings got developed combining characteristics and promise of transdisciplinary and experimental research. They include a broad array of society-based laboratory approaches, such as urban and sustainable living labs (Liedtke et al. 2015; Voytenko et al. 2016), transformation (Charli-Joseph et al. 2018; Pereira et al. 2020), urban transition (Nevens et al. 2013) and real-world labs (Schäpke et al. 2018). Related settingsFootnote 2 share a methodological focus on real-world experiments to understand sustainability problems and develop solution options in science–society collaboration (Caniglia et al. 2017). While many such labs are located in the global north, recently, they emerged as well in global south (e.g. Banson et al. 2018; Mukute et al. 2018; Pereira et al. 2020) (McCrory et al. 2020 for an overview).

Within Germany, real-world laboratories (RwLs) became increasingly promoted as part of a call for science engaging (more) in transformations (WBGU 2011, 2016; Schneidewind and Singer-Brodowski 2013). In 2014, the German federal state of Baden-Württemberg launched two funding lines financing 14 RwL projects (in the following referred to as BW-Labs), as part of a broader “Science for Sustainability” initiative.Footnote 3 Therein, RwLs aim to contribute to understanding and facilitating transformations to sustainability, addressing a broad range of topics (Table 1, for details see Supplement 1).

Table 1 Topical areas addressed by BW-Labs
Full size table

Announced funding criteria asked BW-Labs to focus on societal challenges, to address local sustainability issues within a global scope, use an inter- and transdisciplinary research approach building on collaboration with societal actors and continuous methodological reflection, integrate transdisciplinary teaching elements, have a longer-term orientation and aim for societal impact and knowledge transfer (Baden-Württemberg 2013). Additionally, the second funding line had an explicit focus on urban sustainability and digitalization (Baden-Württemberg 2015). To facilitate methodological reflection and create overarching insights two accompanying research projects got funded as well.Footnote 4 This article results from the collaboration of one of the accompanying projects with the BW-Labs. The current discourse on RwLs engages with many of the crucial issues brought up in debates around sustainability and lab approaches, including transdisciplinarity, experimentation, transformation (Schäpke et al. 2018; McCrory et al. 2020). Thus, respective insights can be of broader relevance. To date, different conceptualizations of RwLs have been developed and some steps to operationalize these understandings, for instance, regarding suitable methods, have been taken (e.g. Defila and Di Giulio 2018a, b, 2019; BMWi 2019). Yet, studies on RwLs combining conceptual and empirical work and building on more than single cases are still in their infancy (early examples: Gerhard and Marquardt 2017; Parodi et al. 2018b). Critical voices call for an empirics-based foundation of this far primary postulated virtues of RwLs, for instance, related factors of success and methods for RwL realization (Jaeger-Erben et al. 2018; Parodi 2019). Furthermore, the danger of an inflationary, unspecific use of the RwL-label is highlighted. Here, the 14 BW-Labs offer a promising learning opportunity.

This article aims at contributing to a comprehensive understanding of how to conceptualize and operationalize the RwL approach. It mainly addresses three research questions (RQs):

  1. 1.

    How can the key conceptual characteristics of an RwL be described?

  2. 2.

    What are relevant success factors for the scholars of the BW-Labs to achieve the desired societal effects?

  3. 3.

    What methods are used in the 14 BW-Labs for the cooperation between science and practice?

To answer these questions, the article builds on insights gained from and in collaboration with the BW-Labs. The accompanying project was concerned with supporting and observing the 14 BW-Labs as a (relatively) novel research format. This included enabling exchange with the (inter-)national research community and providing expertise based on own experiences with transdisciplinary projects. It comprised literature reviews and empirical studies.

Importantly, weFootnote 5 have not carried out an evaluation or assessment, which would have been counterproductive in a project in need of building on mutual trust between accompanying and BW-Lab projects (Defila and Di Giulio 2018b; Weith et al. 2019). Rather, the aim was a mutual learning process about specific requirements, challenges and opportunities when working in an RwL setting. Herein, the accompanying research project collected and compiled data from the BW-Labs (compare Defila and Di Giulio 2018b) and investigated it to provide overview on RwL practices in a diverse sample of labs (see Table 1 and Supplement 1).

Accordingly, we do not draft an ideal type of an RwL building on a comparative evaluation of the BW-Labs. We aim to offer guidance and support for those who want to plan and constitute a new RwL project, as well as to those who seek to reflect a current RwL-practice. Therefore, we provide a decontextualized description of elements, i.e. characteristics, success factorsFootnote 6 and methods. These can be adapted to the upcoming, individual project tasks and context. Or they can be used to put existing labs practices into a broader perspective. Decontextualized element descriptions are combined with examples of BW-Lab application for rendering success factors and methods (more) tangible.

The text is structured as follows: In the chapter “Research methods”, we present the methods used in the accompanying study. Then, we move in four steps from conceptual considerations on RwLs to practical recommendations for RwL work: in the chapter “Real-world labs - key characteristics”, we present key characteristics of RwLs, based on the literature (addressing RQ1); in the chapter “Success factors”, we display RwL success factors described by BW-Labs, integrate key characteristics and success factors and derive related design principles (addressing RQ2); in the chapter “Methods for collaboration”, we formulate challenges for RwL practice and outline method families and individual methods to meet these challenges (addressing RQ3). We conclude by reflecting the state of the art and proposing research needs to further enhance the practice of RwLs in the chapter “Concluding discussion”.

Research methods

The overall approach of the funding lines of BW-Labs formed the broader context of this study and was based on principles of co-creation and mutual learning (Parodi et al. 2018a; Zimpelmann 2018; Wagner and Grunwald 2019). Success, for instance, was co-defined by the funder, the scientific advisory body to the funder and the RwLs. We, as accompanying researchers, had a consultancy function in this process. Additionally, we aimed at understanding what RwLs are, how they can be structured methodically and conceptually, and how success factors can be described.

For this article, we used a mixed methods approach comprising three interlinked components:

  1. (1)

    The presented key characteristics of RwLs are based on a previous publication (Schäpke et al. 2018). In the latter, characteristics were derived from a comprehensive literature review, considering 58 peer-reviewed and non-peer-reviewed articles and book chapters addressing the topic of “Reallabore” and Real-world labs, respectively. This includes theoretical-conceptual and empirical/case-based works. Literature got identified by searching Scopus and Google Scholar (we used “real-world lab*” OR “reallabor*” AND “sustainab*” as a search string), then excluding literature with only peripheral reference to real-world labs (for details see supplementary material to Schäpke et al. 2018). Discussions with researchers from BW-Labs and sustainability scholars during workshops and conferences helped to further ground criteria empirically (Wagner et al. 2016; Schäpke et al. 2017). Key characteristics, drawing on prior work, are used as they are based on the only existing comprehensive RwL-review (to our knowledge). This review as well considered the small number of studies proposing similar sets of RwL-characteristics (Parodi et al. 2016; Wanner et al. 2018). Characteristics have proven to be resourceful to analyze RwLs and to compare RwLs to similar labs in real-world settings (e.g. Urban Transition, Transformation and Sustainability Living labs, Schäpke et al. 2018).

  2. (2)

    We carried out an empirical study on the 14 BW-Labs to identify success factors of RwLs and methods used for collaboration between science and practice actors. We used a semi-structured survey amongst BW-Labs’ researchers consisting of open as well as closed questions (questionnaires in Supplement 3). This allowed questioning all 14 RwLs systematically and with enough time for them to reflect their answers. The inductivity of the approach corresponds to the fact that the RwL is still a relatively new research approach by leaving space for open answers. Long-term experience with transdisciplinary research (Jahn et al. 2012; Lang et al. 2012; Bergmann et al. 2012) and detailed experience with evaluative research and survey development oriented the set-up of the survey questionnaires. This also applies to the questions on the categories of success in the questionnaires, although these were supplemented by success criteria resulting from the observation of the work of the RwLs. The survey addressed questions concerning the classification of the RwL, success factors for working in RwLs, and methods for cooperating with practitioners as well as other scientists in RwLs. All the 14 BW-Labs responded and the answers were evaluated using a summarizing qualitative content analysis. Findings from the surveys only allow to draw conclusions about the processes during the performance of the RwLs, not on ex-post results such as sustainability effects.

  3. (3)

    Insights from further exchange with the 14 BW-Labs during the accompanying research complemented the literature review and the survey results. This included, for instance, workshops, joint publications, dedicated conference sessions and informal conversations. These processes provided findings on key characteristics, success factors and challenges of RwLs in a more informal and unstructured but open and perceptive way.

The present approach has a few limitations. First, the geographical focus on Western Europe/Germany limits the applicability of insights to other contexts, e.g. other regions globally. Second, while all 14 cases considered are self-proclaimed RwLs, comparability has limitations due to strong variations in, for instance, topical focus (see Table 1 and Supplement 1). Lastly, the non-evaluative accompanying research approach prohibits to defining ideal types of effective real-world lab configurations or of assessing success and failure of RwLs beyond self-evaluation. In effect, we do not offer a systematic differentiation (e.g. outlining which method to use in which context), but we provide a comprehensive overview of RwL success-factors and methods and examples of their contextual application.

Real-world labs—key characteristics of a transdisciplinary and transformative research setting

In a prior study, we had distilled five main characteristics of RwLs from the literature (Schäpke et al. 2018). Building on these characteristics, RwLs can be understood as a research approach that (1) aims to contribute to societal transformation, (2) uses experiments as core research method and (3) transdisciplinarity as core research mode, (4) has a long-term orientation and seeks scalability, and transferability of the results while (5) building on learning and reflexivity. Below we describe the characteristics, ways for their implementation and related critical aspects (reproduced in strongly modified form from Schäpke et al. 2018).

Contribution to sustainability transformation

Two interrelated streams of research can be differentiated regarding how they seek to advance sustainability transformation, namely transformation and transformative research (WBGU 2011). Transformation research analyzes dynamics and processes of change in primarily descriptive-analytical ways (Schneidewind and Singer-Brodowski 2013). Transformative research seeks to address problems of unsustainability challenges via inventing and assessing possible solutions and by creating related actionable knowledge, including “strategies that can solve (or mitigate) certain problems” (Caniglia et al. 2017, p. 42). RwLs seek to combine transformation and transformative research. Thus, they follow a twofold aim: “to contribute to societal transformation by experimenting with potential solutions” and “to produce scientific evidence about the social robustness of solutions, as well as about their scalability and transferability” (Schäpke et al. 2018, p. 86f.; see as well Luederitz et al. 2017). A critical task for the scientists is not to get stuck in the work for societal change, but also to derive insights on generic strategies to facilitate successful transformations.

Experiments as core research method

“Experiments are scientific practices that rely on an intervention and aim at producing empirical evidence” on causes and effects (Schäpke et al. 2018, p. 87 building on Caniglia et al. 2017). The forms of experiments that are used by sustainability scientists can be differentiated regarding the control over and the aim of experimentation. The control of researchers on the intervention and their context can differ between full, external control, participatory control and no control (Caniglia et al. 2017). The aims relate to what the “experiments seek to generate evidence about” (Schäpke et al. 2018, p. 87). This is either evidence about sustainability problems, which takes shape in descriptive-analytical knowledge, or about sustainability solutions in form of action-oriented knowledge.

RwLs can be understood as providing a specific setting for experiments in terms of time, place, interrelations and resources. Here, different types of experiments can be carried out, including “experiments designed to generate evidence related to action fostering sustainability transformations” (Schäpke et al. 2018, p. 87). A critical task relates to the context-specific nature of experiments with only limited, participatory control of many factors, which challenges the generation of comparative and transferable insights (Gross et al. 2005 chapter 1).

Transdisciplinarity as core research mode

In transdisciplinary research, scholars from various disciplines work together with societal actors to understand and develop solutions for real-world problems (Lang et al. 2012). A core function of transdisciplinarity is to identify, differentiate and integrate relevant knowledge held by various scientific fields and societal actors to produce knowledge that can guide action. Scholars differentiate three phases of collaboration in an “ideal-type transdisciplinary processes […]: co-design, co-production, and re-integration” (Schäpke et al. 2018, p. 87 building on Bergmann et al. 2012). Co-design includes developing a shared understanding of the problem at hand, aims and questions of a research project. Co-production refers to the generation of new knowledge and re-integration refers to the diffusion and (potential) uptake of generated knowledge in societal and research spheres. How intensive societal actors are involved in this process has been differentiated, reaching “from information transfer through consultation, cooperation, collaboration, to empowerment” (Schäpke et al. 2018, p. 87). This intensity may be varied according to a different aims and situation in the different phases of a project (Stauffacher et al. 2012).

“RwLs realize transdisciplinary research to differentiate and integrate scientific and societal knowledge, related to a real-world problem” (Schäpke et al. 2018, p. 87). Therein, a meaningful engagement of societal actors is argued for. The concrete intensity of collaboration will differ according to project circumstances. Challenges are frequently emerging in RwLs when jointly doing real-world experiments. These include issues of ownership on aims and results of the experiment, transparency about potential risks and benefits, and the management of conflicts arising. To differentiate: while experiments are a (core) research method of RwLs, transdisciplinarity is not a method, but a mode or principle of research. This mode of research is supported by various methods, which are especially helpful in tasks of integration (cognitive, social, communicative). The execution of experiments is one such method (Bergmann et al. 2012: 105 ff.).

Long-term orientation, scalability and transferability of results

Research concerning transformations should allow for a long-term and large-scale perspective. Insights and developed solutions should consider the potential of transfer or scaling (Luederitz et al. 2017). This requires to generalize contextualized insights and to understand relevant contextual factors. “Scaling concerns increasing the reach of solutions in the original context or beyond and depends on insights on scalable features of solutions” (Schäpke et al. 2018, p. 87). Thus, scaling may include to expand the geographical reach of a solution from the household, to the neighborhood and city level, and beyond. Transfer means to expand the scope of solutions, for instance, from sustainable mobility to urban development.

To contribute to sustainability transformations is a key aim of RwLs. While a larger-scale impact does not necessitate a long-term existence of a respective RwL, the uptake of developed innovations and solutions is essential. Accordingly, Rws should develop solutions with a long-term horizon, which potentially go beyond the lab existence. The situatedness and contextual embeddedness of RwLs do pose challenges towards transferability and scalability, which depend on some degree of generalization of insights.

Learning and reflexivity

Learning is considered relevant for sustainability transformations in various ways. Barth and Michelsen (2013) differentiate: (1) the development of individual competencies, including knowledge as well as capacities and skills to perform action; (2) social learning as collective learning process of involved actors, leading to change of understanding of these actors sparked by their interaction, and (Reed et al. 2010); and (3) learning on how to collaborate in transdisciplinary research (reflexivity), by “reflecting on the influence that actors’ values, norms and epistemologies have on the collaboration” (Schäpke et al. 2018, p. 88).

All three levels are relevant for RwLs (Singer-Brodowski et al. 2018): Labs can facilitate individual competency, for instance, by creating learning spaces, such as transdisciplinary seminars. Relevant competencies, such as anticipatory, normative or system-thinking competency, can be strengthened by actively involving students in experimental and (potentially) transformative research processes. “Through building a common learning environment, RwLs can offer an ideal space for analyzing and negotiating divergent meaning perspectives, thereby going beyond given assumptions” (ibid., p. 25). Thus, social learning may be enabled if RwL succeed to facilitate a dialogue amongst participants and to offer “a protected space to build trust, allowing for mistakes and iterations, and mediating conflicts” (Schäpke et al. 2018, p. 88).

A specific social learning process relates to transdisciplinary collaborations, involving actors from research and society that engage in a joint intervention and learning journey, albeit with different agendas and epistemic frames. Reflexivity becomes crucial “and includes confronting, interrelating, and integrating different epistemic cultures, values, or goals” (Schäpke et al. 2018, p. 88).

Overall, learning has a cross-cutting character and supports the other key characteristics. Learning may be hampered by “structural barriers in the educational system, different epistemologies, timeframes and goals of the actors that should be part of the learning community or a dominant control paradigm that is in opposition to experimentation” (Schäpke et al. 2018, p. 88).

In our understanding, all five characteristics are relevant for a research approach to be considered an RwL. Additionally, we consider the five characteristics as sufficient to essentially portray RwLs. This allows to compare and distinguish RwLs from other transdisciplinary and transformative research approaches (compare Wiek and Lang 2016; Wanner et al. 2018). Importantly, mentioned characteristics are broad and can be realized in various ways, making the characteristics-based understanding of RwLs more of a broad umbrella than a strict definition.

Success factors—empirical findings from the Baden-Württemberg real-world laboratories

In the following, we first present the results of the surveys concerning the success factors and second their comparison with the primarily literature-based key characteristics.

By "success factors", we mean factors that contribute to a well-functioning, high-quality science-practice collaboration which is essential to prompt or initiate changes in an RwL’s respective field of action. They can be read as recommendations for a successful process design of collaboration in RwLs and have to be adapted to the specific context.

“Success factors” are not to be understood as necessary conditions in that sense that if one factor is not fulfilled, the project cannot be successful. The context-dependent work of an RwL, positioned most often in relation to complex societal challenges, does prohibit such linear and causalistic ways of thinking. Rather, success factors offer an orientation to increase the probability of success in RwL performance.

Success factors

The analyses of the BW-Lab surveys resulted in eleven success factors (SFs). As pointed out by Zscheischler et al. (2018, p. 1068 ff.), an assessment of success in transdisciplinary research (TDR) projects depends on the assessing person or institutions (e.g. scientist or practitioners). The presented success factors are based on judgements of BW-Lab researchers and are dependent on their understanding of success.Footnote 7

We provide illustrative examples from two RwL projects showing the contextual relevance of SFs, how they got approached and related difficulties. The projects are Knowledge Dialog Northern Black Forest (WiNo) and Future City Lab—RwL for Sustainable Mobility Culture (RNM) (see Table 1 and Supplement 1). These two RwLs were chosen due to their particularly fruitful answers in the surveys. Related statements are taken from these surveys (own translation). In summary, the examples can be read as a micro-narrative that gives an impression of empirical examples for success factors from two RwLs.

Find the right balance between scientific and societal goals (SF1)

Ten out of 14 RwLs aim for both: a direct contribution to sustainable development in the setting of the RwL and a better understanding of successful transformation strategies. RwLs need to balance the former, rather societal and the latter, rather scientific goal. Yet, the surveys show a general tendency that the societal impact has a higher priority than the scientific utilization of the findings from the RwLs. This can be interpreted as a consequence of the close cooperation with practitioners who are particularly interested in rapid implementation. Because of the double aim, RwL researchers should constantly be aware of their role(s) in the project, i.e. whether they act as scientific observers or change agents performing interventions or both. Questions of legitimacy concerning the roles must be clarified. 13 responding RwLs were generally striving for scientific insights or effects but also had to realize that the enormous pressure to bring about real-world effects hindered this. For example, Project WiNo states in the questionnaire on success factors: “Scientific yield appears to be of secondary importance compared to classical research: It is necessary to break down the pressure to exploit research results, for example, in scientific publications.”

Address the needs, interests and restrictions of practitioners (SF2)

Addressing the needs, interests and restrictions of practitioners are regularly mentioned as important for a successful collaboration in an RwL. First, it allows to gain the support of relevant practitioners, including citizens, administrations and municipalities, by raising attention and motivation to participate. Project RNM states that attractive media presence, events (for information and participation) and a topic relevant to many actors are important. Moreover, strong motivation could be seen with participants who are already working in ongoing projects that support the subject of the study. 4 RwLs emphasize the importance of the relevance of the object of investigation for practical actors, 5 projects of the funding line on cities consider the interests of municipal actors to be particularly important.

Answers of the RwL representatives show, that motivation and commitment of practitioners benefit from:

  • defining goals that have the potential of short-term impact;

  • addressing relevant and timely problems or topics;

  • and developing products and results along existing demand by relevant actors that are highly visible and long-lasting.

Second, the answers in the questionnaires show some evidence that it is important not to lose existing support and commitment of practitioners. Therefore, it is relevant to show understanding for restrictions in the work structures of involved practitioners and to consider their time constraints. For practitioners being involved as private individuals, RwLs recommend to consider an adequate financial or material compensation in the planning of an RwL—keeping in mind possible consequences for the voluntary engagement.

Make use of the experimentation concept (SF3)

The results show that the research design of RwLs can be structured as one continuously conducted experiment, as a sequence of several single experiments or as a thoughtful mixture of both. Typically, experiments were mentioned to consist of an intervention and the observation and analysis of the impact of this intervention to draw conclusions for aspects contributing to successful or unsuccessful transformation paths. Experiments allow to apply and test developed solution options in direct and flexible ways. Successful experimentation requires to develop a common understanding among participants on the character of experimental research associated with uncertainty about the outcome. Respondents associated widely differing aspects with the term ‘experiment’.

RNM reported that due to the great importance of experiments in their project, transdisciplinary teams were formed to accompany and examine the experiments. This as well could allow to detect non-intended effects which not necessarily were seen as something negative. “Rather, they can help to illuminate problems from an unintended angle or point to synergies for solving other problems”. However, it should also be added that of the six planned experiments that RNM wanted to conduct, two failed because the practical actors involved lost interest or were unable to provide the necessary support in time. Already organized civil society projects and initiatives enable a faster start to experiment with practice partners, project RNM found out.

Actively communicate (SF4)

Following the answers given, communication is of highest importance for RwLs, especially due to the heterogeneity of the participants. 12 RwLs stress transparent and regular communication. This includes 'one-way' flow of information from science to practitioners (such as newsletters, leaflets or websites) as well as consultation of practitioners and other forms of communication (e.g. workshops, excursions, face-to-face exchange) which represent ‘two-way’ flows of communication. Communication formats fitting specific RwL contexts are to be co-developed with involved practitioners and should be applied regularly, said respondents. The application example from WiNo, which is described in detail in the section “How to make use of the methods”, shows that intensive communication is particularly important in the early project phases: The process of joint problem description and project constitution ('co-design') with many, very heterogeneous actors takes up a lot of time, but is indispensable for later successful cooperation. Thus, communication competencies of the involved scientists should be fostered. Strategies for conflict resolution including mediation can help to support the aspired transformative processes, noted project RNM.

Develop a ‘collaboration culture’ between science and society (SF5)

Integration of scientific and societal knowledge is central for RwLs and happens through co-design, co-production and joint efforts for implementation—according to the responses to the survey. This begins when developing the project proposal: RwL goals, questions and a shared understanding of the to-be addressed problem should be developed collectively. This collaboration can create several benefits: increased mutual understanding, collective targets, participation, societal relevance of the research, as well as permanency, higher acceptance and compatibility of solutions. Furthermore, intensive co-design and co-production create trust and transparency. “Co-design was decisive for the success of the project, because it brought completely different questions into the focus of the RwL than those initially expected by the scientists”, observed WiNo project.

Cooperative research demands the researcher to have transdisciplinary communication, moderation and integration skills as well as methods to prepare and support the processes methodologically and conceptually. 12 RwLs mentioned the culture of participation as a particularly important point, with different emphasis (5 accentuated commitment and willingness to explore the specific requirements of the transdisciplinary RwL approach, 6 underlined formulation of a common view of the problem to be addressed, a common question and a common goal).

Be attached to concrete sites (SF6)

All 14 projects’ answers show the importance of a clearly defined site to which the RwL relates. It is seen as beneficial to have a concrete physical location or infrastructure available for the project. These locations:

  • Allow hosting meetings with citizens (potentially) interested in the RwL.

  • Constitute the infrastructure where RwL experiments are taking place.

  • Increase the visibility of the RwL and its activities, and thus foster public attention.

  • Support trust building through offering places for face-to-face communication.

Respondents report that showing regular presence at these places enhances successful experimentation. Acquired funding and building upon already existing local initiatives are ways mentioned to get access to spaces and infrastructure. However, RNM points out that too many small size sites not only hamper to create mentioned benefits, but can also lead to the unsuccessful realization of experiments.

Create lasting impact and transferability (SF7)

As the BW-Labs were still ongoing when the surveys were conducted, statements about their factual and stable impacts are limited. But the surveys show that perpetuation and transferability are seen as key demand for RwL success, because they correspond to the tangible and enduring societal change. If they are not fulfilled at least to some degree, the RwL would only be a local, singular, and short-lived project. Respondents named long-term transformation plans and to early on consider the potential of the products, concepts, and methods to be developed for the time after the project (12 answers) and for other contexts (2 answers) as a promising way to guide long-term development of results.

Transferability benefits from considering the broader interest and demand for the RwL results and products, as well as from communicating results broadly to increase possibilities for replication in other contexts, for instance, through high-publicity events. Project RNM recommends establishing comparability between different experiments by developing theory-based evaluation criteria to support transferability. For each experiment and at laboratory level, they synthesized "lessons learned" as well as scientific knowledge that aimed at successful and generalizable transformation strategies.

Provide and acquire sufficient time and financial means (SF8)

The projects have identified a sufficiently long duration as an important success factor to address the specific local or regional development opportunities and to be able to conduct an adequate co-design. It does allow for iterations and experimentation, and to be able to adequately perpetuate the results of RwLs. WiNo emphasized that the three-year-duration of many RwLs should be extended to give time to sound co-design and co-production. For instance, the development of a common understanding within the WiNo project team and the co-design phase with numerous very different practical actors took 9 months of the 36-month project duration alone—5 months longer than expected. In addition, both projects note that there are numerous tasks which—compared to classical research projects—require additional, freely available funds, such as for public events, expense allowances for practitioners, external moderation or for the concrete implementation of experiments and results. In the eyes of the respondents a funded pre-phase for co-design would help to foster relevance, acceptance, and impact of the projects.

Be prepared for adaptability (SF9)

Flexibility and adaptability are considered a prerequisite for successful collaboration in an RwL. This openness is beneficial since many aspects might change during the research process, including the societal problems addressed or the composition of the RwL participants. Unintended effects might appear during the project. The same applies to sustainability solution strategies that are tested jointly and afterwards have to be adapted according to the experiences made. WiNo emphasizes that this is the central challenge of RwL work, that goals, contents, actors and processes are constantly changing and that the real added value is learning about adapting goals, procedures and dialogue partners. All 14 RwLs agreed on the need for recursiveness and self-reflection.

Iteration and adaptability need to be balanced with some degree of planning security. Adaptability also concerns the interaction of researchers and research funders. Here a continuous and jointly conducted quality management benefits RwL success more than non-flexible guidelines. Respondents indicate that current funding guidelines do not sufficiently take into account the requirement of adaptability.

Provide research-based learning and reflection in RwL settings (SF10)

Enabling learning opportunities can be a success factor leading to societal impact, scientific insights and education/training. Respondents outlined that testing of scientifically developed solutions in practical experiments has the potential to spark learning of participants of an RwL and lead to the adaptation of goals and tested solutions according to the test-insights. RNM provides an example of a social learning process: the project applied for special permits for the installation of ‘parklets’ (street furniture on former parking lots) in Stuttgart. As there was no comparable precedent, this required intensive consultation with the Office of Public Order and other local authorities responsible for traffic safety. Following the precedent, the city administration issued a regulation easing future parklet installation.Footnote 8

Transdisciplinary teaching formats can be used in the RwL as a way of learning and to pass newly generated conceptual knowledge on to students and young researchers in a systematic way. Both, WiNo and RNM integrated some teaching units at their universities into the project. “Research-based learning was used in the project and was very positively received by students: the practical relevance and also the contributions to a research project were highly appreciated.” (WiNo) Practitioners can benefit from learning and capacity building opportunities in RwL, too. Surveys show that scientists learn a lot in RwLs: through the close cooperation with practitioners they gain new, practical perspectives and knowledge and reflect themselves in new ways.

Consider dependency on external factors (SF11)

Finally, there are factors of high relevance for RwL success, but beyond the influence of an individual RwL.

Most often the support from key actors such as public authorities or administrations is mentioned as a general condition for RwL implementation. Those actors have their specific interests as well as work flows and logics that sometimes clash with the time constraints of the projects. In particular, the RwLs of the second funding line with a focus on cities mentioned a strong dependency on municipal administrations. Depending on these actors the project conceptualization should incorporate uncertainties and possibilities of failure to allow for adapting the RwL design if needed. WiNo adds an example: A practice actor central to the project demanded more sensitivity from the scientists for research in a highly political environment and had reservations about research on aspects that examined developments at the project site too critically.

Intermediate juxtaposition: relating key characteristics and success factors

A comparison of the primarily literature-based key characteristics and the primarily empirics-based success factors allows three observations (Table 2).

Table 2 RwL characteristics, related success factors and design notes
Full size table

First, characteristics and success factors correspond regarding many core aspects. Aspects put forward in the literature as RwL characteristics are also described as important for RwL implementation by respondents from the BW-Labs. Thus, conceptual ideas about what characterizes an RwL are grounded in empirical experience. This convergence is not to outline “the perfect RwL”, but to show learning and experiences from a specific group of RwLs, the BW-Labs. A detailed comparison between characteristics and success factors allows to detect differences and nuances between both. For instance, the reported success factors for the characteristic of ´transdisciplinarity´ emphasize aspects that are less seen in the literature (e.g. the relation of an RwL to a concrete place). This offers possibilities for further investigation towards conceptual enrichment and improved RwL realization.

Second, and adding more nuance, success factors are underpinned with often diverse and more concrete design notes outlining ways to their realization (Table 2). Thereby the critical tasks related to the characteristics find practical answers. For instance, respondents mentioned the challenging double aim of RwL, to both facilitate societal change and produce scientific evidence. Related success factors emphasize the right balance of both aims and the societal relevance of RwL work. Stated design notes show that societal goals are often prioritized over scientific contributions. This emphasis demands researchers to deal with questions of legitimacy and clarity on roles.

Third, interrelations between key characteristics and related success factors become explicit, highlighting complexities in designing and realizing RwL. For instance, contributions to transformation demand long-term orientation and transferability of results. It is important to consider interrelationships in the concrete RwL work.

Methods for Collaboration in RwL—empirical findings from the Baden-Württemberg Real-World Laboratories

In a Mode 1 science understanding, scientific methods can be regarded as established and validated instruments of knowledge generation. Disciplines apply methods to develop discipline-specific knowledge under control of the respective scientific community (Weingart 2003, p. 41 ff). Transdisciplinary research, often considered as representative of Mode 2 science, builds on integrative and participatory processes between science and practice actors and requires a wider definition of methods. Processes require specific methods for a joint knowledge production by appreciating, validating and intertwining knowledges from different disciplines, epistemologies, research and practice. This is also true for RwLs, which include particularly close co-operation.

The following method types are important to foster knowledge integration, social, and communicative integration (cf. Lang et al. 2012; Bergmann et al. 2012, 2005):

  • Methods for analyzing complex societal structures;

  • Methods to facilitate common learning processes;

  • Assessment methods to include different perspectives and value systems;

  • Methods of process organization, communication and negotiation.

Below we present the results of the surveys regarding methods of RwLs.

Overview: frequency of application of different methods in different phases

The BW-Labs were asked which kind of methods they used in which phases of their projects. Methods facilitate different intensities of collaboration between actors from science and practice, and reach from providing information for actors from practice over a consultation of the practitioners to collaboration between science and practice. Survey results show varying intensities during the different stages of project realization—from the initial identification of practice actors to the final transfer of knowledge (Fig. 1).

Fig. 1
figure 1

Application of methods with different intensities of cooperation sorted by project phases. Numbers indicate absolute frequency of mentions

Full size image

Most frequently, methods for consultation were applied—before all in the beginning of projects, for co-design, problem description and definition of targets (Fig. 1).

Methods for information of practice actors were also mentioned relatively often. Communication in the sense of mutual knowledge exchange is particularly important at the beginning of the project in the co-design phase and at the end for knowledge transfer. Discussions with the 14 BW-Lab projects have shown that intensive communication (especially in workshops between science and practice) is of great importance for successful cooperation with stakeholders, especially at the beginning of the project. The projects also argued this in the survey as shown in success factor SF4: a transdisciplinary project, especially one with the claim of a particularly close cooperation between science and practice, will encounter problems if the co-design is not carried out in a direct exchange.

Methods to facilitate close collaboration are mentioned with the lowest frequency. This may appear surprising, since the RwL setting is propagated to allow for particularly close science-practice collaboration. Yet, a differentiated view on methods of consultation reveals a high frequency of interactive consultation workshops (Fig. 2). Summed up, these methods indicate the relatively strong engagement of practice actors in RwLs. While surveys were generally used less often, they appear to be relevant in realizing interventions.

Fig. 2
figure 2

Application of methods supporting different consultation approaches used in different project phases. Numbers indicate absolute frequency of mentions

Full size image

Challenges and methods of realizing RwLs

The key characteristics and success factors described above constitute the requirements for practical work in RwLs. In other words, they are the challenges that need to be met when realizing RwLs to contribute to sustainability transformation by achieving high-quality work in the specific RwL setting. Using appropriate methods can substantially help meeting these challenges, whereby several methods allowing to approach a certain challenge can be summarized to a challenge-oriented method family (Table 3, for a detailed overview see Supplement 2).

Table 3 Empirically derived challenges of RwL research and families of methods to cope with the challenges
Full size table

In the following, we present the central findings on the methods and processes used in the BW-Labs based on survey results. Some challenges and related methods were not covered in the original survey, but became apparent during later stages of empirical work. We focus on methods supporting science and practitioners working together in correspondence with our interest in RwLs as a setting of transdisciplinary research.

Methods to address and balance scientific and societal needs, interests and restrictions (M1)

Methods used in the co-design phase when setting-up an RwL were most frequently mentioned. This indicates the need for special methodological attention when starting projects, initiating collaboration and considering practical and science-oriented needs, interests and restrictions.

The following methods were used most frequently and worked particularly well to identify and integrate practitioners’ and scientists’ problem perceptions, objectives and knowledge:

  • Topic-specific information events, such as lectures, expert and stakeholder workshops and citizens’ consultation hours.

  • Interviews and surveys with societal experts.

  • Direct, personal contact to provide specific information to practice partners.

  • Visioning Workshop: 1. joint vision search; 2. from visions to scenarios; 3. evaluation of scenarios by the participants.

Co-authors from BW-Labs supplemented the following methods:

  • Round table with regular meetings to find ways to work collaboratively on local challenges.

  • Building `tandems´, combining researcher and practice partners, when doing excursions, workshops or presentations.

Methods for experiments and interventions (M2)

Some of the projects structured and systematized their overall process by carrying out ‘experiments’. What the respondents associate with the term ‘experiment’ differs among the analyzed RwLs but what can be said is that they typically consist of an intervention in the project setting on a specific or central aspect of the RwL. The reaction of the actors, social structures and artefacts addressed by the intervention is then observed and evaluated to draw conclusions about aspects of (un-)successful transformation paths. A limited number of single methods were mentioned in the survey, e.g.:

  • Organizing a competition among practice actors with own activities concerning the RwL targets; competition procedure consisting of development and tender, organizational and content-related support and accompanying research.

Additional practice supplemented from BW-Lab Co-authors:

  • Experimenting by including students to develop, present and iterate creative ideas, since experience shows that societal actors are more willing to listen and discuss even provoking ideas of young people than to those of experts.

See Sect. “How to make use of the methods” below, for examples of procedural combinations of different methods to carry out an experiment.

Methods for active communication (M3)

Developing a close and transparent collaboration among the heterogeneous actors calls for high sensitivity concerning communication. Information, consultation and collaboration require a targeted handling of diverse, adapted communication formats. According to the respondents, the following methods and practices were used most frequently and successfully:

Methods of information (one-way flow of information from scientists to practitioners):

  • Articles and advertisements in print media, press reports, press conferences.

  • Information in public space and by mail (flyers, posters, direct/ bulk mail, monitors in public transport, distribution in town halls).

Methods of consultation (two-way flow of information from practitioners to scientists and vice versa):

  • Quantitative, standardized surveys (face-to-face and online).

  • Qualitative interviews with different groups (experts, citizens, users, etc.) and on different occasions and phases.

Methods of information and consultation (two-way flow of information):

  • Personal communication (face-to-face, mail, telephone) with citizens and professional actors in the RwL’s site.

  • Lectures, conferences, workshops, technical discussions, consultation and information hours for citizens to inform about the project and its goals.

  • Exhibitions and fairs to display point of departure and developments.

  • Excursions and walks in the RwL’s site to become acquainted with the situation.

  • Use of social media for online exchange.

Respondents pointed out that these methods often best worked in their combination, i.e. making use of different 'channels' at a time. Methods of information and consultation were overall mostly seen as beneficial. The results of the survey indicate that interactive face-to-face formats are of particular importance, as they allow information to always flow in both directions allowing for immediate, on-site exchange between science and practice. In addition, face-to-face contact with stakeholders is also a key point in building trust, especially as important details can be explained in a personal exchange of information, thus clarifying points of view and concerns. Therefore, digital media methods have some disadvantages, even if built interactively.

Methods for supporting a ‘collaborative culture’ (M4)

The involvement of practice actors in the management or development of the RwL could be regarded as being decisive for the realization of the different key characteristics and to create an impactful RwL. It may contribute to the empowerment of participants and potential self-perpetuation of the RwL in the hands of the former practitioners. The strong involvement of practice actors is particularly challenging, too.

Here are some examples for closer collaboration from the surveys:

  • Establish a plan for participation so that

    • the participation process follows clearly defined objectives;

    • has enough resources available;

    • the time schedule can be controlled and

    • the tasks contained therein are distributed appropriately.

  • Co-design workshops with various stakeholder groups to finding ideas and solutions adapted to the RwLs’ site and achieving open exchange between the groups.

  • ‘Participatory design’ attempts to involve the end user or citizen in the design and decision-making process through democratic, collaborative and transparent design processes.

  • Smaller, cooperative working groups with topic-specific selected practice actors with concrete reference to a single topic.

  • Reflection workshops with external moderation to clarify roles in the team and relieve the team leader.

Additional methods supplemented from BW-Lab Co-authors:

  • Informal, recurring meetings, such as a series of regulars’ tables (German: ‘Stammtisch’) to establish trust.

  • Collaboration among practice actors (without scientists), e.g. in form of a Blog or other cloud-based collaboration tools.

Methods for anchoring in a concrete location (M5)

Relating to a concrete location is very supportive for an RwL. Collaboration between scientists and practice actors can be fostered strongly by installing an easy to access location providing a visible and continuous meeting point. Exemplary settings and methods are:

  • Concrete infrastructures like shops, offices, community centers.

  • Joint walks through the district and excursions to look at problems in physical space and provide occasions for informal exchange.

  • Mobile apps for citizens to participate in data collections on mobility behavior in the location of the RwL.

  • Display of RwL-related information in public spaces (e.g. screens in public transport).

Additional methods supplemented from BW-Lab Co-authors:

  • Interim uses of facilities, moving between different locations in the RwL area.

  • Shared offices with the (external) partner institutions.

Methods for increasing impact and transferability (M6)

RwLs aim to provide knowledge and solutions for complex sustainability problems that ideally can be implemented in other contexts beyond the individual case. The following methods support the transfer and broad application of successful strategies:

  • Mentors and intermediaries supporting the transfer to comparable settings.

  • Networking among practitioners with peer-to-peer exchange and learning.

  • Occasions for informal exchange between societal actors inside and outside the RwL.

  • Strategic plans to continuously support RwL settings, such as municipalities, in longer-term transformation processes.

  • Written products including guidelines and documentation materials adapted to the needs of participating practice partners and suitable for use in other contexts.

  • Development of technical tools (apps, websites, digital participation instruments, real time simulations) that have the potential for being used by practitioners from other/similar contexts that are interested in the transformation strategies developed.

Additional methods supplemented from BW-Lab Co-authors:

  • Films.

  • Presentation of results and/or products to the public (workshops, social events, fairs, newspaper supplements).

  • Appointing transfer managers, for instance, at local organizations responsible for networking and transfer of knowledge from RwL to broader society.

Methods for acquiring adequate resources (M7)

RwLs do depend on a variety of resources, such as finance and infrastructure, but also social capital (trust, support) and adequate human capital (e.g. competencies of researchers). Respective methods relate to both planning and acquisition of initial project resources, as well as their management and acquisition of resources for continuation of activities. Originally not part of the survey, this aspect proved relevant in the empirical findings and respective methods got supplemented by BW-Lab Co-authors.

  • Public-relations and fund-raising activities informing about and inviting to participate in/support the RwL (e.g. face-to-face, posters & leaflets, social media).

  • Continuous media presence (also beyond IT-based media channels).

  • Diversified funding strategy,

    • combining various sources, including from public, private, crowd sourced and industry sources;

    • combining different aims, including research, economic development and innovation, addressing local societal challenges;

    • split overall RwL activities into several sub-activities drawing on different sources;

    • combine funding with different time horizons;

    • network and joint application with other ongoing research and teaching activities at host the university with similar thematic frame or innovative learning formats to extend workload and/or add additional funds to the RwL.

  • Lean and effective project management structures and use of (external) coaching supporting RwL planning and management to counteract tendency of overburdening RwL researchers with high ambitions and multiple tasks and aims.

  • Creation of synergies when engaging practice partners, for instance, to coach students based on personal interests or development of joint activities with practice partners.

Methods for increasing adaptability (M8)

The real-world 'laboratory' does not correspond to the basic scientific laboratory concept with fixed boundary conditions. Social contexts are subject to constant change. RwL actors must regularly review their plans and goals and adapt to changing RwL contexts. Iterative and recursive processes regularly compare original plans, intermediate results and track changes in the research process. They support an open, learning research process as well as a close link between research and practice. As the RwL setting is new and unfamiliar, respondents had to constantly react to unforeseen changes in personnel and organization, as well as external context conditions (see methods section M10). Supportive methods are:

  • Iterative SWOT analysis to repeatedly ascertain the opportunities and risks of one's own approach in the team.

  • Reflection workshops to discuss difficulties and uncertainties, preparation of decision-making for the partial re-orientation of the project.

  • Reflection workshops with the presentation of the results of an experiment to discuss obstacles and effects achieved to benefit similar experiments.

  • Monitoring of ongoing project processes and ex-post evaluation to assess overall project impacts.

Additional methods supplemented from BW-Lab Co-authors:

  • Agile project management and agile methodologies (e.g. scrum, Kanban, design thinking).

Research-based learning in RwL settings (M9)

In the long term, the impact and durability of knowledge how to run an RwL is concretely based on the interaction of teaching/capacity building and research. Only if students and (young) scientists are sensitized to research in transdisciplinary settings (e.g. an RwL) and can practice this in the course of their education, the insights and skills can be enriched and forwarded in science. Furthermore, RwLs support continuous learning of experienced scientists introducing them to transdisciplinary work and related capacity building of practice partners. Some of the BW-Labs integrated university teaching units into their research, namely through:

  • Student idea competition concerning exhibitions or the visualization of future scenarios.

  • Transdisciplinary teaching formats linking research in an RwL with the Education for Sustainable Development programs.

  • Citizen workshops supported by students concerning knowledge on sustainability issues and organization.

Additional methods supplemented from BW-Lab Co-authors:

  • Explorative learning formats in collaboration with practice partners, e.g. including practitioners mentoring student groups and presentation of ideas by students in public events.

  • Gamification approaches (e. g. sustainability walks, scenario-based business game, ‘serious gaming’, role plays, and geocaching).

  • Sustainability experiments (as self-experiments) where citizens take part in sustainability-oriented actions over several month learning how they can influence their own life and that of their personal environment.

  • Cooperation between students and artists regarding on-site artistic interventions regarding sustainability issues.

Methods for considering and handling dependency of external actors (M10)

RwLs incorporate many actors from science, politics, economy and society. These actors have different interests, perspectives, resources and timelines. It is sometimes an enormous task to bring this all together. Dependencies from external actors can arise that may be hard to handle. Originally not part of the survey, this aspect proved relevant in empirical experience and respective methods got supplemented by BW-Lab Co-authors. Methods to reduce the dependency on external actors are:

  • ´Flexibility by Design´ as planning for adaptability based on diversified partners/contents/experiments (e.g. failing experiments free resources for other activities).

  • Setting project boundaries that allow a functioning lab even in case of partner drop-out, securing all affected stakeholder groups are represented.

  • Diversification, regarding

    • partners: for instance, partnership with more than one city (or city district); more than one access (contact person) to the city admin; more than one (or few) NGO as partners;

    • content: project landscape should offer various options with differing central focus: urban development; sustainability research; science communication; participatory formats; etc.;

    • experiment: use several at least partly independent experiments within an RwL, e.g. located in different settings or relying on different partners.

  • Use methods to establish good collaboration and adapt to needs of societal partners (cf. M1 and M4) to reduce risks of unresolvable conflict, break of support, etc.

  • Careful expectation management, for instance, regarding RwL targets and promised effects, to avoid frustration of partners and hampered collaboration.

How to make use of the methods

The individual methods listed here can be of use for designing a new RwL or to adapt or realign an existing RwL. The descriptions above—which can only be cursory here—shall allow identifying methods to address specific challenges—and therewith to realize specific success factors and characteristics of RwL. Yet, method descriptions cannot be used as a blueprint to design an ideal-type, successful RwL. Application of methods should be oriented towards the goals of the RwL and in most cases require a prioritization where to focus efforts. Selection and successful application of methods requires transdisciplinary researchers to take into account various dimensions of appropriateness, for instance, to the context, the aims of the RwL, participants as well as geographical and temporal scales (Beecroft et al. 2018). Consideration of different temporal scopes in combination with various aims of the RwL (e.g. basic and permanent lab infrastructure, continuous research, shorter-term experimentation) may guide a targeted method selection as well (Beecroft et al. 2018). Furthermore, an RwL is regularly enacted by combining various methods. Below we give examples for how to merge different methods for complex and ambitious RwL processes (Application examples 1 and 2). Thereby various challenges of practical RwL realization (Table 2) get addressed simultaneously.

Application example 1: Co-design as a sequence of steps: An example from WiNo-knowledge dialogue black forest (Rhodius and Pregernig 2018)

This BW-Lab described its co-design phase to develop concrete research questions and related research processes as a combination of methods in five steps:

  1. (1)

    A survey of regional actors identified six priority thematic areas and formed scientific teams.

  2. (2)

    The scientists in coordination with authorities in the RwL site as well as with representatives of regional businesses and other key stakeholders developed research ideas in the six thematic areas.

  3. (3)

    A “Knowledge Fair” as a core element of the co-design was carried out to co-develop concrete research questions of high relevance for science and the region together with regional actors and citizens.

  4. (4)

    Based on insights from the fair, research teams revised their project ideas and submitted their project outlines to an advisory board consisting of external scientists.

  5. (5)

    Based on the recommendations of the advisory board, an internal steering board selected research projects for manifestation and determined the distribution of funds.

Application example 2 from FutureCityLab (RNM) for preparing, executing and reflecting (on) a real-world experiment (Puttrowait et al. 2018)

  1. A.

    “The path to the experiment”

    The process of determining the contents and objectives of a real-world experiment had three steps:

    1. (1)

      Identifying key players in a workshop with scientists, civil society practitioners and municipal administration;

    2. (2)

      Consolidating cooperative multi-actor-constellations, ideas for real-world experiments and structural or content-related similarities during a “Market of Ideas” including researchers, civil society practitioners and municipal administration;

    3. (3)

      Selecting project ideas by a transdisciplinary jury based on a defined list of criteria.

      The real-world experiments prepared in this way had a project duration of about three months with a preceding preparation phase of another three months. Thereafter, the effects were assessed, evaluated, discussed and reflected upon (ibid.: 196).

  2. B.

    Transformation(-design) workshop

    Goal: Collaborative definition of goals and corresponding survey methods for real-world experiments in cooperation with scientists and civil society practitioners.

    Time in the project: Before the start of the real-world experiments, in their conception and preparation phase.

    Procedure: a workshop day is organized at a neutral location. Here, civil society practitioners determine what defines the real-world success of the experiment for them. Accordingly, methods are identified by which the success can be measured and evaluated. In the workshop, the procedure for conducting the real-world experiments is put in concrete terms. The workshop thus also provides the basis for the scientific consideration of the real-world experiments´ societal impact.

  3. C.

    Reflection workshop


Aim: To present the preliminary results of the real-world experiment and to discuss the success criteria and general lessons learned with regard to obstacles and effects achieved; thus, future real-world experiments can benefit from practical experience and related recommendations for action for politics and administration can be developed.


Time in the project: After execution of the experiment.


Procedure: The workshop will again take place at a neutral location. Before the workshop, the civil society practitioners formulate a short report describing the effects achieved and obstacles met during the execution of the real-world experiment. At the same time, the team of scientists systematically describes the real-world experiments and evaluates them on the basis of their individual reports. The transformative effects (successes) are reflected and difficulties, findings and other unintended effects of the real-world experiments are compiled.

Concluding discussion

In the above, we addressed three research questions, regarding conceptual characteristics, success factors and methods to realize RwLs. Key characteristics are interrelated with BW-Lab experiences regarding success factors and design notes, resulting challenges for research practice and examples of methods to manage challenges. Therewith, we contribute to a conceptual and empirics-based understanding of RwLs. In combination, characteristics, success factors, design notes and methods can provide orientation for RwL practice as well as for research policy and funding of RwLs. The following discussion section relates above insights to the discourse on sustainability labs and transdisciplinary research. We reflect limitations of our work and outline future research needs.

RwLs as experimental path of transdisciplinary research: state of the debate

We understand transdisciplinary research as aiming to contribute to solving complex societal problems at the same time to create scientific innovations and insights (Bergmann 2012, p. 41). Thus, we go beyond narrow understandings portraying RwLs primarily as regulation-free space for technical innovation (BMWi 2019; but Parodi 2019). RwLs enable a transdisciplinary research practice that aims to contribute to societal change towards sustainability as well as identifying successful transformation strategies that may be transferred to other contexts. The current discourse explores how to integrate RwLs into an overarching conceptual understanding of the ideal–typical transdisciplinary research process (Jahn and Keil 2016; Ukowitz 2017; Rogga and Zscheischler 2018; Wanner et al. 2018). Some authors see RwLs as places to implement previously well-researched sustainability innovations (Jahn and Keil 2016) or to test “knowledge and possible solutions resulting from a preceding TDR process” (Rogga and Zscheischler 2018). Here, RwLs quasi-function as an add-on to existing transdisciplinary research processes. Other authors see RwL as a specific form of transdisciplinarity encompassing all phases of an ideal-type transdisciplinary research process from co-design to co-production and reintegration (Wanner et al. 2018).

Experience from our accompanying research shows that RwLs can be both, an attempt to implement existing knowledge in a highly participative manner or a transdisciplinary research setting deeply anchored in the field of social action and closely cooperating with its actors to create new knowledge. The discussion on the testing and implementation character of RwLs does hint towards a specific dimension of the RwL-setting: the experimentation. While RwLs must be seen in the context of numerous formats that make use of experimental approaches that have emerged over the last decade (Schäpke et al. 2018; McCrory et al. 2020), approaches and impacts of experiments in RwLs are partly underexplored. RwLs are an experimental approach towards creation of knowledge and contribution to societal change rooted in the transdisciplinary research mode. If this dual task can be fulfilled by RwLs, it is a particularity of this approach.

Success and impact of RwLs: in need of further research

In our study, we relied to a strong degree on self-reported perceptions of RwL practitioners on what supports the successful realization of a collaborative RwL. This allows to derive first directions on elements that can be crucial for collaboration in an RwL. This approach has similarities with recent studies on perceptions of researchers and practitioners engaged in labs on strategies and mechanisms how labs create impact (Parodi et al. 2018a, b; Heiskanen et al. 2018; von Wirth et al. 2019). But it does not provide evidence based on a systematic evaluation of processes, outputs and impacts or on a comparison between cases. Current research investigates the impacts of transdisciplinary research in relation to applied research methods and processes and the influence of the research setting, e.g. in terms of intensity of practitioner involvement and the use of knowledge integration methods (Lux et al. 2019; Newig et al. 2019). Yet, insights on whether working with new formats, such as RwLs, lead to faster or even better effects than ‘conventional’ TDR are lacking.

The analysis of impacts of transdisciplinary research is hampered by difficulties to discern a direct link between research and societal change, a time delay for impacts to occur and the difficulty to demarcate the temporal boundaries of the research process (cf. Lux et al. 2019). RwLs, for instance, constitute continuous efforts to provide impulses for change. In the case of the BW-Labs, it can be stated that they have—to varying degrees—focused more on direct collaboration between science and practice and their societal effects than on the scientific yield. The relatively short duration of the projects as well as the novelty of the approach and the requirements of the funding institution and the science itself are certainly decisive for this.

Consequently, a number of tasks for further research appear to be essential: (1) A procedure must be developed for evaluating the medium- and long-term societal and scientific effects of RwLs (as of transdisciplinary research in general). This is also useful for estimating the advantages or disadvantages of the RwL approach over time. (2) To have sufficient material available for this, it is necessary to conduct a targeted testing and evaluation of RwLs with experiments and teaching/learning formats to create experience-based learning cycles. (3) There is a need for clarification—and possibly the creation of a typology—of formats of experiments in RwLs and for guidance on how to design, set up, implement and evaluate them. (4) Funders should supplement their programs with possibilities to enabling long-term observation concerning societal capabilities. (5) Accompanying research in RwL programs concerning the above questions. (6) Targeted comparisons between the results of the accompanying research with the lessons learned in corresponding investigations on transdisciplinary case studies.

Ambitions, trade-offs and demands: balancing flexibility and structure in RwLs

The comprehensive list of characteristics, success factors and related design notes as well as methods points towards the high aspirations that are related to RwLs. These aspirations correspond to high demands on researchers and practitioners engaged in RwLs. To some degree, the variations between mostly literature-based characteristics and empirically derived success factors and design notes reveal that in practice, RwLs decide on priorities and contextual fit. This is necessary as there are diverse interdependencies and therewith potential trade-offs and synergies between different success factors. Furthermore, the high demands not only concern the RwLs directly, but have implications for actors outside the labs (see Parodi et al. 2018a, b). These are institutional requirements regarding the education of RwL-researchers and development of respective skillsets. Appropriate funding policies to enable, for instance, intensive co-creation on longer-term perspective of RwL are another institutional demand.

Looking at the different interpretations of RwLs, the various similar approaches and the open research questions related to the RwL approach, one might question the need of introducing this new research setting. Yet, during our accompanying research and our own work in RwLs, we experienced that exactly the openness of this concept provides spaces for experimentation and learning for sustainability transformation and allows for bridging often existing barriers between academic fields, methodological approaches or social-sub systems. This constant search-process, the iterative design, rejection and implementation processes are inherent to RwLs and have a key role to play for RwLs aiming to contribute to societal change.

With the different conceptual and empirically informed elements presented in this paper, we therefore aim at contributing to structuring RwLs as a research setting without undermining its flexibility. In this way, we want to provide orientation that helps in the design of RwLs and facilitate their successful, method-driven implementation.