Bold ideas and critical thoughts on science.

Dr.-Ing. Mennatullah Hendawy

Achieving (Partial) Generalizability  

Mennatullah Hendawy on six decisions that made her findings from her dissertation, a case study on interdisciplinary urban planning in Cairo, more generalizable.
7 November 2022

To what extent are research findings and theories transferable and generalizable beyond the contexts they were designed from and for? This is a main question that we, researchers, constantly think about and find it hard to answer. Schwandt (1997, p. 57) describes generalization as a “general statement or proposition made by drawing an inference from observation of the particular”.

Mennatulla Hendawy
Mennatullah Hendawy

In this article, I will share my experience of how I sought to achieve generalization in my dissertation in the field of engineering at the Technical University of Berlin. My thesis was on the topic of interdisciplinary urban planning. It aimed at developing a ground-up understanding of cities’ visualizations in the mediatized world, taking Cairo as a case study (Hendawy, 2022). In the thesis, I sought to achieve generalization beyond Cairo by making six choices, which this article will explain next.  

Decision 1: Using multiple sub-case studies to gain a further depth

Generally, the use of case studies in research is regarded as not being generalizable because they usually focus on a small-N sample (Gerring, 2007; Steinmetz, 2004; Stoecker, 1991). Although this perspective is correct, it is only  applicable to studies that require large N samples aiming for statistical generalizations and/or within-population generalizations. However, for studies which aim for cross-populations generalizations, they can be built on small-N samples, which can be achieved using a single case or multiple case studies  (Tsang, 2014a, see also Eisenhardt, 1989; Flyvbjerg, 2012; Yin, 2009).[1]

In my thesis,  while I focused on the context of Cairo, I used five multiple sub-cases in Cairo. In my earlier article at Elephant in the Lab, I showed how I used the cumulative format of my dissertation to investigate five sub-case studies (all in Cairo) to study my research topic. Selecting different sub-case studies in Cairo allowed me to conduct cross-analysis that helped in developing results that could be generalized beyond the studied situations (Lawrence, 2010). For example, in one of the five articles, I investigated the project posters by students in a planning program to investigate which city are made visible inside universities, and, in another article, I investigated real estate TV advertisements. Looking at the students’ posters in relation to (and in comparison with) the TV real estate advertisement, it allowed me to understand that in both universities and public media, an exclusive image of the city is produced and promoted. 

The decision to use multiple sub case studies (in the same context) helped me to gain further depth into the research topic from different angles, which accordingly helped me to claim generalization.

Decision 2: Implementing an extra layer of analysis to generate abstract reflections

In a cumulative dissertation, researchers are requested to not only collect and analyse the data within the individual research papers but also to have an overarching introduction and conclusion, contributing to the overarching research topic. Therefore, after I finished the data collection and analysis of each paper, I started investigating the overall reflections across the papers. I could have stopped at this point of the study and handed in my dissertation. However, since the start of my phd journery, I used to think about the importance of building an urban planning theory from an understudied planning context like Cairo (and the Global South). 

Accordingly, it was important for me to be able to generalise the research findings beyond Cairo. Therefore, it was important to not only compare the results from the sub-cases but also to provide abstract reflections and inference. 

To do so, an analysis beyond the empirical sub-cases was needed to provide abstract reflections. Accordingly, after having a first layer of analysis to analyze the data within the papers, and then having a second layer of analysis to analyze and compare the data across the papers, I implemented an extra third round of analysis with the aim of generating more meta findings, moving beyond the papers and their comparisons. This made the generalizability of the study’s findings more possible as they became more abstract and transferable, and I think this was one of the main reasons for being able to achieve a summa cumm laude in the German PHD system. 

The decision to add an extra layer of analysis beyond the analysis of the empirical data helped me to gain a more meta level insight into the research topic and, in turn, helped me to understand the golden thread that connected the five papers. Hence, allowed me to claim the transferability of the research findings. 

Decision 3: Choosing to build a Middle Range Theory to think beyond the empirical case

One can say that research lies in a spectrum between being too empirical (applicable) or too theoretical. The more empirical a research is, the less abstract it is and hence the less generalisable it can be . As such, the more theoretical and abstract a research is, one can say that it is the more it is preceieved to be generalisable.

As I wanted to move beyond the empirical nature of my papers, I found out about Middle Range Theories (MRT), which refer to “theories that lie between the minor but necessary working hypotheses that evolve in abundance during day-to-day research and the all-inclusive systematic efforts to develop a unified theory that will explain all the observed uniformities of social behavior, social organization, and social change”  (Merton, 1968, p. 39). Based on this understanding of MRT, the goal of the previously mentioned extra layer of analysis was to build one. MRTs are in fact thought of as generalizable (Robb, 2015), as they postulate “mechanistic explanations” (Bunge, 1996) that illustrate ‘the ways of acting of things’ (Bhaskar, 1978, p. 14).

Hence, my choice to build a MRT enabled me to think beyond the grounded empirical specificities of my sub case studies, to build a theory that explains the researched phenomena. 

Decision 4: Using the analytical techniques of the grounded theory method to reach conceptual generalizations

To develop a MRT theory, I had to use specific analytical techniques to keep the research process scientific. Therefore, I used the analytical techniques of the grounded theory (GT) method because the GT is mainly about grounded data (which was available in the empirical case studies in the papers). According to Glaser – one of the prominent scholars in GT, the analytical techniques of GT enable the abstraction of the ‘particular’ via conceptual generalizations instead of descriptive generalizations (Glaser, 2006). 

Generalizability is widely seen as the ability to recollect and reanalyze data and achieve the same results (Ferguson, 2004), . Glaser claims that regenerating data from a field is in fact not possible. He adds that what could actually be possible, is being able to conceptualize descriptions, which is possible GT (Glaser, 2006; Masco, 2007). This is because the grounded theory encourages its adopters to see beyond the field of their work to provide conceptualizations that are “timeless in their applicability” (Glaser, 2002, p. 319). [2]

The choice to use the analytical techniques of the grounded theory method, in combination with the goal of building a mid-range theory,  helped me to see beyond my field in Cairo on higher conceptual levels, which is crucial for generalization. 

Decision 5: Adopting a retroductive reasoning logic (research strategy) to reflect back and forth on the data

Most research adopts inductive (data to theory), deductive (general /theory to particular/ data), or abductive reasoning logic (theory <-> data) for investigation. [3] 

As my research design was cumulative and very iterative, I could neither follow an indicative nor a deductive research strategy. Therefore, I  chose to follow a retroductive reasoning logic as my research strategy. 

Retroduction refers to “the entire abductive–deductive–inductive cycle” (Chiasson, 2001, para. 54).  Glynos & Howarth (2018, p. 9) describe retroduction as a “cycle – a kind of restless ‘spiral’– because as we move from one ‘moment’ to the next, and back again, revising aspects of our account in light of adjustments made in other moments, we never return to the same spot”. In line with my goal to build a new theory, this reasoning logic allows for producing new knowledge and generating theory (Danermark, et al., 2002; Lawson, 1997; Meyer & Lunnay, 2013), [4] which accordingly increases transferability and generalizability. Scholars argue that retroduction helps in developing theories that are transferable. The adoption of retroduction allows for generalizing cross cases (Mukumbang, 2020; Byng et al., 2005).

In order to move beyond the empirical cases, using retroduction, I kept reflecting on the data and findings of the three layers of analysis mentioned above back and forth. As Tsang (2014b, p. 6) states, “when a finding is observed in more than one case, its generalizability is enhanced.” The choice to not use inductive nor deductive reasoning logic, but instead implement teroductive inference helped me to derive more and broader generalizations.

Decision 6: Conceptualizing Cairo as a glocal case to see beyond geographic boundaries 

During my research, I realized that the case of planning visualizations in the context of Cairo is not merely a local case study but also global. Inspired by previous interdisciplinary work and discussions during my PhD journey, I started to not only regard and deal with Cairo as a Global South context, but instead as what I framed as a glocal (local yet global) context. This perception allowed me to see the  research problem in relation to wider networks of mediatization, digitalization, and urban neoliberalism that is happening in many parts of the world.         

The decision to conceptualize Cairo as a glocal case allowed me to view the topic beyond the geographic constraints of the Global North and South.

A note on the six decisions

The above six decisions were made to claim the transferability and generlizability of my research findings and the proposed theory. While all the decisions are linked and affect (and are affected by) one another, it is noteworthy to mention that the decisions were not made in chronological order. I can say that, overall, it was a cumulative iterative process of decision making.

Boundaries of generalization

Although I have made the above six decisions to allow for more generalization of my research findings, I am aware that generalizability has its limits and that it is important to understand that there are bounderies to it that need to be approached with caution so as not to claim universality. This is a particular problem of knowledge colonialism and uneven knowledge production in the field of urban planning (see, among others, Watson, 2016). This can be reflected upon using Whetten’s (1989) simple illustration of what is termed “boundary conditions” to generalization, referring to the   “Who, Where, When ” of the findings. These conditions influece the range in which (theoretical) predictions can be most accurate (Busse et al., 2017). 

It is accordingly incumbent upon researchers to describe (1) what factors and findings in a specific context/topic are found elsewhere outside the studied case/context and (2) what is distinctive and special to the studied case/context and hence can not be generalized. It accordingly is crucial to frame the case (and the context) and in turn discuss what can or cannot be generalized  beyond them ref.  In that sense, one can say that even though many scholars claim that it is hard to achieve full generalizability, partial generalizability can be achieved by making the necessary decisions. 

So, which parts of your work are generalizable to the world and which findings are specific to the research context/case? This is a question that is worth thinking about in one’s own research. 

Author info

Dr.-Ing. Mennatullah Hendawy is an interdisciplinary urban planner working on the intersection of cities and technology toward equity and sustainability. She received her Ph.D. in Planning Building Environment from TU Berlin, Germany, with a summa cum laude. Before pursuing her doctorate studies, in 2015, Mennatullah completed an MSc. in Integrated Urbanism and Sustainable Design with a focus on urban policies from Stuttgart University. Hendawy holds a Bachelor of Science in Architectural Engineering from Ain Shams University in Cairo, Department of Urban planning and design. she was also a research associate at the Chair of Urban Design, TU Berlin. Between 2017-2021, Hendawy was a visiting researcher at the Leibniz Institute for Research on Society. Since 2012, she is affiliated with the Department of Urban Planning and Design at Ain Shams University in Cairo, Egypt.

Digital Object Identifier (DOI)

https://doi.org/10.5281/zenodo.7290759

Cite as

Hendawy, M. (2022). Achieving (Partial) Generalizability. Elephant in the Lab. DOI: 10.5281/zenodo.7290759

References

Collapse references

Bhaskar, R. (1978) A Realist Theory of Science, 2nd edn (Brighton, The Harvester Press).

Byng, R., Norman, I., & Redfern, S. (2005). Using realistic evaluation to evaluate a practice-level intervention to improve primary healthcare for patients with long-term mental illness. Evaluation, 11(1), 69–93. https://doi.org/10.1177/1356389005053198

Bunge, M. (1996) Finding Philosophy in Social Sciences. New Haven, CT and London: Yale University Press .

Busse, C., Schleper, M.C., Weilenmann, J. and Wagner, S.M., 2017. Extending the supply chain visibility boundary: Utilizing stakeholders for identifying supply chain sustainability risks. International Journal of Physical Distribution & Logistics Management.

Danermark, B., Ekstrom, M., Jakobsen, L. and Karlsson, J. (2002) Explaining society: critical realism in the social sciences. London: Routledge.

Eisenhardt, K. M. (1989) ‘Building theories from case study research’, Academy of Management Review, 14, pp. 532–550.

Ferguson, L. (2004) External validity, generalizability, and knowledge utilization. J Nurs Scholarsh, 36, 1622.

Flyvbjerg, B., 2012. Five misunderstandings about case study research, corrected. Qualitative Research: The Essential Guide to Theory and Practice, London and New York: Routledge, pp.165-166.

Gerring, J. (2007). The case study: what it is and what it does. In Boix, C. and Stokes, S.C. (eds), Oxford Handbook of Comparative Politics. New York, NY: Oxford University Press, pp. 90–122.Ferguson, L., 2004. External validity, generalizability, and knowledge utilization. Journal of Nursing Scholarship, 36(1), pp.16-22.

Glaser, B. (2002) Constructivist grounded theory? Forum Qualitative Social Research, 3(3). Available online at: http://www.Qualitative-Research.net/fqs/

Glaser, B. (2006) ‘Doing Formal Grounded Theory’, In A. Bryant and K. Chamaz (eds.) The SAGE Handbook of Grounded Theory. London: Sage, pp. 96-113.

Glynos, J. and Howarth, D. (2018) ‘The Retroductive Cycle: The research process in poststructuralist discourse analysis’, In Marttila, T (ed.) Discourse, Culture and Organization:
Inquiries into Relational Structures of Power. London: Palgrave, pp. 105-125.

Hendawy, M. (2022) Spatio-Visual Co-constructions: Communication and Digitalization of Urban planning in a mediatized world, Cairo as a glocal case. PhD thesis, TU Berlin

Lawrence, J. (2010) ‘The factors that influence adoption and usage decision in SMEs: evaluating interpretive case study research in information systems’, Electronic Journal of Business Research Methods, 8(1), pp. 25–36.

Merton, R. (1968) Social theory and social structure (3rd ed). Glencoe, Illinois: Free Press.

Meyer, S. B. and Lunnay, B. (2013) ‘The Application of Abductive and Retroductive Inference for the Design and Analysis of Theory-Driven Sociological Research’, Sociological Research Online, 18(1), pp. 86-96.

Mukumbang, F.C., 2020. The need for retroductive thinking in implementation sciences.

Robb, J. (2015) ‘What Do Things Want? Object Design as a Middle Range Theory of Material Culture’, Archaeological Papers of the American Anthropological Association, 26, pp. 166-180. Lawson, T. (1997) Economics and Reality. London: Routledge.
Schwandt, T.A. (1997). Qualitative Inquiry. Thousand Oaks, CA: Sage.

Steinmetz, G. (2004). Odious comparisons: incommensurability, the case study, and ‘small N’s’ in sociology. Sociological Theory, 22, pp. 371–400.

Stoecker, R. (1991). Evaluating and rethinking the case study. Sociological Review, 39, pp. 88–112.

Tsang, E.W., 2014a. Case studies and generalization in information systems research: A critical realist perspective. The Journal of Strategic Information Systems, 23(2), pp.174-186.

Tsang, E.W., 2014b. Generalizing from research findings: The merits of case studies. International Journal of Management Reviews, 16(4), pp.369-383.

Watson, V. (2016) ‘Shifting Approaches to Planning Theory: Global North and South’, Journal of Urban Planning, 1(4), pp. 32-41. doi:10.17645/up.v1i4.727

Whetten, D. A. (1989). What constitutes a theoretical contribution? Academy of Management Review, 14(4), 490-495.

Yin, R.K., 2009. Case study research: Design and methods (Vol. 5). sage.

0 Comments

Continue reading

What happens to science when it communicates?

What happens to science when it communicates?

In August 2023 Benedikt Fecher conducted an interview with Clemens Blümel from the German Centre for Higher Education Research and Science Studies (DZHW) on the topic of ‘what happens when science opens up and communicates’ and the emerging challenges for future scientific communication.