WHY “WHAT WORKS” WON’T WORK: EVIDENCE-BASED PRACTICE AND THE DEMOCRATIC DEFICIT IN EDUCATIONAL RESEARCH

Introduction: Education as an Evidence-Based Practice

The idea that education should be or become an evidence-based practice and that teaching should be or become an evidence-based profession has recently come to prominence in several countries around the world.¹ In Britain the push for evidence-based education partly came in the wake of critical reports about educational research that were commissioned by the Department for Education and Employment (the Hillage Report) and the Office for Standards in Education (OFSTED).² These reports vented serious doubts about the quality and relevance of educational research, arguing, among other things, that educational research did not provide answers to the questions the government asks in order to develop educational policy; that it did not provide educational professionals with clear guidance for their work; that it was fragmented, noncumulative, and methodologically flawed; and that it often was tendentious and politically motivated.³

Questions about the quality and relevance of educational research were not only raised by policy makers and educational practitioners, but they also came from within the educational research community itself. For example, in a lecture on teaching as a research-based profession, David Hargreaves accused educational research of not having generated the cumulative body of relevant knowledge that would enable teaching to become a research-based profession.⁴ It is important to see that Hargreaves’s criticism was not only directed at educational research but also implied a message for educational practice. On the one hand, his criticism suggested that educational research should not be left to educational researchers but should be subject to centralized agenda-setting, both with respect to its contents and its methods, so that it can become more practically relevant. At the very same time, however, his criticism suggested that educational practice should not be left to the opinions of educators but that their work should be based upon research evidence. Hargreaves called, in other words, for a transformation of educational research so that educational practice could be transformed into an evidence-based practice. The call for a double transformation of both educational research and educational practice lies at the very heart of the idea of evidence-based education.⁵

In Britain the call for the transformation of educational research and practice has led to a range of initiatives aimed at narrowing the gap between research, policy, and practice. Among these are attempts to synthesize the findings of educational research through the conduct of systematic research reviews (for example, the work of the Evidence for Policy and Practice Information and Co-ordinating Centre at the Institute of Education in London⁶), and attempts to make the outcomes of research more readily available to different educational constituencies (for example, Evidence-Based Education UK [EBE Network], a network for teachers who want to know “what works” in education⁷). It also includes attempts to centrally set the agenda for educational research, both with respect to its contents and its methodology. Regarding the latter, there is a strong push for experimental research that, according to proponents of evidence-based education, is the only method capable of providing secure evidence about “what works.”⁸

Similar concerns about the quality and impact of educational research have been raised in the United States, and the implications of these discussions have been far more dramatic than in Britain and have, according to some, radically changed the landscape of educational research.⁹ Although the idea that research in education should be able to tell us “what works” had already been articulated in the 1980s,¹⁰ it was not until the late 1990s that this way of thinking began to shape legislation regulating federal research funding. Since the reauthorization in 2001 of the Elementary and Secondary Education Act (“No Child Left Behind”), the “gold standard” of randomized controlled field trials seems to have become the preferred — if not prescribed — methodology for educational research.¹¹ Although there is some indication of the emergence of a broader and more encompassing definition of what counts as scientific research in education,¹² the call for causal analysis by means of experimental research in order to find out “what works” remains dominant.¹³

The case for evidence-based practice in education has generated much discussion on both sides of the Atlantic. Proponents of evidence-based education stress that it is about time that educational research starts to follow the pattern that has created “the kind of progressive, systematic improvement over time that has characterized successful parts of our economy and society throughout the twentieth century, in fields such as medicine, agriculture, transportation, and technology.” They suggest that the “most important reason for the extraordinary advances in medicine, agriculture, and other fields is the acceptance by practitioners of evidence as the basis for practice,” and particularly the randomized controlled trial that can establish “beyond reasonable doubt the effectiveness (or lack thereof) of treatments intended for applied use.”¹⁴ Some proponents go as far as to say that any practice not based upon scientific knowledge is inferior and should ultimately be banned. The EBE Network’s Manifesto for Evidence-Based Education states, for example, that education “is too important to allow it to be determined by unfounded opinion, whether of politicians, teachers, researchers or anyone else.” They call for a culture “in which evidence is valued over opinion” and argue that any approach to decision making that is not evidence-based is simply “pre-scientific.” In a similar vein they write that “the ease with which politicians, policy makers — and even teachers — have been able to get away with implementing their prejudices without even token consideration of the evidence, let alone engaging in a serious and informed debate about its quality and importance, is a disgrace.”¹⁵

Opponents of the idea of evidence-based education have raised many questions about the appropriateness of the evidence-based approach for the field of education. Some have questioned the homology between education and medicine and have pointed to the different meanings of evidence in these fields.¹⁶ Others have questioned the positivistic assumptions underlying the idea of evidence-based education and have criticized the narrow conception of research entailed in evidence-based education.¹⁷ Still others have criticized the managerial agenda of evidence-based education and its linear, top-down approach to educational improvement.¹⁸ Finally, many have objected to the lack of an acknowledgment of the crucial role of values in educational research and practice.¹⁹

One positive outcome of these ongoing discussions is that some proponents of an evidence-based approach in education have begun to talk in a more nuanced way about the link between research, policy, and practice, using notions such as “evidence-informed,”“evidence-influenced,” and “evidence-aware” practice.²⁰ While this does suggest a certain understanding of the complex ways in which research might inform policy and practice,²¹ many still maintain that the only road toward real improvement in education is to be found in the “gold standard” of the randomized controlled field trial as the methodology that can show us the effectiveness of treatments “beyond reasonable doubt.”²²

In this essay I wish to take a critical look at the idea of evidence-based practice and the ways in which it has been promoted and implemented in the field of education. Although I do believe that there is scope for improvement of the ways in which educational research and educational practice communicate and interact — an issue that has been central ever since education became an academic discipline²³— I am not convinced that evidence-based practice as it is currently being presented and promoted provides the most appropriate matrix for addressing this issue. I am particularly concerned about the tension between scientific and democratic control over educational practice and educational research. On the research side, evidence-based education seems to favor a technocratic model in which it is assumed that the only relevant research questions are questions about the effectiveness of educational means and techniques, forgetting, among other things, that what counts as “effective” crucially depends on judgments about what is educationally desirable. On the practice side, evidence-based education seems to limit severely the opportunities for educational practitioners to make such judgments in a way that is sensitive to and relevant for their own contextualized settings. The focus on “what works” makes it difficult if not impossible to ask the questions of what it should work for and who should have a say in determining the latter.

To develop my argument I will examine three key assumptions of evidence-based education. I will first ask to what extent the practice of education can be compared to the practice of medicine, the field in which the idea of evidence-based practice was first developed. I will then look at how we should understand the role of (research) knowledge in professional action, giving special attention to the question of what kind of epistemology is most appropriate for professional practices informed by the outcomes of research. Third, I will look at the expectations about the practical role of research that are implied in the idea of evidence-based education. All three cases raise questions about the nature of educational decision making and the role research can play in such processes. All three cases also raise questions about who is and who should be allowed to participate in decisions about what is educationally desirable. Evidence-based practice provides a framework for understanding the role of research in educational practice that not only restricts the scope of decision making to questions about effectivity and effectiveness but that also restricts the opportunities for participation in educational decision making. This is why, as I will argue in my conclusion, we need to expand our views about the interrelations among research, policy, and practice in order to keep in view the fact that education is a thoroughly moral and political practice, one that needs to be subject to continuous democratic contestation and deliberation. From this point of view an exclusive emphasis on “what works” will simply not work.

Professional action in education

The idea of evidence-based practice has its origins in the field of medicine. It was initially developed to teach medical students, but evidence-based medicine has rapidly become the main paradigm in clinical practice and clinical decision making. In addition to the spread of evidence-based practice from medicine to most other health fields (such as dentistry, nursing, physiotherapy, and occupational therapy), it has also been advocated and adopted in more distant fields of professional activity, such as social work, probation, human resource management, and, last but not least, education.²⁴

Although evidence-based practice may at first sight seem to provide an attractive framework for bringing research and professional practice closer together, there is a real question as to whether it offers a neutral framework that can simply be applied to any field of professional activity, or whether it is a framework that brings with it a particular view of professional practice.²⁵ If the latter is the case — and I will argue below that it is — we must consider whether this framework is appropriate for the field of education.

What is the model of professional action that is implied in evidence-based practice? Central to evidence-based practice is the idea of effective intervention.²⁶ Evidence-based practice conceives of professional action as intervention, and looks to research for evidence about the effectiveness of interventions. Research needs to find out, in other words, “what works,” and the main if not the only way of doing this, so it is often argued, is through experimental research, most notably in the form of randomized controlled trials. As one proponent of evidence-based education writes:

The Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) at the Institute of Education in London also argues that the current interest in evidence-informed education “is part of a general move in the UK and elsewhere towards basing policy and professional practice on sound evidence of effectiveness.”²⁸ Martin Oliver and Grainne Conole summarize the situation as follows:

The idea of professional action as effective intervention suggests, first, that evidence-based practice relies upon a causal model of professional action.³⁰ It is based on the idea that professionals do something — they administer a treatment, they intervene in a particular situation — in order to bring about certain effects. This is why the question of effectiveness, the question of “what works,” is so central in the whole discussion about evidence-based practice. Effective interventions are those in which there is a secure relation between the intervention (as cause) and its outcomes or results (as effects). It is important to note that “effectiveness” is an instrumental value: it refers to the quality of processes but does not say anything about what an intervention is supposed to bring about. This means, among other things, that it is meaningless to talk about effective teaching or effective schooling; the question that always needs to be asked is, effective for what?

This relates to the second characteristic of the idea of professional action as intervention, which is that evidence-based practice appears to rely on a separation between the means and ends of professional action.³¹ Evidence-based practice assumes that the ends of professional action are given, and that the only relevant (professional and research) questions to be asked are about the most effective and efficient ways of achieving those ends. In this respect, evidence-based practice entails a technological model of professional action.

While both assumptions may be valid in the field of medicine — although I must add that I think they are only valid given a particular conception of medicine and a particular definition of health³²— I do not think that they can easily be transposed to the field of education. This first problem with this approach is the role of causality: apart from the obvious fact that the condition of being a student is quite different from that of being a patient — being a student is not an illness, just as teaching is not a cure — the most important argument against the idea that education is a causal process lies in the fact that education is not a process of physical interaction but a process of symbolic or symbolically mediated interaction.³³ If teaching is to have any effect on learning, it is because of the fact that students interpret and try to make sense of what they are being taught. It is only through processes of (mutual) interpretation that education is possible.³⁴ Despite the attempts of many to transform education into a causal technology (often based on the idea that we only need more research in order to find and ultimately control all the factors that determine learning), the simple fact that education is not a process of “push and pull”— or, in the language of systems theory, that education is an open and recursive system — shows that it is the very impossibility of an educational technology that makes education possible.³⁵

One may object that this argument is only valid with respect to the content of education, and that it breaks down when we look at other “factors” that influence learning, such as teaching style or group size or even the architecture of schools. Yet, while it may be possible to find strong correlations between these factors and learning, this still does not prove that the relation between the two is a causal one. The underlying “mechanism-which-is-not-a-mechanism” that provides the connection between “input” and “output” is to be found in the interpretations of the learner, in the diverse ways in which learners make sense of the situations they encounter. While we may want to refer to the activities of teachers as interventions — and one could argue that teaching always intervenes in some way or another in an existing course of events — we should not think of these interventions as causes but as opportunities for students to respond and, through their response, to learn something from them.³⁶

This brings me to the second assumption about professional action implied in evidence-based practice: the idea that education can be understood as a technological process in which there is a clear separation between means and ends, and in which it is assumed that the ends are given and the only relevant (professional and research) questions pertain to the most effective and efficient way of achieving these ends. There are two problems with applying this line of thinking to education. The first is that even if we were able to identify the most effective way of achieving a particular end, we might still decide not to act accordingly. A substantial amount of research evidence suggests that the most influential factors in school success are the home environment and, even more important, children’s experiences in their first years. This would suggest that the most effective way to achieve success in education would be to take children away from their parents at an early age and put them in an “ideal” environment. Although many strategies try to intervene in the home environment and to shape children’s experience in their earliest years, most societies find it undesirable to choose the most effective road toward educational achievement. This shows that knowledge about the effectiveness of interventions is not, as such, a sufficient basis for decisions about educational action. There is always the question as to whether particular interventions are desirable.³⁷

In the case of education (and this is the second problem with applying means/ends thinking to it) we not only need to ask whether our educational activities, strategies, and — if one wishes to use the word — interventions are desirable in themselves; we also always need to ask what are the educational effects of our actions. We may well have conclusive empirical evidence that in all cases physical punishment is the most effective way of deterring or controlling disruptive behavior. Yet, as David Carr argues, “the practice should nevertheless be avoided because it teaches children that it is appropriate or permissible in the last resort to enforce one’s will or get one’s own way by the exercise of violence.”³⁸ This example shows that in education means and ends are not linked in a technological or external way but that they are related internally or constitutively. The means we use in education are not neutral with respect to the ends we wish to achieve. It is not the case that in education we can simply use any means as long as they are “effective.” As Carr makes clear, the means we use “contribute qualitatively to the very character…of the goals which they produce.”³⁹ This is why education is at heart a moral practice more than a technological enterprise.⁴⁰

These considerations suggest that the model of professional action implied in evidence-based practice — that is, the idea of education as a treatment or intervention that is a causal means to bring about particular, preestablished ends — is not appropriate for the field of education. What is needed for education is a model of professional action that acknowledges the noncausal nature of educational interaction and the fact that the means and ends of education are internally rather than externally related. What is needed, in other words, is an acknowledgment of the fact that education is a moral practice, rather than a technical or technological one — a distinction that dates back to Aristotle’s distinction between phronesis (practical wisdom) and techne (instrumental knowledge).⁴¹ The most important question for educational professionals is therefore not about the effectiveness of their actions but about the potential educational value of what they do, that is, about the educational desirability of the opportunities for learning that follow from their actions (and what should be prevented at all costs is the situation in which there is a performative contradiction between what they preach and what they practice). This is why the “what works” agenda of evidence-based practice is at least insufficient and probably misplaced in the case of education, because judgment in education is not simply about what is possible (a factual judgment) but about what is educationally desirable (a value judgment). Ian Sanderson summarizes the problems with the instrumental rationality underlying evidence-based practice in the following way:

Professional action in education and, as Sanderson makes clear, in many other professional fields always needs to take the normative dimension into consideration. Professionals need to make judgments about “the most appropriate course of action in the specific circumstances in a context of informal rules, heuristics, norms and values.”⁴³ Sanderson therefore concludes that “the question for teachers is not simply ‘what is effective’ but rather, more broadly it is, ‘what is appropriate for these children in these circumstances.’”⁴⁴ To suggest that research about “what works” can replace normative professional judgment is not only to make an unwarranted leap from “is” to “ought”; it is also to deny educational practitioners the right not to act according to evidence about “what works” if they judge that such a line of action would be educationally undesirable.⁴⁵

Professional judgment and practical epistemology

The conclusion that professional judgment is central to educational practice, and that the nature of this judgment is moral rather than technical, does not imply that professional judgment in education may not be informed by the outcomes of educational research. So far, the only conclusion that has been reached is that education should be understood as a moral, noncausal practice, which means that professional judgments in education are ultimately value judgments, not simply technical judgments. The second issue that I want to explore, therefore, is how we should understand the way in which research outcomes may affect educational practice. For this we need to turn to epistemological questions — and it is remarkable how little attention has been paid to this dimension in the discussion so far.⁴⁶ The main question here is what kind of epistemology might be appropriate for an adequate understanding of the role of knowledge in (professional) action. To develop an answer to this question, I will take a closer look at the work of John Dewey who, in my view, has developed one of the most powerful and sophisticated “practical epistemologies” available in Western philosophy.⁴⁷

Before I do so, I want to emphasize that evidence-based practice represents a rather broad spectrum of ideas about how the link between research/evidence and practice should be understood and established. At one extreme, there are those who think that research will be able to give us “the truth,” that “the truth” can be translated into rules for action, and that the only thing practitioners need to do is to follow these rules without any further reflection on or consideration of the concrete situation they are in. In England, the National Literacy Strategy and the National Numeracy Strategy (introduced by the government to “raise standards” in English and mathematics in primary schools) exemplify such a cookbook approach. (The fact that it is increasingly becoming clear that these strategies are not able to achieve what they were established to achieve is an indication that there may be something seriously wrong with this approach.) Evidence-based medicine takes a more considered approach to the use of research evidence in medical practice, which is mainly due to the fact that evidence-based medicine was developed in the context of clinical problem solving. David Sackett et al.’s definition of evidence-based medicine clearly shows that research evidence is seen as one factor in a process of clinical decision making, rather than the only factor to drive clinical practice:

Elsewhere they write:

In education the picture appears to be less nuanced. While it is acknowledged that individual teachers may need and want to adapt research findings to their particular situation, proponents of evidence-based education argue that in the case of policy, that is, in the situation

What is most significant here is that although there may be different views about how research can and should be used in educational practice, there seems to be an almost unanimous expectation that research can tell us “what works,” that it can provide “sound evidence” about the likely effects of policy and practice, and “sound evidence of effectiveness” more generally. Whether these expectations are warranted ultimately depends on the epistemological assumptions one brings to the understanding of what research can achieve. It is here that Dewey’s ideas are relevant, with respect to both what we can expect from research and how research can be “used” in educational practice.

The most important aspect of Dewey’s theory of knowing lies in the fact that it is not premised on the dualism between immaterial mind and material world — a dualism that has been the framework for modern epistemology at least since René Descartes divided reality into res cogitans (the knowing “stuff”) and res extensa (the “stuff” that occupies space). Dewey offered a theory of knowing that does not start with the impossible question of how “a knower who is purely individual or ‘subjective’ and whose being is wholly psychical and immaterial…and a world to be known which is purely universal or ‘objective’ and whose being is wholly mechanical and physical” can ever reach each other.⁵¹ Instead, he approached the question of knowledge from within an action-theoretical framework, one in which knowing is understood as “a way of doing”— which is why we may want to refer to Dewey’s position as a theory of knowing and not a theory of knowledge.⁵²

The central concept in Dewey’s theory of knowing is the notion of experience. Experience is not about consciousness or mental awareness but refers to the transactions of living organisms and their environments. What is distinctive about these transactions is that they constitute a double relationship:

According to Dewey, experience is the very way in which living organisms, including human organisms, are connected with, are part of, and are involved in “the world.” Contrary to what is suggested in the dualistic worldview, experience is not“a veil that shuts man off from nature,” but “a means of penetrating continually further into the heart of nature.”⁵⁴ This is the central insight of Dewey’s “transactional realism.”⁵⁵ Dewey’s transactional understanding of experience provides a framework in which knowing is no longer about an immaterial mind looking at the material world and registering what goes on in it — a view to which Dewey referred as the spectator theory of knowledge. For him, knowing is not about a world “out there,” but concerns the relation between our actions and their consequences — which is the central idea of Dewey’s transactional theory of knowing.⁵⁶

Because knowing is about grasping and understanding the relation between our actions and their consequences, knowing can help us to gain better control over our actions — better at least, that is, than in the case of blind trial and error. “Where there is the possibility of control,” Dewey observed, “knowledge is the sole agency of its realization.”⁵⁷ It is important to see that “control” here does not mean complete mastery, but the ability to intelligently plan and direct our actions. This ability is most important in those situations in which we are not sure how to act. This is expressed in Dewey’s definition of knowing as having to do with “the transformation of disturbed and unsettled situations into those more controlled and more significant.”⁵⁸ Knowing is also important, however, in order to achieve a more intelligent approach in the other domains of experience. This is expressed in Dewey’s claim that knowing “facilitates control of objects for purposes of non-cognitive experience.”⁵⁹

The main reason Dewey’s transactional theory of knowing is important to our discussion is that it provides us with a framework for understanding the role knowledge plays in action. To understand Dewey’s approach, it is important to see that we do not need to have any knowledge at all in order to act. We do not need to have information about “the world” before we can act in it. As living organisms, we simply are always already active; we simply are always already in transaction with our environment. This does not mean, of course, that we do not learn as a result of our transactions with the world. The whole idea of experience is precisely that we undergo the consequences of our “doings” and that we change as a result of this. Dewey explained that experience results in “change in the organic structures that conditions further behavior.”⁶⁰ He referred to such changes as habits. On this view habits are not patterns of action, but predispositions to act:

Habits therefore are “the basis of organic learning.”⁶²

We basically acquire our habits through processes of trial and error — or, in more theoretical language, through experimentation. In a very fundamental sense, experimentation is the only way in which we can learn anything at all: we learn because we do and subsequently undergo the consequences of our doings. Yet, for Dewey, there is a crucial difference between blind trial and error — experimentation without deliberation and direction — and what he calls intelligent action. The difference between the two has to do with the intervention of thinking or reflection, that is, with the use of symbolic operations.

To understand Dewey’s ideas about the role of thinking in action, it is important to see that we only learn, or acquire new habits, in those situations in which the organism-environment transaction is interrupted. After all, as long as the transaction goes on “smoothly”— that is, when there is coordination between our doings and undergoings — we apparently have all the habits we need. The situation is different when we are not able to maintain coordinated transaction, when we do not “know,” in other words, how to respond. In everyday language, it is the situation in which we encounter a problem. Dewey referred to such cases as indeterminate situations. To resolve these situations we need to find an appropriate response, that is, a line of action that will restore coordination. Put differently, in order to find the appropriate response, we must know what the problem is. Yet, according to Dewey, finding the solution and finding the problem are two sides of the same coin, because we will only know what the problem really was after we have found an adequate response.

One way to find an adequate response is through trial and error. Sometimes this will be successful; sometimes it will not. Apart from the fact that trial and error may not be a very efficient way of problem solving, there is also the risk that some attempts to solve the problem may be irreversible, which means that if those attempts do not solve the problem, we may not be able to solve the problem at all. The way out of this predicament, according to Dewey, is through experimentation with different lines of action at a symbolic level rather than through overt action. This is precisely what thinking does: it is the “dramatic rehearsal (in imagination) of various competing possible lines of action.”⁶³ The choice of a specific line of action should be understood as “hitting in imagination upon an object which furnishes an adequate stimulus to the recovery of overt action.”⁶⁴ Whether this choice will actually lead to coordinated transaction — whether the problem will be solved — will, of course, only become clear when we actually act. Thinking, deliberation cannot solve problems, nor can it guarantee that the chosen response will be successful. What it can do is make the process of choosing more intelligent than it would have been in the case of blind trial and error.

It is precisely because our experimental problem solving is embedded in symbolic operations — in thinking and deliberation, in language, stories, theories, hypotheses, explanations, and the like — that we not only learn at the level of our habits; we also add to our “symbolic resources” for addressing future problems. We could say that we have gained knowledge, as long as we do not forget that this is not knowledge about “the world” but about the relations between our actions and their consequences in this particular situation. After all, according to Dewey’s transactional framework, this is the one and only way in which the world will ever “appear” to us.

The foregoing account of reflective experimental problem solving — a process that Dewey called inquiry— is the matrix for Dewey’s account of the acquisition of knowledge. It reveals why, on his view, knowing is something we literally do, since we only ever acquire knowledge as a result of our actions. Yet knowledge, understood as a symbolized account of the relation between our actions and their consequences, only comes about when we embed our “existential operations” in “symbolic operations”— when, in other words, we embed our actions in thinking, deliberation, and theorizing. One of the main implications of this view is that inquiry, or research, does not provide us with information about a world “out there,” but only about possible relations between actions and consequences. In the case of everyday problem solving, we learn about possible relations between our actions and their consequences. In the case of randomized controlled trials, we learn about possible relations between experimental treatments and measured results. In neither case, however, do we learn truths about a world “out there.” Rather, both types of inquiry yield “warranted assertions” about relations between what we did and what followed from it.⁶⁵ This means that inquiry and research can only tell us what is possible — or, to be even more precise, they can only show us what has been possible. Research, in short, can tell us what worked but cannot tell us what works.

Dewey’s account of the process of inquiry is, however, not only an account of how we acquire knowledge. It is at the very same time an account of how we solve problems. From the latter point of view, Dewey’s account also provides us with a model of professional action and, more important, with a view of the role of knowledge in action. There are three important aspects of Dewey’s account. First, for Dewey professional action is not about following tried and tested recipes, but about addressing concrete and, in a sense, always-unique problems. Dewey’s transactional view implies that although there is structure, form, and duration in our transactions with the world, we cannot and should not expect that situations will stay the same over time, and we should definitely not expect this in the social realm.

Second, it is important to see that knowledge acquired in previous situations — or knowledge acquired by others in different inquiry or research situations — does not enter the process of reflective problem solving in the form of a rule or prescription. Dewey’s assertion that “no conclusion of scientific research can be converted into an immediate rule of educational art” not only means that all research can give us is an understanding of possibilities — of what worked, not what will work.⁶⁶ It also means that in reflective problem solving we do not use “old” knowledge to tell us what we should do; we use “old” knowledge to guide us first in our attempts to understand what the problem might be and then in the intelligent selection of possible lines of action. What “old” knowledge does, in other words, is help us approach problem solving more intelligently. Yet, the proof of the pudding always lies in the action that follows. This will “verify” both the adequacy of our understanding of the problem and, in one and the same process, the adequacy of the proposed solution.⁶⁷

This may seem to suggest — and this is the third aspect — that Dewey would not object to a technological view of professional action, as long as we do not expect too much or the wrong thing from research and as long as we keep in mind that professional judgment is always about situations that are in some respect unique. But for Dewey, problem solving is not simply about finding the right means for achieving a particular end. On his view intelligent problem solving should include both means and ends. It is not only that we need to judge “existential materials” with respect to their function “as material means of effecting a resolved situation.” At the very same time, and in the same process, we need to evaluate ends “on the basis of the available means by which they can be attained.” The point of the process of inquiry is to institute “means-consequences [ends] in strict conjugate relation to each other.”⁶⁸ What we need in the process of inquiry, in other words, is what Dewey referred to as “ends-in-view.” Ends-in-view are hypotheses that are essential for the proper conduct of inquiry.

The upshot of this is that neither in our role as researcher nor in our role as professional educator should we accept given problem definitions and predetermined ends. Dewey argued convincingly that both in research and professional practice any ends are of the nature of a hypothesis and that such hypotheses have to be formed, developed, and tested “in strict correlation with existential conditions as means.”⁶⁹ Similarly, we should approach given definitions of a problem as hypotheses that may alter as a result of the inquiry process. Dewey argued, in other words, that we should not only be experimental with respect to means but also with respect to ends and the interpretation of the problems we address. It is only along these lines that inquiry in the social domain can help us find out not only whether what we desire is achievable but also whether achieving it is desirable. Dewey’s “pragmatic technology” is therefore not about social engineering or social control in the narrow sense of the word.⁷⁰ Action in the social domain can only become intelligent action when its intrinsic relation with human purposes and consequences — that is, when the political nature of inquiry in the social domain — is fully taken into account.

Dewey’s practical epistemology thus provides us with an interesting alternative for the model of evidence-based education. There are two crucial differences. First, Dewey showed that “evidence”— if such a thing exists — does not provide us with rules for action but only with hypotheses for intelligent problem solving. If, to look at it from a slightly different angle, we want an epistemology that is practical enough to understand how knowledge can support practice, we have to concede that the knowledge available through research is not about what works and will work, but about what has worked in the past. The only way to use this knowledge is as an instrument for undertaking intelligent professional action. The second difference between Dewey’s approach and traditional views of evidence-based practice is his recognition that neither research nor professional action can or should only focus on the most effective means to bring about predetermined ends. Researchers and practitioners should also engage in inquiry about ends, and this in close relation to the inquiry into means. Systematic inquiry into what is desirable is not only a task for educational researchers and educational practitioners but, in the case of education, extends to society at large. Sanderson makes a similar point when he argues for the need

A democratic society is precisely one in which the purpose of education is not given but is a constant topic for discussion and deliberation. As I have argued elsewhere, the current political climate in many Western countries has made it increasingly difficult to have a democratic discussion about the purposes of education.⁷²

The practical roles of educational research

There is one more assumption implicit within the idea of evidence-based practice that I wish to discuss briefly. This has to do with the ideas and expectations about the way in which research can be used in and be useful for educational practice. The idea behind the “what works” slogan is that research should provide information about effective strategies for educational action. I have already shown that educational practice is more than the simple application of strategies or techniques to bring about predetermined ends — there is always the question about the desirability and educational value of such strategies, and there are questions about the specificity of particular contexts in which problems need to be addressed. I have also already shown, using Dewey’s work, that research can only indicate what worked, not what works or will work, which means that the outcomes of research cannot simply be translated into rules for action. Knowledge about the relation between actions and consequences can only be used to make professional problem solving more intelligent — nothing more and nothing less. While I have argued that research should not only investigate the effectiveness of educational means but should at the same time inquire into the desirability of educational ends, evidence-based practice only focuses on the first task and, in doing so, assumes that the only way in which research can be relevant for educational practice is through the provision of instrumental or technical knowledge.⁷³

In his discussion of how social science research can be of practical relevance, Gerard de Vries refers to this particular way in which research can inform social practice as the technical role of research.⁷⁴ In the technical role, research is a producer of means, strategies, and techniques to achieve given ends. De Vries argues, however, that this is only one of the ways in which research can be practically relevant, and that there is at least one other way in which research can inform practice. This is by providing a different way of understanding and imagining social reality. He refers to the latter as the cultural role of research.

The first thing that is important about de Vries’s distinction is that it allows us to see that the provision of instrumental knowledge is not the only way in which educational research can inform and be beneficial for educational practice. While there is an important task for educational research in finding, testing, and evaluating different forms of educational action, research can also play a valuable role in helping educational practitioners to acquire a different understanding of their practice, in helping them to see and imagine their practice differently. To see a classroom through the lens of behavioral objectives or through the lens of legitimate peripheral participation can make a huge difference. By looking through a different theoretical lens, we may also be able to understand problems where we did not understand them before, or even to see problems where we did not see them before (think, for example, of the ways in which feminist scholarship has helped us precisely to make problems visible). As a result, we may be able to envisage opportunities for action where we did not envisage them before. The cultural role of educational research is thus no less practical than the technical role. A key problem with the idea of evidence-based practice is that it simply overlooks the cultural option. It focuses on the production of means for given ends⁷⁵ and reduces research questions “to the pragmatics of technical efficiency and effectiveness.”⁷⁶ It only has a technological expectation about research.

There are two other aspects of de Vries’s distinction between the technical and the cultural role that are important for our discussion. The first point is that although the two roles can be distinguished from each other, this does not mean that they should necessarily be thought of as separate. On the one hand, de Vries shows that different interpretations often help us to see new problems and new possibilities for action and therefore can lead to different or more precise “technical” questions for further research. In this respect the cultural role of educational research can pave the way for the technical role. On the other hand, if research is successful in performing its technical role — if, in other words, research does bring about strategies and approaches that successfully solve problems — this may well convince us to see and understand the situation in terms of the framework that informs this particular approach. More often than not, therefore, the technical and the cultural approaches mutually inform and reinforce each other.

The foregoing may suggest that the technical and the cultural role are two options available to researchers to choose from. This, however, may not always be the case. De Vries argues that the role educational research can play depends to a large extent upon the micro- and macropolitical conditions under which researchers operate. In cases in which there is a strong consensus about the aims of education or, to look at it from a different angle, where the aims of education cannot be questioned, the only possible role for research is a technical role. When such a consensus does not exist, or when an existing consensus begins to break down, it becomes possible for research to play a cultural role by providing different interpretations of the situation.

This shows that whether research can play a technical or a cultural role does not solely depend upon the decisions and intentions of researchers but is influenced in a significant way by the environment in which researchers operate. De Vries connects this analysis with the idea of democracy. He suggests that a democratic society is a society in which social research is not restricted to a technical role but can also perform a cultural role. A democratic society is, in other words, characterized by the existence of an open and informed discussion about problem definitions and the aims and ends of our educational endeavors. Thus the fact that the whole discussion about evidence-based practice seems only to have technical expectations about the practical role of research is a worrisome sign from the point of view of democracy.

Conclusions

In this essay I have examined three key assumptions that underlie the idea of evidence-based education. I have examined the model of professional action that is implied in the idea of evidence-based education. I have argued that education cannot be understood as an intervention or treatment because of the noncausal and normative nature of educational practice and because of the fact that the means and ends in education are internally related. This implies that educational professionals need to make judgments about what is educationally desirable. Such judgments are by their very nature normative judgments. I have argued that to suggest that research about “what works” can replace such judgments not only implies an unwarranted leap from “is” to “ought,” but also denies educational practitioners the right not to act according to evidence about “what works” if they judge that such a line of action would be educationally undesirable. The problem with evidence-based education, therefore, is not only that it is not sufficiently aware of the role of norms and values in educational decision making;⁷⁷ the problem is that it also limits the opportunities for educational professionals to exert their judgment about what is educationally desirable in particular situations. This is one instance in which the democratic deficit in evidence-based education becomes visible.

A similar issue became clear in the discussion about the epistemological assumptions of evidence-based practice. Using Dewey’s practical epistemology, I showed that research cannot supply us with rules for action but only with hypotheses for intelligent problem solving. Research can only tell us what has worked in a particular situation, not what will work in any future situation. The role of the educational professional in this process is not to translate general rules into particular lines of action. It is rather to use research findings to make one’s problem solving more intelligent. This not only involves deliberation and judgment about the means and techniques of education; it involves at the very same time deliberation and judgment about the ends of education — and this in a strict and conjugate relation with deliberation and judgment about the means. Dewey’s practical epistemology therefore challenges the idea of evidence-based education in two ways: it challenges the way in which evidence-based education thinks about what research can achieve in relation to educational practice, and it challenges the technocratic model in which it is assumed that the discussion can and should be restrained to technical questions about “what works.” Dewey’s work helps us to see that normative questions are serious research questions in their own right — questions, moreover, that need to be part of a full, free, and open normative debate among all those with a stake in education (which includes not only those with a direct interest in education, but should include all citizens). The fact that the current matrix for evidence-based education does not seem to lead to such conclusions shows another dimension of the democratic deficit in the evidence-based approach.

The idea that the link between research, policy, and practice is not restricted to technical questions, but can also be established through the ways in which research can provide different understandings of educational reality and different ways of imagining a possible future, was central in the third step of my discussion, in which I looked at how evidence-based education conceives of the relation between research, policy, and practice. I not only suggested that evidence-based education seems to be unaware that research can play both a technical and a cultural role and that both can have very real and practical consequences. I also showed that the extent to which research can perform a technical and a cultural role can be taken as an indication of the democratic quality of a society. This is why the current climate in which governments and policy makers seem to demand that educational research plays only a technical role can and should indeed be read as a threat to democracy itself.⁷⁸

This is why there is a real need to widen the scope of our thinking about the relation between research, policy, and practice, so as to make sure that the discussion is no longer restricted to finding the most effective ways to achieve certain ends but also addresses questions about the desirability of the ends themselves. With Dewey I wish to emphasize that we always need to ask the question of whether our ends are desirable given the way in which we might be able to achieve them. In education the further question that always needs to be asked is about the educational quality of our means, that is, about what students will learn from our use of particular means or strategies. From this perspective it is disappointing, to say the least, that the whole discussion about evidence-based practice is focused on technical questions — questions about “what works”— while forgetting the need for critical inquiry into normative and political questions about what is educationally desirable. If we really want to improve the relation between research, policy, and practice in education, we need an approach in which technical questions about education can be addressed in close connection with normative, educational, and political questions about what is educationally desirable. The extent to which a government not only allows the research field to raise this set of questions, but actively supports and encourages researchers to go beyond simplistic questions about “what works,” may well be an indication of the degree to which a society can be called democratic. From the point of view of democracy, an exclusive emphasis on “what works” will simply not work.