Time for a paradigm shift in school education?
Geoff Masters AO
Australian Council for Educational Research
Progress in life is often made in small, incremental steps. But occasionally, advances are made in leaps—for example, when the penny drops and something is seen in an entirely new light. When this happens, some aspect of experience is never quite the same again.
In his book The Structure of Scientific Revolutions, Thomas Kuhn observes that progress in science, too, occurs not only in small increments, but through occasional fundamental shifts in how scientists see and understand some aspect of the world. [i] These shifts may resemble a gestalt switch; the drawing initially seen as a duck suddenly is seen as a rabbit.
Kuhn refers to such shifts in perspective as changes in the ‘paradigm’ within which scientists work. At any given time, the prevailing paradigm consists of current theories, concepts, supporting technologies, methods and literature, which together enable what Kuhn refers to as the practice of ‘normal’ science.
But occasionally normal science encounters unexpected observations that are inconsistent with existing theory. These anomalies may call into question the appropriateness and usefulness of the paradigm itself.
When this occurs, the initial response is not to reject the current paradigm, but to attempt to modify it to accommodate the problem. In Kuhn’s words, an attempt is made to force nature into the box the current paradigm provides.[ii] If this cannot be done, the existing paradigm comes under increasing pressure and, within the relevant scientific community, there may be a growing sense of crisis.
It is under these conditions that an alternative paradigm may emerge offering to better address the problem that has led the old paradigm into crisis. This proposed paradigm offers a different way of thinking and, ultimately, a new approach to ‘normal’ science. The new paradigm may incorporate concepts and terminology from the earlier paradigm, but these often take on new meanings.
Kuhn observes that the introduction of a new paradigm is always controversial. There is inevitably opposition to paradigm change, especially from those who have spent their scientific careers working within the earlier paradigm. Some scientists may never accept the new contender. Change, if it occurs, usually takes considerable time and occurs through a growing shift in the distribution of professional allegiances.
The relevance of this to school education is that schools also can be thought of as operating within a prevailing paradigm. This paradigm has theoretical underpinnings, concepts, rules and methods that guide ‘normal’ practice. However, as occasionally occurs in a field of practice, a significant mismatch currently exists between theory and observation; normal practice in school education is not delivering the results it was expected to deliver. For some, this constitutes a crisis. This mismatch is commonly attributed to factors other than the paradigm itself—for example, to underperforming teachers, underfunded schools, changes in society or a lack of student effort. While these may all be contributors, the thesis of this essay is that the schooling paradigm itself is in need of review and that the answer may lie in a shift in how we think about teaching and learning.
The Prevailing Paradigm
Most schools operate under a shared set of intentions, practices and ways of thinking about teaching and learning. In this sense, they operate within a common paradigm.
One element of these shared intentions, practices and ways of thinking is the school curriculum. The role of the curriculum is to spell out what teachers should teach and students should learn. Different curricula specify teaching and learning intentions in different degrees of detail, but curricula share a common structure. This structure mirrors the structure of schools, with a curriculum being developed for students in each year of school.
The role of teachers under the prevailing paradigm is to deliver the relevant year-level curriculum. Their task is to bring the curriculum to life, to make it interesting and engaging, and to ensure that every student is exposed to, and has an opportunity to learn, the content of the year-level curriculum.
The role of students is to learn what the curriculum specifies and teachers teach. It is accepted that some students will be more effective in this than others. Better learners will learn most of what teachers teach; less able learners will learn less. This is often seen as an inevitable consequence of natural student variability, with some students being more ‘academically inclined’ than others.
The role of assessment is to determine how well students have learnt what teachers have taught. Assessments of student learning may be made at various times during the school year and may relate to different aspects of the curriculum. Students who demonstrate most of the year-level curriculum are awarded high grades; those who demonstrate relatively little are awarded low grades.
If there is a ‘theory’ underlying this paradigm, it is that all students in a year level are able to engage meaningfully with, and benefit from, the year-level curriculum, and are capable of mastering the content of this curriculum if they make the necessary effort. It is recognised that some students may require additional support to be able to do this, and teachers are encouraged to ‘differentiate’ their teaching to provide additional assistance to students who require it. But the assumption underlying normal school practice is that all students are capable of success on year-level curricula with the requisite effort and appropriate support.
The Anomaly
The anomaly in current practice is widely recognised by teachers and those familiar with the empirical evidence. In a nutshell, observations of student learning and performance are not consistent with the assumptions and expectations of the paradigm under which schools operate.
In Kuhn’s terms, something has gone wrong.
The problem is readily apparent in well-constructed measures of student attainment such as those displayed in Figure 1. The distributions in Figure 1 were produced by a team of mathematics education researchers.[iii] The researchers began by constructing a scale for measuring levels of mathematical proficiency. This numerical scale was then divided into eight levels, with level 1 representing the lowest level of proficiency, and level 8 the highest. The resulting eight levels represent absolute levels of mathematics knowledge and skill, independent of age or year level.
They then tested samples of Australian students in each of Years 5, 6, 7, 8 and 9 and used students’ test performances to allocate them to one of the eight proficiency levels. The distributions in Figure 1 are the result of those allocations.
From Figure 1 it can be seen that, in each year group, some students performed at each of the eight proficiency levels. There was a small percentage of students in Year 5 already performing at level 8, and a small percentage of students in Year 9 still performing at level 1, and everything in between. The researchers concluded that the spread of student proficiency within each year group was ‘equivalent to seven years of schooling’.
Conclusions of this kind are reached routinely in analyses of student performance, not only in mathematics, but in all areas of learning for which reliable measures are available. A general conclusion is that, in each year of school in Australia, the difference between the tenth and ninetieth percentiles represents about five to six years of learning. In other words, the most advanced 10 per cent of students in any year group are between five and six years ahead of the least advanced 10 per cent of students. There is thus a significant mismatch between the assumption underlying normal practice—that students in the same year group are more or less equally ready for, and able to benefit from, the same year-level curriculum—and the empirical evidence.
Consequences
This mismatch has unfortunate consequences for both teaching and learning.
Currently, many students are not ready for their year-level curriculum because they lack prerequisite knowledge, skills and understandings. They commence the school year not having mastered the previous year’s curriculum. For some students, this occurs year after year, with learning gaps opening up over time. This is a direct consequence of the timed nature of the school curriculum, which specifies not only what every student is to learn, but also when they are to start learning that content, and how long they have to learn it. If students reach the end of the school year not having mastered the year-level content, they move to the next year’s curriculum regardless. As they progress through school, many students are presented with curricula that are increasingly beyond their reach.
As a result, by the time they reach 15 years of age, one in five Australian students has failed to reach even a minimally acceptable level of reading or mathematics. Another one in five does not meet a ‘proficient’ standard in these foundational skills.[iv] Many of these students have struggled with year-level curricula throughout their schooling, often slipping increasingly far behind over time. These include students at the bottom right of Figure 1.
There are also unfortunate consequences for teaching. Under the prevailing paradigm, the role of teachers is to deliver the year-level curriculum to all students in a year level. But the year-level curriculum can be a poor guide to where teachers should be focusing their teaching for some students. This is a particular problem when there is a large amount of year-level content teachers are expected to cover and so limited time to establish and respond to individual learning needs. The current paradigm often makes it difficult for teachers to meet individuals at their points of need. This is not only an issue in Australia, as one American researcher observes:
In math, when students miss key steps along the way in this progression or learn at a pace that is faster or slower than the state standards anticipate, the standards alone do not provide guidance to teachers on where to focus instruction. They signal to a seventh-grade teacher, for example, that all seventh-grade students should be taught seventh-grade content—whether they happen to be performing two years behind grade level or two years ahead. [v]
Even more seriously perhaps, normal practice in schools sends unfortunate signals to some students about learning itself and the relationship between effort and success. Less advanced students who begin the school year two or three years behind year-level expectations inevitably struggle. Most begin the year on track to receive low grades. Some receive low grades year after year. In an absolute sense, these students may be making good progress, but the current paradigm defines success not in terms of progress, but against age-based expectations. A student who receives a ‘D’ year after year not only is unable to see any absolute progress they may be making, but also may be sent a message that there is something stable about their ability to learn—they are a ‘D’ student. Many less advanced students, who may have lagged age-based expectations since before they began school, internalise this message and eventually disengage.
At the other extreme, students who begin each school year two or three years ahead of year-level expectations (top left of Figure 1) are already on track to receive high grades for the year, sometimes with relatively little effort. For these students, year-level curricula can be insufficiently challenging. In fact, there is Australian evidence that some of the lowest rates of year-on-year progress are made by some of the most advanced students, despite their high grades.[vi] For these students, too, the definition of success in terms of age-based expectations rather than progress can undermine an appreciation of the relationship between effort and success.
An Alternative Paradigm
The basis for an alternative paradigm is contained in Figure 1. Under this alternative, rather than developing a curriculum for all students in each year of school, a mathematics curriculum is developed for each of the eight proficiency levels in Figure 1. This requires a switch in focus—from the horizontal axis (time) to the vertical (proficiency). Instead of inferring what a student is ready to learn next in mathematics from their year level, this is inferred from their demonstrated level of mathematics proficiency, whatever their age.
Research in the learning sciences offers support for this alternative. In general, learning is maximised when learners are given stretch challenges appropriate to the point they have reached in their learning.[vii] As long ago as 1968, educational psychologist David Ausubel wrote, ‘If I had to reduce all of educational psychology to just one principle, I would say this: The most important single factor influencing learning is what the learner already knows. Ascertain this and teach him accordingly’. [viii] Psychologist Lev Vygotsky made a similar point in defining a learner’s ‘zone of proximal development’—the region in which the learner’s likelihood of successful learning is maximised.[ix] In contrast, today’s year-level curricula present many students with content well beyond their current readiness, and many others with content well within their comfort zones.
In essence, this alternative paradigm makes mastery rather than time the basis of advancement through the curriculum. Rather than being required to move to the next year’s curriculum because time has elapsed, students do not progress to the next proficiency level until they have mastered the prior level. The consequence is that students may progress at different rates. Those who require more time have it; those ready to move to the next level are able to do so. Mastery-based rather than time-based advancement may result in some students progressing more slowly, but this is considered preferable to requiring all students to move in lock-step, thereby disadvantaging those who have not yet established prerequisites for success in the next stage of learning.
Figure 1 suggests that a mathematics curriculum developed for any one of these eight proficiency levels is likely to be appropriate for some students in each of Years 5 to 9. Curricula for the higher levels will no doubt be appropriate for many students in Year 10, and for the lower levels, for many students in Year 4. In other words, curricula developed at these eight levels are likely to be appropriate for most students across at least seven years of school, meaning that the number of mathematics curricula to be developed under this alternative would be similar to the current number of year-level curricula.
A crucial feature of the eight proficiency levels in Figure 1 is that they were developed from an empirical analysis of students’ performances in mathematics. They represent increasing levels of demonstrated knowledge, understanding and skill and, together, provide an empirically-based picture of increasing mathematics proficiency. This is a distinguishing feature of this approach; it is strongly informed by evidence as well as theory. Although the sequencing of content is also a feature of year-level curricula, this sequencing is often not strongly empirically based and may instead reflect conventions for introducing topics in particular years of school, involve the repetition of content, and be less focused on defining a progression of learning—that is, on ensuring each year’s curriculum builds on the prior year’s and lays foundations for the next.
A new ‘normal’?
This alternative is more than a different way of structuring the curriculum; it invites new ways of thinking about teaching, learning, assessment and reporting—in a sense, a redefinition of ‘normal’ practice.
The implications for teaching are that a high priority is placed on first ascertaining the points individuals have reached in their learning and then using that information to identify appropriate next steps for teaching. For example, if a curriculum were developed for each of the eight mathematics proficiency levels in Figure 1, the first teaching task would be to establish the highest level an individual had reached but not yet mastered. This would be considered the level at which they were currently working. The associated curriculum is likely to provide appropriately challenging content targeted to the student’s present learning needs.
Teaching is thus conceptualised less as the process of ‘delivering’ the same curriculum to all students in a year level, and more as a process of first establishing where individuals are in their learning and then ensuring each student is taught and given stretch challenges appropriate to their current level of attainment.
This immediately raises the question of how a teacher could manage a classroom in which students were not all working at the same level and engaged in the same activities. In practice, most teachers could probably manage students working at two or three different levels—particularly if they are sometimes engaged in the same activities in ways appropriate to their different levels of attainment. In such classes there is likely to be an amount of explicit, whole-group teaching. But it is difficult to imagine most teachers managing more than this. However, in practice, the variability observed in an entire year cohort is unlikely to be found in many individual classrooms. In some classrooms, all students may, in fact, be working at more or less the same level. Schools in which students vary widely in their levels of attainment may need to find ways of having teachers collaborate to ensure every student is taught at a level appropriate to their current learning needs.
There are also implications for how learning is conceptualised. Rather than defining success only as performance against year-level expectations, a student is considered to have learnt successfully if they make excellent progress over time. In other words, successful learning is equated with, and inferred from, increasing levels of knowledge, understanding and skill. A consequence of this definition of learning is that less advanced students who make excellent progress over the course of a year are considered to have learnt successfully, even if they are still performing below the level hoped for by this time in their schooling. And more advanced students who make limited progress over a year are considered to have learnt less successfully, despite their relatively advanced status.
Assessment under this alternative paradigm is the process of ascertaining where individuals are in their learning. Assessments do not only follow teaching; they also precede it. The aim is to establish the level a student has reached. Ideally, this is accompanied by more diagnostic investigations of a student’s current strengths and weaknesses. What specifically do they not yet know and understand, and what skills are they yet to develop? This may include an exploration of misunderstandings and skill deficits that present obstacles to that learner’s further progress. The essential purpose of such assessments is to identify appropriately challenging next steps for teaching and learning and to establish what a student still needs to learn in order to master the curriculum on which they are working.
When the fundamental purpose of assessment is to identify where students are in their long-term learning at any given time, assessments made at different times can be used to monitor learning progress. Progress may be reflected in the fact that a student has now mastered the curriculum on which they were working and moved to the next level (between-level progress), or it may be reflected in growing mastery of the content of the curriculum on which they are currently working (within-level progress).
Reporting also takes on a new meaning. Currently, most reporting is based solely on year-level expectations and indicates how much of the year-level content a student has mastered and is able to demonstrate. This may be reported as a percentage, often with students who can demonstrate 50 per cent of the course content being considered to have ‘passed’. Or it may be conveyed as a letter grade. In contrast, under the alternative being considered here, reporting indicates the highest level a student has mastered and the progress they have made since the last report (between levels and/or within a level). The principal focus is on where students are in their long term progress in an area of learning.
Importantly, although reporting is primarily against the sequence of curriculum levels through which all students are progressing, this does not rule out the possibility of also providing information about how a student is progressing in relation to other students and/or in relation to the levels intended or expected of students by particular times in their schooling. This additional information may be of particular interest to parents.
The inevitable hesitancy
Thomas Kuhn observed that, in science, a mismatch between theory and practice initially does not lead scientists to question the paradigm on which normal science is based. Instead, every effort is made to accommodate anomalies within existing theory. In much the same way, data of the kind displayed in Figure 1 may not be seen as a reason to abandon the assumption underlying normal school practice (that is, that all students in a year level are able to engage meaningfully with, and benefit from, the same year-level curriculum), but to redouble efforts to make this intention work.
In practice, this is likely to mean exhorting and supporting teachers to ‘differentiate’ their teaching to address students’ differing levels of attainment and learning needs. However, differentiation may be difficult for teachers, especially if there is a significant volume of common year-level content they are expected to deliver to every student.
Some countries make greater efforts than others to address the problem inherent in the current paradigm. For example, Finland provides schools with specialist teachers whose role is to work with students who are slipping behind in their learning. Teachers also meet regularly, often with psychologists and social workers, to discuss the progress of individual students. At the end of the year, students who do not meet an adequate standard in one or more subjects are able to take a separate test. If they still do not demonstrate an acceptable standard, they usually are retained in that year level. In another high-performing country, Estonia, students who struggle with year-level curricula are often taught intensively in small groups of around eight students. The intention in both countries is to ensure that every student masters the content of the year-level curriculum and establishes solid foundations for the next. However, despite these efforts, students remain widely dispersed in their levels of attainment. By 15 years of age, the difference between the tenth and ninetieth percentiles in mathematics is 11% smaller in Finland than in Australia, and 12% smaller in Estonia,[x] representing perhaps 4.5 to 5.5 years of learning and suggesting that, in these countries too, many students are not yet ready for, and many others are not sufficiently challenged by, common year-level curricula.
In science, Kuhn argued, resistance to an alternative paradigm is both ‘inevitable and legitimate’. Change does not occur easily and, when it does occur, is not justified by ‘proof’ but by the perceived possibility of a better way of addressing an existing problem.[xi] Perhaps similarly, a changed way of thinking about teaching and learning in schools should not occur easily. In particular, the expectation that every student in the same year of school will be taught identical content is deeply ingrained, and a range of arguments can be anticipated for continuing this practice and attempting to address the anomalies within it. These arguments are likely to include some predictable concerns.
Concern 1: changing the structure of the curriculum will mean abandoning year levels.
Uncoupling the structure of the curriculum from the structure of schooling does not necessarily require change to the structure of schooling. There are often good social reasons to have the majority of students progress through school with their age peers. The intention would not be to dispense with year groups, but to better identify and address students’ varying learning needs within each year group. Currently, schooling is structured into time periods that include school years, terms and weeks, and curricula are developed to mirror this structure. But this has not always been the case. For hundreds of years, many children were taught in one-teacher village schools, with a wide range of ages in the same room. Individuals were at different points in their learning, often unrelated to their age. Younger students could be taught, and expected to learn, the same material as older students. By the twentieth century, most students attended larger schools where they were now grouped by age, and curricula were developed for these different age groupings. In the future, learning at school may be different again, particularly as technologies play a more influential role, with learning becoming less rigidly anchored to time periods and students in the same year group being able to progress at different rates.
Concern 2: teachers will be unable to manage classrooms in which students are not all working on the same content at the same time.
It is important to note that the alternative being described here is not a proposal to introduce more variability into year-level classrooms. The variability already exists. Most teachers are already doing the best they can to address this variability and meet individual learning needs. But they work within the constraints of a curriculum that assumes all students in a year level are more or less equally ready for the same content. As we have seen, this is patently not the case. In addition, year-level curricula are often overcrowded. In practice, many teachers already have students work on different tasks to address their different learning needs, sometimes by grouping students who are at different stages in their learning. As noted earlier, most teachers can manage students working at two or three levels within the same class, but beyond that, teacher collaboration and whole-school responses may be required to meet every student’s learning needs.
Concern 3: some students will be disadvantaged if students are not all taught the same content at the same time.
Underpinning this concern is a belief that it is ‘equitable’ to teach the same content to all students of the same age. However, this belief confuses equity with equality. An equitable school system recognises and addresses students’ different learning needs, rather than assuming they are all the same. It expects every student to make excellent ongoing progress in their learning, and eventually to achieve high standards, but is not blind to individuals’ varying needs and starting points. And it recognises that, when students are taught what they are not yet ready to learn, those who are already disadvantaged are disproportionately impacted.
Concern 4: a restructured curriculum will result in ‘streaming’ and/or require the development of individual learning plans.
A restructured curriculum does not require the introduction of separate streams. It also does not require individual learning plans. The intention is that all students should work through the same inclusive curriculum sequence, without assuming every student will be at the same point in this sequence at the same time. The problem with streaming is that students can be locked into low-level streams and lowered expectations. In contrast, the expectation here is that every student should make excellent progress and eventually achieve the same high standards. Teachers would decide the best ways to manage students progressing on different timelines. It is true that, under this system, students who progress more slowly or more quickly may be easily identified, but teachers and students already understand the variability that exists in classrooms. Again, the benefits of mastery-based rather than time-based progression are considered to outweigh the costs of explicitly recognising that individuals are at different points in their learning.
Concern 5: a restructured curriculum will lower educational standards.
Underpinning this concern is a belief that the best way to lift performance in schools is to hold all students in the same year of school to the same high standards. However, this is current practice and there is little evidence that it has worked. Despite the setting of achievement standards for each year of school in Australia, 15 year olds’ reading, mathematics and science achievements have been in steady decline, both in absolute terms and relative to the OECD average. One response to this observation has been to call for the setting of still higher achievement standards, but this ignores international evidence that simply holding students and teachers accountable for achieving high standards is of limited effectiveness in practice.[xii] The performance of Australian students is more likely to be improved by ensuring every student’s learning needs are being identified and addressed with well-targeted teaching.
Concern 6: it will not be possible to do this in some subjects.
The concern here is that, while it may be possible to describe increasing levels of proficiency in some subjects, such as mathematics, which are considered naturally sequential, it will not be possible to do this in other, ‘less sequential’ subjects. Those expressing this concern sometimes refer to subjects that consist of topics that could be taught in almost any order and that are introduced in particular years of school largely by convention. It is true that most subjects, including science and history, could be taught as independent, unrelated topics such as the American Civil War or Coral Reefs. Such topics provide students with important factual knowledge. But it is also true that subjects are intended to build over time students’ understandings of essential concepts, principles and methods at the heart of a discipline. These usually transcend and are developed through the study of topics. A sequence of proficiency levels gives primacy to these essential elements of a discipline and describes and illustrates their intended development across the school years. Proficiency levels organise otherwise isolated topics into a ‘curriculum’—in its original sense of being a course along which individuals progress.
Making the change
According to Kuhn, a crucial step in making the change to a different way of thinking and working in science is the development of a ‘clear exhibit’ of what scientific practice would be like under the alternative.[xiii] Analogously, any change to the structure of school learning is likely to depend on a clear illustration of what change would look like in practice, including the implications for future teaching and learning.
For this reason, change should be introduced cautiously. Ideally, work would begin in just one or two areas of learning, with curricula in these areas being redeveloped as proofs of concept. An obvious starting point would be mathematics, given considerable research into the progression of learning in that subject. A second subject should be one in which work has been done to elucidate the nature of increasingly sophisticated knowledge, deeper understandings and higher levels of skill. Work of this kind has been done in a range of subjects, including science, English (reading and writing), second language learning and music. Essential to restructuring any curriculum will be a dependable body of empirical research into how knowledge, skills and understandings in that subject unfold and are best developed over time.
At the heart of a restructured curriculum will be a relatively small number of essential concepts, principles and methods to be developed in increasing depth over a number of years of school. Important factual and procedural knowledge usually will be organised around these core understandings. A key curriculum development task will be to decide on these essential concepts, principles and methods in a subject and to develop empirically-and theoretically-based descriptions of their long-term development. Concrete examples of development—in the form of samples of student responses and work—will be essential to illustrate the levels of the restructured curriculum. And, ideally, these descriptions and illustrations will be accompanied by extensive advice, resources and associated professional learning for teachers.
[i.] TS Kuhn, The structure of scientific revolutions, University of Chicago Press, Chicago, 1962.
iii. D Siemon, A Barkatsas & R Seah (eds), Researching and using progressions (trajectories) in mathematics education, The Netherlands, Brill, 2019, p. 13 (figure 1.1).
[iv.] S Thomson, L De Bortoli, C Underwood & M Schmid, PISA 2018: reporting Australia’s results: volume I student performance, Australian Council for Educational Research, Melbourne, 2019.
[v.] J Rose & C Rush, The iceberg problem: how assessment and accountability policies cause learning gaps in math to persist below the surface....and what to do about it, New Classrooms, New York, 2019.
[vi.] B Preiss, ‘Teachers little help to brightest students’, The Age, 25 October 2013, viewed 2 September 2020,
[vii.] J Bransford, AL Brown, RR Cocking, MS Donovan & J Pellegrino (eds), How people learn: brain, mind, experience, and school: expanded edition, National Academies Press, Washington DC, 2000.
[viii.] DP Ausubel, Educational psychology: a cognitive view, Holt Rinehart & Winston, New York, 1968, p. vi.
[ix.] LS Vygotsky, Mind in society: the development of higher psychological processes, eds. M Cole, V John-Steiner, S Scribner & E Souberman, Harvard University Press, Cambridge, Mass., 1978.
[x.] OECD, PISA 2018 results: what students know and can do, vol. 1, OECD Publishing, 2019, table I.B1.5, viewed 2 September 20202,
[xii.] M Fullan, Choosing the wrong drivers for whole system reform, Seminar series 204, Centre for Strategic Education, East Melbourne, Vic., 2011.
ACER Occasional Essays - November 2020
Geoff Masters AO is CEO of the Australian Council for Educational Research
© 2020 Australian Council for Educational Research
Citation
Masters, G. (2020). Time for a paradigm shift in school education? Australian Council for Educational Research. https://doi.org/10.37517/91645.2020.1
ISSN 2652-8916
All rights reserved. Except under the conditions described in the Copyright Act 1968 of Australia and subsequent amendments, no part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the written permission of the publishers.
