Licensure Tests for Special Education Teachers:
How Well They Assess Knowledge of Reading Instruction and Mathematics

Sandra Stotsky
University of Arkansas

November 7, 2008

Pre-publication draft: To be published in the Journal of Learning Disabilities, 2009

---

Abstract

The 1998 reauthorization of the Higher Education Act requires all states to report annually to the U.S. Department of Education the number of prospective teachers at each teacher training institution who pass their own state tests for licensure. However, the law left decisions on what tests to require in each field, what to assess on them, and their passing scores up to each state.

To determine the content knowledge in reading and mathematics that subject tests for prospective special education teachers assess, this study drew on information available on the Web sites for Educational Testing Service, the American Board for Certification of Teacher Excellence, and National Evaluation Systems. It examined the extent to which these tests address three research-based components of reading pedagogy (instruction in phonemic awareness, phonics, and vocabulary knowledge), the weights attached to knowledge of these three components, and the quality of the sample questions provided for them. It estimated the percentage of test items addressing mathematics content and these three components on each test. It also analyzed the descriptions of ETS’s tests of “principles of teaching and learning” to see whether these tests assess understanding and use of educational theories that underlie effective research-based practices.

The findings of this study suggest that one solution for ensuring that prospective special education teachers begin their teaching careers with a substantial amount of research-based knowledge of reading instruction is to require them to take a dedicated test of research-based reading instructional knowledge, as do California, Massachusetts, and Virginia, in addition to any other tests required for their license. To ensure an adequate amount of relevant mathematical knowledge, states might consider replicating the 40-item test of mathematics content that Massachusetts will require of all aspiring elementary and special education teachers in 2009. States will also need to design their own tests of basic principles of teaching to ensure that they assess knowledge of educational theories that underlie effective research-based practices.

Introduction

Licensure tests—which typically assess the basic substantive knowledge needed for professional practice—are a key measure of quality control for entry into most professions (see, e.g., Hampel 2005, for the role played by state medical licensing exams in upgrading and maintaining the quality of medical schools and their graduates in this country). There are two major reasons for teacher licensure tests: to protect the public (as with most licenses) and to make teacher training programs accountable for the initial academic competence of those who complete their programs (Stotsky, 2004; 2006; 2007a; 2007b). Licensure tests in other professions have not always been motivated by legislators' interest in making their professional training programs accountable for graduates’ performance on these tests (e.g., law schools are typically not held accountable by the state in which they are located for their graduates’ pass/fail rates on the state’s bar exam). But, this was a major reason for the development of these tests for prospective teachers. E.g., legislators explicitly mandated tests of prospective teachers' academic skills and content knowledge in the Massachusetts Education Reform Act of 1993 to assure a minimal level of academic competence in new teachers and to hold their preparation programs accountable for this level (Stotsky, 2004).

States began to require the passing of a licensure test for entry into the teaching profession about two decades ago. A provision in Title II in the 1998 reauthorization of the Higher Education Act then compelled all states to require licensure tests for new teachers.[1] Each state henceforth had to report annually on the pass rates on tests of its own choosing for each cohort of prospective teachers completing training programs in the state’s own teacher training institutions. However, the provision in Title II also allowed each state to decide what licensure tests it would require, what it would assess on them, their passing scores, and when the tests could be taken. The expectation was that a requirement to report pass/fail scores on state licensure tests annually would upgrade the quality of the teacher preparation programs in each state. However, as several studies noted below indicate, this has clearly not happened to elementary licensure programs.

During the 2000s, comprehensive reports were issued by national panels of distinguished scholars and researchers on the research base for the pedagogy used in teaching the two major subjects in the schools. The National Reading Panel (NRP) issued its report, Teaching Children to Read, in April 2000, indicating the major elements supported by high quality research for improving beginning reading instruction. The research findings in this report highlight the role of phonemic awareness, knowledge of sound/letter relationships, fluency, vocabulary knowledge, and general comprehension of written language in the development of reading skill. As the report indicates, most children must receive systematic instruction in phonemic awareness for distinguishing the sounds in words and in phonics for identifying printed words; regularly read aloud to demonstrate fluency; practice enough to acquire decoding skills to the point of automaticity; and receive systematic instruction through the grades to develop their knowledge of word meanings. Figure 1 shows the key research findings of the NRP as summarized by two reading researchers.

Figure 1. Key Research Findings on Beginning Reading Instruction

• Phonemic awareness can be taught and learned. • Phonemic awareness instruction helps children learn to read. • Phonemic awareness instruction helps children learn to spell. • Phonemic awareness instruction is most effective when children are taught to manipulate phonemes by using the letters of the alphabet. • Phonemic awareness instruction is most effective when it focuses on only one or two types of phoneme manipulation, rather than several types. • Systematic and explicit phonics instruction is more effective than non-systematic or no phonics instruction. • Systematic and explicit phonics instruction significantly improves kindergarten and first-grade children’s word recognition and spelling. • Systematic and explicit phonics instruction significantly improves children’s reading comprehension. • Systematic and explicit phonics instruction is effective for children from various social and economic levels. • Systematic and explicit phonics instruction is particularly beneficial for children who are having difficulty learning to read and who are at risk for developing future reading problems. • Systematic and explicit phonics instruction is most effective when introduced early. • Phonics instruction is not an entire reading program for beginning readers. • Repeated and monitored oral reading improves reading fluency and overall reading achievement. • No research evidence is available currently to confirm that instructional time spent on silent, independent reading with minimal guidance and feedback improves reading fluency and overall reading achievement. • Children learn the meanings of most words indirectly, through everyday experiences with oral and written language. • Although a great deal of vocabulary is learned indirectly, some vocabulary should be taught directly. • Text comprehension can be improved by instruction that helps readers use specific comprehension strategies. • Students can be taught to use comprehension strategies. • The following six strategies appear to have a firm scientific basis for improving text comprehension: monitoring comprehension, using graphic and semantic organizers, answering questions, generating questions, recognizing story structure, and summarizing.

Source: Armbruster, B., & Osborn, J. (2001). Put reading first: The research building blocks for teaching children to read. Urbana, IL: Center for the Improvement of Early Reading Achievement, University of Illinois, Urbana-Champaign.

Despite the research support for this body of reading instructional knowledge, two studies that examined syllabi for reading methods courses after 2000 (Steiner & Rozen, 2004; National Council on Teacher Quality, 2006) found that few schools of education expected an understanding of this body of knowledge in their reading methods courses.

The National Mathematics Advisory Panel (NMAP) issued its report, Foundations for Success, in March 2008, indicating what the evidence from high quality research supports for increasing mathematics achievement in pre-school, elementary school, and middle school. Among other things, it identified the 27 major topics of school algebra and set forth the crucial mathematical concepts and skills in PreK-7 whose mastery could lead to success in Algebra I. These foundational concepts and skills were organized in three categories. The first, Fluency with Whole Numbers, includes understanding of place value; fluency in composing and decomposing whole numbers; understanding of the meaning of the basic operations of addition, subtraction, multiplication, and division; automatic recall of number facts; fluency with the standard algorithms for addition, subtraction, multiplication, and division; and knowledge of how to apply these operations to problem-solving. The second, Fluency with Fractions, includes a thorough understanding of positive and negative fractions as well as the ability to locate fractions on the number line, to represent and compare fractions, decimals, and related percents, and to estimate their size. Students also need to know that sums, differences, products, and quotients (with non-zero denominators) of fractions are fractions, why and how (finite) decimal numbers are fractions, and the meaning of percentages. The third category, Particular Aspects of Geometry and Measurement, includes experience with similar triangles because sound treatments of the slope of a straight line and of linear functions depend logically on the properties of similar triangles.

In the area of instructional practices, the NMAP highlighted the following findings. (1) Students with learning problems benefit from explicit and systematic instruction. (2) Formative assessment is beneficial, especially at the elementary level. (3) Small group work and the use of "real-world" problems may be beneficial but only under very specific conditions, at certain grade levels, and chiefly for developing computational skills. (4) Calculator use does not promote conceptual development, calculation skills, or problem solving. In addition, it found no body of research showing that teacher-directed learning prevents students from understanding mathematics.

The NMAP report also reaffirmed teachers’ knowledge of mathematics as, so far, the only identifiable characteristic of an effective mathematics teacher. Yet, a study on the preparation of elementary teachers in mathematics in 77 institutions in 49 states (Greenberg & Walsh, 2008) judged only ten of these institutions to provide adequate mathematics coursework for these aspiring teachers. This study examined the time spent on the four areas of mathematics that an elementary teacher needs to understand: 1) numbers and operations, 2) algebra, 3) geometry and measurement, and 4) data analysis and probability. Of the four areas, the study found algebra instruction the weakest, with over half of all schools (52%) devoting less than 15% of class time to algebra, with another third effectively ignoring it entirely, devoting less than 5% of class time to that area. According to the mathematics advisory group for this study, algebra should comprise roughly 25% of the preparation in mathematics for elementary teachers because they need to understand it as the generalization of the arithmetic they address and its connection to many of the properties, relationships, rules, and models that elementary students study.

To judge by these studies on reading methods courses and mathematics coursework in elementary preparation programs, it appears that Title II’s provision has apparently done little if anything to upgrade the quality of our teacher education programs. Indeed, Arthur Levine’s report, Educating School Teachers, issued in September 2006, concluded that the vast bulk of the 1200 education schools in this country have incoherent curricula as well as excessively low admission standards.

Given this dismal picture of the academic quality of reading methods courses and mathematics coursework for prospective elementary teachers, it is logical to look at what is assessed on the licensure tests taken by most prospective PreK-8 special education teachers for several reasons. In many, if not most, states today, special education teachers work side by side with elementary teachers in the regular classroom. But whether they work in the regular classroom or in resource rooms of their own, they should know, academically and instructionally, exactly what elementary teachers should know. Moreover, they tend to work chiefly with children who have great difficulty in reading (and writing). The next most difficult subject for learning disabled children is mathematics, and their problems in learning mathematics today are compounded by the emphasis in most current mathematics programs on a great deal of reading (of mathematics problems) and mathematics-related writing. Finally, prospective special education teachers may well take the same reading methods courses and mathematics coursework required of prospective elementary teachers. The content assessed on their licensure tests should therefore serve as a measure of quality control for their academic competence and as one predictor of their future effectiveness.

However, no one has systematically examined all the tests designed for, or commonly taken by, prospective special education teachers to determine to what extent their research-based reading instructional knowledge and relevant mathematics knowledge is assessed. Two studies (Rigden, 2006; Stotsky, 2006) examined the licensure tests assessing reading instructional knowledge most commonly taken by prospective elementary teachers, but neither of these studies systematically examined all the tests designed expressly for, or commonly taken by, prospective special education teachers. An earlier study (Mitchell & Barth, 1999) examined the contents of a number of different skills and subject tests (the only study to date to do so), but it, too, did not examine the licensure tests designed for prospective special education teachers.

The purpose of this study, therefore, was to determine the extent to which licensure tests for prospective special education teachers assess three of the five major components of beginning reading instruction that the NRP found supported by high quality research--the development of phonemic awareness, phonics, and vocabulary knowledge--as well as relevant mathematics knowledge. These three components of beginning reading instruction have been ignored, devalued, or distorted for many years in most basal reading programs and teacher preparation programs; "skills" instruction was among the major casualties in the rise of the whole language movement, as Pearson (2004) noted in an account of the "reading wars." These three components are also easy to identify in a test description if they are mentioned at all. [2]

The basic question this study explored was whether licensure tests of subject matter knowledge required of prospective PreK-8 special education teachers adequately assess their knowledge of research-based reading instruction and the mathematics they need for teaching purposes. However, it also examined licensure tests designed to assess prospective or new teachers' knowledge of basic teaching practices in order to see whether these tests assess understanding and use of educational theories that underlie effective research-based practices. It should be noted that this study provides more information on reading than mathematics because reading instructional issues in elementary education have received far more research attention historically than have issues in mathematics instruction in the elementary grades.

GENERAL INFORMATION ON TEACHER LICENSURE TESTS
Some background information on teacher licensure tests will help readers to understand their differences and their limitations. Two private testing companies develop most of the teacher tests used by the states. Educational Testing Service (ETS) provides licensure tests for over 35 states, chiefly states with small populations, while National Evaluation Systems (NES) contracts to provide tailor-made tests for over 12 states, chiefly the most populous states. Well over 50% of U.S. teachers are licensed in NES states (Mitchell & Barth, 1999). The American Board for Certification of Teacher Excellence is a recent addition to the small group of organizations that provide teacher tests. In states that have approved use of their tests for an initial license, endorsement, or master teacher status, it provides tests or endorsement certificates for prospective or practicing teachers who do not wish to enroll in a traditional preparation program and for current teachers seeking master teacher status.

Most states require teaching candidates to take at least two different tests for initial licensure: one assesses the candidate’s basic reading, writing, and arithmetic skills, the other (sometimes more than one content test is required) assesses the content knowledge presumed needed for teaching the field of the license at the grade levels it covers. These licensure tests are taken at different junctures in teacher preparation, typically not at the completion of the program, unlike most professional licensure tests. Because many, if not most, states do not mandate when their teacher tests are to be taken, a growing number of teacher training institutions use the state-required skills test to screen admission into their licensure programs. In states contracting with ETS, PRAXIS I is used for this purpose. In states contracting with NES, a skills test developed by NES is used for this purpose. The content knowledge test, which often includes items on teaching methods, is increasingly being used to screen admission into student teaching in undergraduate licensure programs. However, both the skills test and the content test are usually required for admission into post-baccalaureate programs for the initial license. College graduates who want to become teachers via a post-baccalaureate program (e.g., a M.Ed. program) but cannot pass the content knowledge test for the field they are interested may be still be admitted but are expected to take the right academic coursework and pass the content knowledge test before student teaching is allowed. The basic problem with using a skills test for admission into a preparation program (at the undergraduate or graduate level) is that it serves as an extremely minimal quality control; it is typically at a middle school level in overall difficulty (Mitchell & Barth, 1999). As Mitchell and Barth also found, the subject matter test, depending on the field, may not be much greater in difficulty; they judged those required for elementary licensure as a whole at about the tenth grade level.

Most passing scores don’t provide clear and comparable information on academic competence. Each ETS state determines its own cut or pass score, which may differ from that of another state using the same ETS test. NES states also determine their own cut or pass score. Only pass scores on the tests prepared by the American Board for Certification of Teacher Excellence (ABCTE) are pre-determined, and states may not alter them. There are no data across all states on how many test items need to be correct for a passing score on each of the different tests that states require. Test formats differ across tests and testing companies. In some but not all NES states, tests may have about 80 multiple-choice items and two short essay questions. ETS tests tend to have mainly multiple-choice items, but ETS does offer tests with essay questions as well. Compensatory scoring is used for most ETS and NES tests and for ABCTE tests, that is, the test-taker's raw score (before conversion to a scaled score) is the total number of items answered correctly, not a weighted number depending on the number of items answered correctly in each section of the test.

By default, therefore, licensure tests may strongly influence the content of the mathematics coursework aspiring special education teachers are required to take and the content of what is taught in their reading methods courses. Their required tests may also influence how these aspiring teachers will teach mathematics or reading if the tests contain pedagogical items. Indeed, these tests may tell future teachers exactly what should be taught in the name of reading or mathematics instruction and how.

LICENSURE TESTS TAKEN BY PROSPECTIVE SPECIAL EDUCATION TEACHERS
What kinds of tests does an aspiring special education teacher take? In all states, most if not all prospective teachers are required to take a test of their general reading, writing, and arithmetic skills. In addition, depending on the state, prospective PreK-6 or PreK-8 special education teachers may have to take a subject matter test designated for them. Or they may be required to take, in addition or instead, one or more tests required as well for the aspiring elementary teacher. For example, in California, Massachusetts, and Virginia, they must take a dedicated test of reading instructional knowledge as well as another subject test. In Illinois, they must take an elementary level curriculum test that clearly assesses subject matter, or content, knowledge, including mathematics, not professional knowledge such as teaching methods or skills. In a few ETS states, they may be required to take a similar type of test (PRAXIS 0511). For the secondary level in Massachusetts, the prospective special education teacher may take either an elementary or middle school subject matter test, but such an option doesn't seem to be available in other states. Finally, in a growing number of states, the special education teacher must also take a test of “principles of learning and teaching.” Because of the bewildering differences in state regulations across states, we must look at many different kinds of tests.

Tests of Prospective Teachers’ Skills in Reading, Writing, and Arithmetic
Only one study has examined the content of the tests produced by ETS and NES for this purpose. Although Mitchell and Barth (1999) do not provide any details on the quality of the test items used to assess prospective teachers' reading skills, they judged two-thirds of the mathematics items on ETS’s PRAXIS I to be at the middle school level, noting that it contained fewer items on algebra and geometry than did the 1996 grade 8 National Assessment of Educational Progress (NAEP) mathematics test. They praised the sample items on NES’s skills test for prospective teachers and administrators in Massachusetts, which they considered more complex and demanding than any of the others they looked at. Nevertheless, they did not judge even one of the skills tests they examined as close to the level of a graduating college senior. Overall, they judged tests of teachers’ reading, writing, and arithmetic skills at the “8th to 10th (sometimes 7th) grade level” (p. 10).

Tests of Prospective Teachers’ Knowledge of Reading Instruction and Mathematics
Tests assessing reading instructional knowledge
Appendix A contains profiles of 13 tests assessing reading instructional knowledge wholly or in part. The profiles indicate their major categories (how their objectives are organized), their weights (how much the items in each category count as a percentage of the test), and where the three components of reading instructional knowledge are addressed in the online descriptions of these tests. Four of these tests are offered by ETS as part of the PRAXIS II series. A large majority of the states require one or more of these four ETS subject tests for an initial elementary license and sometimes for an initial special education and/or early childhood license as well. Seven of the tests are used by states that contracted with NES for their development. Because the most populous states as well as a few smaller states contract with NES, this study includes those required by California, Illinois, Massachusetts, Michigan, New York, Oklahoma, and Virginia to cover a variety of NES states. It also includes two tests offered by ABCTE because some states now allow them as an alternative to the required tests for an elementary or special education teacher. In all cases, information was drawn from the descriptions available on the Web sites of these organizations because it is not possible to examine the actual contents of licensure tests (which must be secure) and report on details in their test items.

As indicated in the introductory section, this study examined the weights for the development of phonemic awareness, phonics knowledge or decoding skills, and vocabulary knowledge, not only because each is supported by a large, consistent, and credible body of research evidence (e.g., see Chall, 1996a; 1996b; and 2000, as well as the NRP report), but also because of their central importance to beginning reading instruction in the elementary school, and because these three areas have been neglected in the preparation of teachers of elementary-age children for decades, as suggested by their emphasis in the Reading First initiative. All three components are easy to identify if they are mentioned at all. As also indicated in the introductory section, the weights for reading comprehension or fluency were not examined because the former is assessed on all tests assessing reading instruction and thus noting its presence and weight provides no distinctive information, while the latter may be listed under a variety of synonymous words or phrases and is thus not easily identifiable.

Table 1 provides an estimate of the total percentage of the score on each of these 13 tests that may be accounted for by test items on the three components. Table l also notes the number of states that either require the test or make it one of two or three test options. Keep in mind that many states require more than one PRAXIS subject test for the elementary or special education license. If they require PRAXIS 0012, which consists of four essay questions, they tend to require another test as well.

To arrive at an estimate, I started with the weight(s) indicated on the Web site for the section(s) on reading instruction. I then calculated and added up the weights for the subsections that mentioned these three components. If any one of these three components was mentioned in a long laundry list of topics in a subsection, I calculated what percentage of the weight for the whole subsection relevant test items might reflect; in all cases I tried to err on the side of generosity when it was totally unclear how many test items might reflect any of these three components. Finally, I converted all the subsection percentages I had calculated to percentages of the total test and added them up for the final total.

Table 1: Estimated Percentage of Test Addressing Phonemic Awareness, Phonics, and Vocabulary Knowledge
ABCTE Elementary Test: 9-10%	California RICA:  45-50%
ABCTE Reading Test:  38%	Illinois 110:  5-6%
	Massachusetts 90: 54%
PRAXIS 0011:  7% (17 states)	Michigan 83:  2%
PRAXIS 0012:  1% (7 states)	New York 02:  12%
PRAXIS 0014:  3% (22 states)	Oklahoma 50:  25%
PRAXIS 0201:  39% (1 state)	Virginia VRA:  25%

These 13 tests differ markedly in what they expect a new teacher of elementary-age children to know. As can be seen, the estimated percentages range from 1% or 2% to 50% or 54%. To judge by these percentages, there is a huge variation in the importance these tests attach to an understanding of the implications of an alphabetic writing system for reading instruction and the role of vocabulary knowledge in developing reading skill. As can also be seen, states contracting with NES tend to expect prospective teachers of elementary-age children to acquire more knowledge of these three critical components of beginning reading instruction than do states using ETS tests. Three of these states (California, Massachusetts, and Virginia) try to ensure a high level of knowledge in these three areas by requiring a dedicated test of reading pedagogy in addition to a general, or multi-subject, test. Oklahoma also has a high percentage because its particular multi-subject test consists chiefly of reading and the language arts. On the other hand, at least two NES states (Michigan and Illinois) have no higher expectations than those states using ETS’s elementary tests. The content of a NES test is determined by the state contracting with NES.

The percentages on these tests tell us even more, given the strong possibility that passing scores are not set so high as to fail a majority of those who take the tests for the first time. Clearly, prospective special education teachers taking the NES elementary test required in Michigan and Illinois, or any of ETS’s elementary tests, need not worry if they have learned little about phonemic awareness or phonics (or decoding)—two basic components of beginning reading. Nor do they need to be anxious about how little they may have learned about the nature of the vocabulary of the English language and the variety of approaches needed for developing vocabulary knowledge—the basic element in reading comprehension at every educational level in every subject area. These aspects of reading instruction receive such minimal attention that test-takers could fail every question on these topics and, because of compensatory scoring, still pass these tests no matter where the passing score is set. Put another way, there are no negative consequences for their professional preparation programs if education faculty have included little about these three components in their methods courses and insisted that reading vocabulary be taught only “in context” or on the basis of “prior knowledge” (i.e., not through explicit instruction), injunctions that appear in ETS’s test descriptions whenever vocabulary is mentioned at all.

An independent analysis of the contents of eight licensure tests assessing reading instructional knowledge supports the analysis and conclusions presented above. In a report for the National Council for Accreditation of Teacher Education, Rigden (2006) examined five ETS tests as well as the information that NES provides on the reading tests it developed for California, Massachusetts, and Virginia. She wanted to see if these eight tests address the knowledge base for effective reading instruction. Rigden found that only the three NES tests and one ETS test (PRAXIS 0201), a dedicated reading rest required only in Tennessee (and for which test-takers get credit simply by taking it), have items that address the five components of research-based reading instruction. She observed that PRAXIS 0011 “is not a good measure of a teacher candidate’s knowledge of the five components of effective reading instruction.”

This empirical evidence of professional indifference to a crucial part of the knowledge base for reading instruction on the PRAXIS elementary tests and on some NES state tests would not be so troublesome if states compensated for the miniscule attention to reading instructional knowledge on them by requiring all prospective teachers of elementary-age children to take, in addition to other subject tests, a test emphasizing these crucial areas. But this does not seem to be the case in almost all of the states. Only six states (California, Connecticut as of 2009[3], Massachusetts, Oklahoma, Virginia, and Tennessee) require a separate reading test for licensing prospective elementary teachers and sometimes other teachers of elementary-age children (although only about 70% of Oklahoma’s test actually assesses reading and the language arts, and the number of items in each subsection varies widely across test administrations).

Tests assessing mathematics or mathematical instructional knowledge
There are as yet no dedicated tests of mathematical instructional knowledge similar to those that assess reading instructional knowledge, and prospective special education teachers are not required to take a mathematics subject test in any state. Moreover, there is almost no information available on the nature and quality of the items assessing mathematics or mathematical instructional knowledge on the tests described in the previous section that may be required of prospective special education teachers as well as elementary teachers (e.g., PRAXIS 0011, PRAXIS 0012, PRAXIS 0014, ABCTE's Multiple Subjects Exam, and New York's Multi-Subject Test 02). Rigden’s 2006 study, for example, does not provide any information on how ETS’s commonly used multi-subject tests for elementary licensure address the mathematics or mathematical instructional knowledge needed for teaching mathematics in grades 1-6.

The profiles of these tests on their Web sites do indicate that their sections on mathematics knowledge and mathematical instructional knowledge (which are usually mingled) tend to be organized according to the major categories found in state PreK-12 mathematics standards documents (e.g., numbers and operations, measurement, geometry, patterns and relationships, and data analysis). Beyond that, however, we know nothing. Because of test security issues, the NMAP itself could obtain no information on the mathematical content and quality of licensure tests for those who teach mathematics at any educational level and to any population. Nor have researchers examining the relationship between elementary students' achievement in mathematics and their teachers' scores on the licensure tests they took provided this information, as Stotsky (2007) noted in a critical review of these studies.

Table 2 shows the estimated percentage of test content addressing mathematics knowledge and/or mathematical instructional knowledge for the 13 tests described in the previous section, as well as the estimated percentage of test content addressing the three components of reading instructional knowledge examined in this study, as summarized earlier in Table 1. For the estimate of mathematics content, the percentage for mathematics content given in the test description itself was generally used. But, to interpret these percentages, keep in mind that, for some tests, some of the objectives listed in their descriptions assess knowledge of instructional strategies rather than of mathematics content itself. Note that Table 2 also shows the estimated percentage of test content addressing reading instructional knowledge and mathematics or mathematical instructional knowledge in the tests described in the next two sections.

Table 2: Estimated Percentage of Test Addressing Three Components of Reading Instructional Knowledge and Mathematics Content
Licensure Tests Assessing Reading Instructional Knowledge for Prospective Elementary and Sometimes Other Teachers  (see Appendix A)	Percent in Three Areas of Reading	Percent of Mathematics Content
PRAXIS 0011 (Elementary Education: Curriculum, Instruction, and Assessment), ETS      (17 states)	7%	22%
PRAXIS 0012 (Elementary Education: Content Area Exercises), ETS  (7 states)	1%	25%
PRAXIS 0014 (Elementary Education: Content Knowledge), ETS   (22 states)	3%	25%
PRAXIS 0201 (Reading across the Curriculum: Elementary), ETS   (1 state)	39%	0%
Multiple Subjects Exam (for Elementary Education), ABCTE**	9-10%	27%
Reading Endorsement for K-6, ABCTE	38%	0%
California RICA, NES**   (1 state)	45-50%	0%
Illinois 110 (Elementary/Middle), NES  (1 state)	5-6%	20%
Michigan 83 (Elementary Education), NES    (1 state)	2%	20%
Massachusetts 90 (Foundations of Reading), NES***     (1 state)	54%	0%
New York 02 (Multi-Subject Test: Grades PreK-9), NES (1 state)	12%	18%
Oklahoma 50 (Elementary Education Subtest I), NES    (1 state)	25%	0%
Virginia VRA, NES *** (1 state)	25%	0%
Licensure Tests for Prospective Special Education Teachers
PRAXIS 0353 (Education of Exceptional Students: Core Content Knowledge), ETS	0%	0%
PRAXIS 0351 (Special Education: Knowledge-Based Core Principles), ETS	0%	0%
PRAXIS 0511 (Fundamental Subjects: Content Knowledge), ETS	1%	25%
Special Education (K-6), ABCTE	9%	0%
Illinois 155 (Learning Behavior Specialist I), NES	1%	1%
Michigan 63 (Learning Disabled), NES	4%	1%
New York 60 (Students with Disabilities, CST), NES	1%	1%
Licensure Tests of Teaching Skills for Beginning Teachers
PRAXIS 0521 (Principles of Learning and Teaching: Early Childhood), ETS	0%	0%
PRAXIS 0522 (Principles of Learning and Teaching: Grades K-6), ETS	0%	0%
PRAXIS 0523 (Principles of Learning and Teaching: Grades 5-9), ETS	0%	0%
PRAXIS 0524 (Principles of Learning and Teaching: Grades 7-12), ETS	0%	0%
New York 90 (Elementary Assessment of Teaching Skills), NES  (1 state)	2%	0%

* The number of states requiring the test for prospective elementary teachers is in parentheses after its title.
** Required of both prospective elementary and special education teachers in the state.
*** Required of prospective elementary, early childhood, and special education teachers in the state.

 

Tests of special education: Their reading instructional and mathematics content
Most of the many states that use PRAXIS II tests for most if not all of their subject tests require a test designated for aspiring special education teachers. However, two of the PRAXIS II subject tests for aspiring special education teachers examined for this study (see Table 2) contain little if any reading instructional or mathematics content, according to their descriptions. To judge by what is on the ETS Web site, other PRAXIS tests for special education teachers have no reading instructional or academic content except for PRAXIS 0511, which is not designated specifically for special education teachers but is now beginning to be used for them. The situation is the same for the special education test in many NES states (e.g., Illinois, Michigan, and New York). One can only hope that licensure programs for special education in all states require some coursework in reading pedagogy and mathematics. But prospective teachers in these programs are clearly not being held accountable for knowledge in these areas on licensure tests designed specifically for them in an ETS state, no matter which PRAXIS II test their state requires for them.

Only in states that require special education teachers to take a dedicated reading test in addition to a test designed for them will they be likely to be assessed adequately on research-based reading instructional knowledge. California requires all prospective special education and elementary teachers to take its state-specific reading test. Virginia and Massachusetts require all future early childhood, special education, and elementary teachers to take its state-specific reading test. Of the states that use PRAXIS II tests, only Tennessee requires its aspiring special education teachers to take a licensure test that includes an assessment of reading instructional knowledge, but since Tennessee has set no cut score for that test, the test has no consequences.

The fact that special education teachers are not held accountable for any mathematics knowledge on their own licensure tests was noted obliquely in the NMAP report. It commented that the “Praxis II exams for those who will teach mathematics as content specialists or as generalists vary in the amount and level of mathematical knowledge assessed” and that some of these tests do not assess any mathematics content at all. (p. 37). The report strongly recommended strengthening the mathematics preparation of elementary and middle school teachers (p. xxi), and it spelled out the mathematics that should be taught in preparation programs for early childhood, elementary, middle school, and special education teachers and assessed on their licensure tests (p. xviii).

It is true that there may be about 25% mathematics content on some of the state tests that aspiring special education teachers are required to take that are also required of other generalist teachers. But as noted in the previous section, we have no information in any study on licensure tests about the quality of the test items in this area and their level of difficulty. Nevertheless, each state can do what Massachusetts is now doing to strengthen its preparation programs for elementary and special education teachers. Since 2003, Massachusetts has required prospective special education teachers to take the same general curriculum test taken by prospective elementary teachers. In 2007, because of concern that even 25% of a general test was no longer adequate and because there was no clear information on the level of difficulty of this 25%, the state’s board of education approved the construction of a new 40-item mathematics test that is to count for 50% of this general test. This test will have its own cut score. The board of education also approved a set of guidelines for the contents of the test to help elementary and special education licensure programs to develop appropriate mathematics coursework (Massachusetts Department of Education, 2007). This is a national “first.”

Tests of Prospective Teachers' Knowledge of the Principles of Learning and Teaching: Their Reading Instructional and Mathematics Content [4]
A majority of states now require all beginning teachers to pass a licensure test of basic teaching knowledge in addition to a subject matter test (and a test of the teacher's own skills). This kind of test is important to examine because it assesses new or prospective teachers' knowledge of the educational theories or principles that guide classroom practice and instructional programs in every subject area. We need to know whether this kind of test is constructed in ways that favor or disfavor particular teaching or learning theories or principles. It may support or undermine what has been taught in methods or content courses.

Most NES states have developed their own test (e.g., New York). Other states require one of the tests ETS offers as part of a PRAXIS series called Principles of Learning and Teaching. This set of tests is designed to assess “what a beginning teacher should know about teaching and learning.” While a few of the states listed on the ETS Web site as requiring these tests indicate that they are to be used for the second level of licensure (i.e., after a new teacher has begun teaching), the others require the grade-relevant test for initial licensure. As Table 2 shows, there are four tests in this set of tests, one for early childhood (0521), one for grades K-6 (0522), one for grades 5-9 (0523), and one for grades 7-12 (0524). Each consists of 24 multiple-choice questions and four “case histories” that are each followed by three short-answer questions scored on a scale of 0 to 2. Test content is organized in four categories:

I. Students as Learners (33%, 22% of which is based on short-answer questions)
II. Instruction and Assessment (33%, 22% of which is based on short-answer questions)
III. Teacher Professionalism (22%, 11% of which is based on short-answer questions)
IV. Communication Techniques (11%, based solely on short-answer questions).

These ETS tests assess no reading instructional or mathematics content. But they appear to have serious limitations from a special education (and elementary) perspective. They promote student-directed learning and downgrade teacher-directed instruction, even though the research base for both reading and mathematics instruction supports explicit and systematic instruction for struggling students. The NMAP found that “explicit instruction with students who have mathematical difficulties has shown consistently positive effects on performance with word problems and computation. Results are consistent for students with learning disabilities, as well as other students who perform in the lowest third of a typical class” (p. xxiii). Moreover, the NMAP found no body of research evidence to support an emphasis on either a "student centered" or "teacher directed" approach.

Two examples will show how sample test questions and answers serve to promote a pedagogy favoring student-directed learning and/or to discredit teacher-directed instruction. The following question and choices follow the description of PRAXIS 0522:

Which of the following kinds of instruction is frequently cited as the opposite of discovery learning?
(A) Simulation games
(B) Expository teaching
(C) Mastery learning
(D) Schema training

As ETS does on all its Answer pages for all its sample test questions for all its tests, the Answer page for this example carefully explains why the best answer is B. “The method of teaching most often seen as the opposite of discovery teaching is expository teaching. Discovery learning allows students to explore material on their own and arrive at conclusions. In expository teaching, students are presented with subject matter organized by the teacher.” Not only is this an odd definition of an uncommon phrase (“expository teaching”), it is an indirect slap at direct instruction and leaves anyone familiar with mastery learning in the dark about why it didn’t qualify as the best answer.

The following sample test item accompanies all four test descriptions. It subtly discredits any approach to instruction other than an approach favoring student-directed learning in the answers to the questions that follow two paragraphs, which are presented as being taken from a debate about the advantages and disadvantages of a constructivist approach. Here are the two passages and the two questions following them:

Why constructivist approaches are effective
The point of constructivist instruction is to have students reflect on their questions about new concepts in order to uncover their misconceptions. If a student cannot reason out the answer, this indicates a conceptual problem that the teacher needs to address. It takes more than content-related professional expertise to be a “guide on the side” in this process. Constructivist teaching focuses not on what the teacher knows, but on what and how the student learns. Expertise is focused on teaching students how to derive answers, not on giving them the answers. This means that a constructivist approach to teaching must respond to multiple different learning methods and use multiple approaches to content. It is a myth that a constructivist teacher never requires students to memorize, to drill, to listen to a teacher explain, or to watch a teacher model problem-solving of various kinds. What constructivist approaches take advantage of is a basic truth about human cognition: we all make sense of new information in terms of what we already know or think we know. And each of us must process new information in our own context and experience to make it part of what we really know.

Why constructivist approaches are misguided
The theory of constructivism is appealing for a variety of reasons—especially for its emphasis on direct student engagement in learning. However, as they are implemented, constructivist approaches to teaching often treat memorization, direct instruction, or even open expression of teacher expertise as forbidden. This demotion of the teacher to some sort of friendly facilitator is dangerous, especially in an era in which there is an unprecedented number of teachers teaching out of their fields of expertise. The focus of attention needs to be on how much teachers know about the content being taught. Students need someone to lead them through the quagmire of propaganda and misinformation that they confront daily. Students need a teacher who loves the subject and has enough knowledge to act as an intellectual authority when a little direction is needed. Students need a teacher who does not settle for minimal effort but encourages original thinking and provides substantive intellectual challenge.

Question One: The first passage suggests that reflection on which of the following after a lesson is an essential element in constructivist teaching? (The correct answer is C.)
(A) The extent to which the teacher’s knowledge of the content of the lesson was adequate to meet students’ curiosity about the topic.
(B) The differences between what actually took place and what the teacher planned.
(C) The variety of misconceptions and barriers to understanding revealed by students’ responses to the lesson.
(D) The range of cognitive processes activated by the activities included in the lesson design and implementation.

Question Two: The author of the second passage would regard which of the following teacher behaviors as essential for supporting student learning?
(A) Avoiding lecture and memorization
(B) Allowing students to figure out complex problems without the teacher’s intervention
(C) Emphasizing process rather than content knowledge
(D) Directly guiding students’ thinking on particular topics

There are several problems with this sample test item and its answer options. First, one must note the way in which the passages are titled—“Why constructivist approaches are effective” and “Why constructivist approaches are misguided”—implying that there is a research base supporting constructivist approaches (even if critics think they are misguided) and pre-empting any challenge to this assertion. However, as noted above, one key finding of the NMAP report is that high-quality research does not support the exclusive use of either a “student centered” or “teacher directed” approach; i.e., there is no body of evidence to favor the wholesale promotion of either approach (p. xxii).

Second, assertions in each passage are made to appear as a contrast to each other, implying that student achievement is the concern of the constructivists, not their critics. What teacher would fail to see constructivism as the clear winner in this debate, based on these two paragraphs? Needless to say, the ETS Web site provides no reference for this debate, if it actually took place.

Third, the supposedly correct answer to the second question, D, is in fact not an answer to the question posed but will likely convince prospective test-takers studying these sample questions (as well as school supervisors and those making the decision to require these tests for any level of licensure in the state) that non-constructivist teaching is undesirable on ethical and civic grounds. Nowhere does the second passage say or imply that critics of constructivism want teachers to directly guide “student thinking on particular topics.” If anything, it implies the exact opposite in its final sentence. But the Answers page explains why D is the best answer. “The best answer is D. The second author maintains that students require teacher guidance and a direct expression of the teacher’s expert content knowledge in order to learn most effectively. Choices A, B, and C are not consistent with this approach to teaching. Direct guidance of student’s thinking is consistent with the second author’s approach.” In other words, critics of constructivism support indoctrination.

There are probably several reasons for the way D has been worded: first, to make sure that anyone reading the second passage wouldn’t be carried away by the last sentence in the passage and come down on the side of the critics; and second, to make the test-taker recoil from any desire to be on the side of the critics. (After all, D could have been worded to reflect what the critic of constructivism does say in the last sentence.) No normal American teacher would want to be seen as an indoctrinator if that is how a teacher will be described who thinks students should be taught how to read carefully in order to understand what an author has written, rather than as someone who inculcates democratic values by letting students decide for themselves the meaning of what an author has written.

DISCUSSION AND CONCLUSIONS
The purpose of this study was to determine the extent to which licensure tests for prospective PreK-8 special education teachers assess knowledge of three major components of beginning reading instruction identified by the NRP--the development of phonemic awareness, phonics, and vocabulary knowledge--and relevant mathematics content. The question the study explored was whether their licensure tests adequately assess the knowledge they need for research-based reading instruction and for teaching mathematics. The answer is no.

Table 2 lists all the licensure tests whose descriptions were analyzed for this study, together with an estimate of the percentage of the test items on each test addressing these three components and mathematics content (often mingled with some mathematical instructional knowledge). As can be seen, the tests that are designed for aspiring special education teachers have virtually no content from either area on them. Nor do the ETS tests of “principles of learning and teaching” contain objectives referring to reading (or mathematical) instructional knowledge (one would not expect mathematics content on such tests). Only among tests that are for the most part designed for elementary teachers do we find a few that adequately address important elements of beginning reading instruction as well as some mathematical content. And in many states these tests are often required for the prospective special education teacher as well.

A major problem in the effort to upgrade the academic content of licensure programs for special education teachers is to determine exactly what their licensure tests should address and to encourage all states to enact similar testing requirements. An assessment of their professional knowledge may not be the best use of a subject matter test for aspiring special education teachers. In the eyes of some special education experts, that may seem to be the focus of a subject matter test for them. But proficiency with this kind of knowledge may more appropriately be the focus of the tests they take in their special education coursework and of their student teacher evaluations. In my professional judgment, the focus of their subject matter tests should be the subject matter knowledge that they are most likely to draw on in their work with special education students: relevant mathematics content and research-based reading instructional knowledge.

A requirement that prospective special education teachers pass with a high cut score a dedicated test of reading pedagogy that assesses evidence-based instructional knowledge, such as the test used in California, Massachusetts, or Virginia to license aspiring special education as well as elementary teachers, will help to assure the public that these new teachers will have sufficient reading instructional knowledge to be effective teachers of reading in their first three years of teaching, before value-added measures can be used to gather empirical evidence on effectiveness. But in light of compensatory scoring, the percentages on mathematics content in the tests they now take (whether in Table 2 or not) do not assure us that prospective teachers who pass these tests will necessarily have an adequate knowledge of relevant mathematics content. No current NES or ETS test for non-mathematics teachers in K-8 comes close to the 40-item mathematics section just constructed for a redesigned general curriculum test for prospective elementary and special education teachers in Massachusetts.

It is clear that the licensure tests taken by most prospective special education teachers cannot make their preparation programs teach them what research indicates they should know with respect to mathematics content or reading instructional knowledge. Instead, Title II's vague requirement may, by default, be undermining the efforts of Reading First, a very important programmatic piece of the No Child Left Behind Act of 2001, and contributing to the basically flat scores on the grades 4 and 8 long-term trend reading tests given by the National Assessment of Educational Progress for over 30 years.

Worse yet, current tests may point new special education teachers away from the programmatic requirements for Reading First. This grim conclusion is supported by the results of Rigden’s 2006 report, which found little alignment between four of the five ETS tests she examined and the requirements for Reading First, and by my analysis of ETS’s teacher evaluation instruments. The latter tools promote an educational philosophy unsupported by high quality research evidence that should have been declared a failure long ago for students with learning disabilities and abandoned after consideration of the massive federal, state, and private funds that have been allocated to efforts to improve students’ reading skills in the past three decades. Clearly, the Reading First initiative fights an uphill battle to retrain special education and elementary teachers who may be mistrained in both reading and mathematics pedagogy in their preparation programs, licensed by tests that validate their mistraining, and given professional development that reinforces their mistraining.

RECOMMENDATIONS
Since teacher licensure is a state responsibility, each state needs to undertake its own critical examination of whatever group of tests it requires for those who teach special education children or who supervise or support those who do, regardless of the test developer. It should first determine whether and to what extent the tests reflect the research-based knowledge underlying sound reading instruction and whether and to what extent they assess the mathematical knowledge they need for teaching mathematics.

Second, each state should examine the pedagogy embedded in the tests of general pedagogical knowledge that it may require along with a subject test, as well as the pedagogy promoted in the observational instruments it may also require schools to use to assess a new teacher’s classroom performance. If the tests of general pedagogical knowledge used for licensing teachers, or the observational instruments used for rehiring new teachers, subtly coerce them into adopting an exclusively student-directed approach to learning and eschewing a teacher-directed approach to teaching, these tests and instruments will undermine the benefits of sound mathematics coursework, sound reading and mathematics methods courses, as well as soundly constructed tests assessing research-based reading and mathematical instructional knowledge. The NMAP report has much to say about the need for all children to acquire “fluency with the standard algorithms” in order to achieve computational proficiency (p. xix).

Third, each state should examine its own professional teaching standards. These standards should reflect the research-based recommendations in the NRP and NMAP reports. If not, they should be revised, and the tests that the state uses should be revised to reflect these recommendations.

Finally, Title II should be amended to provide criteria for the content of all the licensure tests taken by prospective elementary, reading, early childhood, and special education teachers as well as reading specialists to ensure that these tests assess research-based reading and mathematical instructional knowledge as well as mathematics knowledge itself and thereby promote, not sabotage, the goals for Reading First and Math Now. These amendments might also recommend model tests as determined by the Institute for Educational Sciences, with financial incentives for states that develop or use sound tests. Requiring all states to use sound criteria in the development or choice of tests they require prospective teachers to take for licensure might be the most useful step Congress could take to raise the academic achievement of the students in our public schools.