FREE ELECTRONIC LIBRARY - Abstract, dissertation, book

Pages:   || 2 |

«What the NAEP Civics Assessment Measures and How Students Perform by Peter Levine with support from the S.D. Bechtel, Jr. Foundation January 2013 ...»

-- [ Page 1 ] --

What the NAEP Civics Assessment Measures and

How Students Perform

by Peter Levine

with support from the S.D. Bechtel, Jr. Foundation

January 2013

Only eight states currently test their students on American government or civics

(usually as part of a much broader social studies test), and so relatively little is

known about young people’s civic knowledge, skills, behaviors, and values.1 Given

the paucity of state data, the federal National Assessment of Educational Progress

(NAEP) in Civics receives predominant attention. The fact that only about one quarter of students typically reach the “proficient” level on the NAEP Civics assessment is probably cited more than any other statistic about civic education, and it is often used to support proposals for adding civics requirements.

Indeed, civic education deserves increased attention, and students’ knowledge may be problematic, but these interpretations of the NAEP are based on misunderstandings. This fact sheet explains how to interpret its results.

Background on the NAEP Civics assessment The NAEP system was designed to give a regular national “report card” on students’ academic proficiency to inform citizens and policymakers. Each NAEP is a test-like assessment on one subject given to randomly sampled American students. In Civics, the NAEP assessment is now administered to more than 21,000 students every three years. Equal thirds of the participants are fourth graders, eighth graders, and 12th-graders.

Compared to other prominent tests and surveys in civics, the NAEP is the most carefully designed and validated and has the largest national sample. Whereas existing state civics tests rely exclusively on multiple-choice questions, the NAEP Civics assessment also includes short essays, which are better measures of advanced skills.

Although participation is voluntary, response rates are generally high. Of the fourth graders in public schools who were asked to participate in 2010, 99% did so, along with 92% of 12th graders in public schools (but just 62% of 12th graders in private schools).2 A more significant source of bias may arise from a lack of motivation. The assessment has no stakes for individuals, schools, or districts; in fact, teachers and students are not even told their own performance. Given the lack of consequences, it is very unlikely that students study or prepare for the NAEP. Teachers have no reason to align their curricula to the NAEP’s content, unless it happens to match requirements of their state or district.

The NAEP scores do not indicate whether students’ civic knowledge is sufficient Typically, the release of NAEP Civics results is treated as evidence that students know far too little about civics. The most recent results (from the 2010 assessment) generated an article in The New York Times entitled “Failing Grades on Civics Exam Called a ‘Crisis.’”3 Only 24% of 12th graders were deemed proficient, and a similar number reached proficiency in 4th and 8th grade.4 Many reporters cited these statistics, along with specific questions on which students performed poorly. The Times story began, “Fewer than half of American eighth graders knew the purpose of the Bill of Rights.” These interpretations of the NAEP are misleading. The test is designed to produce the overall results that it yields. When the current NAEP Civics Framework was developed in 1998, a committee of teachers and other knowledgeable citizens decided how difficult each proposed question ought to be for students at each grade level, and then decided what overall score should qualify a student as having “basic,” “proficient,” or “advanced” knowledge. If members of the committee had agreed on a more—or less—demanding idea of what qualifies as success in civics, they would have set the cutoff scores either higher or lower, and those decisions would have been equally valid. The committee was informed by empirical data, but ultimately it had to make a value-judgment about what questions to ask how much knowledge is satisfactory.5 Each subsequent NAEP Civics assessment has then been carefully constructed to be as equally difficult as the 1998 assessment. Thus 24% percent of 12th graders were rated “proficient” in 2010 because the NAEP 12th grade test had been designed to yield roughly that proficiency rate.

On the most recent NAEP Economics assessment (2006), 42% of seniors reached the proficient level—18 points higher than the scores for the most recent NAEP Civics.6 That does not imply that students are more knowledgeable about economics (which receives less attention than civics in most American schools). It simply means that the economics committee defined “proficient” less rigorously than the civics committee did. Neither committee was right or wrong.

For a citizen or policymaker who wants to decide whether students know enough, there is no substitute for looking at individual NAEP items and making an independent decision about whether most students at a given grade level should know the answers. Citizens may find some NAEP items surprisingly difficult, but they may think that other items are easy and should yield mostly correct answers. At the end of this fact sheet, some examples of items are provided as illustrations.

What the NAEP results do show Although overall scores on the NAEP Civics assessment do not tell us objectively how well students perform, the assessment is highly informative if interpreted

correctly. The results can be used to learn:

1. Which students perform better and worse than the norm for their grade. The National Center for Education Statistics (NCES) reports that White students, female students, students whose parents had completed college, and students not eligible for free or reduced-price lunch scored better than their counterparts on at least one of the grade levels of the NAEP Civics in 2010.

For more detail on demographic differences in NAEP outcomes, see the NCES’s The Nation’s Report Card: Civics 2010. Other research finds that students from more advantaged background receive the most intensive and engaging opportunities to learn civics in schools; unequal inputs may in part explain unequal results.7

2. How students’ knowledge has changed over time. Some questions have been repeated on each NAEP Civics assessment since 1998 to establish trend lines. The difficulty of the whole test is then calibrated by comparing performance on the other items to these repeated questions. Although the way the NAEP is designed makes it resistant to change in the mean scores, some shifts have occurred. At 4th grade, the trend has been upward since 1998; at 12th grade, scores have declined since 2006. Those findings are surprising in light of the fact that time devoted to social studies has shrunk in the early grades—under pressure from testing and standards—but it has increased in high school.8 For details on trends in scores, see The Nation’s Report Card: Civics 2010.

3. Which educational practices are related to higher scores. Students who take the NAEP are asked about the frequency of various instructional practices, from completing worksheets in class to conducting debates. Their teachers are also surveyed about these practices, giving two separate perspectives on what happens in the classroom. In a subsequent fact sheet, CIRCLE will explore how instructional practices relate to students’ scores on the NAEP.

4. How well students understand various specific topics. By examining performance on individual items, we can learn more about which topics American students are mastering and which ones may need more attention.

We will turn to that issue at the end of this fact sheet. But first, it is important to understand what the NAEP measures.

What the NAEP Measures

According to the Framework document that has governed the NAEP since 1998,9 20% of the NAEP Civics assessment should focus on “American constitutional government and its history” and the “distinctive characteristics of American society and American political culture” that “are linked to American constitutional democracy.” Another 25% of the items should assess students’ understanding of how “the government established by the Constitution [embodies] the purposes, values, and principles of American democracy.” Thus a total of 45% of the test should relate to the US constitutional system, defined broadly.

Twenty-five percent of the items should involve the role of citizens. Because the NAEP may not assess students’ actual civic participation, students are instead asked to identify and explain actions taken by citizens in historical or hypothetical situations. For example, students cannot be asked whether they volunteer, but they could be asked why some citizens volunteer. Another 10% of the items must address the nature of civic life, politics, and government; the remaining 20% of the test investigates the role of the US in the world.

Perhaps the most succinct summary of what the NAEP measures comes from an

official definition of civics “proficiency” at the 12th-grade level:

Twelfth-grade students performing at the Proficient level should have a good understanding of how constitutions can limit the power of government and support the rule of law. They should be able to describe similarities and differences among constitutional systems of government, and they should be able to explain fundamental American democratic values, their applications, and their contribution to expanding political participation. They should understand the structure of American government and be able to evaluate activities of political parties, interest groups, and media in public affairs. They should be able to explain the importance of political participation, public service, and political leadership. They should be able to describe major elements of American foreign policy and the performance of major international organizations.10 What qualifies as a “good understanding” is a matter of judgment, not statistical fact.

Fig. 1: Comparing the Civic Mission of Schools, state standards, and the NAEP Figure 1 puts those specifications in comparative context. The NAEP’s breakdown is shown in the right-hand column. At the left are the outcomes deemed essential in the Civic Mission of Schools report, which was written by 60 diverse experts and led to the formation of the Campaign for the Civic Mission of Schools.11 The outcomes that

they selected were:

 civic skills (such as working with other people to address a public problem),  civic values (such as concern for the common good and the rights of others),  civic engagement (such as voting and volunteering), and  civic knowledge, including knowledge of law, history, politics, and community life.

The Civic Mission of Schools did not suggest how these four categories should be weighted, so they are shown as equally important in the left-hand column.

Immediately, one can see that the Civic Mission of Schools outcomes differ from the measured topics in the NAEP Civics Assessment. The NAEP is weighted far more heavily toward knowledge, and especially knowledge of the US Constitution. In fact, the entire NAEP could be described as a test of knowledge, although students are asked some factual questions about civic engagement under the heading “role of citizens” and may be asked to display such academic skills as interpreting a cartoon or giving reasons for political position. Since the NAEP is a paper-and-pencil test taken privately by individual students, it does not directly assess students’ ability to collaborate or deliberate in groups. By statute, it may not ask about behaviors outside of school or about values.

The middle column indicates the topics that are typically emphasized in state standards. All states have standards for civics, and most draw on the voluntary National Curriculum Standards for Social Studies from the National Council for the Social Studies.12 CIRCLE recently reviewed all the states’ standards and identified specific elements that matched the categories in the National Curriculum document.

In figure 1, those categories are shown in proportion to how many states require them. For example, all states and the District of Columbia have standards for “civic skills,” but only 12 states have standards about “individual identity and development,” so civic skills are shown as a larger portion of the column. Overall, the central column of the graph gives a rough, aggregate portrait of what American students are expected to learn.

Comparing the three columns reveals that:

“Civic skills” are emphasized in the Civic Mission of Schools report and are  mentioned in most state standards, but they do not represent one of the tested categories in the NAEP. (Some NAEP items arguably require civic skills, such as interpreting a speech, to answer correctly.) “Civic engagement” is cited as an explicit outcome in the Civic Mission of  Schools and is often included in state standards, but it is not directly measured in the NAEP.

 One fifth of the NAEP is devoted to international or global issues, which were not mentioned at all in the Civic Mission of Schools and are not common in state standards.

The “role of citizens” represents 25% of the NAEP and overlaps with two  categories in state standards (“civic skills” and “civic ideals and practices”) as well as two outcomes in the Civic Mission of Schools framework (“civic skills” and “civic values/dispositions”). However, the NAEP items on this topic are much more academic, consisting of questions about how and why citizens act, instead of measuring actual actions by students.

 The US Constitution and the foundations of the US political system are much more prominent in the NAEP than in state standards or the Civic Mission of Schools.

Is the NAEP aligned with what students learn?

The NAEP is a federal assessment with no consequences for students or schools. In contrast, state standards are legal documents adopted by state governments that are meant to govern curriculum, instruction, textbooks, and tests. Nevertheless, the NAEP Framework may reflect the typical emphasis in classrooms better than the state standards do.

In a large, nationally representative survey of 4,483 young adults conducted in 2012, CIRCLE found that 44.9% of young adults remembered “the Constitution or the U.S.

Pages:   || 2 |

Similar works:

«Whyalla Vintage & classic Car Club Inc JUNE 2015 RAY AND SHILEY HISCOKE’S 1941 CHEVROLET Cover supplied by Ziegler Office Products Playford Ave, Whyalla Whyalla Vintage & Classic Car Club Inc P.O.BOX 2288, WHYALLA NORRIE, SA, 5608 Email address; whyallavccc@hotmail.com. Website; Whyallavcc.com COMMITTEE MEMBERS PRESIDENT PETER KELLAWAY 0428 838 910 V.PRESIDENT JOHN PEATFIELD 8645 5939 / 0403 455 939 SECRETARY SHIRLEY RICHARDSON 8645 5068 TREASURER JEAN TAYLOR 8645 5601 EDITOR BOB MORRIS...»

«Proletarier aller Länder, vereinigt euch! Eure proletarische Internationale i s t nicht untergegangen, und sie w i r d nicht untergehen! Es lebe die III. Internationale Lenins und Stalins! Es lebe die Komintern/ML! Weltrevolution proletarische Strategie und Taktik Generallinie der KOMINTERN / ML (Kommunistische Internationale / Marxisten-Leninisten) am 7. November 2009 = Namensänderung KOMINTERN (SH) (Kommunistische Internationale [ Stalinisten -Hoxhaisten ] ) (erste deutsche Fassung vom 6....»

«Double-Resonance Studies on Compact, High-performance Rubidium Cell Frequency Standards Thejesh N. Bandi This thesis presents experimental studies on continuous-wave (CW) laser-microwave double-resonance (DR) spectroscopy and metrology in rubidium (87Rb) vapor cells in view of new high-performance, compact Rb-cell atomic clocks. Two different approaches were studied in this work; the wall-coated cell approach and the enlarged cell buffer gas cell approach. New magnetron-type cavity microwave...»

«David Lloyd Roberts David Lloyd Roberts STAYING ALIVE Safety and security guidelines for humanitarian volunteers in conflict areas International Committee of the Red Cross 19 Avenue de la Paix 1202 Geneva, Switzerland T +41 22 734 6001 F +41 22 733 2057 E-mail: icrc.gva@icrc.org www.icrc.org Q ICRC,1999 Revised and updated edition, 2005 DEDICATION To my wife Charlie. NOTICE: This book contains a series of guidelines for the safety and security of staff in the field. It does not cover all...»

«Drinking Water Quality Monitoring Item type text; Electronic Dissertation Authors Kilungo, Aminata Peter Publisher The University of Arizona. Rights Copyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author. Downloaded 29-Apr-2016 15:41:49 Link to item...»

«Gewalt gegen Kinder in der EU Wo stehen wir heute? EINGEHENDE ANALYSE EPRS | Wissenschaftlicher Dienst des Europäischen Parlaments Autor: Anna Dimitrova-Stull DE Wissenschaftlicher Dienst für die Mitglieder November 2014 – PE 542.139 (or. FR) In diesem Dokument soll ein Überblick über das Problem der Gewalt gegen Kinder in der EU gegeben werden. Ausgangspunkt hierfür ist die besondere Verletzlichkeit von Kindern, aus der sich eine spezielle Schutzbedürftigkeit ergibt. Die qualitative...»

«NO FAITHFUL OATHS: A COMPARISON OF ESAU’S SPEECH IN JUBILEES 37:18-23 WITH ACHILLES’ SPEECH IN ILIAD 22.260-272 ! by JAMES R. HAMRICK ! A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS in THE FACULTY OF GRADUATE STUDIES Master of Arts in Biblical Studies ! We accept this thesis as conforming to the required standard ! ! ! Dr. James M. Scott, Dr. theol.; Thesis Supervisor ! ! ! Dr. Dirk Büchner, D. Litt.; Second Reader ! ! ! ! ! ! ! TRINITY...»

«Networking in Sport Management: Ideas and Activities to Enhance Student Engagement and Career Development D.O.I: http:dx.doi.org/10.4127/ch.2013.0074 Alan S. Kornspan, Michael A. Duve Department of Sport Science and Wellness Education, University of Akron Abstract The primary purpose of this paper is to present information regarding the development of networking skills to enhance the career development of sport management students. Specifically, literature is reviewed which supports the...»

«11.12.2015: Gültigkeit der Leitlinie nach Prüfung durch das Leitliniensekretariat bis zum 14.02.2017 verlängert Magenkarzinom Diagnostik und Therapie der Adenokarzinome des Magens und ösophagogastralen Übergangs AWMF-Register-Nummer (032-009OL) Kurzversion 02.2012 0 Inhaltsverzeichnis Inhaltsverzeichnis 1. Informationen zu dieser Kurzversion 1.1. Herausgeber 1.2. Finanzierung der Leitlinie 1.3. Federführende Fachgesellschaft 1.4. Kontakt 5 1.5. Besonderer Hinweis 1.6. weitere...»

«Prioritäten des Europäischen Forschungsraums: Sachstand in den Organisationen, die Forschung fördern und betreiben Der Europäische Forschungsraum (EFR) wurde im Jahr 2000 ins Leben gerufen, und Organisationen wie Ihre machen diesen Tag für Tag zur Realität. Allerdings sind noch viele weitere Fortschritte notwendig. Deshalb enthält die Mitteilung der Kommission über Eine verstärkte Partnerschaft im Europäischen Forschungsraum im Zeichen von Exzellenz und Wachstum...»

«THE WANDERING IMAGE: CONVERTING THE WANDERING JEW By Joanna L. Brichetto Thesis Submitted to the Faculty of the Graduate School of Vanderbilt University in partial fulfillment of the requirements for the degree of MASTER OF ARTS in Religion May, 2006 Nashville, Tennessee Approved: Professor Jay Geller Professor Leonard Folgarait To Michael and Margaret ii ACKNOWLEDGEMENTS This work would not have been possible without the generous financial support of the Vanderbilt Graduate Department of...»

«Hoffman, J., & Vallejo Peña, E. (2013). Too Korean to be White and too White to be Korean: Ethnic identity development among transracial Korean American adoptees. Journal of Student Affairs Research and Practice, 50(2), 152–170. doi:10.1515/jsarp-2013-00012 Available at http://dx.doi.org/10.1515/jsarp-2013-00012 Innovations in Research and Scholarship Feature Too Korean to be White and Too White to Be Korean: Ethnic Identity Development Among Transracial Korean American Adoptees Joy Hoffman,...»

<<  HOME   |    CONTACTS
2016 www.abstract.xlibx.info - Free e-library - Abstract, dissertation, book

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.