Date: Mon, 23 Apr 2001 21:47:07 -0500
From: Annie Selden 
To: ARUME-LIST@ENTERPRISE.MAA.ORG
Subject: [ARUME] Wanted:  Information on placement tests

The MAA is no longer providing placement tests, but there seems
to be a great need for them.  [See message below.]
Does anyone know anything about them?

>   Dear Professor Selden,
>
>
>   I have contacted the MAA webpage on mathematics education seeking
>information on placement examinations in mathematics.
>Since the MAA will no longer produce placement tests.
>Penn State,  like perhaps many othe universities,
>needs to identify a new system of placement testing.
>Administering examinations on the web
>has become  very appealing to university adminstrators.
>The math department at Penn State is concerned, though,  with giving up
>examination security, and of course, the potential instructional impact on
>precalculus and calculus.
>I am looking for any studies, articles, materials, names of people that
>may have dealt with such issues.
>I would very much appreciate any tip along such lines.
>
>
>   Sincerely,
>
>
>    Carme Calderer
>    Professor of Mathematics
>    Associate Chair for Undergraduate Mathematics
>    mcc@math.psu.edu

Date: Mon, 23 Apr 2001 21:28:21 -0600
From: Lynne K Ipina 
To: ARUME-LIST@ENTERPRISE.MAA.ORG
Subject: Re: [ARUME] Wanted:  Information on placement tests

AMATYC has a standing committee on placement ... below is the info about
that committee from their website:

        Placement and Assessment
To provide information and serve as resource and support for members and
their institutions on matters regarding placement and assessment.

Chair: Judy Marwick
Morton College
3801 S. Central Avenue
Cicero, IL 60804
(708) 656-8000 ext. 331
marwickj@morton.cc.il.us
PAC web page:

The colleges in Wyoming have been using the COMPASS exam.  For inofrmation
about the ins and outs of that you might contact Jeanette Reisenburg
(email address below); she is frequently asked by the COMPASS people to
train others in the implementation and she is the heart of the WY network.

                        jeanette@uwyo.edu

Hope this is useful.
                                        Lynne Ipina
                                        Math Dept
                                        Univ of Wyoming
                                        

Date: Mon, 23 Apr 2001 20:37:17 -0700
From: Bruce Cooperstein 
To: ARUME-LIST@ENTERPRISE.MAA.ORG
Subject: [ARUME] placement examinations

The University of California, some California Community Colleges, and
perhaps California State Universities (they do have their own exam,
Entry Level Math [ELM]) use the Mathematics Diagnostic Testing
Project examinations.  These run from algebra readiness through
calculus readiness.  I forwarded the query to Alfred Manaster,
Professor at UC San Diego and the executive director of MDTP.

Bruce Cooperstein, Professor
Math Dept
UC Santa Cruz

Date: Mon, 23 Apr 2001 21:33:14 -0700
From: Cathy Kessel 
To: ARUME-LIST@ENTERPRISE.MAA.ORG
Subject: Re: [ARUME] Wanted:  Information on placement tests

ETS deals with security issues, also test design (some ETS people 
just gave talks at AAAS about these, so I think it's probably a hot 
topic for them). Abstracts of some articles on this subject are on 
the ETS Web site:

 
(It might also be a good idea to check out the GRE Web site, 

 Another issue that should 
concern people who use placement tests is predictive validity, in this 
case, how well the test predicts performance in the course. I got 
the impression that MAA didn't do many studies of how well its placement 
tests predicted performance (anyone who knows otherwise, please correct 
me if I'm wrong--it's been a while since I tried to get any information 
from MAA about this). The lack of such studies might be one reason why 
MAA stopped providing placement tests.

Here are the names of a few ETS articles that concern computerized testing.

Stocking, M. L., Smith, R., & Swanson, L. An investigation of approaches 
to computerizing the GREŽ Subject Tests (GRE Board Technical Report 
No. 93-08P, ETS RR-00-4). [This article concerns design.]

Gallagher, A., Bridgeman, B., & Cahalan, C. The effect of computer-based 
tests on racial/ethnic, gender, and language groups (GRE Board Professional 
Report No. 96-21P; ETS RR-00-8).

Wainer, H. CATs: Whither and whence (ETS RR-00-12). ["CAT" stands for 
computer adaptive test.]

--Cathy

Date: Tue, 24 Apr 2001 05:50:21 -0400
From: Ann Marie Murray 
To: ARUME-LIST@ENTERPRISE.MAA.ORG
Subject: Re: [ARUME] Wanted:  Information on placement tests


At Hudson Valley Community College we use the ASSET test which is 
part of the American College Testing Service (ACT).  It is geared 
toward two-year programs but has an elementary algebra and advanced 
mathematics section (college algebra and  geometry).  We administer 
the college algebra test  to all  Calculus I students during the first 
week of the semester.  It correlates highly with performance in Precalculus.  
We are presently studying the correlation between the test and performance 
in Calculus I.

Ann Marie Murray

Date: Tue, 24 Apr 2001 07:02:11 -0500
From: Mercedes McGowen 
To: ARUME-LIST@ENTERPRISE.MAA.ORG
Subject: Re: [ARUME] Wanted:  Information on placement tests

We've been using COMPASS as our placement test at Harper for the past
three years.  The Math Department reviews the placement cut-off
scores for all our courses each year, occasionally making adjustments
to our cut-off scores for various courses.

I've been conducting an analysis of three years of success in a given
course via method of placement:  ACT, SAT or teacher recommendation,
COMPASS, prior course.

Preliminary analysis indicates that, for our developmental courses
(Beginning and Intermediate Algebra), a student's ACT or SAT score is
the best predictor of success as a method of placement: Beginning
Algebra 58.8%, 40% and 50% for the three years 1998, 1999, 2000 and
Intermediate Algebra: 78%, 67%, and, this past year, 35% (I'm still
investigating this).  COMPASS was more consistent, though it also was
less a good predictor the past two years (54%, 53% and 45%-Beginning
Algebra and 62%, 54% and 52% for Intermediate Algebra).

For the College Algebra course, a student's ACT or SAT score was the
most consistent predictor with success in the course via that method
of placement:  56%, 60% and 59%.  COMPASS was 59%, 43% and 51%.

For Calculus, a student's ACT or SAT was the best predictor: 67%,
62%, and 65%.  Those who entered Calculus I via placement by COMPASS
had success in the course: 58%, 55% and 48%.

As we adjusted our COMPASS cut-off scores a year ago, I still need to
analyze the effect of those changed cut-off scores on the success
rates.  As the 2000 COMPASS success percentages were lower for every
class, with the exception of College Algebra,  the changed cut-off
scores might have had some impact, but, I won't know until further
analysis is completed. However, since the ACT/SAT placement success
rate also dropped in 2000 in most courses,  other factors probably
are impacting those rates.


As it is almost the end of the semester, I will not be continuing my
analysis until this summer.  However, I hope to have the study
completed, taking into account the impact of changed cut-off scores
by the end of July.  If anyone is interested, they can contact me.

Mercedes McGowen
--
Dr. Mercedes A. McGowen
Associate Professor, Department of Mathematical Sciences
William Rainey Harper College
1200 West Algonquin Rd.
Palatine, IL 60067

voice mail: (847) 925-6526
FAX: (847) 925-6049

e-mail: mmcgowen@harper.cc.il.us
web: http://www.harper.cc.il.us/mathdept/index.htm

M. McGowen continues, later the same day: 

I will send you results of whatever we've found out.  However, I
suspect that the overall conclusion will be that NONE of the
currently available placement methods is a great predictor of
success. Our results vary from year to year, and, even when we've
raised cut-off scores, it seems the succdess based on placement tests
OR based on the ACT.SAT results are lower than BEFORE the cut-off
scores were raised.

Obviously there are other factors involved.

An interesting analysis (not yet completed) is to compare success via
placement (COMPASS or ACT/SAT) with success via placement from prior
course at your own institution.

The hard reality to face is that none of the placement methods is
doing the job at Harper for all courses consistently over several
years and that our success rates, regardless of method are less than
we would like to see.

I'll be happy to share the data as soon as we complete the study this summer.

Regards,
Mercedes McGowen

Date: Wed, 25 Apr 2001 13:23:38 -0500
From: Annie Selden 
To: ARUME-LIST@ENTERPRISE.MAA.ORG
Subject: [ARUME] Placement Tests -- MDTP

Thanks to all of you who responded to my
request for information on placement tests.
This has been forwarded to Carme Calderer
at Penn State who made the initial inquiry of me.

Below find some information on the California
Mathematics Diagnostic Testing Project (MDTP)
from Alfred Manaster at UCSD.

Annie Selden

>Dear Annie,



>As you can see, Bruce Cooperstein forwarded your query on the ARUME
>list-serve to me. Here are my thoughts concerning placement tests for
>colleges that were well-served by the MAA Placement Testing Program until
>the MAA discontinued that program. I will also try to address some of the
>issues Carme Calderer raised.



>I am convinced that as soon as a test appears on the internet it cannot be
>considered at all secure. Since it can be viewed on any screen, it can be
>copied by hand, by screen-shots, and by photographs. That said, the
>position that security should not be an issue is arguable since the
>purpose of a placement test is to help students select the courses best
>suited to their current knowledge; from that perspective cheating only
>hurts the student. Of course there are many counter-arguments. A leading
>one is that appropriate placement not only benefits the individual student
>who is as well placed as possible given our knowledge of his or her
>preparation but also benefits all students and faculty by minimizing the
>need for on-the-spot remediation or downgrading of courses to accommodate
>ill-prepared students as well as by minimizing the negative effects on the
>class environment that can come from over-prepared students.



>Extremely secure tests are of necessity expensive because they need to be
>changed frequently. Test development is also expensive since a good test
>needs to be well designed by both content and assessment experts at both
>an item and a test level. After initial design and assembly, a good test
>has to be field-tested on appropriate student populations, refined, and,
>often, field-tested again to establish its validity.
>One of the strengths of the PTP was that it offered not only tests but
>also the ability to custom-design tests from a large item bank. Very few
>schools took advantage of this service directly although we know some
>schools created their own versions of PTP tests, sometimes adding their
>own items. I do not know of a similar product that is readily available to
>schools.



>The California Mathematics Diagnostic Testing Project (MDTP), on which I
>have worked for the past 20 years, has a series of tests that have been
>very well received in California. These tests measure readiness for
>courses from first year algebra through beginning calculus. They are not
>secure but are far more secure than any internet-delivered test. The tests
>are currently available for use by educational institutions in
>paper-and-pencil format and will very soon be available in a computer
>delivered (which is not internet delivered) format. More information,
>including license and cost information is available at the MDTP web-site:
>

>Annie, I hope this is helpful to you and your readers. Please let me know
>if there is other information that I can provide.

>Cordially,
>Alfred

>Alfred Manaster
>Department of Mathematics
>University of California, San Diego
>9500 Gilman Drive
>La Jolla, CA 92093-0112
>Voice: (858)534-2644 Fax, include name clearly displayed: (858)534-5273

From kessel@SOE.BERKELEY.EDU Wed Apr 25 14:05:20 2001
Date: Tue, 24 Apr 2001 12:14:05 -0700
From: Cathy Kessel 
Reply-To: MAA ARUME-list 
To: ARUME-LIST@ENTERPRISE.MAA.ORG
Subject: Re: [ARUME] Wanted:  Information on placement tests

Hi Mercedes,

Have you thought of analyzing your data by gender? There are two reasons: 
the (old) SAT tends to underpredict women's course grades relative to those 
of men, and there may be legal issues involved. There are several studies 
showing the former, one with a large data set for courses in remedial 
mathematics, regular mathematics, precalculus, calculus, and courses 
beyond calculus. Of the legal issues, Connor and Vargyas say, "Even 
where validity studies may exist, fairness remains a major concern in 
the analysis of gender discriminatory test uses. For example, users who 
do not separately validate their use of SAT scores by gender when such 
use results in a disparate impact take a substantial risk" (p. 57). I've 
often wondered if people using placement tests worry about such things--or 
for that matter, people who use SAT scores in research. (There would be 
a similar concern about ethnicity.)

--Cathy

Connor, K. & Vargyas, E. (1992).  The legal implications of gender bias 
     in standardized testing.  Berkeley Women's Law Journal, 7.