Based on the research cited above, one can argue that of the nearly two million applicants who take the SAT each year, some will produce SAT scores that are reflective of their true ability to succeed over time in college, and some will produce scores that are inaccurate in one way or another. Of particular interest to this study is the "false negative" SAT score, i.e., a score which predicts poorer academic performance than a student will actually display.
Approximately two-thirds of colleges and universities in the United States employ the SAT as at least one measure in their selection process. According to Gandara and Lopez (1998), SAT scores will be weighted anywhere from "almost not at all" to "heavily" in the admission decision, depending upon the college or university. Thus, a false negative SAT will surely have an adverse impact on some number of admission decisions. For example, in the early 1980s, Connecticut's technical colleges had a cut-off for admittance of 900 on the SAT total score. Because of this, about 80% of the black students who took the test in 1984 would have been denied admittance (Crouse & Trusheim 1988). In California, where Latinos comprise 40% of the public school enrollment, only 3.9% of graduating Latino students were eligible for enrollment into the University of California system in 1994, based on their test scores and high school grades (Latino Eligibility Study 1994). In cases such as these, it is reasonable to conclude that at least some of those who were not admitted to college probably had false negative SAT scores.
Other research, however, shows that differences between minority and non-minority scores are not as great as once thought. In 1993, Weis and Fine conducted a study in reaction to research which suggested that the SAT displayed bias against certain groups. Their results indicated that SAT scores, when used for predicting college grades, do not show systematic differences between minority and non-minority students. Thus, in the absence of a predictable bias in test results, the likelihood is raised that whites and non-whites are both just as liable to fall into the false negative category. Because of this, it is impossible to tell by race or ethnicity which students' low SAT scores are faulty indicators of true ability.
It may be presumed that the false negative phenomenon is rooted in either the test, the test taker, or in the interaction between the two. For example, a test or its scoring might be biased, a possibility suggested in some of the research cited above. Or, a test-taker may fall into the category of "underachiever," a person whose performance on one measure, such as the SAT, is below what one would expect from some other, prior measure of ability, such as high school grades. Researchers speculate that underachievement results from lack of internal locus of control (Rose, Hall, Bolen and Webster 1996; Rotter 1966) and/or poor functioning in competitive situations (Rimm 1997).
Another possibility is that some students with the ability to succeed in college simply do not do well on SAT-like standardized tests for any of a variety of reasons unrelated to underachievement. A representative discussion of this line of thinking can be found in Gorman (1996). A student who does not perform well on the SAT, but subsequently succeeds in college, may be referred to as an Exmet (from the Latin: ex, outside of, and metricus, measuring). Operationally defined, for the purposes of this study an Exmet is a college student whose most recent combined total SAT score was 1000 or below prior to admission, but whose college grade point average after at least two or more semesters was 2.8 (on a 4.0 scale) or above. Unlike the underachiever, the Exmet does better on one measure, such as college grade point average, than predicted by another prior measure, such as the SAT.
There clearly are students, even though their precise numbers are not known, whose academic achievements in college exceed the predictions of one of the most widely used selection devices, the SAT. Anecdotal evidence gathered from college faculty and administrators as well as self-reports of students overwhelmingly confirms the Exmet construct.
Because it is known that there are Exmets, and that by definition the SAT will not accurately assess their true ability to do college work, it follows that a search for other more accurate predictor(s) of college performance would be worthwhile.
The original Scholastic Aptitude Test was intended as an intelligence test which could be used for college admission (Lemann 1999). High scores in certain "aptitudes" were assumed to demonstrate the intellectual abilities needed to be successful in higher education. They were also assumed to be independent of the level of a test taker's specific prior training, whether superior or poor (Lawler and Richman 1997). The SAT measures such aptitudes as comprehension in reading, vocabulary and verbal reasoning, and quantitative problem solving (Hanford 1985).
Certain non-test measures have been shown to correlate with freshman college grade point average as well or better than SAT scores. In particular, high school grade point average and high school class rank have been demonstrated in various studies to be about equal to, and sometimes better than, SAT scores as predictors of success in college (Baron & Norman 1992; Educational Testing Service 1994).
Other alternative measures which have been advanced as possible predictors of college performance include academic self-concept (Rodriguez 1996; Sedlacek 1989), grades in specific high school courses (Elliott and Strenta 1990), educational aspirations (Arbona and Novy 1990), and the ability to cope with racism (Fuertes and Sedlacek 1994; Tracey and Sedlacek 1989).
Sedlacek (1994), citing Sternberg's work on intelligence (1985, 1986), argued as a general principle that non-traditional groups may demonstrate their true academic aptitudes in ways not shown on the most widely-used standardized tests such as the SAT. Sternberg suggested that there were at least three different types of intelligence. Analytic intelligence is the ability to interpret data in a hierarchical fashion in well-defined and stable contexts. His second ("synthetic") and third ("systemic") types of intelligence involve the ability to interpret information in changing contexts, and the ability to adapt to a dynamic environment. Sternberg further contended that typical standardized tests were never intended to, and apparently do not, measure intelligence other than the analytical type.
The purpose of this study was to search for an alternative to the SAT which displayed greater power to predict the college performance of Exmets. Specifically, the intent was to find an alternative test or testing protocol with a significant correlation between its score and college grade point average, a correlation that was higher than that between the SAT and grade point average for Exmets. The approach followed Sternberg's general concept, i.e., to identify measures other than analytic intelligence that would correlate significantly with Exmets' college performance. Selected for evaluation were measures of: (1) critical thinking ability, (2) motivation, and (3) locus of control. Critical thinking ability was judged to be an indicator of Sternberg's "synthetic" intelligence. The latter items were judged to be congruent with Sternberg's "systemic" intelligence.
A number of commercially available validated tests that measured critical thinking ability, motivation and locus of control were screened for inclusion in this study, Among them were: Adult Personality Inventory, California Critical Thinking Disposition Inventory, Campbell Interest and Skill Survey, Christophel's Student Motivation Scale, Guilford-Zimmerman Temperament Survey, Rotter's Locus of Control, 16 Personality Factor Questionnaire, and Work Motivation Inventory.
Second and third-year college students enrolled in a variety of majors were recruited in spring, 1998 to take a series of tests from among those listed above. One sample was from a large state university (n = 299), and the other from a small, liberal arts college (n = 110) both located in the eastern United States. Each school was rated as "selective" in terms of its admission policies by a well-known national guide to colleges and universities. The students ranged in age from 19 to 25, and included European-, African-, Latino/Latina- and Asian-Americans of both genders.
Participants were asked to report their present college grade point averages (response options ranged from 1.1 to 4.0), their last combined total SAT scores (response options ranged from 700 to 1550), and their high school grade point average (again, response options ranged from 1.1 to 4.0). Each participant was then given a package of four test instruments selected from the test packages previously mentioned.
Tests were scored and the results analyzed. The persons who completed the entire protocol generated 29 individual sub-scores, each of which reported on one purported characteristic of synthetic or systemic intelligence. The 29 sub-scores represented indices of characteristics that the various tests purported to measure. Typical examples of these were Competitive, Assertive, Scientific, Independent, Tough-Minded, and other such attributes.
Correlations were computed to assess the relationship between each of the 29 sub-scores and college grade point average. Five of the sub-scores (Creative, Truth-Seeking, Adjusted, Adapting, and Social) yielded correlation coefficents that were moderately to highly significant. These five sub-scores were combined into a test variable named ACCESS. A series of correlation coefficients was then calculated between the composite ACCESS variable and the college grade point averages of several subsets of students: (1) ACCESS and grade point average of all students in the sample, (2) ACCESS and grade point average of all Exmets (last combined SAT score of 1000 or less; present college grade point average of 2.5 or better) in the sample, and (3) ACCESS and grade point average of Exmets who were twenty years old or less. Table One displays these correlation coefficients.
As Table One demonstrates, the ACCESS variable was found to be a strong predictor of college grade point average for students who fall into the Exmet category, especially those Exmets who were age twenty or younger. By way of comparison, according to the College Board's research, the SAT's correlation with freshman grade point average for all students was r = .53, but the correlation between SAT and freshman grade point average for what the College Board called the "low" academic group (Ramist, Lewis & McCamley-Jenkins 1994), was only .43.
To compare the predictive ability of the ACCESS score with other known indicators of college performance, correlations were also computed among Exmets' scores on the ACCESS variable and their college grade point averages, their SAT scores, and their high school grade point averages. The association between SAT scores and college grade point average was found to be non-significant for this group (r = .05). However, a moderate relationship was observed between high school grade point average and college grade point average for Exmets (r = .35, p (less than or equal) .001). Given these findings, a second order partial correlation was conducted to control for any confounding effects of high school grade point average and SAT scores on the predictive capability of the ACCESS variable. This correlation, r = .58, p (less than or equal) .001, indicates that, for Exmets, the "pure" relationship between ACCESS and college grade point average was strong.
Finally, a step-wise regression analysis was performed to identify which of three variables, ACCESS, SAT or high school grade point average, emerged as the best predictor of college grade point average for students classified as Exmets. The regression equation was highly significant F = 17.87, p (less than or equal) 001 . Of the three, ACCESS was the only significant predictor of college grade point average (r = .61, (less than or equal) .001; r2 = .37, p (less than or equal) .001; r2 change = .37, p (less than or equal) .001). Neither SAT nor high school grades were found to be predictors of college grade point average of nearly the same magnitude as was the ACCESS score.
The ACCESS variable correlated with college grade point average substantially higher than did the SAT for Exmets. Also, the ACCESS variable correlated much higher with college grade point average for Exmets than it did for non-Exmets. Moreover, the ACCESS variable was an even more effective predictor of college grade point average with younger test subjects. These data show that the ACCESS variable captures something about Exmets (but only Exmets) that the SAT misses when attempting to predict their performance in college. Such results strongly suggest that a testing protocol that measures the ACCESS variable, if administered post-SAT to high school students whose combined SAT score is less than 1000, will serve to discriminate among those for whom the SAT is an accurate predictor, and those for whom it represents a false negative. Applicants with SAT scores between 700 and 1000 are among the most difficult cases for college admission officers to decide.
In general terms, each of the sub-scales of the ACCESS variable assessed characteristics of test-takers that appeared consistent with both of Sternberg's synthetic and systemic intelligence constructs. One ACCESS component measured the extent to which the respondents felt comfortable in settings that required them to interact with others. A second measured the respondents' level of absorption in ideas, imagination and creativity. A third measured the extent to which respondents were secure, and able to succeed independent of support and approval from external sources. A fourth measured the propensity of respondents to seek knowledge, even that which was contrary to self-interest, personal beliefs or preconceptions. The fifth measured respondents' tolerance to divergent views. The results of this study may provide general empirical support for Sternberg's postulates.
Finally, it is important to note that all of the research on ACCESS has been conducted ex post facto, i.e., on students who had already been admitted, and who had completed at least one year of college before the ACCESS test was administered to them. The most conclusive evidence of the ability of the ACCESS variable to predict the college performance of applicants who are Exmets will come from a longitudinal study that tracks over time the college grade point averages of high scorers on the ACCESS test.
Jeffrey G. Hunter is an assistant professor of management in both the undergraduate and graduate divisions of business studies at Assumption College (MA). He has also taught at La Salle University (PA), Villanova University (PA) and the University of California at Berkeley. From 1991-1998, Dr. Hunter served as a consultant and advisor of higher education for the government of the Republic of Khakassia in the Russian Federation. Dr. Hunter earned his D.B.A. at Golden Gate University (CA). He has published articles in International Educator and the Journal of Contemporary Business Research.
Wendy Samter is an associate professor of communication at the University of Delaware. She has also taught at the University of San Francisco (CA), University of Illinois, the University of Maryland and the University of Rhode Island. Dr. Samter earned her Ph.D. at Purdue University (IN). She has published numerous articles and books on communicative development throughout childhood and adolescence.
Table One Correlation Coefficients Between ACCESS Variable and College Grade Point Averages for Student Sub-Samples.
(TABLE)TEST ALL STUDENTS EXMETS EXMETSUNDER 21ACCESS .28(FN*) .61(FN**) .80(FN**).
* indicates p (less than or equal) .05.
** indicates p (less than or equal) .001.
Arbona, C. & Novy, D.M. 1990. Noncognitive dimensions as predictors of college success among black, Mexican American and white students. Journal of College Student Development 31: 415-422.
Baron, J., & Norman, N.F. 1992. SATs, achievement tests, and high school class rank as predictors of college performance. Educational & Psychological Measurement 52 (4): 1047-1056.
Crouse, J. & Trusheim, D. 1988. The case against the SAT. Chicago: University of Chicago Press.
Elliott, R., & Strenta, A.C. 1990. Is the SAT redundant with high school record in college selection? Research and Development Update. New York: The College Entrance Examination Board.
Fuertes, J. & Sedlacek, W. 1994. Predicting the academic success of Hispanic college students using SAT scores. College Student Journal 28: 350-352.
Gandara, P., & Lopez, E. 1998. Latino students and college entrance exams: How much do they really matter? Hispanic Journal of Behavioral Sciences 28 (1): 17-39.
Gorman, A. 1996. Low SAT scores can't hold them back. Bronx Beat Online, January, 15, 1966.
Hanford, G. H. 1985. Further comment: Yes, the SAT does help colleges. Harvard Educational Review 55 (3): 324-331.
Latino Eligibility Study. 1994. Report No. 4 Santa Cruz, CA: University of California Press.
Lawlor, S. & Richman, S. 1997. The validity of using the SAT as a criterion for black and white students' admission to college. College Student Journal 31 (4): 507-516.
Lehmann, N. 1999. The big test: The secret history of the American meritocracy. New York: Farrar, Straus, and Giroux.
Ramist, L., Lewis, C., & McCamley-Jenkins. 1994. Student group differences in predicting college grades: Sex, language and ethnic groups. New York: The College Board.
Rimm, S. 1997. An underachievement epidemic. Educational Leadership April 1, 1977, 26-29.
Rodriguez, N. 1996. Predicting the academic success of Mexican-American and while college students. Hispanic Journal of Behavioral Sciences 18 (3): 329-343.
Rose, R., Hall, C, Bolen, L., Webster, R. 1996. Locus of control and students' approaches to learning. Psychological Reports 79 (1): 163-172.
Rotter, J.B. 1966. Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs: General and Applied 80 (1): 1-28.
Schwartz, B. 1999. The test under siege. The New York Times, January 17, 1999.
Sedlacek, W.E. 1989. Noncognitive indicators of student success. Journal of College Admission 125: 2-9.
Sedlacek, W.E. 1994. Issues in advancing diversity through assessment. Journal of Counseling & Development 72 (5): 545-554.
Sternberg, J. 1985. Beyond IQ. London, England: Cambridge University Press.
Sternberg, J. 1986. What would better intelligence tests look like? In Measures in the college admissions process. New York: The College Entrance Examination Board.
Tracey, T.J., & Sedlacek, W.E. 1984. Non-cognitive variables in predicting academic success by race. Measurement and Evaluation in Guidance, 16: 171-178.
Weis, L. & Fine, M., Eds. 1993. Beyond silence voices: class, race and gender in United States schools. Albany: State University of New York.