Should Our Public Universities be Cadillacs or Chevrolets?

Do students of today deserve Cadillac or Chevrolet public universities?

Whether you agree or disagree with the various ranking systems for K-12 and higher education— educators, policymakers and other stakeholders are paying attention to them. The U.S. News and World report rankings have had a variety of critiques levied against them over the years, but they are probably the most visible ranking system of higher education.

In the most recent U.S. News ranking, the University of Texas at Austin is ranked #52 and Texas A&M is ranked #70.

Occasionally you hear racialized dog whistles relative to the rankings.

For example, arguments have been put forth in Texas that the policy formerly known at the Top 10%, plan that was implemented to create a more diverse environment, has negatively impacted the ranking of the University of Texas at Austin and Texas A&M, the crown jewels of Texas public higher education.

The Top 10% plan is a policy that has given Texas students that graduate in the Top 10% of their high school class admission to any public university in Texas— with some recent caveats. (Abigail Fisher wasn't eligible for 10% plan and sued)

I previously wrote about the enrollment of Latinos, African Americans, and English Learners in the midst of the Top 10% plan in the posts Actuating equity?: Historical and contemporary analyses of African American access to selective higher education from Sweatt to the Top 10% Law and From Jim Crow to the Top 10% Plan: A historical analysis of Latina/o access to a selective flagship university and Immigrant DREAMs: The Texas 10% admissions plan, ELL student college choice and academic success

Our peer-reviewed research demonstrates that the success of the Top 10% plan has been mixed— some progress, but not enough relative to the gigantic demographic changes in Texas over the past 30 years.

Back to the dog whistle.

Would the abolition or modification of the Top 10% plan have any bearing on the national rankings of the University of Texas at Austin or Texas A&M? What actually goes into the ranking and how could policymakers and educators in Texas improve the relative standing of their flagship public institutions?

Ever wonder what is actually in the U.S. News rankings of universities? I went to the U.S. News website here for the rankings methodology. The table below focuses on the methodology for "national universities."  The left column are the overall ranking indicator categories in the U.S. News system. The sub categories on the right are the factors that are weighted into the ranking indicator.

The U.S. News rankings methodology lists the categories alphabetically, so I will address them in that order. I will determined whether the 10% plan impacts the metric and discuss each.

Acceptance rate: The ratio of the number of students admitted to the number of applicants for fall 2014 admission. The acceptance rate is equal to the total number of students admitted divided by the total number of applicants.

The totals for both applicants and acceptances include only first-time, first-year students. A lower acceptance rate – indicating a school is more selective in whom it admits – scores higher in the ranking model.

Weight: 10% of 12.5%=1.25% of total ranking

10% plan: The 10% plan could have some bearing on the acceptance rate because of automatic admission.

Discussion: While the admission rate could be impacted by the Top 10% policy, at 1.25% the overall impact on the rankings is miniscule. Investments to increase the number of applications to the flagships could alleviate some of the impact.

Average alumni giving rate: The average percentage of undergraduate alumni of record who donated money to the college or university. Alumni of record are former full- or part-time students who received an undergraduate degree and for whom the college or university has a current address. Graduates who earned only a graduate degree are excluded.

Undergraduate alumni donors are alumni with undergraduate degrees from an institution who made one or more gifts for either current operations or capital expenses during the specified academic year.

The alumni giving rate is calculated by dividing the number of alumni donors during a given academic year by the number of alumni of record for that same year. The two most recent years of alumni giving rates that are available are averaged and used in the rankings. For the 2016 edition, the two separately calculated alumni giving rates that were averaged (added together and divided by two) were for giving in the 2012-2013 and 2013-2014 academic years.

The percentage of alumni giving serves as a proxy for how satisfied students are with the school. A higher average alumni giving rate scores better than a lower rate in the ranking model.

Weight: 100% of 5%=5% of total ranking

10% plan: The 10% plan probably has no net negative bearing on the alumni giving rate.

Discussion: One the one hand it could be argued that the 10% plan restricts affirmative action for alumni, but investigations at UT-Austin demonstrated that alumni have still gained access to UT-Austin through official channels. On the other hand it could be argued that a more diverse campus will results in alums of color donating due to (and others) more positive feelings about the institution.

Average first-year student retention rate: The percentage of first-year students who returned to the same college or university the following fall. Average first-year student retention rate indicates the average proportion of the first-year classes entering from fall 2010 through fall 2013 who returned the following fall.

If a school submits fewer than four years of first-year retention rate data, then the average is based on the number of years that are submitted by the school to U.S. News. A higher average first-year retention rate scores better than a lower average retention rate in the ranking model.

Weight: 20% of 22.5%=4.5% of total ranking

10% plan: The 10% plan probably has a positive impact on first year retention.

Discussion: In the article, Desire to finish college: An empirical link between motivation and persistence, Allen found that the student’s high school rank (and first-year college GPA, and a self-reported measure of desire to finish college) accounted for 68% of the variance in the retention of minority students from the first to second year of college. Policymakers can also improve retention by providing resources for more financial aid which research shows increases student retention.

Average graduation rate: The percentage of entering first-year students who graduated within a six-year period or less, averaged over the classes entering from fall 2005 through fall 2008. This excludes students who transferred into the school after their first year and then graduated.

If a school submits fewer than four years of graduation rate data, then the average is based on the number of years that are submitted. A higher average graduation rate scores better than a lower graduation rate in the ranking.

Weight: 80% of 22.5%=18% of total ranking

10% plan: The 10% plan probably has a positive impact on graduation (Also see discussion above for retention).

Discussion: Graduation is a biggie in the rankings— nearly a fifth of the formula. Policymakers and educators must find resources to bring to bear here. Dr. Victor Saenz, a UT-Austin Professor, in his new book talks specifically about a few factors that ensure success of Latino students.

• Policy and programmatic interventions that attend to the needs of students both long before they arrive on campus and also immediately after they arrive.
• Consider how support is extended through social networks (e.g., college access programs, financial aid).
• We should also carefully design “on-ramp” experiences for Latino students that immediately gets them engaged and connected on our college campuses.

Class size, 1-19 students: The percentage of undergraduate classes, excluding class subsections, with fewer than 20 students enrolled during fall 2014. A higher percentage of small classes scores higher than a lower percentage in the ranking model. In other words, the more small classes, the better.

Weight: 30% of 20%=6% of total ranking

10% plan: The 10% plan probably has no impact on 1-19 class size.

Discussion: The biggest impact on class size would likely come from cutting tuition revenue while simultaneously increasing the enrollment and not pacing state contributions to balance this approach.Texas and Texas A&M would need new resources targeted to hire faculty to reduce classes sizes to improve their standing here. When I was at UT-Austin, the political pressure from above was usually to teacher larger classes, not smaller classes.

Class size, 50-plus students: The percentage of undergraduate classes, excluding class subsections, with 50 students or more enrolled during fall 2014. A smaller percentage of large classes scores higher than a larger percentage in the ranking model. In other words, the fewer large classes, the better.

Weight: 10% of 20%=2% of total ranking

10% plan: The 10% plan probably has no impact on 50 plus class size.

Discussion: See 1-19 class size discussion above.

Expenditures per student: Financial resources are measured by the average spending per full-time-equivalent student on instruction, research, public service, academic support, student services and institutional support during the 2013 and 2014 fiscal years. If a school submits fewer than two years of data, then one year is used.

The number of full-time-equivalent undergraduate and graduate students is equal to the number of full-time students plus one-third the number of part-time students.

We first scale the public service and research values by the percentage of full-time-equivalent undergraduate students attending the school. Next, we add total instruction, academic support, student services, institutional support and operations and maintenance (for public institutions only) and then divide by the number of full-time-equivalent students. After calculating this value, we apply a logarithmic transformation to it prior to standardizing. 

Financial resources enable schools to provide students with a high-quality college experience. Consequently, higher average expenditures per student score better than lower expenditures in the ranking model. However, the use of the logarithmic transformation means schools that have expenditures per student that are far higher than most other schools' values see diminishing benefits in the ranking calculations. 

Weight: 100% of 10%=10% of total ranking

10% plan: The 10% plan probably has no impact on average per student expenditures.

Discussion: The discussion in Texas has been whether the state needs Cadillac flagships or Chevrolet flagships. Which I think infers less spending? Less spending per student , which is a 10% of the ranking, of course negatively impacts U.S. News rankings.

Faculty compensation: The average faculty pay and benefits are adjusted for regional differences in cost of living. This includes full-time assistant, associate and full professors. The values are taken for the 2013-2014 and 2014-2015 academic years and then averaged.

If a school submits fewer than two years of faculty salary data, then only one year is used. The regional differences in cost of living are taken from indexes from Runzheimer International.

Higher average faculty salaries after adjusting for regional cost of living score better than lower average faculty salaries in the ranking model.

Weight: 35% of 20%=7% of total ranking

10% plan: The 10% plan probably has no impact on faculty pay.

Discussion: The biggest impact on faculty pay would also likely come from cutting tuition revenue while simultaneously increasing the enrollment and not pacing state contributions to balance this approach. Texas and Texas A&M would need new resources targeted to PAY faculty better to improve their ranking. When I was at UT-Austin, the political pressure from above was usually to hold the line on faculty pay— which caused quite a bit of turnover. During my eight years at UT-Austin the University had given up across the board pay raises. Low pay, lack of permanent raises, and average benefits were partial factors in my 2014 departure from the University of Texas at Austin.

Faculty with a Ph.D. or terminal degree: The percentage of full-time faculty members with a doctorate or the highest degree possible in their field or specialty during the 2014-2015 academic year. Schools with a larger proportion of full-time faculty with the terminal degree in their field score better than schools with a lower proportion.

Weight: 15% of 20%=3% of total ranking

10% plan: The 10% plan probably has no impact on faculty with terminal degrees.

Discussion: At large comprehensive research universities like UT-Austin and Texas A&M, historically this isn't a glaring issue. However, it typically becomes an issue when an institution doesn't have the resources to hire tenure track faculty and instead has to hire adjuncts without Ph.Ds.

Graduation rate performance: The difference between the actual six-year graduation rate for students entering in fall 2008 and the predicted graduation rate. The predicted graduation rate is based upon characteristics of the entering class, as well as characteristics of the institution.

This indicator of added value shows the effect of the college's programs and policies on the six-year graduation rate of students after controlling for spending per student, the proportion of undergraduates receiving Pell Grants, standardized test scores and high school class standing. 

If the actual graduation rate is higher than the predicted rate, the college is enhancing achievement or is overperforming. If its actual graduation rate is lower than the predicted rate, then it's underperforming.

A school with a higher ratio of its actual graduation rate compared with its U.S. News predicted graduation rate (actual graduation rate divided by predicted rate) scores better than a school with a lower ratio in the ranking model.

Graduation rate performance has been used in the National Universities and National Liberal Arts Colleges ranking categories since the 1997 edition of Best Colleges, and in the Regional Universities and Regional Colleges ranking categories starting with the 2014 edition.

Weight: 100% of 7.5%=7.5% of total ranking

10% plan: The 10% plan probably has no impact on graduation rate performance.

Discussion: This measure should give UT-Austin and Texas A&M a more comparable measure of graduation to Stanford and Harvard because it controls for characteristics of the student population. However, refer back to the important resource commitments to improve graduation rates noted above.

High school class standing: The proportion of students enrolled for the academic year beginning in fall 2014 who graduated in the top 10 percent (for National Universities and National Liberal Arts Colleges) or 25 percent (Regional Universities and Regional Colleges) of their high school class.

A higher proportion of students from either the top 10 percent or top 25 percent of their high school class scores better than lower proportions in the ranking model. Colleges reporting high school class standing based on less than 20 percent of their entering classes had their scores discounted before standardization.

Weight: 25% of 12.5%=3.125% of total ranking

10% plan: The 10% plan probably has a positive impact on high school class standing.

Discussion: This measure should give UT-Austin and Texas A&M a bump compared to other state institutions in the ranking because the Top 10% plan guarantees students in the top ranking of their high school class.

High school counselor rating score: Opinions of high school guidance counselors are only factored into the rankings of National Universities and National Liberal Arts Colleges. These ratings by public and private independent school counselors are used as an indicator of academic reputation in these two categories, along with separate ratings from college admissions deans, provosts and presidents.

Scores for each school are totaled and divided by the number of counselors who rated that school.

This year, for the first time, the three most recent years of survey results, from spring 2013, spring 2014 and spring 2015, were averaged to compute the high school counselor reputation score that is used in the rankings. This was done to increase the number of ratings each school received and more fully represent the views of guidance counselors, as well as to reduce the year-to-year volatility in the average counselor score. Previously, the two most recent years of data were used.

The counselors' one-year response rate was 7 percent for the spring 2015 surveys. A higher average high school counselor reputation score does better than a lower score in the ranking model. The Regional Colleges and Regional Universities rankings do not have a high school counselor ratings component.

[See more on the high school counselor scores in the 2016 rankings.]

Weight: 33.3% of 22.5%=7.49% of total ranking

10% plan: The 10% plan probably has no impact on overall counselor rating but could positively impact statewide ratings from Texas.

Discussion: The fact that schools in rural and urban areas of Texas hadn't always been able to send students to UT-Austin and Texas A&M before Top 10%, you might expect counselors in the state of Texas to be more positive about the flagship institutions.

Peer assessment: A measure of how a school is regarded by administrators at peer institutions. A school's peer assessment score is determined by surveying presidents, provosts and deans of admissions, or officials in equivalent positions, at institutions in the school's ranking category.

Each individual is asked to rate peer schools' undergraduate academic programs on a scale from 1 (marginal) to 5 (distinguished). Those individuals who do not know enough about a school to evaluate it fairly are asked to mark "don't know."

A school's score is the average score of all the respondents who rated it. Responses of "don't know" count neither for nor against a school.

This year, for the first time, the two most recent years of peer assessment survey results, from spring 2014 and spring 2015, were averaged to compute the academic reputation peer assessment score that is used in the rankings. This was done to increase the number of ratings each school received and more fully represent the views of high-level academics, as well as to reduce the year-to-year volatility in the average peer assessment score. Previously, only the most recent year's data were used. 

In spring 2015, 40 percent of those surveyed responded. The response rate was 42 percent for the 2014 survey.

A higher average peer assessment score does better than a lower peer assessment score in the ranking model. The academic peer assessment rating is used in the National Universities, National Liberal Arts Colleges, Regional Universities and Regional Colleges rankings.

Weight: 66.7% of 22.5%=15% of total ranking

10% plan: The 10% plan probably has no impact on peer assessment.

Discussion: I suspect that all of factors discussed above and below probably impact the public discourse and impressions of the flagships in Texas. It is probably not helpful that arguably the biggest story out of Texas universities for higher education faculty and administrators during the past year has been the recommendation that faculty not discuss sensitive topics or "anger" students in class due to the new carry on campus law. Faculty are very protective of academic freedom and their duty to freely create knowledge for society. Nevertheless, the common denominator of most of the topics we have discussed above require Texas and Texas A&M to be Cadillac institutions instead of Chevrolets, which means resources must be invested by policymakers to improve the institutions' metrics to enter the Top 50 universities in the US.

Proportion of full-time faculty: The proportion of the 2014-2015 full-time-equivalent faculty that is full time. The number of full-time-equivalent faculty members is equal to the number of full-time faculty members plus one-third of the number of part-time faculty members.

We do not include faculty in preclinical and clinical medicine; administrative officers with titles such as dean of students, librarian, registrar or coach, even though they may devote part of their time to classroom instruction and may have faculty status; undergraduate or graduate students who are teaching assistants or teaching fellows; faculty members on leave without pay; or replacement faculty for those faculty members on sabbatical leave.

To calculate this percentage, the total full-time faculty is divided by the full-time-equivalent faculty. A higher proportion of faculty members who are full-time scores better than a lower proportion in the ranking model. 

Weight: 5% of 20%=1% of total ranking

10% plan: The 10% plan probably has no impact on proportion of full-time faculty.

Discussion: At large comprehensive research universities like UT-Austin and Texas A&M, historically this isn't a glaring issue. However, where it typically becomes an issue is when an institution doesn't have the resources to hire tenure track faculty and instead hires temporary adjuncts.

SAT/ACT scores: Average test scores on both the critical reading and math portions of the SAT and composite ACT of all enrolled first-time, first-year students entering in fall 2014 are combined for the ranking model.

Before being used as a ranking indicator, the scores from both tests are converted to the percentile of the national distribution corresponding to that school's scores on the critical reading and math portions of the SAT and the composite ACT. The SAT writing section is not used in the ranking model.

To better represent the entire entering class, we use a calculation that combines the values of both the critical reading and math portions of the SAT and the composite ACT of all fall-entering students based on the percentage of the fall entering class that submitted each test. If less than 75 percent of the fall 2014 entering class submitted SAT and ACT scores, their test scores were discounted in the ranking calculations. This policy was also used in the 2015 edition.

A higher average entering class test score on the critical reading and math portions of the SAT and composite ACT does better than a lower average SAT and ACT test score in the ranking model. 

Weight: 65% of 12.5%=8.12% of total ranking

10% plan: The 10% may have a negative impact on average SAT and ACT scores.

Discussion: However, a focus on class ranking instead of SAT and ACT scores is probably wise. The research has demonstrated that SAT and ACT scores are not good predictors of student performance. Notably, test optional admissions have become more popular. While admitting students based on ranking may have positive benefits for first-year retention and graduation rates (see above), it could negatively impact the U.S. News rating because approximately 8% of the ranking is related to average scores.

Student-faculty ratio: This is the ratio of full-time-equivalent students to full-time-equivalent faculty members during the fall of 2014.

This excludes faculty and students of law, medical, business and other stand-alone graduate or professional programs in which faculty members teach virtually only graduate-level students. Faculty numbers also exclude graduate or undergraduate students who are teaching assistants.

A lower student-faculty ratio (fewer students per each faculty member) scores better than a higher ratio in the ranking model.

Weight: 5% of 20%=1% of total ranking

10% plan: The 10% plan probably has no impact on student to faculty ratio.

Discussion: The biggest impact on student to faculty would likely come from cutting tuition revenue while simultaneously increasing the enrollment and not pacing state contributions to balance this approach. Texas and Texas A&M would need new resources targeted to hire more faculty to reduce student-faculty ratio to improve their standing here. As I noted in the class size metric above, when I was at UT-Austin, the political pressure from above was always to teacher larger classes, not smaller classes.

Conclusion

In summary, the Top 10% plan appears to have very little negative impact on the U.S. News and World report ranking— and instead likely has a net positive impact.

Nevertheless, the University of Texas at Austin is ranked #52 and Texas A&M is ranked #70— so clearly more investment by the Texas Legislature is necessary to make the flagships more competitive in the various metrics in the rankings.

What is very clear from the  U.S. News metrics is that policymakers in various states can increase investments for public institutions to make them more similar to Cadillacs rather than Chevrolets— if they hope to improve their public flagship's "national university" standing.

Please Facebook Like, Tweet, etc below and/or reblog to share this discussion with others.

Want to know about Cloaking Inequity’s freshly pressed conversations about educational policy? Click the “Follow blog by email” button on the home page.

Twitter: @ProfessorJVH

Click here for Vitae.