“GAMING THE REPORTING”?

In a recent interview with Lee Pacchia of Bloomberg News, U.S. News & World Report’s director of data research Robert Morse explained this year’s only revision to his law school rankings methodology. Morse gave different weights to various employment outcomes for class of 2011 graduates. But he didn’t disclose precisely what those different weights were.

Morse said that such transparency worried him. Full-time, long-term jobs requiring a legal degree got 100 percent credit. But he didn’t reveal the weight he gave other employment categories (part-time, short-term, non-J.D.-required) because he didn’t want deans “gaming the reporting of their results.” It was an interesting choice of words.

Teapot tempests

In some ways, all of the attention to the changes in this year’s rankings methodology is remarkable. Certainly, a school’s employment success for graduates is important. But the nine-month data point for which the ABA now requires more detailed information accounts for only 14 percent of a school’s total U.S. News ranking score. To put that in context, consider some of the more consequential rankings criteria.

Fifteen percent of every school’s U.S. News score is based on a non-scientific survey of practicing lawyers and judges. The survey response rate this year was only nine percent.

Likewise, the “peer assessment” survey that goes to four faculty members at every accredited law school — dean, dean of academic affairs, chair of faculty appointments, and most recently tenured faculty member — accounts for 25 percent of a school’s ranking score. It asks those four individuals to rate all ABA-accredited law schools from 1-to-5, without requiring that any respondent know anything about the schools he or she assesses.

Taken together, the two so-called “quality assessment” surveys comprising 40 percent of every school’s ranking are a self-reinforcing contest for brand recognition. As measures of substantive educational value, well, you decide.

Game of moans

But if, as Morse suggests, his concern is “gaming the reporting,” he must be worried that some deans would either: 1) self-report inaccurate data; or 2) otherwise change their behavior in an effort to raise their school’s ranking. He’s a bit late to both parties.

Scandals engulfed prominent law schools that submitted false LSAT and GPA statistics for their entering classes. But how many others haven’t been caught cheating? No one knows. As for permissible behavior that accomplishes similar objectives, examples abound.

For years, deans seeking to enhance the 12.5 percent of the rankings component relating to median LSAT scores for J.D. entrants have been “buying” higher LSATs through “merit” scholarships. Need-based financial aid has suffered. Ironically, those merit scholarships often disappear after the first year of law school.

Likewise, the faculty resources component is 15 percent of every school’s ranking. But it encourages expenditures — and skyrocketing tuition — without regard to whether they benefit a student’s educational experience.

Whom to blame

Morse establishes the criteria and methodology that incentivize behavior producing these and many other perverse outcomes. But he doesn’t think that any of the current problems confronting the profession are his fault.

“U.S. News isn’t the ABA,” he told Pacchia. “U.S. News doesn’t regulate the reporting requirements…[W]e’re not responsible for the cost of law school, the state of legal employment, the impact that recession has had on hiring, or the fact that 10 or 20 new law schools have opened over the last couple decades. We’re not responsible for the imbalance of jobs to graduates. No, I think we’re not responsible. I think we’ve helped prospective students understand what they are getting into than they were previously.”

Of course, the problem isn’t just the flawed rankings methodology itself. Also culpable are the decision-makers who regard a single overall ranking as meaningful — students, deans, university administrators, and trustees. Without their blind deference to a superficially appealing metric, the U.S. News rankings would disappear — just as the U.S. News & World Report print news magazine did years ago.

Cultural obsession

Pervasive throughout society, rankings may be a permanent feature of the legal profession. But it’s worth remembering that they’re relatively new. Before the first U.S. News list of only the top 20 law schools in 1987, prospective students and law schools somehow found each other.

Today, rankings facilitate laziness. The illusory comfort of an unambiguous numerical solution is easier than engaging in critical thought and exercising independent judgment. Forgotten along the way is the computer science maxim “garbage in, garbage out.”

THE LAW SCHOOL STORY OF 2012 — DEANS IN DENIAL

Doubling down on a losing hand is rarely a good move. Case Western Reserve University Law School Dean Lawrence E. Mitchell generated a flurry of criticism — including my earlier post, “The Lawyer Bubble” — for his November 28, 2012 op-ed in the New York Times. On January 4, 2013, he took to the airwaves in a Bloomberg Law interview. It made me wonder whether he hears his own words as he speaks them.

Mitchell has made himself the poster child for deans in denial — the law school story of the year. It emerged in a big way last June when, for the first time, the ABA released meaningful jobs data. Nine months after graduation, only about 50 percent of the law school class of 2011 had full-time, long-term jobs requiring a legal degree. Deans everywhere began dissembling, as reported in the Wall Street Journal.

Sometimes offense isn’t the best defense

As the growing lawyer bubble made headlines, a handful of wise deans followed the lead of University of California Hastings School of Law Dean Frank Wu, who had previously acknowledged, “The critics of legal education are right. There are too many law schools and too many law students and we need to do something about that.”

In contrast, Dean Mitchell went on offense, most recently in a 15-minute interview with Lee Pacchia. To his tenuous op-ed points, Mitchell added a few more.

What oversupply?

For example, he said, “It’s not clear to me that there’s an oversupply problem at all.” As support, he cited low-income people who go without legal services. Pacchia asked him how debt-ridden graduates paying Case more than $40,000 in annual tuition could take on such work full-time.

It’s a mistake, Mitchell responded, to “measure the worth of higher education by the dollar return on the investment.” Perhaps he has a point, but it’s not really an answer. Earlier in the interview, Mitchell said this about high tuition cost: “Ninety percent of my class receives financial aid. The mean offer is $25,000 a year.” Critics focus on the sticker price, he said, “but law schools discount fairly heavily.”

What proportion of those financial aid packages is grants, rather than loans that can’t be discharged in bankruptcy? Mitchell didn’t say, but here are two clues.

In his op-ed, Mitchell reported accurately that overall average private law school student debt is $125,000. In his April 3, 2012 blog post, he boasted that Case graduates have “almost 22 percent less debt than graduates of other private law schools.” The resulting arithmetic implies that Case’s financial aid packages result in average student loan debt of about $100,000 for its law graduates.

Cost spiral

In another defense of soaring tuition, Mitchell argued that, in 1985, medical school was four times more expensive than law school. So what? In the intervening 25 years, law school tuition has caught up with and, in some cases, surpassed that of medical school. Does that make sense to anyone other than Mitchell?

He also said that schools must pay top dollar for law professors because their opportunity costs are high: they could be making big bucks in big firms. But the only relevant question is, do they want to?

Mitchell’s own experience may provide a partial answer. His CV lists six years as an associate at three different New York law firms from 1981 to 1987. Sometime during that period, he said, it became “hard to get out of bed in the morning and I didn’t like going to work.” So he “took a two-thirds pay cut and went into teaching.”

How about decent jobs?

Throughout the year, Mitchell travels the country, “like Willy Loman in Death of a Salesman,” meeting with hiring partners of big law firms. He interviews his students and writes personal letters of recommendation to help them get jobs. Doesn’t the need for such efforts tell him something?

Yet for all of Mitchell’s laudable sales pitches, Pacchia noted, the Law School Transparency Project reports that 38 percent of 2011 Case Western graduates were still unemployed or underemployed nine months later.

“I haven’t myself taken a snapshot a year out,” Mitchell said, “but I’ve talked to my admissions staff about this a lot and I suspect if you looked a year out, things would change dramatically. I’m really confident if you looked a year and a half out, they would.”

Mitchell offered no supporting data, but he “suspects” and is “really confident” that, eventually, things will turn out just fine.

Optimism untethered to reality

Why is Mitchell convinced that things are better than the available facts suggest? Because, for example, most of his 1981 Columbia Law School class took jobs in big law firms. Ten years later, his class reunion book revealed that “almost nobody was at a law firm.”

It’s hard to know where to begin dissecting Mitchell’s anecdote, but start with the fact that his students aren’t graduating from Columbia Law School.

Just another business

Finally, Mitchell observed, “Of course, we’re running a business at the end of the day.” Without acknowledging the destructive impact of short term business-type metrics, such as the annual U.S. News & World Report rankings, he argues that “using business sense in managing law schools is going to help us get some of these problems under control.”

Until Mitchell and many other deans with similar attitudes get past denial over what is happening to the profession, they’ll never reach, much less overcome, the subsequent stages of grief — anger, bargaining, depression, and acceptance. Perhaps another reading of Death of a Salesman will help.

BLOOMBERG LAW INTERVIEW

Last week, Bloomberg Law’s Lee Pacchia interviewed me on two topics that are frequent subjects of this blog. Two segments are now up and running on the Bloomberg Law channel. Here are the links:

Law Prof: Why Discharging Student Debt in Bankruptcy is Good for Lenders
http://youtu.be/z4Q3Y4PRwqQ

Ex-Partner: $1M Salaries Should Satisfy BigLaw Partners
http://youtu.be/2TErRjB9j6Y