The much anticipated ranking of Canadian law schools by Maclean’s was released this past week.

Also as anticipated, an enormous furor arose among faculty and professionals across the nation over the approach used.

An American Approach to Canadian Schools

The rankings was that they were conducted by Brian Leiter, of the University of Texas at Austin Law School.

Leiter was a leader in critiquing rankings of American law schools by the renowned U.S. News and World Report.

Even these rankings have been largely rejected by the Association of American Law Schools (AALS) due to methodological concerns, especially on the arbitrary manner in which weight is assigned to various criteria.

But rankings play an enormous role in the U.S., where there are three tiers of law schools. Poorly ranked schools in the U.S. can suffer financially, and experience cutbacks and downsizing.

In Canada’s publicly subsidized education system, this approach simply does not work.

A Consensus of Objection

To be fair, Maclean’s did use a different approach than used in the U.S. for rankings there. Their approach was much more simplistic and rudimentary, using the following formula:

• Faculty quality (50%)
• Student (more aptly graduate) quality (total of 50%)
  • elite firm hiring (25%)
  • national reach (15%)
  • Supreme Court Clerkship hiring (10%)

Ben Alarie, of the University of Toronto faculty blog, has already raised concerns with the use of these criteria.

Some of the strongest to date are that faculty are evaluated on the basis of publications, but only Canadian publications are reviewed. Schools with strong international emphases or specialized areas of study naturally suffer with this approach. The definition of elite law firms was also suspect, and not reflective of the specializations that occur in different law firms.

Alarie has also questioned the manner in which the composite rankings are formed, and the dangers of using rankings at all,

I conclude with a general observation that if one must have rankings… then it is better to have rankings that are at least measuring meaningful things.

These sentiments were echoed by Russ Brown of the University of Alberta and Frank Pasquale of Concurring Opinions.

Dr. Ian Holloway, Dean of the Law at the University of Western Ontario, also pointed out some factual errors, where the institution’s faculty was erroneously inflated by over 20%, severely affecting their faculty ranking.

Holloway also commented on a distinction between Commonwealth countries and the U.S. in that the former considers publications of books worth of academic merit, whereas the latter does not, again artificially deflating the academic presence of faculty.

Nor are Supreme Court clerkships as indicative in Canada of a school’s quality as they are in the U.S., as many judges tend to favour their own alma matter. There are also special seats assigned regionally, which inflates some schools above others.

In short, while some conceded the need for an assessment of Canadian law schools, most felt this approach too simplistic and not reflective of the academic landscape in Canada.

A Measure of What?

Maclean’s has stated,

Our law ranking is not, however, a ranking of which schools are the hardest to get into. It is, instead, about measuring the quality of the output of each school.

Patrick Monahan of Osgoode Hall responded by noting that the rankings excluded evaluation of the school curriculum, students enrolled and extracurricular activities, or the teaching and educational experience.

Apples and Oranges

And because some schools, such as the University of Windsor and the University of Victoria, emphasize completely different aspects of both the admissions process and the educational experience, any uniform evaluation comparing Canadian law schools to each other on these basis alone are likely to be misleading.

In a letter to Maclean’s, Peter Showler of the University of Ottawa pointed out the inconsitency of the magazine criticizing Bay Street firms earlier this year, and currently using them as a comparator. He explained,

I teach refugee law and have the annual pleasure of teaching some of the best students in the school. They are smart, dedicated, passionate about social justice and would not go near a Bay Street firm. Find another yardstick.

Updates

Tony Keller, Managing Editor of Special Projects at Maclean’s, wrote in with some corrections. Keller states,

We asked Leiter to assist us in preparing the first annual Maclean’s law school rankings because he is a leading critic — arguably THE leading critic — of the US News methodology. He’s spent the past decade criticizing US News, time and again and in great detail,going so far as to create his own alternative (and superior) system of evaluating law schools.

We thank him for pointing them out, and have modified the post accordingly.