Friday, 25 January 2013

Will the UK be ‘in’ or ‘out’?

Amongst others, The Speech had to the aim to provide clarity. Did it? I tried to visualise what may happen: 
Using this chart, one could also add percentages to each of the arrows - which would then give us a rough idea about the likelihood of an ‘in’ or ‘out’.

Sunday, 20 January 2013

Most cited US legal scholars in German law journals

[Sunday evening procrastination] A couple of months ago the Hein Online Blog posted a list of the most cited US legal scholars. Here is an extract:
(1) Sunstein, Cass, (2) Posner, Richard, (3) Easterbrook, Frank, (4) Epstein, Richard, (5) Coffee, John, (6) Eskridge, William, (7) Lemley, Mark, (8) Delgado, Richard, (9) Pound, Roscoe, (10) Fischel, Daniel ..... (12) Prosser, William... (18) Macey, Jonathan.... (23) Calabresi, Guido .....(42) Henkin, Louis .... (49) Brandeis, Louis
I searched how often the top 50 names have been cited in German law journals, searching all German language journals in Beck Online, which provides a good mix of areas of law with many of the top journals included. Unfortunately, as the German citation style is just to cite the last name, there is the risk of false positives and I had to exclude names such as ‘Coffee’ or ‘Pound’ (not wanting to check everything by hand). So, here are the top ten [citations in square brackets]:  
(1) Posner [105], (2) Brandeis [46], (3) Prosser [29], (4) Lemley [21], (5) Sunstein [20], (6) Henkin [18], (7) Calabresi [16], (8) Fischel [15], (9) Easterbrook [13], (10) Macey [11]
Being a bit suspicious, I also searched other most cited US legal scholars of the Leiter Ranking with the following results: Dworkin [64], Lessig [34], Kraakman [28], Bebchuk [22].
Interpretation: first of all, this shows – with few exceptions - fairly low citation numbers. Not a surprise, in my view, given the strong focus of German legal scholarship on German legal doctrine and German-language publications. Second, as far as there are citations to US legal scholars, these are mainly limited to particular fields/types of research: law & economics, jurisprudence, IP & IT, plus a bit corporate and competition law. Third, of course, some German legal scholars also publish in English, in particular on topics of international, EU and comparative law. Thus, without having counted it, I’d also add these fields as potentially more open to the English-language literature.

Thursday, 10 January 2013

The REF 2014 ranking puzzle – not yet solved

It has just been announced (see Times Higher article) that statistics will be released on the academics eligible to be submitted for the REF 2014. This may be welcomed since it may indicate which universities just 'cherry-picked' selected members of staff, i.e. it may enable a more reliable assessment of which universities are really the best ones. Yet, there is the problem how exactly this forthcoming information can be used to calculate such a ranking. By way of illustration, in the Table below three universities (A, B, C) have the following data (assuming four outputs for each staff member submitted), leading to the following options to rank the universities:

staff submitted
total staff

  1. Ranking according to ‘research power’, i.e. the average * multiplied by the number of staff submitted (see e.g. here for the RAE 2008) - here, A would be the best one. But this option unduly favours big universities. So, it's better to calculate submissions per staff: for example,
  2. Considering data on staff submitted: here too, A is the best university, regardless of the method used, i.e. whether to calculate the average *, rank according to % of 4*, or rank according to % of 3 and 4* (for these options see the RAE 2008 guardian ranking). But, of course, A only submitted 25% of its staff - thus, let’s also consider the total staff, for example:
  3. Top rank according to 4* percentage of total staff: now C ahead with 15% (and B second with 13%).
  4. Top rank according to 3 or 4 * percentage of total staff; now B ahead with 44% (and C second with 40%).
  5. Top rank according to average *, considering total staff. The problem is that we don’t know how the non-submitted staff would have scored. Thus, there are various options: (a) consider them as 0*: here C ahead of B (1.95 and 1.56). (b) consider them as 2* (and, to be fair, also shift all submitted 0 and 1* outputs to 2*): here B ahead of C (2.56 and 2.55). (c) consider only top 25% submissions of each of the universities (since only for those we have full data): here C ahead of B (3.6 and 3.5)
So, what to do? In general, averages of total staff, i.e. (5) would be preferable since they would make better use of the data than just the top ranks. But, as can be seen, here they don’t work in a reliable way. Thus, my preference would be for (4), given that I don’t really believe that 3 and 4* research can be properly distinguished. Yet, my guess would be that (3) may well become the most used option.
Important further caveats: some members of staff do not have to, but can, submit four outputs, e.g., early career researchers, part-time staff, staff on sick leave etc – and it may be unlikely that the staff data will also include such sometimes fairly complex information. Also, according to the comments on the Times Higher site (see above), teaching-only staff will not be included in the staff data and I’m not sure what to make of this: an ‘invitation’ to universities to tweak the data by way of shifting some staff to the teaching category, or a sensible division of labour (also noting that in the top US research universities a significant proportion of the teaching is done by adjunct professors). But, in any case, these and other problems with the total staff data may also have the consequence that (1) and (2) will still have some support.

Friday, 4 January 2013

Happy New Year! – and w.i.m.i.D.

meaning: what interested me in December
A bit short – also I’m wondering whether I’ll continue these lists in the New Year in the current form (or perhaps less frequently on this Blog, or more frequently via Twitter).

PS: update where I am this term (the picture may still change).