Xref: utzoo comp.ai.neural-nets:959 alt.cyb-sys:31
Path: utzoo!attcan!uunet!mcsun!ukc!warwick!ecctp
From: ecctp@warwick.ac.uk (Dr J A K Cave)
Newsgroups: comp.ai.neural-nets,alt.cyb-sys
Subject: Re: Generalization Criteria
Message-ID: <263@orchid.warwick.ac.uk>
Date: 27 Sep 89 12:56:50 GMT
References: <506@uvaee.ee.virginia.EDU>
Reply-To: ecctp@warwick.ac.uk (Dr J A K Cave)
Organization: Computing Services, Warwick University, UK
Lines: 17


        The psychometric literature has spawned an approach known
as G-theory to quantify the generalizeability of structural information to
bigger "universes" - in practical terms it looks like a gussied-up
analysis of variance/covariance, but that's OK for some purposes.
I will try to dig up some references, but for starters you might
search the published works of Rich Shavelson: I know he did a summary
for the NAS that appeared in a paper on job performance measurement
within the last 3 years.

Econometrics has come up with a variety of measures grossly related to
MSE.  Examples include Mallow's Cp, Amemiya's PC, mean-square prediction error
on an independent sample; or Rbar-squared (the correlation coefficient
adjusted for degrees of freedom, using the correlation between predicted
and actual values on a disjoint set of data.  Don't know if this is
even close to what you are seeking (different disciplines, different meanings
for the same words...)