FOA Home
Section §5.2.7 raised several
important cognitive questions arising from attempts to study
``semantic'' interpretation of LSI/SVD dimensions. The connection to the
psychologically important analysis of MDS adds even more. If we
interpret RelFbk information as constraints about which documents a
user likes, how much can we reduce dimensionality without violating the
RelFbk constraints they are giving us? That is, how much can we
reduce the representational space before we're changing somebody's
order? Second, how many RelFbk statements do we need in order to
accurately determine a good compression? How many people have to tell us
things before we have enough information in order to form this reduced
representation? These questions also connect to ones related to learning
these representations (cf. Chapter §7 ),
and to our mechanisms, as part of the interface or as part of a special
experimental system like RAVE, for efficiently collecing large volumes
of RelFbk (cf. Section §4.4 ). Most
of these questions remain unanswered, but a beginning is simply counting
the number of preference constraints provided by each RelFbk
labeling of a hitlist by a user.
First note that the total number of
preferences statements required to completely determine $n$ elements
grows very rapidly as ${{n 2} \choose 2}$, corresponding to the fact
that preference order information is defined over pairs of
pairs of evaluated documents. Against this backdrop, each each
RelFbk labeling produces: \mathname{NPlus} & \equiv & |\oplus| \\
\mathname{NDCare} & \equiv & |\#| \\ \mathname{NMinus} & \equiv &
|\ominus| \\[0.1in] \mathname{NRetr} & = & \mathname{NPlus} +
\mathname{NDCare} + \mathname{NMinus} \\ \mathname{NPref} & \equiv &
\mathname{NPlus} \cdot (\mathname{NDCare} + \mathname{NMinus} ) +
(\mathname{NDCare} \cdot \mathname{NMinus})
When this information is
used as part of error correction learning, another useful quantity is
the number of documents over which the RelFbk preference relation
and the ranked list disagree: \mathname{Disagree} \equiv \left\{
\left\langle \mathbf{d}_{i},\mathbf{d}_{j}\right\rangle \mid \left(
\mathbf{d}_{i} \prec \mathbf{d}_{j} \right) \wedge \left(
\mathname{Rank}(\mathbf{d}_{i}) < \mathname{Rank}(\mathbf{d}_{j})\right)
\right\}
Because this relatively small number of data points will always
be dwarfed by the number of total preferences across the entire corpus,
the goal becomes to constrain the application of RelFbk data to
those subsets of the corpus possibly appropriate to a particular query.
And since we intend to {\em learn} from the browsing users, we can
afford to be patient: the documents are only written once but browsing
users will continue to read them and provide RelFbk for some time.
Top of Page
Information in \RelFbk