Notes for Machine Problem 5
Michael Petropoulos
INPUT FILES: inv-kw.d, queries, rel.d, doclen.d
OUTPUT FILES:
plots-weak/q1.d, …, q11.d, plots-weak/avg.d,
mod-match.d, mod-plots-con/q1.d, …, q11.d, mod-plots-con/avg.d
mod-plots-weak/q1.d, …, q11.d, mod-plots-weak/avg.d
// MATCH.C code file
This is the code from MP4.
You should modify the way it outputs the results.
The results list for each query should contain all documents with nonzero match score, not just
the first 100 most relevant docs, and it should be sorted according to match score in descending order.
Your code should output the results list of each query to the match.d file in the following format:
QueryNo DocId MatchScore
// MOD-MATCH.C code file
Same as match.c file, but you use the Okapi formula to calculate wkd
In order to calculate the average length of the documents and to find the length
of each document for the Okapi formula, please use the doclen.d file.
In order to calculate the weight of query keywords, you use again the average length of documents.
The length of the query is just the number of keywords it contains. BE CAREFUL, you must use Okapi
formula only for LONG QUERIES. For short queries you still use equations 3.34 and 3.32.
// EVAL.C code file
// Structure that holds splay tree node data for rel.d file
typedef struct
{
int notion[11]; // 0 initially, 1 for weak, 2 for conservative
// when it satisfies conservative it also satisfies weak
} sp_data;
// Vars for splay tree
SPTREE Docs; // Keeps the relevant notion of each doc for each query
// The key for each node is a DocId and the data is a sp_data structure
SPBLK *spnode;
sp_data *sdata;
int NDocRel_con[11]; // Number of relevant docs for each query according to conservative notion.
int NDocRel_weak[11]; // Number of relevant docs for each query according to weak notion.
void main (int argc, char **argv)
{
// Initialize splay tree for rel.d file
spuinit( &Docs, (SP_COMPARE_FUNC *)strcmp );
// Parse rel.d file and construct splay tree
while more lines in rel.d {
read QueryNo, DocId and weak_or_conservative attributes of the current line
use splookup to see if document with DocId already in splay tree
if document not in splay tree {
construct a new node with key the DocId and data a new sp_data structure
initialize notion array
set notion[QueryNo] = weak_or_conservative;
add the node to splay tree Docs
}
else { //document in splay tree
for this node set notion[QueryNo] = weak_or_conservative;
}
if (weak_or_conservative == 2)
if (weak_or_conservative > 0)
}
execute the code in match.c to produce the match.d file
execute the code in mod-match.c to produce the mod-match.d file
// Create the Precision / Recal curves
match_rel2pr();
avg_curve();
}
void match_rel2pr() {
// Check table on page 106 of FOA to understand what you are trying to do here
int query = 0;
int rank = 0;
int con_docs = 0, weak_docs = 0;
open the match.d file
while more lines in match.d {
read QueryNo, DocId from current line
if (QueryNo != query) {
if (query != 0)
// You have to add a final point in curve if not all relevant documents have been retrieved (recall < 1)
// The precision for that point is the number of relevant docs for current query over total number of docs in corpus
if (con_docs/NDoc_Rel_con < 1)
output 1.0 (recall) and NDoc_Rel_con[query]/TotalDocsInCorpus (precision) to plots-con/query.d file
if (weak_docs/NDoc_Rel_weak < 1)
output 1.0 (recall) and NDoc_Rel_weak[query]/TotalDocsInCorpus (precision) to plots-weak/query.d file
close the plots-con/query.d file
close the plots-weak/query.d file
query = QueryNo;
open plots-con/query.d file
open plots-weak/query.d file
rank = 0;
con_docs = weak_docs = 0;
}
rank++
check if document DocId is in splay tree Docs
if (DocId in splay tree Docs and notion[query] == 2) {
con_docs++
}
if (DocId in splay tree Docs and notion[query] > 0) {
weak_docs++
}
output con_docs/NDoc_Rel_con[query] (con recall, use double)
and con_docs/rank (con precision, use double) to plots-con/query.d file
output weak_docs/NDoc_Rel_weak[query] (weak recall, use double)
and weak_docs/rank (weak precision, use double) to plots-weak/query.d file
}
close the match.d file
do the same for mod-match.d file and produce the mod-plots-con/q1.d, …, q11.d files
and the mod-plots-weak/q1.d, …, q11.d files
}
void avg_curve() {
double con_avg = 0;
double weak_avg = 0;
open all plots-con/q1.d, …, q11.d files
open plots-con/avg.d file
open all plots-weak/q1.d, …, q11.d files
open plots-weak/avg.d file
open all mod-plots-con/q1.d, …, q11.d files
open mod-plots-con/avg.d file
open all mod-plots-weak/q1.d, …, q11.d files
open mod-plots-weak/avg.d file
for each fixed recall level (10%, 20%, …, 100%) {
for each plots-con/qi.d (i = 1, …, 11) file
if the file does not contain a point for this exact recall level
find the closest points, p1 and p2, that contain this recall level
call linear_interpolation routine and get the con_precision value for
the fixed recall level
}
add the con_precision value to con_avg variable;
}
output the fixed recall value and the con_avg/11 to plots/avg.d file
for each plots-con/qi.d (i = 1, …, 11) file
do the same
for each mod-plots-con/qi.d (i = 1, …, 11) file
do the same
for each mod-plots-weak/qi.d (i = 1, …, 11) file
do the same
}
close all files
}
double linear_interpolation(double x1, double y1, double x2, double y2, double recall_level) {
int y = [[(y2 – y1) / (x2 – x1)] * (recall_level – x1)] + y1
// This is the equation of the straight line that passes through the points (x1, y1) and (x2, y2)
return y;
}
Notes for gnuplot
You can create a script for gnuplot.
For example, you can put in file "plot_script.gp" the following gnuplot commands:
set data style lines
set term postscript
set output "q1.ps"
set title "Query 1"
plot 'q1.d' title "Q1"
...
set output "q11.ps"
set title "Query 11"
plot 'q11.d' title "Q11"
After that you can execute the following commands:
gnuplot plot_script.gp
mpage -8 q1.ps ... q11.ps > all.ps