Assignment 3 – Data Mining Competition

In this assignment, you will learn how to approach and solve real world problems by participating in the UCSD student data mining contest. You are required to solve ONLY the first problem – document classification, but attempts to solve the second problem are encouraged and will be given extra credits. The final grade of this assignment will be based on the report (50%) you wrote and the performance (50%) of your algorithm in the contest.

Part 1

Get familiar with the instructions on the website: http://mill.ucsd.edu/. Register your team, download the dataset, and submit a test file to make sure you know how it works.

You should get this part done as soon as possible, and have ONLY one member email us your team information including team name and members of the team. In the email subject field, please include “CSE 151”.

My email:

shingyan.lam”at”gmail.com

bmcfee”at”cs.ucsd.edu

Note: You can work alone or in a team of up to 3 people.

Part 2

At this point, you are free to explore different algorithms that perform well. We do have some suggestions:

1. Start with an easy-to-implement algorithm as a baseline in case you don’t have enough time to get it done. You should look at how well it does on the training set AND the “quiz set.” The latter is most important, as the quiz set is supposed to give you an estimate of how well it will do on the final test set. For example, you could just memorize the labels on the training set by storing it, and doing 100% on it. No one will be excited by that! So it is important to look at the quiz set results, that’s your benchmark.

2. Do some analysis and try to improve the performance of the current algorithm. Reading papers/books usually gives you a better idea of which approach to try.

3. A paper I found particularly useful: “Tackling the Poor Assumptions of Naïve Bayes Text Classifiers”. (Do a google search and you’ll find it immediately.)

4. Start early and post any question you have on the webboard.

What to turn in:

Report:

1. Follow the NIPS format just as you did in previous assignment.

2. Clearly explain the algorithms you used. Why these algorithms? What preprocessing did you do on the data, if any?

3. Results: Include all the analyses you did. You should at least give performance (% correct), and any analysis of why this algorithm worked well or failed. You can often do this on the basis of the errors it makes.

4. Discussion: What are your conclusions based on your experience? Did things turn out as you expected? How could you improve the results?

5. Be as precise as possible.

6. A separate paragraph by each team member stating: (a) what that team member contributed to the project. (b) what that team member learned from the project.

All the above factors will be taken into account when grading your assignment.

Code:

You are free to use any language you like (according to the website). But you need to turning all the code you wrote, and the code should be well documented and commented.

Grade break down:

50% based on the report, and 50% based on the performance (accuracy) you got from the document classification task. Extra credit will be rewarded for attempting to solve the second problem. Penalties will be given to those who don’t follow the instructions.

Due date is specified in the referencing page.

Prize:

Besides the prize offered by Fair Isaac, we’ll also have a “mini contest” in our class. The team with the highest score will be awarded a “secret” present. It’s so “secret” that we don’t know what it is now, but in the past, such prizes have been on the order of a free lunch at the Faculty Club with Professor Cottrell.