Home
Schedule
Projects
TA OHs

AI Engineering Projects

Overview

This course includes two hands-on AI engineering projects using RapidFire AI, an open-source experimentation engine for LLM application development. The projects are designed to give you practical hands-on experience with two of the most important workflows in modern AI engineering: RAG & context engineering and supervised fine-tuning (SFT).

These projects are modeled on the RapidFire AI 2026 Winter Competition on LLM Experimentation in which students showcased rigorous experimentation for RAG and SFT use cases.

Dates

  • Project 1: RAG and Context Engineering

    • Release Date: Wed, Apr 15

    • Discussion by TA: Wed, Apr 15

    • Early Submission for one-time leaderboard preview: Thu, Apr 30, 11:59pm PT

    • Due Date: Wed, May 6, 11:59pm PT

    • F2F Interviews: Tue, May 5 to Wed, May 13

  • Project 2: Supervised Fine-Tuning (SFT)

    • Release Date: Mon, May 18

    • Discussion by TA: Wed, May 20

    • Due Date: Thu, Jun 4, 11:59pm PT

Grading

Project 1 is graded on two equally weighted components:

  • Submission: Code Repo + Files + Report (10%):

    • Your output file is auto-graded on eval metrics using our scripts (and LLM judge for Project 1).

    • Your automation scripts will be verified for end-to-end runnability.

    • Report graded by TAs for quality of experimental methodology and analysis.

    • The 10% is assessed holistically across code correctness and quality, auto-grading results, and report quality.

    • Fallback manual code review by TAs for partial credits if auto-grading fails due to mundane technical issues.

  • Face-to-Face (F2F) Interview (10%):

    • In-person 10min interviews conducted by TAs based on auto-grading results and your git repo, code, and report.

    • Will focus on diving into your approach, results, and counterfactual questions, e.g. "What would happen if you changed X?", "Why didn't you try Y?".

    • You will not have access to electronics/Web/LLMs during the interview but the TAs will.

    • Outcome: Pass (10%) or Re-interview

    • Re-interview conducted by the professor, with one of three final outcomes: Pass (10%), Half-pass (5%), or Fail (0)

    • Can be over Zoom if you have OSD accommodations or a university-approved reason for exemption.

Project 2 will not have F2F interviews.

Instructions and Caveats

  • Teams:

    • You may work in teams of 2 or individually (team of 1).

    • Submit your team decision via the Google Form (link on Canvas) within 48 hours of each project's release. One submission per team.

    • Team composition can change between Project 1 and Project 2.

    • You are encouraged to find your own teammate. You can also use Piazza for this.

  • Project Docs and Presentations:

    • These will be released on Canvas and announced on Piazza.

    • Try to attend the TA discussion presenting the project details to ask live Q&A.

    • The TA discussions will be recorded and made available via Canvas afterward.

  • There are no late days for either project. Plan your work accordingly.

  • Leaderboard:

    • A leaderboard will be maintained for each project for motivation purposes. Leaderboard position is NOT part of the base grade.

    • Extra credit: The top 10th percentile of the teams will receive 2% extra credit toward the course grand total per project. The rest of the top 20th percentile will receive 1% extra credit per project.

    • An intermediate one-time preview of the leaderboard will be made for your benefit roughly a week before the actual due date based on early submissions made till then. No additional feedback will be provided apart from leaderboard position. (Update: This intermediate preview now applies only to Project 1.)

    • Early submission is strongly encouraged to help you gauge your standing and leave time for iteration. You can resubmit after that if you'd like.

    • Your final leaderboard position (for extra credit) will be based on your final submission.

  • Resources:

    • Each team will receive sufficient AWS credits for Project 1 and SDSC DSMLP GPU resources for Project 2. Details will be provided in each project's release packet.

    • Plan and budget your usage carefully. Rough cost estimates will be provided in each project statement.

    • Do NOT waste the resources, say, by letting them sit idle overnight.

  • Git Repository:

    • You must set up a private git repository (GitHub or BitBucket) and add all TAs and the instructor as collaborators within 24 hours of each project's release.

    • It is okay to reuse Project 1's repo for Project 2 if the team is the same.

    • Practice regular, meaningful commits throughout the project. Your entire commit history must be visible and should reflect genuine iterative development.

    • Gaming of commit history (e.g., fabricating commits, rewriting history to simulate gradual development) is a violation of academic integrity.

Academic Integrity

  • It is okay to discuss the projects with your peers at a conceptual level. It is also okay to post conceptual or high-level questions, logistical questions, and useful references on Piazza. But do NOT share any code across teams and do NOT post any of your solution code for discussion. A team's code submission must be entirely their own.

  • It is okay to use AI coding assistants (Claude Code, ChatGPT, Cursor, etc.) to assist you. However, you must understand every line of code you submit and be prepared to explain and modify it during the F2F interview. Include a brief statement in your report about if/how AI tools assisted you.

  • MOSS will be used to detect code similarity across submissions. Additional program analysis tools may also be used.

  • Please review UCSD's honor code and policies and procedures on academic integrity on this website. If plagiarism is detected in your project submissions, or if any other form of academic integrity violation is identified, the University authorities will be notified for appropriate disciplinary action to be taken. You will also get 0 for that project and get downgraded substantially.