CSC 6203 Large Language Models

What is this course about?

The course will introduce the key concepts in LLMs in terms of training, deployment, downstream applications. In the technical level, it covers language model, architecture engineering, prompt engineering, retrieval, reasoning, multimodality, tools, alignment and evaluations. This course will form a sound basis for further use of LLMs. In particular, the topics include:

• Introduction to Large Language Models (LLMs) - User's perspective
• Language models and beyond
• Architecture engineering and scaling law - Transformer and beyond
• Training LLMs from scratch - Pre-training, SFT, learning LLMs with human feedback
• Efficiency in LLMs
• Prompt engineering
• Knowledge and reasoning
• Multimodal LLMs
• LLMs in vertical domains
• Tools and large language models
• Privacy, bias, fairness, toxicity and holistic evaluation
• Alignment and limitations

Assignments (40%)

• Assignment 1 (20%): Using API for testing prompt engineering
• Assignment 2 (20%): A toy LLM application
Both assignments need a report and code attachment if it has coding. See the relevant evalution criterion as the final project.

Final project (55%)

The final project consists of two parts: Project Presentation (15%) and Project Report (40%) .

• Project Presentation (15%): You are required to design your project poster using the specified Poster template. Your poster presentation will be rated by at least 3 experts (TAs and at least one external professor or scientist from industry). The average rating will be the final credit.
• Content quality (5%): Well-presented posters or slides are highly valued.
• Oral presentation (5%): Clear and enthusiastic speaking is encouraged.
• Overall subjective assesment (5%): Although subjective assesment might be biased, it happens everywhere!
• Project report (40%): The project report will be publicly available after the final poster session. Please let us know if you don't wish so.
• Technical excitement (15%): It is encouraged to do something that is either interesting or useful!
• Technical soundness (15%): A) discuss the motivation on why you work this project and your algorithm or approach. Even you are reproducing a published paper, you should have your own motivation. B) Cite existing related work. C) Present your algorithms or systems for your project. Provide key information for reviewers to judge whether it is technically correct. D) Provide reasonable evaluation protocol, it should be detailed to contexualize your results; E)Report quantitative results and include qualitative evaluation. Analyze and understand your system by inspecting key outputs and intermediate results. Discuss how it works, when it succeeds and when it fails, and try to interpret why it works and why not.
• Clarity in writing (5%): The report is written in a precise and concise manner so the report can be easily understood.
• Indivisual contribution (5%): This is based on individual contribution, probably on a subjective basis.
• Bonus and penalty Note that the project credit is capped at 55%
• TA favorites (2%): If one of TAs nominates the project as his/her favorite, the involved students would get 1% bonus credit. Each TA could nominate one and he or she could reserve his/her nomination. This credict could only be obtained once.
• Instructor favorites (1%): If the instructor nominates the project as his/her favorite, the involved students would get 1% bonus credit. Instructor could nominate at most three projects. One could get both TA favorites and Instructor favorites.
• Project early-bird bonus (2%): If you submit the project report by the early submission due date, 2% bonus credit will be entitled.
• Code reproducibility bonus (1%): One could obtain this If TAs think they could easily reproduce your results based on the provide material.
• Ethics concerns (-1%): If there are any serious ethics concerns by the ethics committee (The instructor and all TAs), the project would get 1% penalty.

Participation (5%)

Here are some ways to earn the participation credit, which is capped at 5%.

• Attending guest lectures: In the second half of the course, we have four invited speakers. We encourage students to attend the guest lectures and participate in Q&A. All students get 0.75% per guest lecture (in total 3%) for either attending in person, or by writing a guest lecture report if they attend remotely or watch the recording.
• Completing feedback surveys: We will send out two feedback surveys during the semester to
• User Study: Students are welcone to conduct user study upon their interest; this is not mandatory (thus it does not affect final marks).
• Course and Teaching Evaluation (CTE): The school will send requests for CTE to all students. The CTE is worth 1% credit.
• Volunteer credit (1%): TAs/instuctor can nominate students for a volunteer credit for those who help the poster session organization, or help answer questions from other students (not writing assignments).

Late Policy

The penalty is 0.5% off the final course grade for each late day.