Discuss model pre-training and the value of continued pre-training vs fine-tuning Define the terms Generative AI, large language models, prompt, and describe the transformer architecture that powers LLMs Describe the steps in a typical LLM-based, generative AI model lifecycle and discuss the constraining factors that drive decisions at each step of model lifecycle Discuss computational challenges during model pre-training and determine how to efficiently reduce memory footprint Define the term scaling law and describe the laws that have been discovered for LLMs related to training dataset size, compute budget, inference requirements, and other factors.
Describe how fine-tuning with instructions using prompt datasets can improve performance on one or more tasks Define catastrophic forgetting and explain techniques that can be used to overcome it Define the term Parameter-efficient Fine Tuning (PEFT) Explain how PEFT decreases computational cost and overcomes catastrophic forgetting Explain how fine-tuning with instructions using prompt datasets can increase LLM performance on one or more tasks
Describe how RLHF uses human feedback to improve the performance and alignment of large language models Explain how data gathered from human labelers is used to train a reward model for RLHF Define chain-of-thought prompting and describe how it can be used to improve LLMs reasoning and planning abilities Discuss the challenges that LLMs face with knowledge cut-offs, and explain how information retrieval and augmentation techniques can overcome these challenges