1. How to adapt a general language model to follow task-specific instructions?

❌ Zero-shot learning using templates
❌ Reinforcement learning with real-time feedback
✅ Instruction-tuning with labeled examples
❌ Transfer learning with parameter freezing

Explanation:
Instruction tuning teaches the model to follow instructions using curated instruction → response examples, making it suitable for chatbots.

---

2. Which function formats instruction–response pairs for validation?

❌ preprocess_train_response
❌ response_validator
✅ formatting_prompts_func_no_response
❌ template_instruction_creator

Explanation:
formatting_prompts_func_no_response creates properly formatted prompts for evaluation or validation during instruction tuning.

---

3. How does reward modeling improve response selection?

❌ Reduces training parameters
✅ Tunes the model’s responses based on user or preference signals
❌ Accelerates execution
❌ Expands vocabulary

Explanation:
Reward modeling trains the model to prefer certain responses—such as responses that support cats—by learning from preference labels.

---

4. Which model identifies preference between two responses?

✅ Bradley–Terry model with sigmoid function
❌ Reinforcement learning with entropy loss
❌ Zero-shot calibration
❌ Pairwise contrastive divergence

Explanation:
The Bradley–Terry model is used for pairwise ranking—core to preference modeling in RLHF.

---

5. What does trainer.train() do in RewardTrainer?

❌ Creates training data
❌ Creates train/test splits
❌ Only logs stats
✅ Initiates and performs the full training loop

Explanation:
trainer.train() runs the complete forward/backward passes, optimization steps, and logging.

---

6. Why apply instruction-tuning before RL in LLM training?

❌ Evaluate unseen questions
❌ Train preference ranking samples
✅ Provide the model with structured instruction–response pairs
❌ Pretrain on unstructured data

Explanation:
Before applying reinforcement learning, the model must first learn to follow instructions. Instruction tuning provides this foundation.

---

7. Which function converts raw input into usable get_response format?

❌ Load evaluation weights
❌ Embed attention vectors
❌ Assign static rewards
✅ Create input–output structure for model readiness

Explanation:
This function prepares raw input into a format the model can score or generate from.

---

🧾 Summary Table

---