L14- Chain of Thought & Tree of Thought | Advanced Prompt Engineering for Complex Reasoning Problems

In Lecture 14 of the AI for Software Engineers series, Bipin Kumar covers the two most advanced prompting techniques in GenAI — Chain of Thought and Tree of Thought. These are the methods that unlock LLM reasoning for truly difficult problems, and they power features like ChatGPT Deep Research behind the scenes.
If you have ever wondered why an LLM gives a wrong answer to a complex question, this class shows you exactly how to fix that.
🔗 Chain of Thought (CoT) — Think Step by Step:
The core idea is simple. Instead of asking the LLM to jump straight to a final answer, you instruct it to break the problem into ordered steps and reason through each one before concluding. The magic phrase that activates this behaviour is "Let's think step by step."
Why it works: just like a human solving a maths problem — we do not see the question and instantly know the answer, we work through it in stages. When you force the LLM to do the same, it catches intermediate errors that would otherwise lead to a wrong final answer.
Real example from class: a bakery sold 3 dozen cupcakes in the morning and 2.5 dozen in the afternoon. Without Chain of Thought, the model gave a number directly — and it was wrong. With "think step by step," the model first calculated morning cupcakes, then afternoon cupcakes, then the total, then the remaining — catching the right answer at each stage.
How to apply it in a prompt:

Add "think step by step" to your question
You can also specify the exact steps to think through — for example: first consider dependency, then reserves, then alternatives, then economic impact
You can combine it with Few-Shot by giving a worked example showing the step-by-step format you want

One-shot CoT example from class: a train speed problem where one worked example was provided first, showing the model exactly how to decompose the reasoning — first hour distance, second hour distance, then combining. The model then followed that exact pattern on a new problem.
Logic puzzle application: "All roses are flowers. Some flowers fade quickly. Can we conclude some roses fade?" — Without CoT the model may guess. With CoT and explicit reasoning steps, it correctly identified that you cannot make that conclusion because the overlap between roses and fading flowers is not established.

🌳 Tree of Thought (ToT) — Explore Multiple Paths:
While Chain of Thought goes in one direction step by step, Tree of Thought deliberately branches out. You ask the LLM to generate several different approaches to a problem, evaluate each one, and then select the best combination.
The structure is always the same three stages:

Stage 1 — Propose multiple strategies or paths (typically 2 to 5)
Stage 2 — Evaluate each path on specific criteria such as cost, risk, short-term effectiveness, or time to result
Stage 3 — Compare all evaluated paths and recommend the best combination
The key difference from Chain of Thought: CoT solves one problem in sequence. ToT solves one problem from multiple angles simultaneously, then merges the best insights.

💸 The Major Disadvantage — Cost:
Both Chain of Thought and Tree of Thought generate significantly more tokens than standard prompting. More steps, more evaluation, more output. This directly increases your API bill.
Because of this, they are not suitable for high-volume applications where thousands of users are sending queries. In most industry scenarios, Zero-Shot with a well-written system prompt is sufficient and far more cost-effective.
Use these techniques when the problem is genuinely complex, appears infrequently, and where accuracy matters more than cost — such as research, strategy analysis, or security-related reasoning.
ChatGPT Deep Research = CoT and ToT:
The Deep Research button in ChatGPT is not magic — it is these exact techniques running in the backend. The model is given structured instructions to think in multiple directions, evaluate results, and synthesise a final answer. That is why it is limited to a small number of uses per day — the token consumption is very high.

⏭️ Next Lecture (Lecture 15):
👉 Structured Output and Output Parsers — getting the LLM to return JSON, typed data, and formatted results

📅 Class Schedule:
Monday, Wednesday & Friday at 7:00 AM

💬 Tried Chain of Thought on a maths problem? Share your prompt and result in the comments — Bipin replies!
📌 Subscribe and turn on notifications so you never miss a class.

#ChainOfThought #TreeOfThought #PromptEngineering #AdvancedPrompting #LangChain #GenAI #LLM #AIforEngineers #BipinKumar #OpenAI #GPT #ReasoningAI #DeepResearch #GenerativeAI #AIInterview

Видео L14- Chain of Thought & Tree of Thought | Advanced Prompt Engineering for Complex Reasoning Problems канала Zero to Deployed