Large Language Models (LLMs)

"Hello, World!" → ["Hello", ",", " World", "!"]           = 4 tokens
"ChatGPT"       → ["Chat", "G", "PT"]                    = 3 tokens
"tokenization"  → ["token", "ization"]                    = 2 tokens
"đ"            → ["å", "đ"]                              = 2 tokens (non-ASCII splits more)

Rule of thumb:
  1 token ≈ 4 characters ≈ 0.75 words (for English)
  100 tokens ≈ 75 words

# Using tiktoken (OpenAI's tokenizer)
import tiktoken

enc = tiktoken.encoding_for_model("gpt-4o")
tokens = enc.encode("Hello, how many tokens is this?")
print(f"Token count: {len(tokens)}")   # 8
print(f"Tokens: {[enc.decode([t]) for t in tokens]}")

Objective: predict the next token
Input:  "The cat sat on the"
Target: "mat"

Input:  "def fibonacci(n): if n <= 1: return n return fibonacci("
Target: "n-1"

Repeat for TRILLIONS of examples from the internet, books, code
→ Model learns grammar, facts, reasoning, coding, math

┌──────────────────────────────────────────────────────────────┐
│                       Transformer                             │
│                                                               │
│  Input tokens: ["The", "cat", "sat", "on", "the"]            │
│       ↓                                                       │
│  Embedding Layer (token → dense vector)                       │
│       ↓                                                       │
│  Positional Encoding (add position information)               │
│       ↓                                                       │
│  ┌─────────────────────────────────────────┐                 │
│  │         Self-Attention Layer             │  × N layers    │
│  │   (each token attends to all others)    │                 │
│  │         Feed-Forward Layer              │                 │
│  └─────────────────────────────────────────┘                 │
│       ↓                                                       │
│  Output: probability distribution over all tokens             │
│  → most likely next token: "mat"                             │
└──────────────────────────────────────────────────────────────┘

Step 1: Supervised Fine-Tuning (SFT)
  Human experts write ideal (prompt, response) pairs
  Fine-tune the base model on these examples

Step 2: Reward Model Training
  Show humans pairs of model responses
  Humans pick which is better
  Train a reward model that predicts human preference

Step 3: RL with PPO
  Use the reward model to score outputs
  Fine-tune the LLM to produce higher-reward responses
  → Model learns to be helpful, harmless, honest

# Temperature = 0: always picks the most likely token (deterministic, focused)
# Best for: code, structured data, factual answers

# Temperature = 1: samples proportionally from the distribution (balanced)
# Best for: most conversational use cases

# Temperature > 1: more random, creative, surprising
# Best for: creative writing, brainstorming

# Top-P (nucleus sampling): only sample from the top P% of probability mass
# top_p=0.9: only consider tokens that together make up 90% of probability

import openai

client = openai.OpenAI()
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write a poem about databases"}],
    temperature=0.9,    # creative
    top_p=0.95,
    max_tokens=200,
)
print(response.choices[0].message.content)

import numpy as np
from typing import List, Tuple, Optional

def decode_next_token(
    logits: np.ndarray, 
    temperature: float = 1.0, 
    top_k: int = 0, 
    top_p: float = 0.0
) -> Tuple[int, np.ndarray]:
    # logits shape: (vocab_size,)
    
    # 1. Handle Greedy Decoding (zero temperature)
    if temperature == 0.0:
        idx = int(np.argmax(logits))
        probs = np.zeros_like(logits)
        probs[idx] = 1.0
        return idx, probs
        
    # Apply temperature scaling
    scaled_logits = logits / temperature
    
    # 2. Apply Top-K filtering
    if top_k > 0:
        indices_to_remove = scaled_logits < np.sort(scaled_logits)[-top_k]
        scaled_logits[indices_to_remove] = -1e9  # mask out
        
    # Convert logits to probabilities via Softmax
    exp_logits = np.exp(scaled_logits - np.max(scaled_logits))
    probs = exp_logits / np.sum(exp_logits)
    
    # 3. Apply Top-P (Nucleus) filtering
    if 0.0 < top_p < 1.0:
        sorted_indices = np.argsort(probs)[::-1]
        sorted_probs = probs[sorted_indices]
        cumulative_probs = np.cumsum(sorted_probs)
        
        # Identify elements exceeding top_p threshold
        sorted_indices_to_remove = cumulative_probs > top_p
        # Shift mask to keep the first token exceeding top_p
        sorted_indices_to_remove[1:] = sorted_indices_to_remove[:-1].copy()
        sorted_indices_to_remove[0] = False
        
        indices_to_remove = sorted_indices[sorted_indices_to_remove]
        probs[indices_to_remove] = 0.0
        probs = probs / np.sum(probs)  # re-normalize
        
    # Sample from the final distribution
    sampled_idx = int(np.random.choice(len(probs), p=probs))
    return sampled_idx, probs

# Example logits for 5 vocabulary tokens: ["the", "cat", "sat", "mat", "door"]
vocab = ["the", "cat", "sat", "mat", "door"]
mock_logits = np.array([2.0, 1.5, 0.5, 3.0, -1.0])

# Test: low temperature (focused), Top-K=3
idx, final_probs = decode_next_token(mock_logits, temperature=0.5, top_k=3)
print("Vocabulary probabilities after filtering:\n", dict(zip(vocab, final_probs)))
print("Sampled next token:", vocab[idx])

messages = [
    {
        "role": "system",
        "content": """You are PrepDeck AI, a world-class software engineering mentor.
Your goal is to help engineers crack FAANG interviews.
Always explain concepts from first principles.
When explaining algorithms, always include:
1. Intuition and analogy
2. Step-by-step breakdown
3. Code in Python
4. Time and space complexity
5. Common interview variations

Be encouraging but technically rigorous. Never give wrong information."""
    },
    {
        "role": "user",
        "content": "Explain binary search"
    }
]

# ❌ Vague prompt
"Explain sorting"

# ✅ Specific, structured prompt
"""
Explain merge sort for a software engineering interview.
Include:
1. The problem it solves (and why quicksort isn't always better)
2. The divide-and-conquer intuition with an example
3. Python implementation
4. Time complexity: O(n log n) — explain why
5. Space complexity and when that matters
6. One trick interview question about merge sort
Keep the response under 500 words."""

# ✅ Chain-of-thought prompting (for complex reasoning)
"""
A train leaves station A at 9am going 60mph.
Another train leaves station B at 10am going 90mph.
The stations are 300 miles apart. When do they meet?

Think step by step:"""

# ✅ Few-shot prompting (show examples)
"""
Classify the sentiment:
Input: "This product is amazing!" → Positive
Input: "Terrible quality, broke in a day" → Negative
Input: "It's okay I guess" → Neutral
Input: "Never buying from here again!" → """

from openai import OpenAI
import os

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

# Simple completion
def ask(question: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a helpful coding mentor."},
            {"role": "user", "content": question}
        ],
        temperature=0.7,
        max_tokens=1000,
    )
    return response.choices[0].message.content

# Streaming (show text as it's generated — better UX)
def ask_streaming(question: str):
    stream = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": question}],
        stream=True,
    )
    for chunk in stream:
        if chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="", flush=True)

class ConversationManager:
    def __init__(self, system_prompt: str):
        self.messages = [{"role": "system", "content": system_prompt}]
        self.max_history = 20  # limit history to control costs

    def chat(self, user_message: str) -> str:
        self.messages.append({"role": "user", "content": user_message})

        response = client.chat.completions.create(
            model="gpt-4o",
            messages=self.messages[-self.max_history:],  # sliding window
        )
        assistant_message = response.choices[0].message.content
        self.messages.append({"role": "assistant", "content": assistant_message})
        return assistant_message

from pydantic import BaseModel
from typing import List

class InterviewQuestion(BaseModel):
    question: str
    difficulty: str   # "easy" | "medium" | "hard"
    topic: str
    hints: List[str]
    time_complexity: str

# OpenAI structured output (guaranteed valid JSON)
response = client.beta.chat.completions.parse(
    model="gpt-4o",
    messages=[{
        "role": "user",
        "content": "Generate a binary search interview question"
    }],
    response_format=InterviewQuestion,
)
question = response.choices[0].message.parsed
print(f"Q: {question.question}")
print(f"Difficulty: {question.difficulty}")
print(f"Hints: {question.hints}")

import json

# Define tools the LLM can call
tools = [{
    "type": "function",
    "function": {
        "name": "search_documentation",
        "description": "Search the PrepDeck documentation for a topic",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Search query"},
                "section": {
                    "type": "string",
                    "enum": ["dsa", "hld", "lld", "backend"],
                    "description": "Which section to search"
                }
            },
            "required": ["query"]
        }
    }
}]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Find articles about hash tables"}],
    tools=tools,
    tool_choice="auto",
)

# Handle tool calls
if response.choices[0].message.tool_calls:
    for tool_call in response.choices[0].message.tool_calls:
        args = json.loads(tool_call.function.arguments)
        result = search_documentation(**args)   # your actual function
        # Add result back to conversation...

Cost management:
  - Use smaller models for simple tasks (gpt-4o-mini over gpt-4o)
  - Cache repeated responses (semantic caching with vector DB)
  - Limit context window usage

Latency:
  - Streaming responses feel faster (first token appears quickly)
  - Parallel calls when queries are independent

Reliability:
  - Retry with exponential backoff on rate limit errors
  - Fallback to smaller models on failures
  - Set max_tokens to prevent runaway costs

Safety:
  - Input validation (don't let users inject system prompt overrides)
  - Output filtering (content moderation API)
  - Never expose your system prompt directly to users

# Conceptualizing Vector Distance
import numpy as np

# A 3-dimensional mock vector representation
vector_reset_password = np.array([0.9, 0.1, -0.05])
vector_forgot_login   = np.array([0.85, 0.15, -0.02])
vector_capital_france = np.array([-0.2, 0.7, 0.8])

# Cosine similarity helper
def cosine_similarity(v1, v2):
    return np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2))

print(f"Similarity (Reset vs Forgot): {cosine_similarity(vector_reset_password, vector_forgot_login):.4f}")
# Outputs close to 1.0 (highly similar)
print(f"Similarity (Reset vs France): {cosine_similarity(vector_reset_password, vector_capital_france):.4f}")
# Outputs close to 0.0 or negative (unrelated)