Lecture 3: Incremental & Decremental Design Strategies [12/01/2026]

Course: Advanced Data Structures and Algorithms

1. Course Context & New Arrivals [00:00:00]

The instructor welcomes new students from Meti (Ministry of Electronics and Information Technology) joining for the 16th edition of the coursework.

Recap: Algorithms scale problem-solving via efficiency. Cryptography involves enormous parameters (2048-bit numbers, 300+ digits), making efficient designs (encryption/decryption) a matter of life and death for systems.
Resource Focus: Time and Space. Memory optimization is critical for chips (smart cards, credit cards), sensor networks, IoT, and mobile devices with limited RAM/energy.

Definition: A "Paradigm" is a pattern of thought/computation. Incremental design involves building a solution by iteratively adding elements.

Pattern: Solve for $A_1$, use that to solve ${A_1, A_2}$, and continue until ${A_1, \dots, A_n}$.
Example: Summation [00:26:38]: $S_i = S_{i-1} + A_i$.

Logic: Assume $A_1 \dots A_{i-1}$ is already sorted. For a new element $A_i$, move it left by swapping until it finds its proper position.
Code Evolution/Bug Fixing [00:44:48]:
- Naive: while A[k-1] > A[k]: swap(A[k-1], A[k]).
- Risk: If $A_i$ is the smallest element, it tries to compare with $A_0$ (non-existent/out-of-bounds), causing a crash.
- Fix 1 (Safe Control): while k > 1 and A[k-1] > A[k]. This adds a bookkeeping check to every iteration.
Tuning for Efficiency (The Sentinel) [00:56:04]:
- Concept: Place an "exceptional value" (Sentinel) at $A_0$.
- Implementation: Set $A_0 = -\infty$. Now, the k > 1 check is unnecessary because the sentinel will always stop the comparison loop. This reduces logic overhead in the inner loop.
Tuning for Efficiency (Shifting vs. Swapping) [01:02:09]:
- Swapping: Requires 3 assignment operations (TEMP variable).
- Shifting: Save $A_i$ in $X$. Shift elements right ($A[k+1] = A[k]$) until the gap for $X$ is found.
- Gain: Shifting uses 1 assignment per step vs 3 for swapping. This effectively triples the speed of the inner loop.

Definition: Progressively working with smaller subsets until the set is empty (Construction by Pruning/Elimination).

Example: Binary Search [01:40:16]: Eliminates 50% of the search space with one comparison. $n \to n/2 \to n/4 \dots$ results in $O(\log n)$ complexity. This is significantly faster than linear search (e.g., 20 steps vs 1 million steps for $10^6$ elements).

Logic:
- Find the maximum element in $A[1 \dots i]$.
- Swap it with the last element $A[i]$.
- Reduce the problem size by focusing on $A[1 \dots i-1]$.
Complexity Analysis [02:08:43]:
- Stage $i$ requires $(i-1)$ comparisons.
- Total Comparisons: $(n-1) + (n-2) + \dots + 1 = \frac{n(n-1)}{2} = \Theta(n^2)$.
- Comparison: Insertion and Selection Sort share the same growth rate ($\Theta(n^2)$), though Selection Sort performs fewer swaps (beneficial if swapping is expensive).

Definition: In a group of $n$ people, a Celebrity knows no one, but everyone knows the celebrity.

Naive Solution: Build an $N \times N$ "Who knows whom" matrix. $O(n^2)$ queries.
Clever Decremental Solution [02:23:41]:
1. Ask: "Does $P_1$ know $P_2$?"
2. If YES: $P_1$ cannot be a celebrity. Eliminate $P_1$.
3. If NO: $P_2$ cannot be a celebrity (since everyone must know the celebrity). Eliminate $P_2$.
4. Observation: 1 query eliminates 1 person.
5. After $n-1$ queries, only 1 candidate remains.
6. Verification: 2 checks per other $(n-1)$ people to verify the final candidate.
Total Complexity: $(n-1) + 2(n-1) = 3n-3$, which is $O(n)$ (Linear Time).
Impact: For $n=1000$, queries drop from 1,000,000 (Naive) to ~3,000 (Clever).

The instructor emphasizes that creativity cannot be "taught" via formulas.

Analogy: A tiger cub learns to hunt by observing patterns (which bush to hide behind) and innovating.
Goal: By studying classic "clever" algorithms, students internalize mental models to attack future, unique problems in their professional careers.

Key Concepts: