Computer Vision

From "what" to "what, where, and how many"

The DL path taught the engine (CNNs, transfer learning). Computer vision the field is about the tasks built on it — a ladder of increasing detail:

• Classification — one label per image: "contains a dog".
• Object detection — a bounding box + label for every object: "3 cars here, 2 pedestrians there". Detectors like the YOLO family ("You Only Look Once") predict all boxes in a single network pass — real-time on video, the workhorse of traffic systems and shop-floor robots.
• Semantic segmentation — classify every pixel (road / sidewalk / person / sky): the scene understanding a self-driving car needs. Instance segmentation further separates each individual object's silhouette.
• Beyond stills — tracking objects across video frames, estimating human pose (joint positions — fitness apps, motion capture), depth estimation, OCR, face verification (your phone's unlock).

Same backbone, different "heads" — one pretrained feature extractor feeding task-specific outputs. Transfer learning in action, everywhere.

Vision systems in production

Deployed vision = ML-workflow discipline (ML path capstone) with domain twists:

• Data is the product. Labeling boxes/masks is expensive; teams lean on augmentation, synthetic data from game engines, and increasingly on pretrained open-vocabulary models to bootstrap labels.
• Domain shift bites hard. A detector trained on sunny California footage degrades in snow, at night, on fisheye lenses. Evaluation must slice by condition (remember slicing?).
• Edge constraints. Vision often runs on cameras, phones, cars — models get compressed (quantization, pruning, distillation) to fit milliwatt budgets at 30 fps.
• Reliability stakes. Perception errors cascade into physical actions; safety-critical vision pairs models with redundancy (multiple sensors — camera + radar + lidar) and monitoring.

The multimodal turn

The newest chapter dissolves the vision/language border. CLIP-style models embed images and texts into a shared vector space (the embedding worldview, completed): "photo of a rusty bicycle" lands near actual such photos — enabling zero-shot classification (no training on your classes!), semantic image search, and content moderation. Diffusion generators (DL path) invert the arrow: text → image. And vision-language models — the multimodal LLMs you can chat with about a screenshot — answer questions, read charts and describe scenes. Vision has become one modality inside general-purpose models; expect the boundaries between "vision engineer" and "AI engineer" to keep fading.

Key takeaways

• The task ladder: classify → detect → segment → track/pose; one backbone, many heads.
• Production vision is dominated by data, domain shift and edge/safety constraints, not architecture.
• CLIP-style shared embeddings + multimodal LLMs merged vision into general AI.