Part V: Foundation Models and Agentic Sensing
Chapter 21  [R]

Multimodal Sensor-Language Models and Sensor Reasoning

"The day I could put my measurements into words was the day you finally believed what I was sensing."

An Articulate Embodied AI Agent

Placeholder chapter page. Sections are produced by the book-skills pipeline.

Sections

  1. 21.1 Aligning sensor representations with language
  2. 21.2 Sensor-to-text: captioning and describing signals
  3. 21.3 Sensor question answering (SensorQA, SensorBench, OpenSQA)
  4. 21.4 LLM-based reasoning over sensor streams (LLaSA, SensorLLM, HARGPT, LLMSense)
  5. 21.5 Prompting and retrieval over sensor context
  6. 21.6 Grounding, hallucination, and verification for physical signals
  7. 21.7 Evaluation of sensor-language systems

Lab 21

build a sensor-QA prototype that answers natural-language questions over an activity/telemetry stream, with a grounding check.