Work for a Member organization and need a Member Portal account? Register here with your official email address.
Large language models excel at understanding text, but most of the world’s data is not text. It arrives as streams of measurements from sensors, satellites, machines, and infrastructure. This research explores how we can help AI understand time-series data and generate explanations that people can actually use in their daily lives.
Our framework augments sensor measurements with statistical summaries, detected patterns, and contextual information about the monitored system and its environment. To evaluate the impact of this additional context, we compared three approaches: a baseline large language model, a fine-tuned model, and a fine-tuned model augmented with retrieval-based contextual information (RAG). Responses generated by each configuration were assessed using a separate evaluation framework that scored individual interpretation segments for grounding, pattern recognition, context utilization, actionability, and calibration. By combining contextual information with specialized model training, the system produced interpretations that were more grounded, actionable, and reflective of real-world conditions.
Our results show that contextual information substantially improves the ability of AI models to interpret sensor time-series data. The best-performing approach, which combined fine-tuning with retrieval-augmented context (RAG), achieved a 96.6% task success rate on synthetic datasets and 91.1% on real-world sensor data, outperforming both baseline and fine-tuned-only models. The context-augmented approach also achieved the highest scores for grounding, pattern recognition, context utilization, actionability, and calibration, while significantly reducing hallucinations and generic responses. Together, these findings demonstrate that combining sensor measurements with environmental and social context can dramatically improve the reliability and usefulness of AI-generated interpretations.