HAVOC-TemiApp

Source code on GitHub.

README

HAVOC: Healthcare Assistant with Video, Olfaction, and Conversation

HAVOC is an advanced Android application designed to transform the Temi robot into an autonomous healthcare assistant. The project leverages Large Language Models (LLMs), real-time communication, and environmental sensor integration to create a robot capable of navigating complex environments, interacting naturally with people, and performing environmental monitoring tasks.

!image

Features

• Dual LLM-Powered Intelligence: The robot's intelligence is driven by a two-tiered LLM system, using CAAI's LLM-Factory:
- Proactive Planning: A "Planner" model analyzes conversation history and the robot's current context to generate a logical, step-by-step action plan (e.g., moving to a new location, speaking, etc.). - Natural Conversation: A "Talker" model enables fluid conversations with users. It gracefully concludes interactions by executing a tool call whenever the conversation is over or the user gives a direct command.
• Centralized Environmental & Safety Monitoring: HAVOC continuously streams the robot's live video feed, positional coordinates, and raw smell sensor data via WebRTC to a companion repository, HAVOC-Server. This server acts as a centralized hub for all heavy environmental processing.
• Autonomous Operation: The robot operates on a schedule, automatically navigating from its charging base to its work location during operational hours (8 AM to 5 PM on weekdays) and returning to charge when its battery is low or at the end of the day.
• Human Detection & Interaction: The robot uses its built-in sensors to detect when a person is nearby for a sustained period and can proactively initiate a conversation.
• Robust State-Driven Logic: The robot's core behavior is managed by a formal state machine, ensuring its essential duties are predictably completed.

---

Example Execution Flow

A typical "day in the life" of a Temi robot controlled by HAVOC unfolds as follows:

• Initialization: The robot begins in the HomeBase state, resting on its charging dock.

• Shift Start: At 8:00 AM on a weekday, a scheduled timer (checkAndMoveTemi) fires. The robot transitions to the MovingToEntrance state and navigates to its designated work_location.

• On Duty: Upon arrival, it enters the Detecting state, actively waiting for user inte...