Futuriti AI Chatbot

A conversational AI assistant for futuriti.org that helps Kentucky students and adults explore careers, education programs, and living wages through natural conversation.

Built to serve two grant requirements simultaneously:

• Action Fund Grant — Living wage calculator delivered conversationally
• Lumina Grant — Anonymous student interest pipeline data collection

Features

• Conversational career exploration — Students ask questions in plain English, the AI responds with Kentucky-specific data about salaries, programs, costs, and next steps
• Living wage calculator — Compares career salaries against county-level cost of living
• FERPA-compliant data collection — Anonymously captures career interests, schools considered, and programs explored for grant reporting
• PDF summaries — Downloadable conversation summaries with action items; re-uploadable to continue a session
• Session persistence — Each session gets a unique URL that can be bookmarked and shared to return later
• Mobile-friendly, WCAG AA compliant — Responsive design with proper ARIA labels, keyboard navigation, and color contrast

Architecture

src/
  scraper/       Multithreaded Playwright-based web scraper for futuriti.org
  rag/           ChromaDB vector store with sentence-transformers embeddings
  chat/          OpenAI-compatible LLM client with 7 tool-calling functions
  analytics/     SQLite-backed anonymous interest data collection
  pdf/           ReportLab PDF generation and PyPDF2 parsing for re-upload
  data/          Excel/CSV loaders and structured data (SQLite)
  web/           FastAPI server with session persistence and chat UI

Prerequisites

• Python 3.11+
• An OpenAI-compatible LLM API (OpenAI, Azure OpenAI, Ollama, vLLM, etc.)

Quick Start

# 1. Clone and enter the project
cd Futuriti

# 2. Create and activate virtual environment
python -m venv .venv
source .venv/Scripts/activate   # Windows/Git Bash
# source .venv/bin/activate     # macOS/Linux

# 3. Install dependencies
pip install -r requirements.txt
playwright install chromium

# 4. Configure environment
cp .env.example .env
# Edit .env with your LLM API credentials (base URL, API key, model name)

# 5. Scrape futuriti.org (~329 pages, ~15-20 minutes)
python scripts/scrape.py

# 6. Ingest all data (web + Excel) into the vector store and structured database
#    Place the Futuriti Excel file at data/csv/ before running
python scripts/ingest.py

# 7. Start the chatbot server
python scripts/serve.py
# Open http://localhost:8000

Configuration

All configuration is in .env. See .env.example for all available options:

Data Ingestion

scripts/ingest.py handles all data ingestion in one step:

1. Web data — Ingests scraped futuriti.org content into ChromaDB
2. Excel data — Loads the Futuriti Excel file into SQLite and embeds structured data chunks into ChromaDB

# Full ingest (web + Excel)
python scripts/ingest.py

# Reset the vector store and re-ingest everything
python scripts/ingest.py --reset

# Only ingest web data (skip Excel)
python scripts/ingest.py --skip-xlsx

# Only ingest Excel data (skip web)
python scripts/ingest.py --skip-web

# Use a custom Excel file path
python scripts/ingest.py --xlsx-file path/to/file.xlsx

The Excel file should be placed at data/csv/ (default: data/csv/Futuriti_Data_Feb 2026_withMonthYearlyCalculator-1.xlsx). If the file is not found, Excel ingestion is skipped with a warning.

Loading CSV data separately

For standalone CSV imports (wages, living wage, programs):

python scripts/load_csv.py data/csv/living_wage.csv living_wage
python scripts/load_csv.py data/csv/salaries.csv career_salary
python scripts/load_csv.py data/csv/programs.csv programs

LLM Tool Calling

The chatbot uses 7 tools via function calling:

Before CSV data is loaded, salary/wage/program tools fall back to RAG search with a note that precise figures will be available soon.

Project Structure

.env.example          Environment variable template
requirements.txt      Python dependencies
scripts/
  scrape.py           Scrape futuriti.org
  ingest.py           Ingest web + Excel data into vector store and SQLite
  load_csv.py         Load standalone CSV data
  load_xlsx.py        Load Excel data only (without web ingestion)
  serve.py            Start the web server
src/
  config.py           Pydantic settings loader
  scraper/            Web scraper (Playwright + ThreadPoolExecutor)
  rag/                RAG pipeline (ChromaDB + sentence-transformers)
  chat/               Chat engine (OpenAI client + tool calling loop)
  analytics/          Anonymous interest data collection (SQLite)
  pdf/                PDF generation (ReportLab) and parsing (PyPDF2)
  data/               CSV loaders for structured data
  web/                FastAPI app, routes, session store, and static UI
data/
  raw/                Raw scraped HTML (gitignored)
  processed/          Extracted text JSON (gitignored)
  csv/                Place CSV data files here
  chroma_db/          Vector store (gitignored)

License

Apache License 2.0. See LICENSE.

Imported from gh:Kentucky-Open-Science/FuturitiPOC. Source last updated 2026-04-10. Synced 2026-07-27.