Skip to content

Changelog

[2026-02-19] - Full Test Suite

Test Infrastructure

  • Framework: pytest + FastAPI TestClient (httpx) with pytest-cov for coverage.
  • Mocking: Library-level mocks for Supabase, OpenAI, and Pinecone — started at module level in conftest.py before any app code imports.
  • Auth: Dependency overrides for get_current_user and require_admin in TestClient fixtures.

Test Coverage (148 tests, all passing)

  • Services (8 files, 72 tests): pdf_processor, tier_config, cache, quality_checker, deduplication, text_normalizer, wallet_reservation, llm.
  • Routers (10 files, 66 tests): health, auth, me, curriculum, wallet, admin_users, admin_ingestion, admin_documents, scraping, chat.
  • Agents (1 file, 10 tests): teacher_agent node functions (decide_path, check_wallet, retrieve_and_rerank, ask_clarifying, finalize).
  • Coverage: 58% overall (100% on pdf_processor, tier_config, dependencies; 79-99% on wallet_reservation, text_normalizer, quality_checker).

Documentation Sync

  • Moved backend_architecture.md from docs root to docs/30_design/.
  • Added Testing Guide runbook (docs/20_runbooks/testing.md).
  • Updated repo structure, progress, backlog, and changelog to reflect current state.
  • Updated mkdocs.yml navigation with all new and previously missing files.

[2026-02-18] - Gap-Filling: UI Integration Endpoints

Documents Table Enrichment

  • Migration: 20260218000021_documents_enrichment.sql adds optional columns (title, major, weight, category, reference_id, status, pinecone_index/namespace/vector_count) to documents table.
  • Documents are now self-contained for student queries — no joins with references needed.
  • Reference metadata is copied during ingestion; NULL for manual uploads, updatable via PATCH /admin/documents/{id}.

Student Content Browsing (3 new endpoints)

  • New router: app/api/routers/curriculum.py at /curriculum.
  • GET /curriculum/subjects — distinct subjects from ready documents.
  • GET /curriculum/textbooks — textbooks grouped by grade, with optional filters.
  • GET /curriculum/textbooks/{id} — single textbook detail with page range from chunks.

Job-Based Ingestion (4 new endpoints)

  • POST /admin/ingest/{reference_id} now creates a queued job instead of running synchronously.
  • POST /admin/jobs/dispatch — picks oldest queued job, runs it as a FastAPI BackgroundTask.
  • GET /admin/jobs — list jobs with status filter and pagination.
  • GET /admin/jobs/{id} — single job detail.
  • POST /admin/jobs/{id}/requeue — requeue failed jobs with retry tracking.
  • Background task follows state machine: queued → parsing → tokenizing → embedding_request_sent → embedding_upserted → ready.
  • Audit trail via ingestion_audit table at each state transition.

Admin User CRUD (3 new endpoints)

  • POST /admin/users — create user via Supabase Auth admin API (profile auto-created by trigger).
  • DELETE /admin/users/{user_id} — delete from profiles + Supabase Auth.
  • POST /admin/users/{user_id}/reset-password — reset via Supabase Auth admin API.

Admin Stats & Document Management (3 new endpoints)

  • GET /admin/stats — dashboard aggregates (documents, chunks, references, users, jobs).
  • PATCH /admin/documents/{id} — update optional metadata on documents.
  • DELETE /admin/documents/{id} — delete document + chunks + Pinecone vectors, reset reference status.

Upload Curriculum Enhancement

  • POST /admin/upload-curriculum now creates a documents record on success.

[2026-02-17] - Architecture Fix: Service Layer Consolidation

Supabase Key Cleanup

  • Removed SUPABASE_ANON_KEY and SUPABASE_KEY from config and entire codebase.
  • Standard: All backend operations now use SUPABASE_SERVICE_ROLE_KEY exclusively.
  • Removed get_user_client() and get_supabase_client() from auth.py.
  • Auth/Me/Scraping/Admin routers now use supabase_service singleton from dependencies.py.

Teacher Agent Rewrite

  • Replaced old wallet.*, retrieval.*, supabase_store.* calls with new service layer.
  • Billing: Agent now uses wallet_service.reserve() / wallet_service.finalize() (atomic reservation pattern).
  • Retrieval: Agent now uses retrieval_pipeline.retrieve() instead of old retrieval.retrieve_context().
  • Logging: Usage logged directly via supabase_service instead of old supabase_store.log_usage().
  • State: Added reservation_id field to TeacherState.

Chat Router

  • Replaced from app.services import llm, wallet, supabase_store with from app.services.llm import generate_answer.
  • Added reservation_id to initial agent state.

Admin Ingestion

  • New endpoint: POST /admin/ingest/{reference_id} - Downloads PDF from URL, chunks, embeds, upserts to Pinecone.
  • Replaced old pdf_processor, embeddings, pinecone_store, supabase_references imports with embedding_service, pinecone_adapter, supabase_service from dependencies.py.
  • Legacy: POST /admin/vector-embedding kept as redirect to new endpoint.

Scraping Router

  • Replaced supabase_references.* function imports with inline functions using supabase_service directly.
  • No dependency on old supabase_references.py module.

Wallet: Transactions & Top-Up

  • New migration: 20260217000020_transactions.sql - Financial transactions table for real-money tracking.
  • New models: TopUpRequest, ExpenseRequest, TransactionResponse, TopUpResponse, TransactionListResponse.
  • New endpoints: POST /wallet/topup, POST /wallet/transactions/expense, GET /wallet/transactions.
  • WalletReservationService: Added topup() method and calculate_output_cost() static method.

Cleanup: Deleted Files

  • Old services (7): wallet.py, supabase_store.py, supabase_references.py, retrieval.py, embeddings.py, pinecone_store.py, reranker.py
  • Stub routers (2): scraper_admin.py, metrics.py
  • Misc (6): report.py, koutoubi_report.json, postman/collection_v2.json, scripts/test_scraping_koutoubi.py, db/verification/, app/db/

[2026-02-16] - DB Audit & Verification

  • Audit: Performed full inventory of Supabase DB state vs Repo intent. No drift detected.
  • Verification: Created db/verification/verify_all.sql and timestamped audit script.
  • Verification: Appended inline SQL verification blocks (as comments) to all 11 migration files.
  • Postman: Added "DB Hardening Verification" folder with RLS/Auth test cases.
  • Docs: Generated supabase_db_verification_report.md.

[2026-02-16] - DB Hardening: Phase 2

  • Security: Enabled strict RLS on all remaining public tables (documents, chunks, scrape_runs, references, personas).
  • Security: Hardened handle_new_user against search_path hijacking.
  • Performance: Added missing indexes for foreign keys on chunks and scrape_runs.
  • Docs: Updated supabase_state.md and created supabase_db_verification_report.md.

[2026-02-16] - Admin Dashboard & Auth MVP

  • Auth: Added /auth/signup, /auth/signin, /auth/logout, and /auth/reset-password endpoints.
  • User: Added /me (GET/PATCH) for profile management with RLS support.
  • Admin: Consolidated all admin endpoints under /admin. Added User Management (List/Role update) and global monitoring for Scrapers and References.
  • Postman: Revamped collection with JWT auto-capture and folder-based organization.
  • Requirements: Added email-validator for Pydantic email validation.

[2026-02-16] - Production Hardening Cycle

  • Auth: Replaced manual x-admin-key checks with require_admin dependency supporting Supabase JWT Roles + Key fallback.
  • Ingestion: Added POST /vector-embedding for reference-driven ingestion; deprecated direct /upload-curriculum.
  • RAG: Implemented "translate-for-retrieval" for Arabic queries to improve recall against French corpus.
  • DB: Created migration 20260216000006_rls_phase_1.sql enabling RLS and adding retrieval_query logging.
  • Docs: Updated all design docs and API inventory to reflect the new state.

[2026-02-16] - Initial Structured Backend

  • Added: Agentic RAG core with LangGraph.
  • Added: Koutoubi scraper with metadata inference.
  • Added: EduWallet system with transaction ledger.
  • Added: SSE Streaming support for real-time chat.
  • Added: Comprehensive /docs structure.

[Pre-2026-02-16]

  • Prototype implementation of FastAPI + Supabase.
  • Initial PDF processing experiments.

Back to Index