NutriSnap AI is a bilingual (Chinese / English) nutrition and canteen management platform designed for Chinese and international university students, nutrition coaches, partner canteens, and campus health administrators in Chengdu.
It addresses two practical campus dining problems:
- Cross-cultural dining barriers: International students often cannot read Chinese menu names, identify pork or peanut allergens, judge Sichuan spice levels, or speak common ordering phrases at the canteen window.
- Inaccurate photo calorie estimates: Standard photo-logging apps underestimate cooking oil in Chinese canteen dishes and ignore portion variation and plate leftovers. NutriSnap AI replaces guesswork with weighed serving scoops, weighed tray returns, lab-measured oil profiles, and transparent calorie uncertainty ranges.
| Module | Description |
|---|---|
| Cross-Cultural Dish Guide (菜品指南) | 68 canteen dishes (61 Chengdu campus dishes + 7 partner halal window dishes) presented as bilingual culture cards with tone-marked pinyin, cultural stories, ordering tips, spice ratings (0-3), 13 allergen categories (contains / may contain, including pork and peanuts), and halal / vegetarian flags. Includes 6 combinable filters, search in Chinese, English, or toneless pinyin, and 12 spoken Mandarin ordering phrases using browser speech synthesis (GET /api/food/guide, nutrisnap_prototype/dish_culture.json). |
| Scoop-by-Scoop Serving Scale (每一勺都称重) | A weighing terminal in SERVE mode sits under the tray at the canteen window. The server taps a dish and adds a scoop; each weight increment is assigned directly to that dish without requiring computer vision. The canteen receives per-dish average portion and scoop-to-scoop variation metrics. |
| Clean-Plate Return Station (光盘回收台) | A weighing terminal in RETURN mode weighs returned trays. Students scan a QR code to deduct uneaten food from their daily intake (eaten = served - left), while canteens track plate waste and clean-plate rates in line with China's Anti-Food Waste Law. |
| Lab-Measured Oil & Uncertainty Ranges | Supports administrative import of GB 5009.6 acid-hydrolysis lab oil measurements (docs/OIL_STUDY_PROTOCOL.md). Every meal and daily total displays an explicit calorie range calculated from portion-measurement error and nutrient-source error (uncertainty.py). |
| Checkout & Campus-Card Integration | Accepts HMAC-SHA256 signed sales events (with timestamp and replay protection) from existing canteen POS and campus-card systems (tools/pos_simulator.py). Students who opt in to link their campus card (stored only as a keyed hash) receive automatic meal logging. |
| Campus Weight Management Programme | Opt-in campus programme supporting fitness-test (体测) BMI records and generating k-anonymous university reports (WS/T 428-2013 BMI categories, weight change, protein intake, and plate waste, masking cohorts under 10 as <10) with Excel export. |
| Bilingual Fridge Chef & Coaching | Generates macro-targeted recipes in Simplified Chinese or English from available pantry ingredients (POST /api/recipe/generate), paired with nutrition coach meal verification, protein calibration, and 1-on-1 chat. |
| Role | Capabilities |
|---|---|
| Student (Member) | Daily calorie & macro dashboard with uncertainty ranges, Dish Guide with spoken Mandarin phrases, photo / search / barcode meal logging, Fridge Chef recipes, nearby partner canteens, 90-day weight trends, adaptive TDEE calibration, streaks & badges, coach chat (PRO), campus card linking, and PIPL data export / deletion. |
| Nutrition Coach | Client roster, meal log verification, protein calibration, personalized dietary feedback, and client analytics. |
| Partner Canteen (Merchant) | Business-licence onboarding, menu management with 4/4/9 kcal/g consistency checks, weighing terminal registration (SERVE, RETURN, TRAY), and 7-day portion consistency and plate-waste analytics. |
| Administrator | Coach and canteen approval, campus map POI management, GB 5009.6 lab data import, POS integration credentials, anonymised campus programme reporting, and Excel export. |
| Weighing Terminal (Kiosk) | Authenticates via X-Kiosk-Key (stored as SHA-256 hash), restricts selections to its assigned canteen window, reads live weights from ESP32 + HX711 load-cell hardware over Web Serial or serial bridge (with a clearly labelled simulator for demos), and generates single-use 10-minute QR claim tickets. |
schema.sql MySQL 8 schema: 27 tables (3NF) covering users, meals, kiosks, trays, returns, lab data, POS integration, and programme
nutrisnap_prototype/ Zero-dependency Python 3 HTTP server + SQLite (27 tables, ~90 REST endpoints)
server.py Core HTTP server, authentication, recipes, and background scheduler
api_growth.py Extended REST API endpoints (Dish Guide, kiosks, lab data, POS integration, programme)
ai_service.py Optional Qwen-VL / OpenAI-compatible / Gemini connector with deterministic local fallback
food_service.py Chengdu dish search, Open Food Facts barcode lookup, and Cross-Cultural Dish Guide loader
kiosk.py Scoop-by-scoop serving scale, return station, tray weighing, QR tickets, and waste analytics
uncertainty.py Calorie and macro uncertainty propagation model
labdata.py GB 5009.6 lab measurement import and validation
integration.py HMAC-SHA256 POS checkout webhook and campus-card hash linking
programme.py Campus weight-management programme and k-anonymous reporting
demo_seed.py Synthetic demonstration dataset generator (30 synthetic students, 7 days of trays)
food_catalog.json Curated Chengdu canteen dish nutrition catalogue (61 campus dishes + 7 halal window dishes)
dish_culture.json Bilingual dish stories, tone-marked pinyin, allergens, spice levels, and 12 ordering phrases
static/ Responsive bilingual single-page frontend (index.html, css/style.css, js/app.js, growth.js, guide.js, i18n.js)
nutrisnap_springboot/ Spring Boot 3.2 + Spring Security + MyBatis-Plus + MySQL + Redis + WebSocket enterprise backend
tests/api_smoke_test.py 167 end-to-end API checks covering all roles, permissions, kiosks, Dish Guide, and bilingual recipes
tests/kiosk_unit_test.py 10 unit tests for scoop-by-scoop weight assignment, tare handling, and portion splitting
tools/pos_simulator.py CLI simulator for HMAC-signed canteen POS checkout events
hardware/ ESP32 + HX711 load-cell firmware (esp32_hx711_scale_bridge.ino) and Python serial bridge
deploy/ Production Docker Compose stack (MySQL, Redis, API, Nginx TLS)
docs/diagrams_v6/ Bilingual system architecture, core flow, and role interaction diagrams (Chinese & English)
docs/OIL_STUDY_PROTOCOL.md GB 5009.6 acid-hydrolysis lab oil testing protocol and CSV template
docs/LAUNCH_PLAN.md Campus pilot rollout and PIPL compliance plan
Requires Python 3.10+ (uses only the Python standard library and bundled SQLite):
cd nutrisnap_prototype
python database.py # optional: reset to a fresh synthetic demo database
python server.py # starts server at http://localhost:8080Run the automated verification suites in a second terminal (against a fresh database):
python ../tests/api_smoke_test.py http://localhost:8080
python ../tests/kiosk_unit_test.py- Execute
schema.sqlin MySQL 8. - Open
nutrisnap_springbootin IntelliJ IDEA or build with Maven (JDK 17). - Configure
MYSQL_PASSWORD(and optionalDASHSCOPE_API_KEY) and runNutriSnapApplication.
cd deploy
cp .env.example .env
docker compose up -d --build- Flash
hardware/esp32_hx711_scale_bridge.inoto an ESP32 connected to an HX711 load-cell amplifier (DOUT -> GPIO 4,SCK -> GPIO 5). Calibrate with a 500 g reference weight by sendingC500over serial. - Open
http://localhost:8080/?kiosk=1and enter a registered terminal key:- Serving scale demo key:
nsk_demo_serve_canteen2_window8 - Return station demo key:
nsk_demo_return_canteen2 - Whole-tray demo key:
nsk_demo_cdtu_canteen2_window8
- Serving scale demo key:
- Connect via Web Serial in Chrome/Edge or use the built-in simulator during software evaluation.
| Username | Password | Role |
|---|---|---|
alex_student |
password123 |
Student Member (PRO) with 7-day meal and weight history |
john_lifter |
password123 |
Student Member (FREE) |
coach_sarah |
password123 |
Certified Nutrition Coach |
canteen_li |
password123 |
Partner Canteen (Canteen 2 Halal Window, 7 dishes) |
admin |
admin123 |
Platform Administrator |
| Variable | Purpose |
|---|---|
DASHSCOPE_API_KEY |
Enables Qwen-VL vision and Qwen text models via Alibaba Cloud Model Studio |
LLM_BASE_URL, LLM_API_KEY, LLM_VISION_MODEL, LLM_TEXT_MODEL |
Connects any OpenAI-compatible vision/text endpoint (such as GLM-4V) |
GEMINI_API_KEY |
Optional Gemini API key for overseas deployments |
AI_PROVIDER |
auto (default), qwen, openai, gemini, or none |
NUTRISNAP_DEMO_DATA |
Defaults to true for evaluation; set to false in production |
NUTRISNAP_INTEGRATION_SECRET |
HMAC secret for POS checkout integration and campus-card hashing |
HMAC_WEBHOOK_SECRET, PAYMENT_SANDBOX |
Subscription webhook verification secret and sandbox toggle |
- API Smoke Suite (
tests/api_smoke_test.py): 167 / 167 checks passing (authentication, role permissions, Dish Guide filters and allergen flags, bilingual recipes, scoop-by-scoop serving mode, return station leftover deduction, lab import validation, HMAC replay protection, k-anonymous programme reports, and PIPL data export). - Scale Unit Suite (
tests/kiosk_unit_test.py): 10 / 10 checks passing (scoop weight assignment, tare handling, invalid weight rejection, and standard-portion splitting). - End-to-End Browser Verification: 21 / 21 Playwright checks passing across Chinese and English interfaces with zero console errors.
- Data Notice: All student records, weighed trays, and programme reports included in demo mode are synthetically generated by
demo_seed.pyto demonstrate system functionality and do not represent live clinical or pilot measurements.