# MDD Hand Lab — four fingers / eight tendon pullers

Prototype source and printable attachments for index, middle, ring and little finger assistance. Two motors per finger: one palmar flexion tendon and one dorsal extension tendon. No thumb, wrist or elbow actuation. **Bench prototype; no worn or patient validation.**

## Deliverables

- [Printable kit ZIP](cad/printable-kit.zip), [STLs](cad/printable/), [dimensions and printing](cad/README.md), [actual mesh previews](cad/previews/printable-parts.png), and [animated assembled model](cad/assembly.glb).
- [ESP32 firmware](firmware/src/main.cpp) with independent local switch/actuator task and protocol v2 Wi-Fi API.
- [iPhone companion source](app/) with four-finger controls, MediaPipe visual calibration, bounded camera-guided assistance target and native local HTTP transport. Generated Xcode project and [compiled unsigned iPhone archive](builds/ios/README.md) included; signing and device installation remain.
- [Simulation results](simulation/report.json), [tension/movement traces](simulation/trace.png) and [test suite](tests/). Synthetic dynamics, no patient force prediction.
- [3D explanation source](explainer/player.html) with a 32-second seekable scene; browser snapshots and HyperFrames checks pass.
- [Build evidence and remaining work](BUILD_STATUS.md).

If using the website downloads, extract `software-source.zip` into a project folder, then extract `printable-kit.zip` into that folder’s `cad/` subdirectory. The repository-only `scripts/sync-site.py` expects the surrounding project board checkout.

## Quick bench simulator

Requires Python 3, a C++ compiler and Node 22+/npm. No sensor hardware is needed for simulation. The downloadable software ZIP includes the vision assets; a fresh Git checkout fetches them once.

```sh
npm --prefix app ci
npm --prefix app run assets  # omit if using the ZIP: model/WASM already included
npm --prefix app run build
python3 simulation/server.py
```

Open `http://127.0.0.1:8765`, enable **Simulator**, enter **123456**, and connect. Acknowledge any fault, check the bench acknowledgement, then Arm. Wait for `ready`, select fingers, and hold Open/Curl. Each finger also has its own hold controls; different fingers may open and curl simultaneously. Rest unwinds; Stop latches a fault. Simulation runs the **same Motion.h** as firmware, behind an illustrative spring/slack finger plant. It does not emulate the ESP32 radio, ADC electronics, real servos or skin mechanics.

The preview builder uses the installed esbuild path on this machine; set `MDD_ESBUILD` to another executable if needed. It bundles the app UI without requiring Capacitor at runtime in a desktop browser. Camera requires MediaPipe WASM/model assets; no tracking is fabricated when assets or permissions are missing.

## iPhone build

Requires Node 22+, internet for dependencies (and assets if rebuilding from git), a Mac with full Xcode 26+, an Apple signing team, and iOS camera/local-network permissions. Existing `capacitor.config.json` has no remote server override; model and WASM must be bundled before joining ESP32's internet-free AP.

```sh
cd app
npm ci
npm run assets
npm run ios:prepare
npm run ios:open
```

In Xcode choose signing team, connect iPhone, enable Developer Mode if requested by iOS and run. Open the app once and permit camera/local network. Join `MDD-Hand-Lab` (no internet is expected); read its per-boot password and pairing code from the ESP32 USB serial monitor. Address is `http://192.168.4.1`. The installed app uses Capacitor's native HTTP client; a Safari page hosted over HTTPS is not a substitute for this native local-HTTP path.

`ios:prepare` sets camera and local-network purpose strings plus an ATS HTTP exception for the local device transport. The app restricts the entered endpoint; keep this prototype on its isolated AP. Review native permission/signing and live camera/network behavior on the target iPhone. Simulator and physical-device targets compile successfully on hosted Xcode 26.6. Real camera/network operation is still unverified because no iPhone is connected. The generated project, offline assets and compiled unsigned archive are supplied.

## ESP32 wiring

Target: **ESP32 DevKit / WROOM-32**, not the pictured ESP8266. Review before using another board.

| Finger | Flexion signal | Extension signal |
|---|---|---|
| Index | GPIO13 | GPIO14 |
| Middle | GPIO16 | GPIO17 |
| Ring | GPIO18 | GPIO19 |
| Little | GPIO23 | GPIO27 |

Servo power comes from a separately regulated supply matching the actual servos' voltage and measured simultaneous current, with a fuse/current limit. **Do not power eight servos from ESP32 USB or its 3.3 V pin.** Tie logic/supply grounds together. ESP32 supplies PWM signals only. A physical power cut-off and mechanical tendon disconnect are separate from the GPIO stop signal.

| Input | GPIO | Wiring / behavior |
|---|---|---|
| Arm / fault acknowledge | 32 | Momentary button to GND; first press acknowledges a fault, next press arms |
| Open | 33 | Hold to GND: extension; release: low preload |
| Curl | 34 | Hold to GND: flexion; **external 10k pullup to 3.3 V** |
| Rest | 35 | Button to GND: unwind all channels; **external 10k pullup** |
| Stop interlock | 25 | Normally closed circuit to GND; opening/broken wire causes fault |
| Finger selection | 26 | Cycle All → Index → Middle → Ring → Little |
| LOCAL / REMOTE | 39 | Ground = LOCAL; high = REMOTE; **external 10k pullup** |

LOCAL mode needs no phone, Wi-Fi client or heartbeat. Switching ownership while outputs are enabled faults the controller. Both motion buttons together fault rather than choose a direction. REMOTE requires a paired session and a command within 1 s. Switching modes never silently hands control to a phone.

### Tension sensing

Default firmware requires eight calibrated tension channels: amplified analogue load cells feeding **two ADS1115 modules at 0x48 and 0x49**, four channels each, on SDA21/SCL22. Power ADCs and their inputs within 3.3 V limits; verify the amplifier's usable range. This hardware is additional to the spider kit. Invalid/missing/stale acquisition or over-limit tension causes a latched fault.

Determine per-channel no-load counts and counts/newton with known reference loads, then save through the app while disarmed. The supplied bench limits (0.2 N preload, ≤1.98 N requested assistance and 3 N fault threshold) are **control-demo constants, not patient-safe settings**. Calibrate amplifier polarity so increased tension increases ADC counts.

For **unloaded motor/spool testing with tendons detached**, `esp32dev-unloaded` explicitly disables force control and is identified as such in app/status. It moves to fixed travel fractions, never reports measured force, and cannot perform camera-adaptive force assistance. Do not attach that build to a person.

### Flash

```sh
cd firmware
pio run -e esp32dev
pio run -e esp32dev -t upload
pio device monitor -b 115200
```

For the unloaded build substitute `esp32dev-unloaded` in both commands. `platformio.ini` pins platform and JSON-library versions. The PWM API is for the pinned Arduino-ESP32 2.x core, not the incompatible 3.x API. Both target builds compile successfully. Prebuilt full flash images, hashes and flashing instructions are in [firmware/releases](firmware/releases/README.md). No board has been flashed here.

### Rest, arm and motion

Rest is a controlled return to each channel's verified release endpoint; normal tension mode also checks residual load is below 0.1 N before reporting rest and disabling PWM. Residual tension faults with `release_still_tensioned` instead of claiming the ropes are loose. Arm starts at release endpoints and takes up slack using measured low tension. The puller for the requested direction waits for the opposing tendon to unload. Travel exhaustion, force faults, missing sensors, physical stop and remote timeout disable PWM and latch a fault. Fault acknowledgement never arms automatically. A disconnected camera stops adaptive remote operation; manual local switches remain independent of vision.

The camera adapts a bounded **assistance target from movement error**; load sensors close the force loop. It cannot discover safe force, interface pressure, contracture or a clinical range. A printed cap or joint occlusion may degrade tracking. No automatic motor-endpoint search is implemented.

## Tests

```sh
c++ -std=c++17 -Wall -Wextra -Werror tests/motion_test.cpp -o /tmp/mdd-motion-test
/tmp/mdd-motion-test
node --test tests/calibration.test.mts
python3 tests/protocol_test.py
c++ -std=c++17 -Wall -Wextra -Werror simulation/plant.cpp -o /tmp/mdd-plant
/tmp/mdd-plant simulation/trace.csv
```

Node 22.18+ supports the TypeScript stripping used by these tests. Run CAD generation/mesh checks with `python3 cad/generate.py` (NumPy required). Tests cover control and protocol logic; they do not establish iOS usability, real-time ESP32 timing, sensor accuracy, motor travel, printed strength or clinical safety.

The shared `LocalSwitches.h` mapper is host-tested: a held Arm on startup does not arm; a fresh debounced press is needed. Rest overrides Arm, conflicting motion buttons fault, and changing LOCAL/REMOTE while enabled latches a fault. The `/api/actions` endpoint accepts four per-finger actions, so different fingers can open and curl at the same time.

The standard [animated GLB model](cad/assembly.glb) embeds the actual printable meshes. Anatomy and motor cases are labeled proxies. The motor tray sits on a lower cover and four posts, with nominal case clearance checked from the chosen proxy dimensions. This must be compared with the actual servo before printing/assembly.


The website software ZIP includes the generated iOS project and bundled MediaPipe assets. Run `npm ci`, then `npm run ios:prepare` before opening Xcode; `npm run assets` is needed only when fetching/updating assets from a fresh Git checkout. The downloaded model is SHA-256 pinned. `app/mediapipe-assets.json` records the packaged WASM/model files. `npm audit` reports zero known vulnerabilities for the pinned dependency tree at this build.

Real browser validation: start `python3 simulation/server.py`, then run `node tests/browser_test.mjs` with Playwright installed (`MDD_PLAYWRIGHT_PATH` if needed). It tests real HTTP and simultaneous touch against the C++ core, plus actual MediaPipe inference on synthetic no-hand frames. This does not substitute for an iPhone camera/network or physical servo test.

Native build evidence: [successful simulator, device and ESP32 CI jobs](https://github.com/vasanthsreeram/BG4122-mdd-project/actions/runs/38716666110). The iPhone archive requires Apple signing before installation; it is not an installable signed IPA.