A free gesture synth that runs online in your browser — no downloads, no plugins, just your webcam and your hands. FAQ ↓
Gesture Synth Weld is a browser-based gesture synth — a hand gesture music synthesizer that runs entirely in your web browser. Using your webcam and MediaPipe hand tracking, it creates a two-hand musical instrument — your left hand controls harmony (key, scale degree, major/minor toggle) and your right hand controls expression (chord type, volume, tone, and octave). No downloads, no MIDI controllers, no plugins required.
Choose from 3 modes: Gesture (two-hand chord playing), Theremin (continuous pitch and volume control), and Mono Piano (single-note melody). Enable the Arpeggiator to sweep through chord notes like a harp, or toggle Auto Bass for an automatic low-end foundation that follows your harmony. Record up to 60 seconds of your performance — audio, full video, or privacy-friendly skeleton animation — ready to download or share to social platforms.
Powered by the Web Audio API via Tone.js for low-latency synthesis and MediaPipe Hands for real-time finger tracking (30fps). All processing happens locally on your device — your camera feed is never uploaded to any server.
What you need: a computer with a webcam, a modern browser (Chrome, Edge, or Firefox recommended), and speakers or headphones. For the best experience, use Chrome or Edge with WebGL2 hardware acceleration enabled.
A gesture synth turns body movement into sound. Browser-based gesture synths use your webcam to track your hands — no hardware controller needed. Gesture Synth Weld pushes the format further: each finger becomes a musical command, your left hand picks harmony while your right hand shapes expression, and everything runs live in the browser with zero installs.
Compare that to hardware gesture synths (gloves, ring sensors, MIDI theremins): they're powerful, but they cost hundreds of dollars and need setup. A webcam gesture synth gets you the same experience for free — as long as your hands are in the frame. That's the trade-off: no hardware cost, just your two hands.
Three ways to play music with gestures — here's how they compare:
| Approach | Cost | Setup | Pros | Cons |
|---|---|---|---|---|
| Browser gesture synth (webcam + hand tracking) | Free | None — open the page, allow camera | Zero install, zero cost, runs on any device | Needs light and clear hand visibility |
| Hardware gesture synth (gloves, ring sensors, MIDI theremins) | $100 – $1,000+ | Drivers, calibration, MIDI wiring | Precise, low latency, works in any light | Expensive, bulky, needs setup |
| DAW + MIDI controller | Software + hardware cost | Install DAW, configure mappings | Full production power, unlimited tracks | Steep learning curve, desk-bound, not gesture-based |
Prefer an instrument you can pick up in seconds? See the full gesture playbook or the FAQ — or just start playing with the controls above.
Click "Enable Camera" and grant camera permission. Your video stays on your device — nothing is uploaded.
Choose a key (C–B, default A). 1–5 fingers select scale degrees I–V. Index+Pinky = VI, +Thumb = VII. Default mode is locked to Major — switch to Scale+Tilt in settings to control major/minor with wrist tilt.
Fingers set chord complexity (1=triad, 2=inversion, 3=7th, 4=9th). Height = volume, tilt = tone. Both hands required for sound — left fist mutes it, right fist restores octave. Or lock chord style
All gesture combinations for Gesture mode. Both hands required — left hand sets harmony, right hand controls expression. Quick start ↑
Here's how to use Gesture Synth Weld as a hand gesture music synthesizer — every gesture, mapped:
Your left hand picks the chord — each finger count is a different scale degree:
| Fingers | Chord | Gesture | Example (A Major) |
|---|---|---|---|
| 1 | I | Any 1 finger raised | A |
| 2 | ii | Any 2 fingers (not VI combo) | Bm |
| 3 | iii | Any 3 fingers (not VII combo) | C#m |
| 4 | IV | Any 4 fingers raised | D |
| 5 | V | All 5 fingers raised | E |
| VI | vi | Index + Pinky ONLY (middle/ring/thumb must be down) | F#m |
| VII | vii° | Index + Pinky + Thumb ONLY (middle/ring must be down) | G#dim |
| 0 (Fist) | — | All fingers curled in | Mute (notes held, resume when fingers return) |
Default is Scale notes only (Major locked) — wrist tilt is off until you switch to Scale+Tilt in Settings.
| Wrist Action | Result | Notes |
|---|---|---|
| Tilt Left (tilt < 0) | Minor | All chords become minor (I→i, IV→iv, V→v, etc.) |
| Tilt Right (tilt ≥ 0) | Major | All chords become major (I, IV, V, etc.) |
| Scale notes only (lock mode) | Fixed | Mode locked (default: Major) — ignores wrist tilt |
Your right hand shapes the sound — chord type, volume, tone, and octave:
| Fingers | Major Chord Type | Minor Chord Type |
|---|---|---|
| 1 | Major Triad | Minor Triad |
| 2 | Major 1st Inversion | Minor 1st Inversion |
| 3 | Major 7th | Minor 7th |
| 4 | Major 9th | Minor 9th |
| 0 (Fist) | Default chord type — the tucked thumb also brings the octave back to normal; sound continues | |
Finger count → chord type requires Right Hand Mode = Finger Layout in Settings — the default is Fixed Chord Style (Major Triad), where the right hand controls volume + tone instead. Or lock to a fixed chord style in Settings — choose from 8 types: Major Triad, Major 1st Inv, Minor Triad, Diminished, Sus2, Sus4, Major 7th, Dominant 7th.
Height controls volume; wrist tilt sweeps the tone filter:
| Movement | Controls | How It Works |
|---|---|---|
| Hand Height (Y-axis) | Volume | Hand higher = louder. Hand lower = quieter. |
| Wrist Tilt (left/right) | Filter Sweep / Tone | Tilt right = brighter (filter opens). Tilt left = darker (filter closes). |
Thumb out drops the octave for low bass lines:
| Thumb Position | Result | Status Display |
|---|---|---|
| Thumb curled in (default) | Normal octave | e.g. A |
| Thumb extended out | Down one octave (-8ve) | e.g. A (-8ve) |
Ways to stop or mute the sound:
| Condition | Behavior |
|---|---|
| Left fist (0 fingers) | Music mutes — notes held, resume when fingers return |
| Right fist (0 fingers) | Thumb tucks in → back to normal octave — sound continues, chord keeps updating |
| One hand leaves frame | Music stops after 80ms grace period |
| Both hands leave frame | Music stops after 80ms grace period |
Both hands required — the left hand picks a single note (interval + key), the right hand triggers it and controls volume. Left fist = silence.
| Left Fingers | Interval | Example (Key A) |
|---|---|---|
| 1 | Root note | A4 |
| 2 | Major 3rd | C#5 |
| 3 | Perfect 5th | E5 |
| 4 | Octave | A5 |
| 5 | 9th (octave + 2nd) | B5 |
Both hands required — right hand height = pitch (C3 → C6, logarithmic), left hand height = volume. Raise your right hand for higher notes; raise your left hand for a louder sound.
Everything you can fine-tune in the Settings panel:
| Control | Options | Notes |
|---|---|---|
| Key | 12 keys (C – B) | Toolbar dropdown — transposes every chord |
| Left Hand Mode | Scale+Tilt · Scale notes only | Scale+Tilt: wrist tilt flips major ↔ minor (default: Scale only, Major locked) |
| Right Hand Mode | Finger Layout · Fixed Chord Style | Finger Layout: 1–4 fingers = triad / 1st inv / 7th / 9th (default: Fixed, Major Triad) |
| Fixed Chord Style | 8 types | Major Triad · Major 1st Inv · Minor Triad · Diminished · Sus2 · Sus4 · Major 7th · Dominant 7th |
| Timbre | Gesture (sawtooth) · Theremin (sine) | Follows the selected mode |
| Arpeggiator | Off · Slow · Normal · Fast | ⟿ toggle — sweeps chord notes sequentially like a harp |
| Auto Bass | Off · On (volume slider) | ∿ toggle — root note two octaves below, independent volume |
| Metronome | BPM 40–240 · 3/4·4/4·5/4·6/8·7/8 · 1–16 bars · Click/Wood/Beep/Hi-hat · TAP · volume | ♪ toggle in Settings. Note: the metronome is audible in recordings when it's on |
| Shortcuts | Space = stop all notes · Esc = reset | Work at any time (browser focus) |
Don't have a webcam — or just don't want to use one? Turn on Settings → No camera? Keyboard mode (desktop) — or switch instantly with the ⌨ Keyboard button in the top toolbar — and play the synth with your physical keyboard. No camera permission, no model download, nothing is captured. In keyboard mode the same toolbar button becomes 📷 Camera: one click switches back to hand gestures.
| Key | What it does |
|---|---|
| 1 – 7 | Scale degrees I – VII — hold to play, release to stop (like a keyboard instrument) |
| [ / ] | Minor / Major mode |
| 8 / 9 / 0 / - | Chord style: triad · 1st inversion · 7th · 9th |
| Shift | Octave down (8vb) |
| ↑ / ↓ | Volume sweep (hold) |
| ← / → | Filter sweep (hold) |
| Space | Stop all notes |
The first time you enter keyboard mode, an interactive guide shows a real QWERTY keyboard with a demo animation — press the keys yourself and they light up with what each one plays. You can replay it anytime from the ? help panel. Recording in keyboard mode captures audio (including your voice if you enable the mic) — there's no camera feed to record as video.
The default settings out of the box:
| Setting | Default | Description |
|---|---|---|
| Key | A | All chords transposed to A major scale (can change via toolbar dropdown) |
| Left Hand Mode | Scale Only (Major locked) | Fingers pick scale degree, mode locked to Major |
| Right Hand Mode | Fixed Chord Style (Major Triad) | Chord type locked, right hand controls volume + tone |
| Arpeggiator | Off | Toggle ⟿ to sweep chord notes sequentially |
| Auto Bass | Off | Toggle ∿ for automatic low-end root note |
The default settings are ready to play instantly: Key = A Major, Left Hand = Scale Only (Major locked), Right Hand = Fixed Chord Style (Major triad). Arpeggiator and Auto Bass are off by default — toggle them on for a richer sound. Just raise both hands — left hand picks the scale degree, right hand triggers the chord.
Enable the Arpeggiator toggle to transform block chords into flowing arpeggios. Each note in the chord plays sequentially from low to high (like a harp sweep). Use Slow speed for ambient pads, Normal for melodic accompaniment, and Fast for energetic runs. The arpeggiator works with all chord types and settings.
Toggle Auto Bass to add a low-end foundation that automatically follows your left-hand harmony. The bass plays the root note of your current chord in a lower octave, giving your performance a full band feel. Adjust the bass volume independently — keep it subtle for background warmth or turn it up for a driving low-end. Auto Bass works especially well with the default Gesture mode. See FAQ for more →
Switch your right hand to Fixed Chord Style to lock into a specific chord type. This is perfect for practicing a particular chord quality (try Diminished or Sus4 for unique textures) or when you want to focus purely on expression without worrying about finger count. You can still control volume, tone, and octave with your right hand — only the chord type is locked.
Try all 3 modes: Gesture for full two-hand chords, Theremin for expressive pitch bends, and Piano for single-note melodies. Combine the Arpeggiator with Auto Bass for a rich, layered sound. Use right-hand wrist tilt to sweep the filter — tilt left for darker tones, right for brighter tones. Record your favorite performances — audio, full video, or skeleton animation — to share or revisit later.
Click the red Record button to capture up to 60 seconds of your performance. A 3-2-1 countdown gets you ready, and a magenta 3-2-1 wrap-up warns you before the recording stops. Pick a mode: audio only, full video (camera with neon skeleton), or skeleton animation that never shows your camera feed — and choose an aspect ratio for your sharing platform (9:16 for TikTok/Reels/Shorts, 16:9 for YouTube, 1:1 for Instagram). Singing along? Your voice is captured on the microphone and mixed into the recording, so you can accompany your own singing with live chords. On mobile you can share the result straight from the share sheet; on desktop it downloads as an MP4/M4A or WebM file. Audio downloads carry a branded cover with our name and website, so every file promotes Gesture Synth Weld. Your camera feed is never uploaded to any server. More questions? Check the FAQ →
You may have previously blocked this site's camera access. Allow it per browser:
Then reload the page and click "Enable Camera" again.
The hand-tracking model (~20 MB) downloads on first use — on slow connections this can take several minutes. The progress bar keeps moving; you can cancel and the download continues in the background (the next click starts instantly). If loading fails, check your internet connection and click Retry.
Hand tracking uses your GPU via WebGL. Close other GPU-intensive apps (games, video editors). If you're on a laptop, make sure it's plugged in — some browsers throttle GPU performance on battery. Chrome and Edge generally offer the best MediaPipe performance.
The audio engine requires a user gesture to start (browser autoplay policy). Make sure you clicked "Enable Camera" before expecting sound. Also check that your browser isn't muted and your system volume is up. Also make sure both hands are visible to the camera — sound only plays when both hands are detected with at least one finger raised.
| Error | Meaning | Fix |
|---|---|---|
| NotAllowedError | Camera permission was denied | Allow camera access (see "Camera access was denied" above) |
| NotFoundError | No camera was found on this device | Connect a webcam / close apps using it (see "Camera not detected") |
| NotReadableError | The camera is busy or hardware-locked | Close other apps using the camera, then Retry |
| NotSupportedError / security | Camera requires a secure context (HTTPS) | Use the site over HTTPS (gesturesynthweld.com) |
A gesture synth (gesture synthesizer) is a tool that turns hand movements captured by your webcam into musical sounds — no physical instrument needed. Gesture Synth Weld is one such browser-based gesture synth that maps each finger to a different note, letting you play music just by moving your hands in the air. Learn what makes it different →
No. All you need is a computer or laptop with a built-in or USB webcam, and a modern browser like Chrome, Edge, or Firefox. No MIDI controllers, no external sensors, no downloads. Your webcam handles all the input — MediaPipe's hand tracking model runs entirely on your device.
Gesture Synth Weld works in any modern browser that supports the Web Audio API and getUserMedia — including Chrome 90+, Firefox 90+, Edge 90+, and Safari 15+. For the best hand tracking experience, Chrome or Edge is recommended since they support MediaPipe's WebGL2 backend most efficiently.
Yes! Click the Record button to start capturing your performance (up to 60 seconds). A 3-2-1 countdown gets you ready, and a 3-2-1 wrap-up warns you before the recording stops. Choose what to capture: audio only, full video (camera with neon skeleton), or skeleton animation (no camera feed — privacy-friendly). Pick an aspect ratio for your sharing platform: 9:16 for TikTok/Reels/Shorts, 16:9 for YouTube, 1:1 for Instagram. Sing along? Your microphone is mixed into the recording (toggleable) — perfect for singing over your own chords. Pick your microphone in the chooser if needed (some Macs default to a virtual device that records silence), and balance voice vs chords with the recording-mix slider — headphones give the cleanest mix. After recording, download the file — or on mobile, share it straight to your apps from the share sheet (the share message carries our brand and gesturesynthweld.com). Your camera feed is never uploaded to any server. More recording tips →
Gesture Synth Weld supports all 12 keys (C through B) with two right-hand modes. In Finger Layout mode: 1 finger = triad, 2 fingers = 1st inversion, 3 fingers = 7th chord, 4 fingers = 9th chord. In Fixed Chord mode, choose from 8 chord types: Major Triad, Major 1st Inversion, Minor Triad, Diminished, Sus2, Sus4, Major 7th, and Dominant 7th. Lock to a fixed major or minor mode, or enable Scale+Tilt in settings to switch between major and minor with left-hand wrist tilt. See the full gesture table →
Yes — Keyboard Mode turns your physical keyboard into the instrument (desktop only, no camera permission and no model download needed). Enable it in Settings → "No camera? Keyboard mode" or with the ⌨ Keyboard button in the top toolbar (in keyboard mode it becomes 📷 Camera — one click switches back to hand gestures), then press 1-7 for scale degrees I-VII (hold to play, release to stop), [ / ] for minor/major, 8/9/0/- for chord styles, Shift for octave down, ↑/↓ for volume and ←/→ for the filter sweep, and Space to stop. A real-keyboard guide with a demo animation shows you every key the first time — replayable anytime from the ? help panel. Recording in keyboard mode captures audio (add your voice with the mic) — video modes need a camera feed, so they're hidden there.
Gesture Synth Weld features a unique two-hand division system: your left hand controls harmony (key selection, scale degree, major/minor toggle) while your right hand controls expression (chord complexity, volume, tone, octave). Unlike basic gesture synths, it includes an arpeggiator for harp-like sweeps, an auto bass generator for low-end foundation, a sawtooth synth engine with real-time filter sweep, support for 9th chords (4-finger), 8 fixed chord styles, and built-in MP4/M4A recording. It's also open source under the MIT license.
Yes! Gesture Synth Weld is free to use. We support the development and hosting of this tool through non-intrusive advertising. No registration, no sign-up, no paywall — just open and play. The source code is also available on GitHub under the MIT license for anyone who wants to study, modify, or contribute.