The Scale Hand

A 3D hand that plays the drill's fingering, next to the numbers the drill shows today · round 1 · 2026-09-28 · round 2, how the hand moves · 2026-09-29 · no spend

Why this lab

The question on 2026-09-28 was where three.js would actually earn its place in the game. The answer was hands that move with the music: a small 3D object whose pose comes from live data and has to answer the player. Everything that is flat and fast is better as Canvas 2D, everything with a fixed set of views is better rendered once in Blender, and the 3D worlds belong to Babylon or Unreal.

The first place for it is the Scales & Arpeggios drill. With ☝ fingerings on, the drill already shows the reviewed fingering from app.scale_fingerings as numbers: a digit over the next key and one under each pip. A learning plan's split card turns them on because the card "is ABOUT a thumb crossing". A number can say 1; it cannot show a thumb travelling under the hand to get there.

What is on the bench

  1. The same run, three ways, flipped with keys 1 to 3 at the same moment: A the numbers as the drill draws them now, B the hand alone, C the hand with the numbers.
  2. The hand plays the drill's own fingering, read from the table the game ships (C major right hand: 1 2 3 · 1 2 3 4 5 up, 5 4 3 2 1 · 3 2 1 down). Nothing is typed in for the lab, and all twelve major scales are on the picker.
  3. 🎹 I play holds the clock at every note until your key goes down, with the finger waiting just above it. A video could not wait for you; this is why the hand is drawn live.
  4. Four cameras, and the thumb seen through the hand in blue while the palm hides it. Beside you, where a teacher would stand, shows the thumb going under best.

Played with a MIDI keyboard, or the pointer on the keys. Nothing here reaches the network: no analytics, no database. A test (tools/scale_hand_test.mjs) plays all twelve majors, one and two octaves, through the real hand: no bone stretches, a pressed fingertip stays within a millimetre of its key, fingers 2 to 5 never pass each other, and every crossing goes under or over.

Loading the 3D hand…

The three

One clock, one run, one fingering. They differ only in what the player is given to read.

A The numbers what the drill does today

The ☝ toggle's digits, drawn as drawKeys and drawLane draw them: the next key's finger in a blue badge above it, and every note's finger under its pip, so the whole run reads as a map.

  • The whole pattern at a glance: 1 2 3 · 1 2 3 4 5 is easy to learn as a shape.
  • Costs nothing: no download, no WebGL, every device.
  • Transfers straight to printed fingering in a score.
  • Says which finger, never how it gets there. The crossing is two numbers side by side, 3 then 1, and the player has to know the thumb goes under.
  • On the 88-key strip the badge is one key wide.

B The hand no numbers at all

The rigged hand plays the run: each finger goes down on its key, the waiting fingers hover beside the one that plays, the thumb travels under the palm to its next key, and on the way down the third finger arches over the thumb.

  • Shows the crossing itself: when the thumb starts to move, where it travels, what the palm does. The thumb shows through the hand in blue while the palm hides it.
  • Waits for the player, at any tempo.
  • All twelve majors, either direction, one or two octaves, straight from the table: nothing is baked.
  • Which finger is which is left to the eye; beginners count fingers.
  • The pattern is gone: you see one note at a time.
  • About 300 KB to download (three.js and the hand), and WebGL.

C The hand and the numbers recommended

B, with the drill's numbers put where they belong: the next key's digit floats over it in the same blue badge, the same digit sits on the finger that will play it, and the drill's pips run along the bottom with every finger under them.

  • The number says which finger, the hand says how. The badge over F and the thumb both read 1, so the mapping needs no teaching.
  • Keeps A's map of the whole run.
  • The busiest picture of the three.
  • Same download and WebGL as B.

Round 2 · how the hand moves (2026-09-29)

Two ideas taken from V9011N's Hand-thesia, a pygame animator of hands over falling notes (no license, so the ideas were rebuilt here, not its code). Both are switches in the panel, off unless you turn them on; the game's 🖐 Show me and the Hand Reader are unchanged until one is picked.

Turn the hand at each crossing

While the thumb passes under, the hand turns about 14° toward the little finger, so the fingers point the way the run is going; while a finger goes over the thumb, toward the thumb. It builds up as the crossing finger travels, holds through the landing and lets go as the hand settles. Today the palm leaned toward the thumb until the thumb landed, then swung the other way in half a beat.

Relaxed fingers

Waiting fingers lie extended further into the keys instead of folding over their marks, a finger's tip rests further along its length from the knuckle, and in the Hand Reader's grips the long fingers play along the arc of their lengths (the little finger nearest the front). Hand-thesia flattens its fingers by lowering the hand; here that only arched them up from the knuckle, so the hand sits a touch higher instead. Try a major arpeggio: its waiting fingers were the claw.

Round 3 · the wrist straight (the same day)

Lionel on round 2: "the 'relaxed' versions are infinitely better. in fact the only one that looks bad is the G7 chord. one thing that needs to be paid more attention to is the wrist position. it should never be skewed or tilted to one side, but always remain in the middle neutral position". So the relaxed hand now keeps its palm within 3.4° of straight, in line with the forearm, and reaches every key by moving rather than turning; in the grips its index and middle fingers reach deeper so they curve instead of hooking (G7), and its long fingers point straight in rather than all leaning one way. The same rule rules out the turn at each crossing: its switch stays for comparison, and with relaxed fingers on it does nothing.

Recommendation

C, and only where a crossing is the lesson: a learning plan's split card, and a 🖐 Show me button beside ☝ on the drill. The drill's race stays numbers-only; nobody races a hand demonstration.

What the round leaves open

  1. Where it lives in the drill. The drill's canvas is 980 × 430 and full; the hand wants its own panel, or the round's start card, or the review of an earlier round.
  2. The left hand. The motion is written for either hand (the side is one sign), but only the right hand has been looked at.
  3. Minor scales, the modes, arpeggios. The table fingers them all and the motion takes any run, but only the majors have been looked at and tested. In an arpeggio the thumb goes under a wider gap.
  4. The thumb seen through the hand. On by default here; it could be the teaching moment itself, or too much.
  5. Which camera ships. At the piano is the player's own view; Beside you shows the thumb best.
  6. A slower crossing. On by default here: the clock slows to under half speed while a thumb or finger crosses.