Vertical animated cartoon short made from scratch in code

GOAL: "task": "Make a vertical animated cartoon short of MY story, built in code: three.js r160 world + canvas-drawn 2D character cards, rendered frame by frame in headless Chromium (Playwright, swiftshader), encoded with ffmpeg. MOTION: 1080x1920, 30fps, exactly as long as my voiceover. Work start to finish; only ask if script and audio don't match.", "my_inputs": { "story": "[your idea, true event, legend or full script]", "script": "[optional exact script]", "voiceover": "[optional: attach mp3/wav]", "length_seconds": 30, "voice_speed": 1.0, "captions": false, "notes": "[optional: era, setting, characters, jokes]" }, "stages": [ "A. No script: write one at ~3.7 words/s (30s~110 words): 1) hook = absurd fact, no setup 2) setting in 1-2 lines with a concrete detail 3) 2-4 escalating beats, others doubt/mock, one exaggeration joke 4) pivot 'But...'/'You see,' revealing the real mechanism with an everyday analogy 5) short ironic twist = last words. Spoken English, contractions, numbers as words, never describe what's on screen. If presented as true, web-check facts and say history/disputed/legend.", "B. No voiceover: give script in a copyable block + word count + est. length, a short shot plan, recording tips (one warm narrator; ElevenLabs speed 1.05-1.1, stability 45-55%; pauses at sentence ends; no music). Stop and ask for the upload.", "C. With voiceover: build everything below without stopping. Task list for progress; send a stills contact sheet before the long render." ], "setup": [ "mkdir -p web/lib work stills frames out", "npm pack three@0.160.0 -> copy package/build/three.module.js to web/lib (fallback: sparse git clone of mrdoob/three.js branch r160, folder build)", "Playwright + Chromium (npm i -g playwright && npx playwright install chromium); launch args --use-angle=swiftshader --enable-unsafe-swiftshader --ignore-gpu-blocklist", "python3 numpy Pillow, ffmpeg (with rubberband)", "Serve web/ with a tiny Node http server; render.mjs loads index.html, waits for window.ready, calls window.renderFrame(f), saves canvas.toDataURL('image/jpeg',.95) to frames/f0000.jpg. Support RESUME=1 (skip existing) and a frame list 'a-b' or '1,5,9'." ], "timing": "If voice_speed!=1: ffmpeg -af rubberband=tempo=<speed> (pitch kept). Claude can't hear audio: align my script to speech using ffmpeg silencedetect (-35dB, 0.12s): split script into phrases at punctuation, DP-match phrases to speech segments by syllable count vs duration, penalise sentence ends inside a segment. Spread words by syllables. Write work/phrases.json; check pauses fall at sentence ends. Frames = round(duration*30). Every reaction lands 0-3 frames before its word; the final payoff lands on the final word.", "architecture": { "files": "web/index.html (canvas#c 1080x1920 + module main.js), main.js (renderer, materials, world, cards, sprites, passes), story.js (SHOTS, LIGHTS, lightFor, camAt, dofAt, worldAt, actorsAt, fxAt: all pure functions of frame f), motion.js, paint.js (canvas textures), toon.js (character + creature drawing), https://t.co/MwT4jjcu22, https://t.co/3V5W7hv6gk, https://t.co/XLDraRXh1Z, https://t.co/vZbd2PyUGV (contact sheets).", "renderer": "WebGLRenderer antialias:false, preserveDrawingBuffer, PCFSoft shadows 2048, sun = DirectionalLight from az/el per light preset, shadow camera centred on each shot's action (ext 8-60m), HemisphereLight, Fog(colour = sky horizon). Camera Perspective, near .08 far 1500.", "materials": "MeshToonMaterial with a 2-texel gradientMap [90,255] Nearest = hard 2-step toon shading. Textures are CanvasTextures painted in code: flat base + soft blotches + dark seams/strokes, repeat-wrapped, colorSpace sRGB. Boxes use world-scaled UVs (constant texel density).", "passes": [ "1 colour: render scene to HalfFloat RT (samples 4).", "2 normal+depth: swap every mesh material to a ShaderMaterial writing vec4(normal*.5+.5, viewDepth) (handle USE_INSTANCING: pos=instanceMatrix*pos). Cards/sprites write rgb 0 + depth with alpha-test (so they block but are never outlined). Hide sky, clouds, blob shadows. Fog and shadows off for this pass.", "3 ink+grade fullscreen: 16 taps on a ring of radius 2.5px*clamp(7/depth,.62,1.25); edge = smoothstep(.006,.02,(dS-d)/d) on the NEAR side of depth jumps only; crease = smoothstep(.88,.72,dot(n,nS))*.9 when depths are within 6%; skip card samples; sky counts as depth 1e4; fade ink 140-520m. colour = mix(c, #030203, e); saturation 1.08-1.15; gain/lift tint; vignette .2-.34; white flash uniform.", "4 lens: per-pixel CoC = cinf*(1-focus/d) (near side capped 1.4x); 48-tap golden-angle disc, each tap weighted by its own CoC so sharp foreground never bleeds; skip when cinf<.5. Output linearToOutputTexel." ], "sky": "inverted sphere following camera: mix(horizon, top, pow(y,.55)) + optional sun disc. Painted cloud billboards far away (650-900m), fog off, facing camera." }, "characters": { "card": "Each character is a 1200x2000 CanvasTexture on a 1.35x2.25m plane with origin at the soles (design space 600x1000, soles at y=975, 1 design unit = 2.25mm). MeshBasicMaterial alphaTest .02, DoubleSide; shadow via customDepthMaterial with the same map (alphaTest .5); soft dark radial blob under the feet. Billboard on yaw only: rotY = atan2(cam.x-x, cam.z-z). Redraw a card only when its key (JSON of q()-rounded pose values) changes.", "proportions": "Bean body: head blob ~236 wide, legs 1.38xHH, torso 1.08xHH (stout 1.12), eyes ~1.47m high. Head = superellipse blob with slight egg + per-seed asymmetry. Torso = egg blob with a tunic skirt below. Legs/arms = thin curved strokes (5.8u/5.2u, u=headWidth/100) in a darker shade of the outfit; round dot hands; small boots.", "ink": "Outline colour #241726 (plum), width 5u, thicker on the side away from the light (light from upper-left, -120deg). Fills: base colour with a soft shadow gradient toward lower-right. Skin default #ffe2c2, also #f5c9a0 #ecd0b6 #c99067.", "faces": "Eyes 13u apart either side of centre. Kinds: dot, lid (half-closed), happy (arc), squeeze, shock (big white + small pupil + raised arc brows), slit, dash. Mouths: dash, frown, smirk, smile, grin (open with teeth line), laugh, o, O, wave (nervous), teeth. Brows are short tapered strokes; angle sets mood. Expressions: neutral, stern, grin, laugh, nervous (+sweat drop), shock, smile, pained, squint, smug/side-eye (half lids, pupils to one side). Blinks every 1.3-2.3s for 4 frames.", "outfits": "Built from parts: tunic colour, belt, buttons, pockets, sash, hat (slouch hat, cap, top hat, bonnet, helmet, crown, hair), beard/moustache, apron, dress, props in hands. Design new outfits for each story in the same flat style.", "arm_poses": "hang, point (+finger), up, wave, hip, belly, cheek (both = shock), salute (flat hand at brim, drawn over the head), thumb (thumbs-up), shrug, reach, lift, cross, prop-holding (pen, stamp, wrench, spear, sign). Whole upper body tilts about the hip; head tilts separately.", "creatures": "Animals are side-view canvas sprites (600x800 design, feet at y=780) on shared textures cached by pose key, so flocks of 50+ cost little: shaggy shaded body blob with fringe strokes, thick inked limbs, small head, big googly white eye with pupil that can look/side-eye, simple beak/snout. Legs via 2-bone IK with joint bent backward; run cycle 16 phases (foot x = hip + stride*cos, lift = max(0,sin)); poses: stand, run, peck/graze, smug. Flip the sprite when travel direction dot camera-right < 0." }, "world": { "build": "Primitives only: boxes, cylinders, cones, squashed spheres, extrusions; group set-pieces with pivots driven by 0..1 progress from worldAt(f) with eased, staggered parts. Instanced meshes for crops/grass/rocks; clear a patch where the action happens.", "background": "low half-buried hill spheres, distant mesas or mountains (cone + snow cap), trees (trunk cylinder + 4-6 squashed sphere canopies), far houses, fog to the horizon colour.", "secondary_motion": "windmills turn, flags/cloth wave (vertex sine), smoke/dust/leaves/Zzz/stars as cached 2D fx sprites, water shader scroll, wheels roll with distance/radius, vehicles bounce.", "text": "Diegetic text (signs, crates, letters, stamps) painted on canvas with real fonts; place on +z faces or planes so it never reads mirrored. Swap textures to animate (e.g. stamp appears on impact)." }, "lights": { "day": "sun az64 el50 #fff3df 2.45, hemi sky #d2e2f6 ground #a8905e 1.1, sky top #2d74d4 horizon #cfe3f5, fog 120-1300, sat 1.12, vig .2", "golden": "sun az30 el11 #ffb46a 2.9, hemi #a99bcc/#7a5040 .98, sky #6474c0 -> #ffc485 + sun disc, gain #fff0e0, sat 1.15, vig .26, tint cards #ffe2c4", "dusk": "sun el5 #ff9a72 .9, hemi #5d6cab/#3a3048 .8, sky #18234f -> #d18a7c, gain #e2e8ff, lift #04050b, vig .34, a warm point-light lamp, tint cards #c9cff2", "use": "day = normal story, golden = decline or time passing, dusk-blue = explanation or ending. Time jump = 6-frame white flash." }, "motion": { "helpers": "smooth(t)=t^3(t(6t-15)+10); easeOut=1-(1-t)^3; back(t)=overshoot s=1.7; spring settle; Catmull-Rom spline through via-points; idle(f,seed)={bob .006 sin(.11f), tilt 1.2deg sin(.05f), headTilt 1.5deg sin(.037f)}; blinkAt(f,seed); q(x)=round for card keys.", "camera": "Keys {f,pos,look,fov,roll,ease,path} inside cut-free segments; glides 18-45 frames; easeOut for whips/reveals; every hold creeps (fov *= 1-.00012*(f-f0)); 3-5deg roll on drama; impact kick = 7-frame alternating look offset + roll that decays. Vehicles: track alongside, match speed, slight judder.", "continuity": "prefer one continuous move: MCU pull-back into a wide, crane up and over a fence into the establishing shot, lateral truck along a lineup, crane down into an insert, rack focus to the background gag. Hard cut only on place/time jumps.", "acting": "anticipation -> back overshoot -> spring settle; pop-ins scale 0->1 with back easing + dust puff; stamp = 2-frame dip + puff + kick; laughs = alternating tilt + bob; head bobs on stressed words; nothing static for 4+ frames.", "walks_runs": "feet planted, bob 3-7%; run phase = speed/stride; fast movers must stay in frame until their beat is done." }, "shot_grammar": { "plan_first": "work/PLAN.md: one row per shot (frames, words, setting, light, camera, cast/poses/expressions, fx, sfx). Usually one shot per phrase; merge tiny ones. Illustrate every sentence literally; props carry the jokes.", "order": "frame 0 = hook MCU of the hero mid-action -> pull-back reveal on the punch of sentence 1 -> wide establishing crane -> lineup/two-shot with pop-ins on each listed item -> reaction close-up (a creature or extra slides in with a side-eye on the question line) -> action wide (chase, scatter, collapse) -> time-jump flash -> insert on the key object -> twist: music stops, low sting, push-in with roll, payoff on the last word.", "framing": "Bean cards 2.25m tall: MCU = camera ~3.5m away at fov 32-34 looking at 1.35m; wides fov 40-52. Vertical frame half-width = d*tan(hfov/2), hfov = 2*atan(tan(vfov/2)*.5625): at 4m and fov 40 only +/-0.8m, so stage people in depth.", "lens": "DOF on close-ups/mediums (cInf 7-14, focus on the eyes), wides sharp, racks ~14 frames." }, "audio": { "stems": "https://t.co/MwT4jjcu22 writes only music.wav + sfx.wav (48k mono, numpy synth). Music: short loop in the story's mood (march: kick/snare, plucked bass, brass pad, little fanfare; or plucky comedy; or eerie pad); stops on the twist line, then a low drone. SFX in frames/30: whoosh (low-passed noise with sine^2 envelope) on big moves, thunk (pitched sine drop + noise) on pops/stamps, rustle/leaves, feet patter, engine (pulse train low-passed), scrape, ting (2-4 kHz bells), sting (41 Hz + noise hit + brass stab) on the twist.", "mix": "https://t.co/3V5W7hv6gk: gated RMS of the voice (0.4s blocks above -40dB of max); scale music so its gated RMS = 0.5x voice (-6 dB); SFX = min(0.5x voice RMS, peaks 0.7x voice peak); sum; one linear gain only. No ducking, no compression. Print the measured levels and confirm.", "finish": "measure integrated loudness (ffmpeg ebur128), apply one volume gain to -14 LUFS + alimiter -1 dBFS; encode: ffmpeg -framerate 30 -i frames/f%04d.jpg -i mix.wav -c:v libx264 -preset slow -crf 17 -pix_fmt yuv420p -c:a aac -b:a 192k -shortest -movflags +faststart. QA: freezedetect (n=.0005, d=.4) and blackdetect must find nothing; make a 2fps tiled QA sheet and look at it." }, "look_check": "Before the full render, render stills at every shot's middle plus key actions, tile the