Three.js + Theater.js WebGL Animation Production Concise Tutorial

In this tutorial, we'll cover the basics of Theater.js and discover how to create stunning animation sequences. We'll demonstrate how to animate a Three.js cube, integrate eye-catching visual effects, modify colors, experiment with HTML elements, and synchronize animations with sound playback at specific intervals.

Recommendation: Use NSDT editor to quickly build programmable 3D scenes

1. Theater.js installation and setup

First, we need a starter template with Three.js and a basic scene. Theater.js has two base packages:

• @theatre/studiois the editor GUI we use to create animations
• @theatre/corePlay the animation we created.

We can add the Theater.js package like this:

# with npm
npm install --save @theatre/core @theatre/studio

# with yarn
yarn add @theatre/core @theatre/studio

Or download the starter template, which includes all required dependencies, and run the following command:

# Install the dependencies 
yarn install

# Start the server
yarn run dev.

2. Create the cube and the floor

The starter template gives us a simple cube, floor, some lighting and track controls.

// Cube
  const geometry = new THREE.BoxGeometry(10, 10, 10);
  const material = new THREE.MeshPhongMaterial({ color: 0x049ef4 });
  const box = new THREE.Mesh(geometry, material);
  box.castShadow = true;
  box.receiveShadow = true;
  scene.add(box);

// Floor
  const floorGeometry = new THREE.CylinderGeometry(30, 30, 300, 30);
  const floorMaterial = new THREE.MeshPhongMaterial({ color: 0xf0f0f0 });
  const floor = new THREE.Mesh(floorGeometry, floorMaterial);
  floor.position.set(0, -150, 0);
  floor.receiveShadow = true;
  scene.add(floor);

// Lights
  const ambLight = new THREE.AmbientLight(0xfefefe, 0.2);
  const dirLight = new THREE.DirectionalLight(0xfefefe, 1);
  dirLight.castShadow = true;
  dirLight.shadow.mapSize.width = 1024;
  dirLight.shadow.mapSize.height = 1024;
  dirLight.shadow.camera.far = 100;
  dirLight.shadow.camera.near = 1;
  dirLight.shadow.camera.top = 40;
  dirLight.shadow.camera.right = 40;
  dirLight.shadow.camera.bottom = -40;
  dirLight.shadow.camera.left = -40;
  dirLight.position.set(20, 30, 20);
  scene.add(ambLight, dirLight);

// OrbitControls
  controls = new OrbitControls(camera, renderer.domElement);
  controls.enableZoom = true;
  controls.enableDamping = true;
  controls.autoRotate = false;
  controls.dampingFactor = 0.1;
  controls.minDistance = 2.4;
  controls.target.set(0, 20, 0);

3. Import Theater and create a project

We need to import { getProject, types } from @theatre/core. Once done, we also need to import and initialize studio from @theatre/studio.

A project in Theater.js is like a saved file. Items are stored in localStorage, so you won't lose your progress if you close and reopen your browser. Create a new theater project and give it a name.
Then we create a new worksheet. The worksheet contains all objects that can be animated.

import { getProject, types } from '@theatre/core';
import studio from '@theatre/studio';
studio.initialize();

// Create a project for the animation
const project = getProject('TheatreTutorial_1');

// Create a sheet
const sheet = project.sheet('AnimationScene');

4. Objects and properties

Every object that needs to be animated has a corresponding Theater Sheet object. These sheet objects contain properties (Props) that can be animated to create motion and other dynamic effects in the scene.

Let's create a new boxObject and name it Box.

const boxObj = sheet.object('Box', {});

Properties correspond to specific characteristics of objects that can be animated, and can be of different types, which can be imported using import {types} from '@theatre/core'.

We're going to add some props. Let's start with rotation, create a property of composite type and add xR, yR and zR of numeric type, value: 0, range: [-Math.PI, Math.PI].

Again, let's add position and scale properties. Adding a nudgeMultiplier to it gives us finer control.

const boxObj = sheet.object('Box', {
    rotation: types.compound({
      xR: types.number(0, { range: [-Math.PI, Math.PI] }),
      yR: types.number(0, { range: [-Math.PI, Math.PI] }),
      zR: types.number(0, { range: [-Math.PI, Math.PI] }),
    }),
    position: types.compound({
      x: types.number(0, { nudgeMultiplier: 0.1 }),
      y: types.number(0, { nudgeMultiplier: 0.1 }),
      z: types.number(0, { nudgeMultiplier: 0.1 }),
    }),
    scale: types.compound({
      xS: types.number(1, { nudgeMultiplier: 0.1 }),
      yS: types.number(1, { nudgeMultiplier: 0.1 }),
      zS: types.number(1, { nudgeMultiplier: 0.1 }),
    }),
});

Now we can see that there is a new Box object under the worksheet.

5. Animate the cube

Time to animate our cube. We need a way to rotate the cube mesh based on the value of the boxObj property. This can be done by listening for boxObj changes using the onValuesChange() hook.

boxObj.onValuesChange((values) => {
    const { xR, yR, zR } = values.rotation;
    box.rotation.set(xR, yR, zR);
    const { x, y, z } = values.position;
    box.position.set(x, y, z);
    const { xS, yS, zS } = values.scale;
    box.scale.set(xS, yS, zS);
});

Moving the slider now affects our cube in real time.

6. Add keyframes

Let's add some keyframes. You can right-click any attribute and click Sequence or Sequence All.

This will open the Sequence Editor with properties. We can resize the sequence timeline, zoom in or out, and use the blue pointer to scrub the sequence.

Drag to move the pointer, then click the yellow button to add a keyframe.

Let's resize the sequence timeline to a little over 2 seconds. Then add keyframes to animate the cube's y position. Again, let's sort the scale and add keyframes to it. You can play around with these values ​​or experiment with them yourself until you feel comfortable.

Then press the spacebar to play the sequence:

7. Diagram Editor

Clicking the button next to each property in the sequence editor will open the Graph Editor or the multi-track curve editor. This comes in handy when we want to refine an animation by manually editing the velocity curve of one or more tracks.

Click the links between keyframes to display a list of default easing curves available.

8. Modify the color

Let's move on and see how to modify colors using Theater.js. Let's create a new object and name it Colors. The background color is types.rgba(). Likewise, we also create properties for the floor color and box color:

const colorObj = sheet.object('Colors',{
    backgroundColor: types.rgba(),
    floorColor: types.rgba(),
    boxColor: types.rgba(),
})

In the onValuesChange() hook, we can set scene.background or the background color of the underlying HTML element. Using setRGB(), we set the color of the floor and box textures. Click and drag the color picker to change the color.

colorObj.onValuesChange((values)=>{
    // scene.background = new THREE.Color(values.backgroundColor.toString());
    // @ts-ignore
    document.body.style.backgroundColor = values.backgroundColor;
    floorMaterial.color.setRGB(values.floorColor.r,values.floorColor.g,values.floorColor.b)
    boxMaterial.color.setRGB(values.boxColor.r,values.boxColor.g,values.boxColor.b)
})

It would be nice if the cube would glow when stretched. Let's create a new Theater object: boxEffects.

const boxEffectsObj = sheet.object('Effects',{
    boxGlow:types.rgba(),
})
boxEffectsObj.onValuesChange((values)=>{
    boxMaterial.emissive.setRGB(values.boxGlow.r,values.boxGlow.g,values.boxGlow.b);
})

Let's sequence it and add two emissive keyframes with #000000 on the first few frames and choose a color for the compressed state. Then it goes back to normal on the last frame.

9. Speed ​​line effect

To add Toon Speed ​​Lines vFx, let's create three cubes and scale them to look like lines, then add them to the scene as a group.

// Swoosh Effect Objects
const swooshMaterial = new THREE.MeshBasicMaterial({color:0x222222,transparent:true,opacity:1});
const swooshEffect = new THREE.Group();

const swooshBig = new THREE.Mesh(geometry, swooshMaterial );
swooshBig.scale.set(0.02,2,0.02)
swooshBig.position.set(1,0,-2)

const swooshSmall1 = new THREE.Mesh(geometry, swooshMaterial );
swooshSmall1.scale.set(0.02,1,0.02)
swooshSmall1.position.set(1,0,3)

const swooshSmall2 = new THREE.Mesh(geometry, swooshMaterial );
swooshSmall2.scale.set(0.02,1.4,0.02)
swooshSmall2.position.set(-3,0,0)

swooshEffect.add( swooshBig, swooshSmall1, swooshSmall2 );
swooshEffect.position.set(0,20,0)
scene.add(swooshEffect)

Let's add some more props to the boxEffect object to adjust the scale, position and opacity of the line. Experiment with that keyframe to get the desired effect.

const boxEffectsObj = sheet.object('Effects',{
    boxGlow:types.rgba(),
    swooshScale:types.number(1,{nudgeMultiplier:0.01}),
    swooshPosition:types.number(0,{nudgeMultiplier:0.01}),
    swooshOpacity:types.number(1,{nudgeMultiplier:0.01})
})
boxEffectsObj.onValuesChange((values)=>{
    boxMaterial.emissive.setRGB(values.boxGlow.r,values.boxGlow.g,values.boxGlow.b);
    swooshEffect.scale.setY(values.swooshScale);
    swooshEffect.position.setY(values.swooshPosition);
    swooshMaterial.opacity=values.swooshScale;
})

10. Comic text effect

Time for some anime text effects: "Boink!"

import {CSS2DRenderer, CSS2DObject} from THREE and create a textRenderer. Let's set style.position to absolute and update the OrbitControls' domElement.

Create a new CSS2D object, add it to the scene, and add an HTML element representing the object. Adds text to the box so that it follows the box's position on the screen.

  <div id="boink">Boink!!</div>

import {CSS2DRenderer,CSS2DObject} from 'three/examples/jsm/renderers/CSS2DRenderer'
let textRenderer = new CSS2DRenderer();
textRenderer.setSize(window.innerWidth,window.innerHeight);
textRenderer.domElement.style.position = 'absolute';
textRenderer.domElement.style.top = "0";
textRenderer.domElement.style.left = "0";
textRenderer.domElement.style.width = "100%";
textRenderer.domElement.style.height = "100%";
textRenderer.domElement.style.zIndex = "2";
document.body.appendChild(textRenderer.domElement)

// OrbitControls
controls = new OrbitControls(camera, textRenderer.domElement);

// Text Effects
const boinkDom = document.getElementById('boink');
const boinkText = new CSS2DObject(boinkDom);
boinkText.position.set(-25,0,0)
box.add(boinkText);

// add this to your render()/tick() function
// textRenderer.render(scene, camera);

Create a new theater.js object: textEffectObj, which contains properties for opacity, text, and scale.

Using onValuesChange(), update the innerText of the HTML element. One interesting thing about Theater.js: it can also be used to modify and animate text. Sort all the properties and add keyframes to make the text pop when the box pops up.

const textEffectObj = sheet.object('text',{
    opacity:1,
    text:"",
    scale: 1
});

textEffectObj.onValuesChange((values)=>{
    if(!boinkDom)return;
    boinkDom.innerText = values.text;
    boinkDom.style.opacity = ""+values.opacity
    boinkDom.style.fontSize = ""+values.scale+"px";
})

11. Mouse sound effect

Finally, to bring everything to life, let's add sound effects. I searched for some free sounds on Pixabay and imported them into the project. Then I load them using Three.js AudioLoader . Here's how I add sounds to my Three.js project:

// importing my sounds as urls
import swooshSound from '../assets/sounds/whoosh.mp3';
import boinkSound from '../assets/sounds/boink.mp3';
import thudSound from '../assets/sounds/loud-thud-45719.mp3';

const listener = new THREE.AudioListener();
const loader = new THREE.AudioLoader(loadingMgr);
let soundReady = false;
const swoosh = new THREE.Audio(listener)
const boink = new THREE.Audio(listener)
const thud = new THREE.Audio(listener)

setupSounds();

function setupSounds() {
  camera.add(listener);

  audioSetup(swoosh,swooshSound,0.3,loader)
  audioSetup(boink,boinkSound,0.2,loader)
  audioSetup(thud,thudSound,0.5,loader)
}

function audioSetup(sound:THREE.Audio, url:string, volume:number, loader:THREE.AudioLoader){
  loader.load(
    url,
    // onLoad callback
    function ( audioBuffer ) {
      sound.setBuffer( audioBuffer );
      sound.setVolume(volume)
      sound.loop=false;
    },
  );
}

Once this is set up, we can continue playing the sound based on the pointer position in the sequence. We can achieve this by utilizing the onChange() hook and monitoring the pointer position changes to trigger the sound to play at certain intervals.

// play the audio based on pointer position
onChange(sheet.sequence.pointer.position, (position) => {
    if(!soundReady)return;
    if(position > 0.79 && position < 0.83){
        if(!thud.isPlaying){
            thud.play();
        }
    }
    else if(position > 1.18 && position < 1.23){
        if(!boink.isPlaying){
            boink.play();
        }
    }
    else if(position > 0.00 && position<0.04){
        if(!swoosh.isPlaying){
            swoosh.playbackRate= 1.7;
            swoosh.play();
        }
    }
})

To add a new event listener for click, set soundReady to true and use sheet.sequence.play() to play the animation with an iteration count of Infinity in the range 0-2.

<style>
.enterSceneContainer{
  z-index: 4;
  position: absolute;
  display: block;
  width: 100%;
  height: 100%;
  text-align: center;
  transition: all 0.5s ease;
}
</style>
<div class="enterSceneContainer" id="tapStart">
    <p>Tap to start</p>
</div>
// Play sequence on click once all assets are loaded
const tapStart = document.getElementById('tapStart');

tapStart.addEventListener(
    'click',
    function () {
        soundReady = true;
        tapStart.style.opacity = "0";
        setTimeout(()=>{
          tapStart.style.display = "none";
        },400)
        sheet.sequence.play({ iterationCount: Infinity, range: [0, 2] });
    }
);

12. Tone mapping and encoders

To enhance the color of the scene, you can specify different toneMappings and outputEncodings for the renderer.

After trying various options, I chose to set them as LinearToneMapping and sRGBEncoding for this particular project.

renderer.outputEncoding = THREE.sRGBEncoding;
renderer.toneMapping = THREE.LinearToneMapping;

To add fog and synchronize it with the scene background, you can include the relevant code in the colorObj.onValuesChange() hook.

scene.fog = new THREE.FogExp2(0xffffff, 0.009);
colorObj.onValuesChange((values)=>{
    // @ts-ignore
    scene.fog.color = new THREE.Color(values.backgroundColor.toString());
    // ... rest of the code here ...
})

13. Deploy to production environment

To complete the project, we need to export the animation and deploy it to production. Just click on the project name in the Studio UI and select "Export to JSON" and save the state.

Import the saved state into main.js and pass the saved state in getProject.

import projectState from '../assets/Saved_TheatreState.json';

project = getProject('TheatreTutorial_1', { state: projectState });

After exporting the animation, you can remove the studio import and studio.initialize(), as they are not needed for production. Alternatively, you can conditionally remove or include them as needed.

let project;
// Using Vite
if (import.meta.env.DEV) {
    studio.initialize();
    // Create a project from local state
    project = getProject('TheatreTutorial_1');
}
else {
    // Create a project from saved state
    project = getProject('TheatreTutorial_1', { state: projectState });
}

Don't forget to review the final code . Alternatively, you can follow this link to watch a video tutorial.

---

Original link: Theater.js animation production — BimAnt