dependent library
- Pixi.js is a front-end graphics rendering library that uses sprite technology to draw high-performance graphics.
- Tone.js is a front-end audio framework that encapsulates the web audio API, which can quickly create audio samples, audio effects, audio analysis and audio playback.
- @tonejs/midi is a plugin for tonejs, which can convert midi files into json format that can be parsed by Tone.js.
midi file analysis
First of all, we need to import midi files into the browser. Due to the security restrictions of the browser, we can only use the file selector to import files.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Midi可视化</title>
</head>
<body>
<input type="file" id="file">
<script>
const input = document.querySelector('#file');
input.addEventListener('change', (e) => {
console.log(e.target.files[0]);
})
</script>
</body>
</html>
In this way, you can get the File object corresponding to the Midi file.
Import tonejs and tone/midi plugins, ready to parse File objects.
<script
type="text/javascript"
src="https://unpkg.com/tone@latest/build/Tone.js"
></script>
<script
type="text/javascript"
src="https://unpkg.com/@tonejs/midi"
></script>
Parse a File object with a MIDI plugin.
// 读取midi文件
function parseMidi(file) {
// 创建文件读取器
const reader = new FileReader();
// 读取文件
reader.readAsArrayBuffer(file);
// 文件读取完成后将文件转化为json对象
reader.addEventListener('load', (e) => {
currentMidi = new Midi(e.target.result);
console.log(currentMidi);
})
}
Is there anyone who understands design patterns and can help me optimize the code? I always feel uncomfortable defining a global variable currentMidi.
This currentMidi stores information such as audio tracks, musical notes, instruments, etc. of midi files. Data can be provided for creating synthesizers and visualizations below.
Audio Player
Audio is played using tonejs to create a synthesizer.
Add a button to the page and bind an event to play audio. The code below uses Tonejs to create a synthesizer to play sound.
play.addEventListener('click', (e) => {
console.log(currentMidi);
// 如果未加载midi文件
if(!currentMidi) {
alert('未加载文件');
return;
}
const now = Tone.now() + 0.5; // 获取当前时间
const synths = []; // 存储合成器
// 遍历midi文件中的轨道
currentMidi.tracks.forEach(track => {
// 创建合成器作为音轨并连接至出口，音色使用Tonejs的默认音色
const synth = new Tone.PolySynth(Tone.Synth, {
envelope: {
// 声音的生命周期：按下按键 - 渐入 - 攻击阶段 - 衰减阶段 - 衰减结束 - 松开按键 - 声音消逝
attack: 0.02, // 渐入时间
decay: 0.1, // 攻击阶段（最大音量）持续时间
sustain: 0.3, // 衰减结束后的最小声音
release: 1, // 从松开按键到声音彻底消失所需的时间
},
}).toDestination();
// 将合成器存储起来，为之后停止播放的功能留下接口。
synths.push(synth);
// 遍历轨道中的每个音符
track.notes.forEach(note => {
// 合成器发声
synth.triggerAttackRelease(
note.name, // 音名
note.duration, // 持续时间
note.time + now, // 开始发声时间
note.velocity // 音量
);
});
});
})
Here you can realize the playback of MIDI files.
visualization
The visualization uses pixijs, first use npm to download pixijs and then import it.
<script src="../../node_modules/pixi.js/dist/pixi.js"></script>
Use pixijs to create a canvas and mount it on the page.
const Application = PIXI.Application; // 应用类，快速创建PIXI应用
const Sprite = PIXI.Sprite; // 精灵类
const Graphics = PIXI.Graphics; // 图形类
// 创建应用程序并挂载
const pixi = new Application({
width: 1000,
height: 600,
backgroundColor: 0x000000
})
// pixi.view 代表画布，是一个canvas元素
document.body.appendChild(pixi.view);
We use the lamp class as a visualization of each note.
const config = {
speed: 1
}
// 定义灯光类作为音符的可视化
class Light extends Graphics {
constructor(color, height, x) {
super();
this.beginFill(color);
this.drawRect(x, 600, 10, height);
this.endFill();
// pixijs的定时器，可以实现每帧执行一次，并且十分稳定
pixi.ticker.add(() => {
this.y -= config.speed * 5;
});
}
}
Just create two light tubes to try the effect.
// pixi.stage 代表舞台，所有的物体必须挂载在舞台上才可以显示。
pixi.stage.addChild( new Light(0xffffff, 50, 400) );
Successfully displayed, and moving all the way up.
After that, notes are created based on the parsed json object.
In order to synchronize with the audio as much as possible and reduce the delay, we use Tonejs audio scheduling.
When traversing a note, create a scheduling task corresponding to this note.
// 在播放按钮的事件中，遍历音符时，创建音频调度，实现音画同步
Tone.Transport.schedule((time) => {
// 根据音调划分颜色，（其实应该根据轨道来划分的）
if(note.midi < 65) {
pixi.stage.addChild(new Light(g.config.leftColor, note.duration * 200 * g.config.speed, (note.midi - 20) * 10))
} else {
pixi.stage.addChild(new Light(g.config.rightColor, note.duration * 200 * g.config.speed, (note.midi - 20) * 10))
}
}, note.time + now);
// 在代码最外层设置音频调度的模式，并启动音频调度。
Tone.context.latencyHint = 'fastest';
Tone.Transport.start();
At this point, the basic functions of the program are completed.
