Skip to content

Usage As Library

Jump to bottom
spessasus edited this page Jun 26, 2024 · 16 revisions

Library usage

The program makes a heavy usage of the ES6 Modules!

To use the program as a library, simply copy the spessasynth_lib folder to your desired destination.

Tip

It is recommended to obtain spessasynth_lib via git rather than releases as it usually has the latest bugfixes. (and bugs)

Minimal setup

The minimal working setup requires Synthetizer class

The setup is initialized as follows:

audioContext.audioWorklet.addModule("./spessasynth_lib/synthetizer/worklet_system/worklet_processor.js");
const synth = new Synthetizer(outputNode, soundFontBuffer);

Simple MIDI player demo

See this demo live

This demo demonstrates how to quickly set up a synthetizer and a sequencer to play a MIDI file.

The demo uses 4 classes: Synthetizer class, SoundFont2 class, MIDI class and Sequencer class.

simple_demo.html

<h1>SpessaSynth demo</h1>
<p id="message">Please wait for the soundFont to load.</p>
<input type="file" id="midi_input" accept="audio/mid">
<!-- note the type="module" -->
<script src="simple_demo.js" type="module"></script>

simple_demo.js

What the script does:

  1. Imports the necessary variables
  2. fetch-es the path/to/your/soundfont.sf2
  3. Parses the read file using SoundFont2
  4. Initializes an AudioContext and a Synthetizer instance with the parsed soundfont
  5. Adds an EventListener for the file input:
    • Parses the MIDI file using MIDI class
    • Initializes a Sequencer instance and connects it to the Synthetizer instance we created earlier
    • Starts the playback via sequencer.play(); 
// import the modules
import { MIDI } from "./spessasynth_lib/midi_parser/midi_loader.js";
import { Sequencer } from "./spessasynth_lib/sequencer/sequencer.js";
import { Synthetizer } from "./spessasynth_lib/synthetizer/synthetizer.js";

// load the soundfont
fetch("soundfont.sf2").then(async response => {
   // load the soundfont into an array buffer
   let soundFontArrayBuffer = await response.arrayBuffer();
   document.getElementById("message").innerText = "SoundFont has been loaded!";

   // add an event listener for the file inout
   document.getElementById("midi_input").addEventListener("change", async event => {
      // check if any files are added
      if (!event.target.files[0]) {
         return;
      }
      const file = event.target.files[0];
      const arrayBuffer = await file.arrayBuffer();                           // convert the file to array buffer
      const parsedMidi = new MIDI(arrayBuffer);                             // parse the MIDI file
      const context = new AudioContext();                                     // create an audioContext
      // add the worklet
      await context.audioWorklet.addModule("./spessasynth_lib/synthetizer/worklet_system/worklet_processor.js")
      const synth = new Synthetizer(context.destination, soundFontArrayBuffer);          // create the synthetizer
      const seq = new Sequencer([parsedMidi], synth);                         // create the sequencer (it can accept multiple files, so we need to pass an array)
      seq.play();
   })
});

It's that simple!

A more advanced demo

See this demo live

The code above is very basic, it only allows to upload a midi file. We can add more features such as play/pause and time controls to our player without much effort.

Let's add some control buttons:

advanced_demo.html

<h1>SpessaSynth demo</h1>
<p id="message">Please wait for the soundFont to load.</p>
<input type="file" id="midi_input" multiple accept="audio/mid">
<br><br>
<input type="range" min="0" max="1000" value="0" id="progress">
<br>

<button id="previous">Previous song</button>
<button id="pause">Pause</button>
<button id="next">Next song</button>

<!-- note the type="module" -->
<script src="advanced_demo.js" type="module"></script>

Now we need to add functionality to those buttons:

  • Input can now accept more files
  • Previous song button
  • Pause button
  • Next song button
  • Song progress slider

advanced_demo.js

// import the modules
import { MIDI } from "./spessasynth_lib/midi_parser/midi_loader.js";
import { Sequencer } from "./spessasynth_lib/sequencer/sequencer.js";
import { Synthetizer } from "./spessasynth_lib/synthetizer/synthetizer.js";

// load the soundfont
fetch("soundfont.sf2").then(async response => {
   // load the soundfont into an array buffer
   let soundFontBuffer = await response.arrayBuffer();
   document.getElementById("message").innerText = "SoundFont has been loaded!";


   // add an event listener for the file inout
   document.getElementById("midi_input").addEventListener("change", async event => {
      // check if any files are added
      if (!event.target.files[0]) {
         return;
      }
      // parse all the files
      const parsedSongs = [];
      for (let file of event.target.files) {
         const buffer = await file.arrayBuffer();
         parsedSongs.push(new MIDI(buffer));
      }
      // create the context and add audio worklet
      const context = new AudioContext();
      await context.audioWorklet.addModule("./spessasynth_lib/synthetizer/worklet_system/worklet_processor.js")
      const synth = new Synthetizer(context.destination, soundFontBuffer);          // create the synthetizer
      const seq = new Sequencer(parsedSongs, synth);                          // create the sequencer without parsed midis
      seq.play();                                                             // play the midi
      seq.loop = false;                                                       // the sequencer loops a single song by default

      // make the slider move with the song
      let slider = document.getElementById("progress");
      setInterval(() => {
         // slider ranges from 0 to 1000
         slider.value = (seq.currentTime / seq.duration) * 1000;
      }, 1000);

      // add time adjustment
      slider.onchange = () => {
         // calculate the time
         let targetTime = (slider.value / 1000) * seq.duration;
         seq.currentTime = targetTime; // switch the time (the sequencer adjusts automatically)
      }

      // add button controls
      document.getElementById("previous").onclick = () => {
         seq.previousSong(); // go back by one song
      }
      document.getElementById("pause").onclick = () => {
         if (seq.paused) {
            document.getElementById("pause").innerText = "Pause";
            seq.play(); // resume
         }
         else {
            document.getElementById("pause").innerText = "Resume";
            seq.pause(); // pause

         }
      }
      document.getElementById("next").onclick = () => {
         seq.nextSong(); // go to next song
      }
   });
});

Adding visualizations

See this demo live

Let's spice up our demo a bit!

visualizer.html

We need to add the canvas and our "keyboard"

<h1>SpessaSynth demo: visualizer</h1>
<p id="message">Please wait for the soundFont to load.</p>
<input type="file" id="midi_input" multiple accept="audio/mid">
<br><br>
<canvas id="canvas" width="500" height="500"></canvas>
<table>
    <tbody>
        <tr id="keyboard"></tr>
    </tbody>
</table>

<!-- note the type="module" -->
<script src="visualizer.js" type="module"></script>

visualizer.js

We use 2 functions of the API to achieve this:

synth.connectIndividualOutputs(audioNodes);

This connects the AnalyserNodes to the synthesizer, allowing visualizations.

synth.eventHandler.addEvent("noteon", event => {/*...*/})

Event system allows us to hook up events (in this case, note on and off to visualize keypresses)

// import the modules
import { MIDI } from "./spessasynth_lib/midi_parser/midi_loader.js";
import { Sequencer } from "./spessasynth_lib/sequencer/sequencer.js";
import { Synthetizer } from "./spessasynth_lib/synthetizer/synthetizer.js";

// load the soundfont
fetch("soundfont.sf2").then(async response => {
   // load the soundfont into an array buffer
   let soundFontArrayBuffer = await response.arrayBuffer();
   document.getElementById("message").innerText = "SoundFont has been loaded!";

   // add an event listener for the file inout
   document.getElementById("midi_input").addEventListener("change", async event => {
      // check if any files are added
      if (!event.target.files[0]) {
         return;
      }
      const file = event.target.files[0];
      const arrayBuffer = await file.arrayBuffer(); // convert the file to array buffer
      const parsedMidi = new MIDI(arrayBuffer); // parse the MIDI file
      const context = new AudioContext(); // create an audioContext
      // add the worklet
      await context.audioWorklet.addModule("./spessasynth_lib/synthetizer/worklet_system/worklet_processor.js");

      // prepare and play
      const synth = new Synthetizer(context.destination, soundFontArrayBuffer); // create the synthetizer
      const seq = new Sequencer([parsedMidi], synth); // create the sequencer (it can accept multiple files so we need to pass an array)
      seq.play(); // play the midi

      const canvas = document.getElementById("canvas"); // get canvas
      const drawingContext = canvas.getContext("2d");
      /**
       * create the AnalyserNodes for the channels
       */
      const analysers = [];
      for (let i = 0; i < 16; i++) {
         analysers.push(context.createAnalyser()); // create analyser
      }

      // connect them to the synthesizer
      synth.connectIndividualOutputs(analysers);

      // render analysers in a 4x4 grid
      function render()
      {
         // clear the rectangle
         drawingContext.clearRect(0, 0, canvas.width, canvas.height);
         analysers.forEach((analyser, channelIndex) => {
            // calculate positions
            const width = canvas.width / 4;
            const height = canvas.height / 4;
            const step = width / analyser.frequencyBinCount;
            const x = width * (channelIndex % 4); // this code makes us a 4x4 grid
            const y = height * Math.floor(channelIndex / 4) + height / 2;

            // draw the waveform
            const waveData = new Float32Array(analyser.frequencyBinCount);
            // get the data from analyser
            analyser.getFloatTimeDomainData(waveData);
            drawingContext.beginPath();
            drawingContext.moveTo(x, y);
            for (let i = 0; i < waveData.length; i++)
            {
               drawingContext.lineTo(x + step * i, y + waveData[i] * height);
            }
            drawingContext.stroke();
         });

         // draw again
         requestAnimationFrame(render);
      }
      render();

      // create a keyboard
      const keyboard = document.getElementById("keyboard");
      // create an array of 128 keys
      const keys = [];
      for (let i = 0; i < 128; i++)
      {
         const key = document.createElement("td");
         key.style.width = "5px";
         key.style.height = "50px";
         key.style.border = "solid black 1px";
         keyboard.appendChild(key);
         keys.push(key);
      }

      // add listeners to show keys being pressed

      // add note on listener
      synth.eventHandler.addEvent("noteon", "demo-keyboard-note-on", event => {
         keys[event.midiNote].style.background = "green"
      });

      // add note off listener
      synth.eventHandler.addEvent("noteoff", "demo-keyboard-note-off", event => {
         keys[event.midiNote].style.background = "white";
      })
   })
});

Quite cool, isn't it?

Render audio to file

See this demo live

Let's make use of SpessaSynth 3.0. It allows us to render an audio file to a file!

offline_audio.html

Nothing new here.

<h1>SpessaSynth demo: offline audio conversion</h1>
<p id="message">Please wait for the soundFont to load.</p>
<input type="file" id="midi_input" accept="audio/mid">
<br><br>

<!-- note the type="module" -->
<script src='offline_audio.js' type="module"></script>

offline_audio.js

Here we use OfflineAudioContext to render the audio to file and audioBufferToWav helper, conveniently bundled with SpessaSynth. Note that we pass the MIDI file directly to the Synthesizer class this time.

// import the modules
import { MIDI } from "../../spessasynth-examples/spessasynth_lib/midi_parser/midi_loader.js";
import { Synthetizer } from "../../spessasynth-examples/spessasynth_lib/synthetizer/synthetizer.js";
import { audioBufferToWav } from "../../spessasynth-examples/spessasynth_lib/utils/buffer_to_wav.js";

// load the soundfont
fetch("soundfont.sf2").then(async response => {
    // load the soundfont into an array buffer
    let soundFontArrayBuffer = await response.arrayBuffer();
    document.getElementById("message").innerText = "SoundFont has been loaded!";

    // add an event listener for the file inout
    document.getElementById("midi_input").addEventListener("change", async event => {
        // check if any files are added
        if (!event.target.files[0]) {
            return;
        }
        const file = event.target.files[0];
        const arrayBuffer = await file.arrayBuffer(); // convert the file to array buffer
        const parsedMidi = new MIDI(arrayBuffer, file.name); // parse the MIDI file
        const sampleRate = 44100; // 44100Hz
        const context = new OfflineAudioContext({
                numberOfChannels: 2, // stereo
                sampleRate: sampleRate,
                length: sampleRate * (parsedMidi.duration + 1), // sample rate times duration plus one second (for the sound to fade away rather than cut)
        });
        // add the worklet
        await context.audioWorklet.addModule("./spessasynth_lib/synthetizer/worklet_system/worklet_processor.js");
        // here we set the event system to disabled as it's not needed. Also, we need to pass the parsed MIDI here for the synthesizer to start rendering it
        const synth = new Synthetizer(context.destination, soundFontArrayBuffer, false, {
            parsedMIDI: parsedMidi,
            snapshot: undefined // this is used to copy the data of another synthesizer, so no need to use it here
        });
        
       // show progress
       const showRendering = setInterval(() => {
          const progress = Math.floor(synth.currentTime / parsedMidi.duration * 100);
          document.getElementById("message").innerText = `Rendering... ${progress}%`;
       }, 500);

       // start rendering the audio
       const outputBuffer = await context.startRendering();
       clearInterval(showRendering);

        // convert the buffer to wav file
        const wavFile = audioBufferToWav(outputBuffer);

        // make the browser download the file
        const a = document.createElement("a");
        a.href = URL.createObjectURL(wavFile);
        a.download = parsedMidi.midiName + ".wav";
        a.click();
    })
});
Clone this wiki locally