pianosu Dev Notes

Phaser3 boilerplate and JS game dev links: https://github.com/Sidaroth/Phaser3.Boilerplate

Phaser 100 line keyboard piano: https://github.com/akrolsmir/Phaser-Piano

Ableton in-browser how to compose music: https://learningmusic.ableton.com/index.html

This is a ton of fun!
  • Has keyboard playing
  • Looping music just sounds great!
  • Syncs beats and piano notes, wow
  • Draggable UI is super toylike
  • Makes music feel approachable
  • Super short explanations with practice prolems
  • Builds up on previous understanding
  • Saves your game state automatically
  • Nice backing track, sheet music display, but tells you exactly what to press. Familiar to sheet music readers
  • Also waits until you hit the right key, so you get it right
  • Nice visualization, also: waits until the right key is hit
  • Also shows sheet music

Tensorflow js AI for piano midi generation: https://magenta.tensorflow.org/performance-rnn

Shawn Wasabi making music on Nintendo Labo: https://www.youtube.com/watch?v=D5KHP0mQ5fE

@March 20, 2021

Youtube tutorial on levels of Jazz:

Jazz Levels

Sheet music as a service: https://flat.io/

Hm... a lot of smart people I know have taken an interest in music.

  • Angie
  • Eric Zhang
  • Eric Jang
  • Is it Asian competition? Does it go deeper?

@March 21, 2021

Pitch Shifting (Autotune) in JS: Paper, GIthub

Music Source Separation

Separating out voices from songs is a great first step

Audacity vocal removal works okay, 7/10

https://vocalremover.org/ works great out-of-the-box, 9/10

LaSAFT: Cool demo, 8/10 paper, but code doesn't repro ☹️

Demucs: No demo, but lots of GH stars and sponsored by FB research, 8/10 https://github.com/facebookresearch/demucs

Waveform to midi notes

Essentia PitchMelodia (C++ song extraction library with WASM bindings): https://mtg.github.io/essentia.js/examples/offline/pitchmelodia/example.html

Suggestion: Manually move 3 songs into


  1. Youtube to Mp3
  2. Mp3 to voice/drums/bass/instruments waveform
  3. Waveform to notes?

Hard things about transcribing:

  • Want to hear the backing music, with a 4/4 beat indicator
  • Want to line up the transcribed notes to the song
  • Want to be able to skip backwards to perfect an area (pretty sure the first section is good)

Unsolved questions:

  • Where does data go?
    • Is it in Vue (and we pass a VueApp reference around?)
    • In Phaser itself?
  • What widgets do we use?
    • HTML/Vue based
      • More familiar, faster dev speed, debuggable
    • Phaser-based eg Rex
      • Better user experience, no weird routing design,


Timeline: Ability to scrub back and forth
  • Escape key to pause and restart
  • Backspace to try again / jump back 5 sec (or "whoops, messed up, try again")
  • Implementation: Need to have objects with long lived times, instead of arcade gravity
    • Unless: We can manipulate scene timeline to do it for us...? Not that likely, no extract feature
      • Hm... Scene.time.update...?
    • paused only stops future timer events; doesn't stop existing physics
    • Scene pause/resume Phaser example


  • Want to hold down notes to play longer
  • Feels like there should be two keys between F and J
    • Could make starting notes G and H, or configurable
    • Maybe QWER or 1234 for the home row?
    • Bryant also had this feedback
  • Virality:
    • Tiktok virality: half video is someone else's, half is yours
      • AKA rounds

Eric: need hit feedback

Adrian: Non diatonic songs?

Adrian: Tools inspire you to build things