This 'Beat Saber' Project Uses AI to Generate Custom Beat Maps for Any Song – Road to VR

  • The Entire VR Industry in One Little Email

    The Daily Roundup is our comprehensive coverage of the VR industry wrapped up into one daily email, delivered directly to your inbox. 

    Manually creating beat maps for your favorite songs in Beat Saber (2018) requires a bit of know-how and time. If you’re looking for a quick and surprisingly effective solution to automatically creating beat maps for any song, look no further than the fan-built service called ‘Beat Sage’.

    Beat Sage is an AI which creates custom beat maps with any song on YouTube, or any mp3 in your possession. Simply navigate to the Beat Sage website, plug in the song’s YouTube URL (or upload your own mp3), tick a few boxes for level difficulty, game modes, and song events, and you’ll have a brand new mess of levels for Beat Saber on SteamVR headsets and Oculus Quest. And it’s free.

    Created by Beat Saber fans Chris Donahue and Abhay Agarwal, Beat Sage is said to use two neural networks to map an audio file into what the team calls “a plausible Beat Saber level.”

    Image captured by Road to VR

    “These neural networks were trained on Beat Saber levels created by humans,” the team explains. “The first neural network listens to the audio and predicts at what points in time blocks should be placed. The second neural network looks at the predicted timings and maps each to a timestamp to a block type (e.g. red up, blue down, red up + blue down).”

    Check out the promo below to see some of the results, timestamped showing Ariana Grande’s song ‘thank u, next’.

    SEE ALSO’Beat Saber’ Gets Free Music Pack ‘OST Vol. 3’ on All Supported Platforms

    Installing the songs is fairly straightforward on PC, as it doesn’t require any modding. Find the ‘CustomLevels’ subfolder in your directory either via Steam or Oculus Home—depending on where you bought Beat Saber. Here’s the default locations.