As a performing musician, incorporating projected visuals within your performances is a sure-fire way of enhancing the experience for your audience. There are many Video Jockey and Projection Mapping software applications designed for doing this, such as Resolume Arena or ArKaos GrandVJ, which allow artists to trigger and control video, graphics and animations in real time, as well as syncing them to music. Most of these applications can be controlled using MIDI, making them extremely useful to musicians who already use MIDI within their shows. However, the MIDI integration is often quite limited and may not let you control the visuals exactly how you want. One way around this would be to create your own custom software application
In this tutorial, I’m going to demonstrate how to create a simple desktop application that generates real-time animations using MIDI data, using a simple but extensive platform that could be used for almost any MIDI-to-visuals ideas that you may have.
The platform we will be using here is called openFrameworks (often abbreviated to oF)—a free toolkit for developing interactive and multimedia applications, designed for artists and designers to create software in a simple way using a textual programming language. We will be using oF over other similar platforms, as it is very easy for beginners to use, has great audio/MIDI capabilities, and uses a common programming language which is very useful to learn for software development endeavors.
Before we get started, here is a simple example of what we will be making—an application that draws animated circles whenever it receives a MIDI note, using note number and velocity to determine color and size, as well as using MIDI mod wheel messages to control other aspects of the visuals. My particular animation is being triggered by MIDI messages coming out of Ableton Live on OS X, which is also triggering drum samples within Native Instruments Battery.
What you will Need:
First you will need to install your IDE. Installing Xcode on OS X is easy—simply download it from the Mac App Store and follow the installation instructions. For Code::Blocks on Windows the installation process is a bit more complicated, however fortunately oF have provided a setup guide here.
To install oF, simply move the downloaded oF folder into any location on your computer. To install ofxMidi, move the downloaded ofxMidi folder into the addons directory within your oF folder.
Next you will need to set up a project for your IDE so that you can develop your oF application. To do this you will use oF’s projectGenerator application that you can find in the projectGenerator directory within the oF folder, which will automatically generate an Xcode/Code::Blocks project for you. Open up projectGenerator and do the following:
There should now be a new folder in the openFrameworks/apps/myApps directory for your newly created project.
Next you need to open your newly created project and add some code to it:
Teaching you how the code for this project works is a bit beyond the scope of this article, so I’ve placed many comments (lines beginning with ‘//’) in the code to describe what each part is doing.
You are almost ready to try out the MIDI visualiser application, however before that you need to build the code into a piece of software. To do this complete the following steps:
In Xcode on OS X:
In Code::Blocks on Windows:
Congratulations—you have just built your own piece of software! Now it’s time to give it a go.
Using the MIDI visualiser application is easy. As the instructions state, simply select the MIDI input you want to use with the numbers on your computer keyboard, pressing ‘s’ to hide the instructions. Send some MIDI note data to the application from the selected MIDI input and you’ll notice it draws a circle for each note played, which fades out over time. The size of the circles change depending on note velocity, and colour of the circles change depending on note number. Try sending mod wheel (CC 1) messages—you’ll notice that this changes the background color. The application window can be resized or made full screen.
So that’s how you can build your own software for generating visuals that are controlled via MIDI. Not only have you created something that could enhance your live performances in a unique way, but you’ve also learnt the very basics of a software toolkit that you could use to create an infinite number of diverse multimedia applications and projects; just check out the openFrameworks gallery to see the amazing things that have been done with it. If you want to know more about developing software with oF I recommend checking out the official openFrameworks tutorials for more information.