philippseifried.com
[Update: Iain Peet pointed out that the definitions of “aliasing” and “anti-aliasing” in audio are more narrow than I thought when I wrote this post. Specifically, aliasing in audio refers to the artifacts you get when you shift or create frequencies beyond half the sample rate, and anti-aliasing refers to low-pass filtering audio signals in order to remove these artifacts. Both of these play no part in this post, and I have updated it to remove the terms.]
In this fifth installment of my series on dynamic audio programming in AS3, I want to take a quick look at the artifacts we introduce when we process audio signals without interpolation. We’ve already had a brief encounter with these, when we implemented a basic flanger effect back in part 3, which turned out to have a somewhat dirty, distorted sound to it. In this article, we’ll take a closer look at where this dirt came from, by looking at a naïve implementation of pitch shifting.
Pitch-shifting, and how not to do it
To be clear, for the purposes of this article, “pitch shifting” is what happens when you slow a recorded sound down (the pitch lowers, by an octave whenever the speed is halved) or speed it up (the pitch rises, by an octave whenever the speed is doubled). Pitch-shifting while preserving the original duration of a sound is an entirely different story, and a good deal more involved.
With that out of the way, let’s look at what a basic implementation of the effect might look and sound like:
In part 3, we had a first look at creating audio effects in AS3 by processing microphone input with robot voice effects. One of the things we did was to create a so-called comb filter by adding a delayed version of the original signal to itself. We then created a flanging effect by oscillating the delay’s offset.
Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.
Audio sample: 2 drum samples, one dry, one processed with a flanger.
Given that flangers are one of the staples of audio effects (especially when applied to electric guitars), you may have been wondering why exactly this worked the way it did. In this article we’ll figure this out.
This gives us an opportunity to review a bit of audio theory, independent of AS3. Next time, we’ll redo our quick and dirty flanger implementation and add antialiasing, but today’s article will be free of code – you’ll just need a little high school math.
In the first two tutorials of this series on dynamic audio in AS3, we’ve covered pretty much everything that Flash’s realtime sound API offers us. Let’s put all of it to use and benefit humankind by building a little app that will turn your voice into a horrible robot!
Pictured: Code sample 5. (image source: http://www.dieselsweeties.com)
Basically, what we’re going to do is take input from the microphone in a stream of sound samples, try to come up with something interesting to do with the samples, and then send the samples on to the sound card’s output.
Rather than building the app so that the user gets record and play buttons with which they can record a take and play back the processed version, we’ll build a real-time effect and process and play back the sound as it comes in. There are two reasons I want to go this route:
First, there’s less chrome involved that doesn’t add to the subject (GUI, application state and such).
Second, real-time processing is actually harder and more interesting than processing pre-recorded audio, because it poses the question how we can make (reasonably) sure that we have received enough input from the mic whenever we’re asked to fill a new output buffer. I’ll go over the intricacies of that in the next two sections – if you want to get right to the part where we do nasty things to your vowels, feel free to skip these and come back to them later.
In this second installment of my series of tutorials on dynamic sound in ActionScript, I’ll discuss the different parts of the sound API and show you how to extract single samples from a sound that’s in memory or coming from the microphone, as well as how to generate simple dynamic audio in real time.
As I wrote in part 1, the sound API introduced in Flash Player 10 is essentially just a set of methods and classes that let you access individual samples in a sound. Just as the introduction of the BitmapData class enabled you to manipulate and read pixels in bitmap images and from a webcam’s input, the sound API lets you process sound in memory or coming from microphone input, at the sample level.
There are two basic ingredients you need to understand in order to cook up dynamic audio – ByteArrays and SampleDataEvents:
In this series of advanced ActionScript tutorials, I’ll give some practical examples on how to work with the sound API introduced in Flash Player 10 to process audio in real time (filtering, adding effects, etc.) or synthesize sound from scratch. My goal is to evolve this into a series of articles that starts at the very beginning but goes a good deal farther than other AS3 audio tutorials, which tend to explain the API and then stop there, as if the rest were easy to figure out on your own.
The current roadmap is as follows (I’ll edit this if plans change):
Part 1 gives an overview of what the API is and what it can and cannot do.
Part 2 will explore the different parts that make up the API.
In part 3, we’ll put all of that to use and create a simple effect that takes input from a microphone and turns it into a robot voice.
In part 4, we review a bit of audio theory and explain how flangers, comb filters and chorus effects work.
In part 5, we’ll discuss interpolation by looking at pitch shifting implementations.
From there on out, we’ll start working on a sound manager that will allow you to mix and seamlessly string together pieces of a song. In a Flash game, this would give you the ability to create much less repetitive music while conserving file size. In the installments following that, we’ll extend the sound manager with a flexible effect architecture. To think of what you’ll be able to do with this, picture a Flash game in which entering a cavern would add a low pass filter and reverb to the game music. Finally, at the end of the series, I’ll give you some pointers on how to write your own software synthesizer in Flash.
We recently completed a project which involved automatically creating heatmaps containing click data from a questionnaire. In this article I’ll discuss the details of how we generated these maps. Sample code will be given in Processing, but the concepts are applicable to any programming language/API that lets you manipulate individual pixels.
Clickmap (left) and corresponding heatmap
The project’s basic premise was that users would get a set of questions related to the area in which they live, questions along the lines of “What’s your favorite spot?” or “Which place do you think could use more trees?”. They answered these questions by clicking on a map. These clicks would be stored in a database and periodically evaluated by a server-side utility written in C++, which would create clickmaps (containing the locations of every click for a given question) and heatmaps (giving a rough overview of the relative click densities in different regions on the map).
In my last post, I wrote about the importance of understanding how the decisions you make about modelling randomness can shape your game and sometimes leave unwanted artefacts in its gameplay. I’d like to linger on the topic of randomness for a bit and take a quick look at Gaussian distribution. If this conjures up somber memories of half-understood highschool math, don’t worry – this article will be very light on math.
Gaussian (or “normal”) distribution is everywhere. The famous bell curve gives us the distribution of a population’s height or intelligence, of amplitudes in electronic noise, or of the number of rainy days in a given month.
But why is the curve shaped like it is, and when is it a good idea to model a random process with normal distribution rather than just make all possible outcomes equally probable?
Between 2008 and early 2010, we must have played around 500 hours of the balloon battle mode of Super Mario Kart (the GameCube version) at the bobblebrook office. We knew the ins & outs of every stage, discovered little-known bugs/features (such as the ability to shoot across low walls and plateaus if you’re standing right in front of them – total game changer on the cookie stage), we tried playing the game in reverse gear only, and on the GameCube stage we even tried playing with closed eyes (which didn’t work that well, in case you’re wondering).
Mario Kart balloon battle on the GC - before Nintendo completely butchered it for the Wii (yes, I had feelings about that). Image source: http://cube.ign.com/articles/458/458922p3.html
When you play a game a lot, small faults become very noticeable. With Mario Kart, our biggest ire was with the random number generator that determined which weapon a player would get when they’d hit an item box. You knew it was Monday when that damn thing gave one player one red shell after another while giving the other player nothing but bananas. We were wondering how such an obviously buggy piece of code made it into such a high profile game.
Except that Mario Kart’s RNG probably wasn’t buggy, at least not from a technical perspective. Continue reading »
The Objective-C/cocos2d ports of Drifts are our first foray into the world of iOS. They were originally released earlier this year (back in March). As this is our first iOS production, we were interested in learning what kind of measures produce what kind of effect on our sales, even more than making the biggest impact at launch.
So we tried doing different things at different times, things like adding Twitter and Facebook functionality (which so far has done nothing for us), experimenting with pricing (lesson: making your game free for a day brings you a pretty huge amount of downloads, as various websites automatically pick up on the price change; this can be repeated several times with good results) or contacting various review sites (not doing that before or at launch was probably our biggest mistake).
A couple of weeks ago, we finally released the lite versions for the game, and there are a few things in the downloads and sales data that surprised me and that I want to share with you:
Shape of download and sales volumes upon introducing the lite versions of "Drifts".
This is the graph for the first few days the lite versions were out. The orange bars show downloads of “Drifts Lite“ and “Drifts HD Lite”, the blue bars show sales for “Drifts” and “Drifts HD” (which at the time were priced at tier 1 and tier 2, i.e. 0.79€ and 1.59€ respectively). Note that the blue bars are shown at a magnification of approximately 10000%.
Last week we took a look at how to create metaballs in Flash, now it’s time for another all-time classic: the specular bloom effect.
(source link: http://bit.ly/pCMLmU)
In the real world, blooming occurs because even a perfect lens can’t focus perfectly, causing adjacent parts of an image to bleed together. At normal light levels, this effect is too subtle to be noticeable. However, if a light source in the image is much brighter than the rest, the bright parts will bleed across the source’s edges, causing the familiar glow that’s seen in so many games throughout the last decade.
A common way of doing bloom is as follows:
Take the input image and create a thresholded version of it, setting pixels below a specific threshold to black. Then blur the result and add it to the original image (using ADD or SCREEN as the blend mode).
This approach is fast and works well for many applications, but it does have one drawback: There is a very noticeable cut off point, where a light source starts to glow.
Implementing specular bloom via thresholding results in a noticeable point at which objects begin to glow (image source: http://bit.ly/qD0qWS)
Shameless Plug
