omino pixel blog

Edit: See note at end of article for warnings about this script AE CS 5.5 on Mac

Hello, pixel pushers and vert herders! Ah, it’s great to be back in After Effects land. Alas in this last year, the day job (got hired by Skype, acquired by Microsoft) and life in general (after throwing elaborate theme-parties for decades, planning a wedding wasn’t actually that hard, but still) got a little busy, but thankfully again have a little bit of time for my favorite things. That is, pixels.

Omino Python Since I released it this year, it’s been downloaded several thousand times, and about a dozen enthusiastic folks have licensed it. Alas, I don’t really have the time to either properly promote or properly support this as a paid product… so I’ll make it completely free soon.

Ok! So, MIDI!

What is MIDI

MIDI is a control protocol and file format invented 20-odd years ago for music playback on synthesizers. It records all the notes. A .mid file is typically quite small, since it has no sound data in it.

Into After Effects?

Yes! The attached script lets you read a MIDI file into After Effects, and provides keyframes with all the notes.

Install:

1. Download this file.
2. Put the file in /Applications/Adobe After Effects CS5/Scripts/ScriptUI Panels/
3. Run AE, and under window, choose om_midi.jsx, and dock the panel where you like.

Recipe

Here’s one way you could use this script. In the om_midi panel, click the little dot button, and choose a MIDI file. I used one from Wikipedia, BWV 772, one of JS Bach’s 2-part inventions.

Then, with a Comp selected, click Apply.

This creates a Null layer named “midi” with several Slider effects full of keyframes.

This particular MIDI file kept the left and right hands in separate channels, which is nice! The “pitch” effect has keyframes at the beginning of each note with a value of 60 for middle C, 61 for C-sharp, and so on. Here we can see the left and right hands’ keyframes.

A fun way to use these keyframes is to drive the Position property of another layer. I took a photo of my hand, and set the Position expression and Opacity expressions:

Position:
 transform.position + 5*(0,thisComp.layer("midi").effect("ch_0_pitch")("Slider")-60)

Opacity:
  transform.opacity * thisComp.layer("midi").effect("ch_0_vel")("Slider")

So the hand will move 5 pixels for each semitone, centered on Middle-C (60).
Then I rendered the audio as AIFF and added it to the comp. (I used Propellerheads Reason. Yeah… if you’re using MIDI you need yet another set of software to manipulate it.)

And here is the result animation:

More Notes

This is just the simplest usage. A more elaborate deployment might be to export just 1 part of a complex sound-track work as MIDI, to get some of the cues into your AE timeline. With more complex scripts or expressions, you could to much more than just move the hands around.

The script will put multiple keyframes with the same time-value! In the example above, you can see the hands flicker slightly as the final chord begins. Most of the the part invention has just 1 note per hand, which was nice for the demo.

Also, inside the script is a general-purpose MIDI file reader, which you could use for other purposes. I’ve generated Illustrator pictures from scores, for example, full of circles and triangles. Drop me an email or comment if you want help using it! It’s pretty fun.

Update 2012.01.08 Warning about AE CS 5.5 on Mac

Under Mac OS X AE CS 5.5, when you choose the file, it deletes it! This is a CS5.5 bug. (Works fine in CS5.)

Best workaround is to use AE CS 5 or AE CS 4.
Another workaround is:

• make a copy of the file, like myfile_copy.mid
• Select it in the script UI
• recopy the file (selecting it deleted it)
• click apply and all is well.

(The parentheses in the post title mean that this post is essentially parenthetical.)

Arggh. Developing Pixel Bender kernels and plug-ins, my test files kept behaving erratically. Expression indexes were to blame. Each time I added or removed a parameter, the numbers all shift.

So you see, first enabling the expression produces effect("my_effect")(8). But it depends on the exact order of the parameters!

But effect("my_effect")("param 9") depends on the name.

I think I like the second one better, because I’m pretty good about nailing down my parameter names, but more likely to move them around order-wise.

(See?)

After Effects is just about my favorite software application. Very elegant model, and a clean & responsive UI.

But I think I can articulate two well-considered features that would boost its utility quite generally.

Feature Request 1: Foldering

When working with many layers, it would often be nice to group several of them together. To do this, I’d like to create a folder for them. This would help manage the grouped layers visually, and collapse to reduce them to a single line item.

You’d be able to apply effects to the folder. Maybe it’s called “groups”.

This is very similar to a precomp, of course, but you can work on the keyframes locally, instead of switching up and down the tree or using a split view.

Feature Request 2: Exportable Rigging Properties

Precomping is great, but it still doesn’t let you “rig” and “instance”. In my particular case, I’ve got two “claws” animated. They’re identical, but need to animate differently. Today, I need to duplicate the layers (either in the comp, or as separate precomps). Which is a shame, because I’d like to fiddle with the claw design, maybe later, and have my changes affect both!

So, this feature lets you expose a keyframable property on the comp, so that when it’s nested you can control it from the parent comp.

You can, if you’re overly clever, use Time Remapping to accomplish this a little bit. But I’d rather see clearly named “rigging” properties on my comp, itself, to animate.

(You could imagine also being able to add rigging properties to a folder, or maybe even any layer. This would be the same functionality as the “Slider” effect, but maybe tidier.)

As well as sliders and checkboxes, you could export “layer” properties, which pipe in a layer of the parent into the child comp. This lets you create an effect out of a comp. Presets can do this, a little bit, but in a flat, non-tweakable non-instanceable fashion.

I’ve heard one AE developer refer to a variation on this idea as a “Capsule”. The idea he described would require (I think?) you to save a comp into a separate file, before being able to reuse it. I think one of the brilliant project features of After Effects is that you get multiple timelines within one project. I like the One Big File workflow! (I wish 3d apps had this; as far as I know, they all work only on a single timeline per document.)

Miscellany

It would be nice to drop in a “reference” layer, that could repeate a different layer within the same comp to a different layer order. This would need to be post effects to be useful. It would let you perform different treatments on the same source… again, without a precomp.

It would also let your manage a folder, and still retain flexibility on its layer order.

To conclude…

After Effects is already quite strong in terms of “instancing”. That’s practically its main power. But it fails for purposes much beyond “tidiness”. “Rigging Properties” addresses this key omission.

Photoshop recently got “foldering”, and lets you set transfer mode and transparency on a group of layers. This is grand! I want it for After Effects.

Here’s a little movie that demonstrates a text track being used to “trigger” the opacity of another track.

Controlling Pip Opacity From Text

I needed a blinking pip. (It’s actually for an interesting optical effect… but for now, we’ll just talk about the pip.) Rather than keyframing each blink, I used a Text track where A is a very dim blink, and Z is a very bright blink. The blinks all have the same luminous decay. Every keyframe on the Text track triggers a new blink.

The expression to do this is actually a little program. (Exactly what some artists hate…)

Here’s the expression for controlling the Pip’s opacity. It’s in 3 parts: 1) Find the most recent Text track keyframe, 2) See what letter it was, from A to Z, and 3) Set the opacity, taking into account the brightness and how long it’s been decaying.

// find most recent key
var s = thisComp.layer("controlTrack").text.sourceText; // score layer
var n = s.numKeys;
var kValue = "A"; // default value
var kTime = 0;
for(var i = 1; i <= n; i++)
{
	var k = s.key(i);
	if(k.time <= time)
	{
		// only care if there's a char in our position.
		var a = k.value.charAt(0).toUpperCase();
		if(a >= "A" && a <= "Z")
		{
			kValue = a;
			kTime = k.time;
		}
	}
}

// now, how long ago was that?
var age = time - kTime;

// map character a-z to 0.0-1.0
kValue = kValue.charCodeAt(0);
kValue -= 65;
kValue = kValue / 25.0;

// decay appropriately
var result = kValue * Math.pow(0.3, age) * transform.opacity;

// done.
result

Why do this tricky control business with a text track, instead of just keyframing the flashes more directly? This way, the behavior is very well specified... and even more so for the next movie.

Controlling Five Pips With a Text Track

You can easily see what's going on here. Five characters control five pip's blinks.

The clever bit was to avoid pasting that monster expression from above into five different tracks. Instead, a single slider control track is driven from the text, and shows the first pip's blinking at time=0, the second pip's at time=300, the third's at time=600, and so on. The whole animation is 20 seconds, so there's plenty of room on the timeline for this.

Each pip's opacity is driven from the appropriate section of this slider channel:

pip0 opacity: transform.opacity * thisComp.layer("controlTrack").effect("scoreReader")("Slider").valueAtTime(time + 0*300)

pip1 opacity: transform.opacity * thisComp.layer("controlTrack").effect("scoreReader")("Slider").valueAtTime(time + 1*300)

pip2 opacity: transform.opacity * thisComp.layer("controlTrack").effect("scoreReader")("Slider").valueAtTime(time + 2*300)

... and so on.

So, why pile this cleverness atop an already tricky expression? Because with an expression that tricky, anything is better than duplicating five times across different tracks!

Here's the expression, which extracts brightness from the text track, for each position, at times separated by 300 seconds.

// Allow different sections of this track to extract the brightness for each pip.
// we divide it up into 5 minute segments (300 seconds). To find out your brightness
// at a particular time, examine *this* track at time + 300 * pipNumber.
//
// This is an alternative to duplicating the rather ornate expression into each pip, at
// the expense, of course, of complexity.
//

var segmentDuration = 300;
var segmentIndex = Math.floor(time / segmentDuration);
var segmentTime = time - segmentIndex * segmentDuration + .01; // local time... plus 1/100 for roundoff

// find most recent key
var s = thisComp.layer("controlTrack").text.sourceText; // score layer
var n = s.numKeys;
var kValue = "A"; // default value
var kTime = 0;
for(var i = 1; i <= n; i++)
{
	var k = s.key(i);
	if(k.time <= segmentTime)
	{
		// only care if there's a char in our position.
		var a = k.value.charAt(segmentIndex).toUpperCase();
		if(a >= "A" && a <= "Z")
		{
			kValue = a;
			kTime = k.time;
		}
	}
}

// now, how long ago was that?
var age = segmentTime - kTime;

// map score character a-z to 0.0-1.0
kValue = kValue.charCodeAt(0);
kValue -= 65;
kValue = kValue / 25.0;

// decay appropriately
var result = kValue * Math.pow(0.3, age);

result

And lastly, here's the self-contained After Effects project.