picture home | art | events | music | rack extensions | downloads | pixel blog | qt_tools

omino pixel blog

pixels, motion, and scripting
david van brink // Mon 2011.12.26 13:56 // {after effects}

AE: Hello again, & MIDI

Edit: See note at end of article for warnings about this script AE CS 5.5 on Mac

Hello, pixel pushers and vert herders! Ah, it’s great to be back in After Effects land. Alas in this last year, the day job (got hired by Skype, acquired by Microsoft) and life in general (after throwing elaborate theme-parties for decades, planning a wedding wasn’t actually that hard, but still) got a little busy, but thankfully again have a little bit of time for my favorite things. That is, pixels.

Omino Python Since I released it this year, it’s been downloaded several thousand times, and about a dozen enthusiastic folks have licensed it. Alas, I don’t really have the time to either properly promote or properly support this as a paid product… so I’ll make it completely free soon.

Ok! So, MIDI!

What is MIDI

MIDI is a control protocol and file format invented 20-odd years ago for music playback on synthesizers. It records all the notes. A .mid file is typically quite small, since it has no sound data in it.

Into After Effects?

Yes! The attached script lets you read a MIDI file into After Effects, and provides keyframes with all the notes.

Install:

1. Download this file.
2. Put the file in /Applications/Adobe After Effects CS5/Scripts/ScriptUI Panels/
3. Run AE, and under window, choose om_midi.jsx, and dock the panel where you like.

Om midi panel

Recipe

Here’s one way you could use this script. In the om_midi panel, click the little dot button, and choose a MIDI file. I used one from Wikipedia, BWV 772, one of JS Bach’s 2-part inventions.

Then, with a Comp selected, click Apply.

This creates a Null layer named “midi” with several Slider effects full of keyframes.

This particular MIDI file kept the left and right hands in separate channels, which is nice! The “pitch” effect has keyframes at the beginning of each note with a value of 60 for middle C, 61 for C-sharp, and so on. Here we can see the left and right hands’ keyframes.

A fun way to use these keyframes is to drive the Position property of another layer. I took a photo of my hand, and set the Position expression and Opacity expressions:

Position:
 transform.position + 5*(0,thisComp.layer("midi").effect("ch_0_pitch")("Slider")-60)

Opacity:
  transform.opacity * thisComp.layer("midi").effect("ch_0_vel")("Slider")

So the hand will move 5 pixels for each semitone, centered on Middle-C (60).
Then I rendered the audio as AIFF and added it to the comp. (I used Propellerheads Reason. Yeah… if you’re using MIDI you need yet another set of software to manipulate it.)

And here is the result animation:

BWV_772.mov

click for
http://omino.com/pixelblog/wp-content/uploads/2011/12/BWV_772.mov

More Notes

This is just the simplest usage. A more elaborate deployment might be to export just 1 part of a complex sound-track work as MIDI, to get some of the cues into your AE timeline. With more complex scripts or expressions, you could to much more than just move the hands around.

The script will put multiple keyframes with the same time-value! In the example above, you can see the hands flicker slightly as the final chord begins. Most of the the part invention has just 1 note per hand, which was nice for the demo.

Also, inside the script is a general-purpose MIDI file reader, which you could use for other purposes. I’ve generated Illustrator pictures from scores, for example, full of circles and triangles. Drop me an email or comment if you want help using it! It’s pretty fun.

Update 2012.01.08 Warning about AE CS 5.5 on Mac

Under Mac OS X AE CS 5.5, when you choose the file, it deletes it! This is a CS5.5 bug. (Works fine in CS5.)

Best workaround is to use AE CS 5 or AE CS 4.
Another workaround is:

  • make a copy of the file, like myfile_copy.mid
  • Select it in the script UI
  • recopy the file (selecting it deleted it)
  • click apply and all is well.
11 comments
Mylenium's Blog » Saved! // Sat 2011.12.31 03:203:20 am

[…] That has been true for the previously mentioned unwrapping cylinder thing and now it seems you can work with MIDI files in After Effects, after all. Not in the way that that person imagined (by tying them live to […]

hyp // Sun 2012.01.1 17:535:53 pm

any chance you can update this to work with CS 5.5? such an awesome script, but it doesnʻt work unfortunately. thanks for your hard work.

nathan // Tue 2012.01.3 15:133:13 pm

this is the coolest thing ever

arto // Wed 2012.01.4 14:352:35 pm

^hyp: Did you make a new comp & load a real .mid file? It works fine in my 5.5.

Thank you for a great script I’ve been searching for a long time! Gotta start learning those expressions now.. :)

david van brink // Sat 2012.01.7 13:501:50 pm

^arto I hope it’s useful for you.

^hyp — I tried AECS5.5 on Mac… and yes it’s broken. It deletes the file. This is a CS5.5 bug. Adding workarounds to the article…

lotzio // Tue 2012.01.10 18:366:36 pm

GREAT STUFF MAN!!! THX FOR SHARING, IT’LL BE SUPER USEFUL FOR SOME OF MY PROPOSALS. REALLY APPRECIATE.

音のデータとAfter Effects その2 | AEP Project // Sat 2012.01.14 08:018:01 am

[…] 楽譜のデータから作る さて、少しだけ変わった方向からAEと音のデータについて考えます。動画だけでなく、音楽を制作される方なら、オーディオよりも楽譜としてデータを持って行けたら、嬉しいと思います。 MIDI、というファイル形式があります。昔のHPで流れてる音楽によく使われたり、カラオケでも使われている形式です。 MIDIとは「ある楽器をこの強さ、この音階で鳴らす」という楽譜の情報です。 もし、オーディオのデータのほかに、この楽譜のデータを持っていたら、より繊細な映像を作ることも可能です。 このMIDIには一曲分の情報を持つことができるので、例えばピアノとボーカルのバラード曲があった場合、オーディオデータだと一緒の波形になるか、個別に用意しないといけませんが、MIDIだとピアノとボーカル、それぞれの値を1つのファイルできちんと取得することができます。 このom_midiは、MIDIのデータをAEにキーフレームの情報として読みこむスクリプトです。 http://omino.com/pixelblog/2011/12/26/ae-hello-again-midi/ヌルレイヤーにスライダー制御エフェクトとして情報が入っています。 このスクリプトも、その1で紹介した「AudioToMarkers」と同様にAEの言語に合わせて中身を書き換える必要があるので注意してください。 また、注意する点として、修正の内容がバージョンによって違います。CS5以上では「スライダー制御」、CS4以前では「スライダ制御」ですので注意してください。 ちなみに、その1と同様でスクリプトの書き換えではExtendScript Toolkitを使用しています。 […]

cheerbot // Mon 2012.01.16 17:035:03 pm

Hey, I was really happy to find this as I’m trying to sync sequences of frames to execute on specific keyframed values of imported midi notes.

Through asking some questions (I’m new to AE), I started a discussion at forums.adobe.com/message/4142986 to get insight.

You may want to take a look, one suggestion is utilizing markers instead of key frames. I would love to see an easier way to automate and include midi values for dynamics and so would like to see this script expanded a little bit, and I would love to get you included as this script seems the easiest way at the moment to get midi data in beyond simple rhythmic uses. Also, on 5.5, deleting original file but easy to work around with a copy of course.

StoryBoy // Tue 2012.01.24 15:353:35 pm

Hi David,

Firstly, this works great!

Is there any chance of getting the JS Script to import CC midi values as well? I’ve got a MIDI file with a PITCHBEND envelope which would be awesome to have in after-effects.

Thanks again!

david van brink // Tue 2012.01.24 16:544:54 pm

SB — Excellent Idea. I’ll add that to the next rev. (Though if you’re inspired and know the MIDI bytes a little, you might be able to add it yourself…)

But yes, PitchBend and Control tracks… Oh the fun!

cheerbot // Tue 2012.01.24 22:4610:46 pm

Hey I just wanted to say thanks again, I’m going to be using this script a lot! I just finished this little thing which was pretty easy to make using the script and expressions your provided.

http://vimeo.com/35613691

I’d also like to point out to others that if you’re going to be doing marker-based animation timemapping off of midi data, these two scripts are incredibly useful paired with David’s script:

Keyframes to Markers:
http://www.redefinery.com/ae/view.php?item=rd_KeyMarkers

Copy markers:
http://www.redefinery.com/ae/view.php?item=rd_CopyMarkers

oh, i dont know. what do you think?


david van brink // Tue 2009.08.4 17:40 // {after effects extendscript}

AE: Scripting Notes

Just a quick note about a bug and an optimization when scripting After Effects CS4 (and probably earlier versions, too).

addProperty() bug

When adding several effects, each addition invalidates the object variable references to earlier ones. Here’s a code fragment which shows the problem and the solution.

var comp = app.project.activeItem;
var layer = comp.layers.addNull();
layer.name = "null_layer";

var slider1 = layer.Effects.addProperty("Slider Control");
slider1.name = "s1";

var slider2 = layer.Effects.addProperty("Slider Control");
slider2.name = "s2";

// At this point, slider2 is valid, but slider1 is mysteriously not!
// Any action or reference to slider1 will cause an "invalid object" error
//
// What can we do?
// 
// Fortunately, they have names by which we can recover them

slider1 = layer.Effects.property("s1");
slider2 = layer.Effects.property("s2");

// Now they're both good to go.

setValueAtTime() Gets very slow!

If you do a whole lot of setValueAtTime() calls to set keyframes, the script will run very slowly. Fortunately, you can just call setValuesAtTimes(), the plural form, to set many at once, which is much more efficient! Makes the minutes seem like seconds, Captain.

// this will be very slow, if myData has more than a few dozen items
for(var i = 0; i < myData.length; i++)
	prop.setValueAtTime(myData[i].t,myData[i].v);

// but if we build up our arrays first...
var timesArray = new Array();
var valuesArray = new Array();
for(var i = 0; i < myData.length; i++)
{
	timesArray.push(myData[i].t);
	valuesArray.push(myData[i].v);
}
// and set them all at once
prop.setValuesAtTime(timesArray,valuesArray);

// it will go lickity-split!

(Thanks creative cow thread!)

oh, i dont know. what do you think?


david van brink // Sat 2008.12.27 10:05 // {after effects extendscript}

AE: A Snowflake

colorfulSnowflake.jpg

download snowflakeSteps.aep

Featuring: Bevel Alpha, omino kaleidoscope, and animated paths.

Sounds like the USA has had a heck of a Christmas weather run! Folks taking their vacations on airport waiting benches, and so forth. In particular, Seattle, home to After Effects. Where I am, it’s 7:33 pm, and I’ve just returned, sweating, from a nice bowl of Mun Eefoo Mee. Total google hits for that dish: 0, but the chef told me it’s his specialty, and it was delicious. Cost, with a cup of iced sweetened barley, 5 Ringitts Malaysian. Current temperature: 80F. Current exchange rate: 3.8 Ringitts to the US Dollar.

It’s my kind of place. “Christmas” is a national holiday, but nobody celebrates beyond a few santas at the mall. On the 25th, the shoe stores and the hardware stores were all open. As were thankfully the restaurants; I’m expated here alone for a few months, no home cooking for this coder.

So, in a fit of boundless atheism, I spent the 25th cobbling together an After Effects project to grow snowflakes!

Step 0: Research

I found this great site, http://snowcrystals.com/ maintained by Kenneth G. Libbrecht, a Caltech physics professor. Fabulous stuff, including movies of lab-grown snow crystals!

After reading his site, and other sources, and general pondering, I jumped in with the scripting. It’s only a 2 day project, so it’s not really going to accurately mimic the physics of snow… but we’ll settle for “suggestive” and “cool-looking”.

(And, here is a short article about snowflake symmetry.)

Step 1: Growth

step1-spikypath.mov

click for
http://omino.com/pixelblog/wp-content/uploads/2008/12/step1-spikypath.mov

This movie shows one branch of a snowflake being “grown”. It’s a layer mask path, animated by ridiculous please-don’t-read-the-code script I wrote… but, truth to tell, you could animate almost anything in there, and, by the end, it’s going to look pretty snow-ish. You can download my script if you like. It’s kinda tweaky; the main trick is that the lengths parameter consists of letters, where a is a very short spike, and z is full length.

Step 2: Hexagonal Symmetry

We all know how to make a paper snowflake, with three or four folds and some scissors. We’ll do the same thing with my plugin, omino kaleidoscope, downloadable here.

(You can also use CC Kaleida, which is included with AE these days, to make an 8-fold snowflake. But use mine for authentic regulation 6-way symmetry.)

The omino kaleidoscope works by using an AE layer’s Mask Path to represent the shape of the mirrors. Any path will work (though curves are treated as straight segments). For snowflake symmetry, we need a wedge which is one twelfth of the pie. Again, I wrote a script, wedgePath to do this. But you could draw the mask manually, if you’re careful. (Be sure to set its mode to “none”, we don’t want it to clip the image.)

And set up omino kaleidoscope as shown in the screenshot above.

Step 3: Moreness

step3_moreness.mov

click for
http://omino.com/pixelblog/wp-content/uploads/2008/12/step3_moreness.mov

So far so good. Next, let’s add a few more layers of animated paths. We’ll superimpose them to get different gray-levels. Some of them are time-shifted, to evolve at different rates. Different transfer-modes can liven this up a little. Again: you can animate almost anything, and the kaleidoscope will make it snowflake-like.

Step 4: Imperfect

step4_imperfect_frame.jpg

To make it look just a shade less mechanical, we can have the snowflake grow a little bit irregularly. Now, it is supposed to be a crystal, so we’ll go gently with this. Real snowflakes often show subtle imperfections like this, presumably due to environmental variations during formation.

The above frame is from Step 3, with Time Displaclement applied. Here’s the layer used as the Time Displacement Map. It ranges from black to 50% grey, ensuring that only negative time displacements will be used.

slownoise.mov

click for
http://omino.com/pixelblog/wp-content/uploads/2008/12/slownoise.mov

Note. Time Displacement is a render-time hog, since AE needs to render many frames past and present for each final frame. It’s helpful to disable this effect while tinkering, turn it back on for the render.

Step 5: Crystallize

This is the last step, really. We’ll use a few applications of Channel Mapper to convert luma to alpha. And then, this is the exciting part, we use Bevel Alpha with several colored lights to get the colorful crystalline look.

The “light angle” for the bevel effect has been animated with an expression

effect("Bevel Alpha")("Light Angle") + time * 90

just to keep it lively.

Also, Channel Mapper has been used to pull a dark grey in for the snowflake’s color. The different brightnesses earlier were mapped to alpha, for the beveling; the final result looks better with colors darker than white.

And I’m not above adding a bit of Find Edges and Glow, no sir. The secret sauce is ketchup. Whatever it takes!

Here’s the final result:

finalMovie.mov

click for
http://omino.com/pixelblog/wp-content/uploads/2008/12/finalMovie.mov

As ever, here’s the After Effects project. It doesn’t require any assets (the finalMovie comp references the photo and music loops, but pay it no mind).
download snowflakeSteps.aep

And so far, it has not been a cold day in Penang, Malaysia.

oh, i dont know. what do you think?


david van brink // Thu 2008.12.25 00:11 // {after effects extendscript}

AE: Mask Vertices from ExtendScript

We’ll take it as axiomatic that scripting After Effects is pretty keen. But sometimes you can get lost in the nest of properties, property groups, values, attributes, and the subtle differences in nomenclature for properties and attributes used by After Effects and the JavaScript.

It is definitely possible to access and manipulate each vertex on a layer mask. Below is a script which displays some mask points, and then modifies a vertex. It will show this alert:

verticesGot.png

And this is the script which displays the dialog, and then modifies the 4th vertex of the mask. It demonstrates the recipe to navigate the Masks, Mask, Mask Path, and so on.

One thing first. Many sites and blogs show only very, very short scripts. I take a slightly different approach. I think code should tell a story. I try to build things up as simply as possible. Despite its apparent length, I think you’ll find the script easy to follow. Just go 1 line at a time.


// Utility to find a comp by name.
function findComp(name)
{
	for(var i = 1; i <= app.project.numItems; i++)
	{
		var item = app.project.item(i);
		if(item != null && item.name == name)
			return item;
	}
	return null;
}

// Show the vertices of the first mask of comp1/layer1.
function main()
{
	var comp = findComp("comp1");
	var layer = comp.layer("layer1");
	var masks = layer.Masks;

	var firstMask = masks.property(1);

	if(firstMask == null)
		return;
	
	var maskPathProperty = firstMask.property("Mask Path");
	var maskPath = maskPathProperty.value; // or valueAtTime(t) if  you like

	var sm = "Vertices\n";
	var vertices = maskPath.vertices; // array of [x,y] pairs
	for(var i = 0; i < vertices.length; i++)
	{
		var p = vertices[i];
		var x = p[0];
		var y = p[1];
		sm += "v[" + i + "] = " + x + "," + y + "\n";
	}

	alert(sm); // Show the vertices.

    // Now, we change a point.
	var p = vertices[3];
	p[1] = 300;
	
	// Must be "put back" bit by bit.
	maskPath.vertices = vertices;
	maskPathProperty.setValue(maskPath);

}

main();

A few notes:

  • To change the vertex, we need to assign it back to the Mask Path and then into the Mask property; changing it "in place" won't alter the actual mask.
  • Chris-g notices that you must assign the vertices before assigning closed to true or false; assigning the vertices "automagically" sets closed to true. Thanks Chris!

Hope that's useful.

oh, i dont know. what do you think?



0.0795s
(c) 2003-2013 omino.com / contact poly@omino.com