omino pixel blog

Here’s another variation on additive lighting tricks…

This one is a little experiment using After Effects’ spotlights on still images which have been rendered with similarly-located light sources.

Here’s one of the several still images.

And here’s a short movie showin a little more of the procedure, and some results.

My dad recently transferred all his (our!) old 8mm movie reels to dvd. We’re talking 1957 to 1972 for this first batch. He sent me copies, three dvd’s worth.

It was pretty painless; he used yesvideo.com, which distributes through Costco. I think the pickup and delivery was at Costco. Quality seems a bit blurry, but maybe that’s the source.

Of course, I want to reedit the content a bit, in Final Cut and After Effects and all that. Turns out it was very easy to import the DVD’s into the Mac, in a usable format, with qt_tools!

(Download my command-line qt_tools here.)

The catch is, you need the Apple MPEG-2 Import Component either for $19.99 from Apple, or installed with Final Cut.

All the video lives in a .VOB file. The biggest ones are usually the content you want; the littlest ones are usually menu loops. Using qt_tools, we convert VOB to quicktime. Well, mp4, anyway.

qt_export --dodialog VTS_01_1.VOB ~/mp4s/VTS_01_1.mp4 --savesettings=mp4_pretty_good.st

I found mp4 at about 5000 kbits/second looked ok.

This only works on personal dvd’s, of course. Unencrypted.

Lately been trying 3d apps, looking for my next toy.

Quick recap of 3d software that I’ve used or recently tried:

LightWave. Fluent. This app has everything except proper instancing. Crufty bizarro-world interface… and yet, gets the job done. Expressions and scripting available.

modo. Fluent. Smooth as silk. Clean and modern UI. Like lightwave, but from this universe, instead of the Amiga. Still lacks expressions, limited animation, and no whizbangery (no bones, particles, or dynamics). Yet.

Cinema 4D (c4d). Comes so highly recommended. Only tried it for an hour or two. Lovely modern app. I could see getting a lot done with this. Scripting and expressions. The UI is modern and responsive.

Maya. I want to like it, because I’ve seen great stuff come out of it (from people who are by far better artists than I, alas). But I feel the weight of history in it, it remains inscrutable to me. I have an expert friend who is going to sit down with me and give me a good dose of Getting-It. I hope. I love scripting, that won’t be the obstacle.

Houdini. Love at first click.

Most apps aspire to give you a clean well-lighted crafts studio, with the kiln warmed up, the tools spread out before you.

Houdini gives you a fully stocked laboratory, soldering iron (1), screwdriver (1), wirecutters (1), and a high-voltage supply rail. Go!

This means there’s a high fixed overhead to get anything done. But it sure lends itself to quirky algorithmic experiments.

The picture at the top, and the bottom, are relatively trivial Houdini documents with some stacked operators. The bottom one uses random expressions to control the height of the blocks-stacks. (The stars part is straight from a tutorial on the company site.)

Long, long ago I saw an article or talk by Paul Haeberli, of SGI. Oh, he covered many interesting and clever graphics tricks. One of them was the idea of “synthetic lighting”, or, “doing lighting in post”.

He has a page about it, here.

The idea is pretty simple. If you have images (or animations) of individual lights, you can add them together to get multiple lights. It appears to be a relatively common render feature these days, but I’d never played with it before. So here goes.

+

=

You can color and mask them, too, to get some pretty nifty effects, all from the same source images.

The pictures above, showing my infantile modeling skills, were done in Luxology modo, using full radiosity render. This can take a while! 10 minutes per low res frame, about.

The youtube video below shows some further examples of getting liveliness out of just a few frames, using this idea of additive post-process lighting.

Here’s a little movie that demonstrates a text track being used to “trigger” the opacity of another track.

Controlling Pip Opacity From Text

I needed a blinking pip. (It’s actually for an interesting optical effect… but for now, we’ll just talk about the pip.) Rather than keyframing each blink, I used a Text track where A is a very dim blink, and Z is a very bright blink. The blinks all have the same luminous decay. Every keyframe on the Text track triggers a new blink.

The expression to do this is actually a little program. (Exactly what some artists hate…)

Here’s the expression for controlling the Pip’s opacity. It’s in 3 parts: 1) Find the most recent Text track keyframe, 2) See what letter it was, from A to Z, and 3) Set the opacity, taking into account the brightness and how long it’s been decaying.

// find most recent key
var s = thisComp.layer("controlTrack").text.sourceText; // score layer
var n = s.numKeys;
var kValue = "A"; // default value
var kTime = 0;
for(var i = 1; i <= n; i++)
{
	var k = s.key(i);
	if(k.time <= time)
	{
		// only care if there's a char in our position.
		var a = k.value.charAt(0).toUpperCase();
		if(a >= "A" && a <= "Z")
		{
			kValue = a;
			kTime = k.time;
		}
	}
}

// now, how long ago was that?
var age = time - kTime;

// map character a-z to 0.0-1.0
kValue = kValue.charCodeAt(0);
kValue -= 65;
kValue = kValue / 25.0;

// decay appropriately
var result = kValue * Math.pow(0.3, age) * transform.opacity;

// done.
result

Why do this tricky control business with a text track, instead of just keyframing the flashes more directly? This way, the behavior is very well specified... and even more so for the next movie.

Controlling Five Pips With a Text Track

You can easily see what's going on here. Five characters control five pip's blinks.

The clever bit was to avoid pasting that monster expression from above into five different tracks. Instead, a single slider control track is driven from the text, and shows the first pip's blinking at time=0, the second pip's at time=300, the third's at time=600, and so on. The whole animation is 20 seconds, so there's plenty of room on the timeline for this.

Each pip's opacity is driven from the appropriate section of this slider channel:

pip0 opacity: transform.opacity * thisComp.layer("controlTrack").effect("scoreReader")("Slider").valueAtTime(time + 0*300)

pip1 opacity: transform.opacity * thisComp.layer("controlTrack").effect("scoreReader")("Slider").valueAtTime(time + 1*300)

pip2 opacity: transform.opacity * thisComp.layer("controlTrack").effect("scoreReader")("Slider").valueAtTime(time + 2*300)

... and so on.

So, why pile this cleverness atop an already tricky expression? Because with an expression that tricky, anything is better than duplicating five times across different tracks!

Here's the expression, which extracts brightness from the text track, for each position, at times separated by 300 seconds.

// Allow different sections of this track to extract the brightness for each pip.
// we divide it up into 5 minute segments (300 seconds). To find out your brightness
// at a particular time, examine *this* track at time + 300 * pipNumber.
//
// This is an alternative to duplicating the rather ornate expression into each pip, at
// the expense, of course, of complexity.
//

var segmentDuration = 300;
var segmentIndex = Math.floor(time / segmentDuration);
var segmentTime = time - segmentIndex * segmentDuration + .01; // local time... plus 1/100 for roundoff

// find most recent key
var s = thisComp.layer("controlTrack").text.sourceText; // score layer
var n = s.numKeys;
var kValue = "A"; // default value
var kTime = 0;
for(var i = 1; i <= n; i++)
{
	var k = s.key(i);
	if(k.time <= segmentTime)
	{
		// only care if there's a char in our position.
		var a = k.value.charAt(segmentIndex).toUpperCase();
		if(a >= "A" && a <= "Z")
		{
			kValue = a;
			kTime = k.time;
		}
	}
}

// now, how long ago was that?
var age = segmentTime - kTime;

// map score character a-z to 0.0-1.0
kValue = kValue.charCodeAt(0);
kValue -= 65;
kValue = kValue / 25.0;

// decay appropriately
var result = kValue * Math.pow(0.3, age);

result

And lastly, here's the self-contained After Effects project.

Here are some super short notes about using Pixel Bender inside of After Effects CS4. This post is just another dicing of the available info culled from the After Effects help, and the Pixel Bender Developer Guide.

But it’s the stuff that I most needed to get started, as well as a handy copy-and-paste source for the parameter control types.

This is about Pixel Bender in After Effects specifically.

Installing

Let’s start at the end. To use a Pixel Bender kernel in After Effects, just drop it in the Plug-Ins folder.

• By default, it shows up in the Pixel Bender effects category.
• It won’t show up until the next time you start After Effects
• If it has any syntax errors, AE will warn you as it starts up, and reject the kernel.
• Any changes to the kernel won’t show up until you quit and re-run After Effects. (Purge-all doesn’t seem to reload it…)
• (And you can put it in a subfolder of Plug-Ins, of course, to keep things organized.)

Authoring

Source Image To work with After Effects, near as I can tell, it must take at least one image4 source. Even if the kernel is a pure generator, it needs to take an image input.

Multiple Source Images If the kernel takes more than one source image, the additional sources can be chosen in AE by a layer control. (Grand!)

Control Types By default, all parameters show up as sliders. But you can request a different control in the parameter description.

Example

Here’s an example of a Pixel Bender kernel that just generates a solid color. It has several parameter controls on it of the different supported types, and accepts a second layer as input. These are all ignored! But we can see how the controls are displayed, which is handy.

Here’s a screen capture, with the About box. (Trivial bug, the display name isn’t used in the About box…)

And here’s the code for the kernel, showing the various descriptors.

<languageVersion : 1.0;>

kernel solid_color
<namespace : "omino";
 vendor : "Omino PixelBlog 2009";
 version : 2;
 description : "Generates a solid color.";
 displayname: "Solid Color";
 category: "omino"; >
{
 input image4 src;
 input image4 src2;
 output pixel4 dst;

 parameter float4 color
  <defaultValue: float4(1,0,0,1);
   aeDisplayName: "Color";
   aeUIControl: "aeColor";>;

 parameter float angle
  <defaultValue: 45.0;
   aeDisplayName: "Angle";
   aeUIControl: "aeAngle";>;

 parameter float2 point
  <aePointRelativeDefaultValue: float2(0.5,0.5);
   aeDisplayName: "Point";
   aeUIControl: "aePoint";>;

 parameter int popup
  <defaultValue: 3;
   aePopupString:"Zero|One|Two|Three|Four";
   aeDisplayName: "Popup";
   aeUIControl: "aePopup";>;

    void evaluatePixel()
    {
        dst = color;
    }
}

Onward

With this amount of flexibility, and the instant cross-platform nature of Pixel Bender kernels, the barriers to entry for effects development have been dropped to almost nothing. Great stuff!

Joy!

I finally have returned from several months away, back into the usual routine. My fresh copy of After Effects CS4 was waiting! (Thanks so much, you know who you are!)

Verrrry brief first impressions.

Darker, Darker

I love how each successive version is a bit darker, more mysterious looking. It’s just like Luke Skywalker, remember how in the first movie he’s wearing something like a beige judo gi, and by Jedi he’s all in black and getting rather professional? AE is following the same sequence here.

But seriously, I like it.

(How long til UI-fashion flips back around to rose-petal-white and no visible edges? Maybe when e-paper starts to be the dominant output format. Bring on the new colors for 2020. Tufte said it would be like this.)

Pixel Bender

It’s so simple, just drop your .pbk files into the Plug-Ins folder and they show up in the Effects & Presets list, in the Pixel Bender category. Love it. More to come on this topic.

Still, first impressions: need a way to influence the parameter control rendering in the UI, to get grouping, and point and color controls.

UPDATE Jeff at Redefinery corrects me again. Pixel Bender filters can define the control type for each parameter, like this (excerpted from the latest pixel bender online help):

parameter float blend_with_original
<
  minValue: 0.0;
  maxValue: 100.0;
  defaultValue: 0.0;
  description: "Amount to blend with original input"; //comment only, ignored in AE
  aeDisplayName: "Blend Factor"; //AE-specific metadata
  aeUIControl: "aePercentSlider"; //AE-specific metadata
>;

Legal values for aeUIControl include aeAngle, aePercentSlider, aePoint, aeColor, and a couple others.

Online Help

Hate it. I do know After Effects pretty well… but also, I do flip to the manual not infrequently. I don’t miss the paper manual, not one bit. But I don’t much care for where CS4’s “After Effects Help…” takes you, some catch-all page of community goodies. The community pages are great, and I’m happy when Google lands me up there.

I just want the table of contents. And CS3’s help viewer seemed tighter and snappier.

UPDATE I see this has been fixed in 9.0.1! Help takes you to the contents page.

A Mysterious ASIO Driver

I never saw this before… but Cubase is offering After Effects 9.0 as an available ASIO driver. I have no idea what this means… More research is needed! Possibly a side effect of being able to preview AE out to an ASIO sound card.

Some Tiny Stuff

Todd K at Adobe posted a pretty comprehensive list of CS3->CS4 changes at http://blogs.adobe.com/toddkopriva/2008/09/an_unordered_approximately_com.html. Many of them are small… but cumulatively it’s good stuff.

Here’s one that really, well, really does make a difference. If your view is scaled to 25%, then the resolution is automatically dropped by the same amount.

(You can imagine the deep philosophical discussions on whether that’s the right thing to do… “But when you scale back up, your cache is gone!” But, praise Darwin, these things have to be tried, and by God, we all of us irrevocably ascend towards perfection.)

Overall

So, overall, After Effects 9.0 aka CS4 seems to be an evolutionary step.

As a relatively light recreational user, many of the subtler workflow changes are outside my domain. I don’t have a render farm, nor major pipeline and multi-tool integration issues.

But on the Eye Candy side… I dare say we can still anticipate some startling Pixel Bender innovations over time, enabled by AE9.

Update: Dear Adobe — please acquire and incorporate ObviousFX’s Copy Image plugin as part of After Effects proper.

The plugin, available for Mac and Windows, adds an Edit menu item, Copy Image, which copies the current composition image to the clipboard.

Thanks, ObviousFX and thanks, Rich, for pointing it out for me. Great stuff.

Original post:

Hopefully, someone will point out that this is already in After Effects CS4. But I’m such a luddite that I haven’t upgraded or even played with AE CS4 yet.

Sometimes, I use After Effects as a really powerful drawing program, no animation. It works in a very… engineer-friendly way, for compositing. I like that I can concoct a tree of layers, and modifiers on those layers, and use expressions to “rig” them together.

Here’s my feature request:

Copy Frame, High Quality

It’s pretty simple. It would place the current frame of the current comp at either full resolution, or maybe current resolution, as a PNG with transparency(or whatever is appropriate, here in 2009), onto the clipboard, ready to be pasted into another drawing app (or an email or whatever).

I realize with modern graphics displays and screen grabs you can get pretty good pastes… but you’ll have to crop carefully, maybe rearrange the workspace to get it visible, &c. And there’s render-to-file, but what a bother.

So I’d like Copy Current Frame as a feature, please. Thanks!

download snowflakeSteps.aep

Featuring: Bevel Alpha, omino kaleidoscope, and animated paths.

Sounds like the USA has had a heck of a Christmas weather run! Folks taking their vacations on airport waiting benches, and so forth. In particular, Seattle, home to After Effects. Where I am, it’s 7:33 pm, and I’ve just returned, sweating, from a nice bowl of Mun Eefoo Mee. Total google hits for that dish: 0, but the chef told me it’s his specialty, and it was delicious. Cost, with a cup of iced sweetened barley, 5 Ringitts Malaysian. Current temperature: 80F. Current exchange rate: 3.8 Ringitts to the US Dollar.

It’s my kind of place. “Christmas” is a national holiday, but nobody celebrates beyond a few santas at the mall. On the 25th, the shoe stores and the hardware stores were all open. As were thankfully the restaurants; I’m expated here alone for a few months, no home cooking for this coder.

So, in a fit of boundless atheism, I spent the 25th cobbling together an After Effects project to grow snowflakes!

Step 0: Research

I found this great site, http://snowcrystals.com/ maintained by Kenneth G. Libbrecht, a Caltech physics professor. Fabulous stuff, including movies of lab-grown snow crystals!

After reading his site, and other sources, and general pondering, I jumped in with the scripting. It’s only a 2 day project, so it’s not really going to accurately mimic the physics of snow… but we’ll settle for “suggestive” and “cool-looking”.

(And, here is a short article about snowflake symmetry.)

Step 1: Growth

This movie shows one branch of a snowflake being “grown”. It’s a layer mask path, animated by ridiculous please-don’t-read-the-code script I wrote… but, truth to tell, you could animate almost anything in there, and, by the end, it’s going to look pretty snow-ish. You can download my script if you like. It’s kinda tweaky; the main trick is that the lengths parameter consists of letters, where a is a very short spike, and z is full length.

Step 2: Hexagonal Symmetry

We all know how to make a paper snowflake, with three or four folds and some scissors. We’ll do the same thing with my plugin, omino kaleidoscope, downloadable here.

(You can also use CC Kaleida, which is included with AE these days, to make an 8-fold snowflake. But use mine for authentic regulation 6-way symmetry.)

The omino kaleidoscope works by using an AE layer’s Mask Path to represent the shape of the mirrors. Any path will work (though curves are treated as straight segments). For snowflake symmetry, we need a wedge which is one twelfth of the pie. Again, I wrote a script, wedgePath to do this. But you could draw the mask manually, if you’re careful. (Be sure to set its mode to “none”, we don’t want it to clip the image.)

And set up omino kaleidoscope as shown in the screenshot above.

Step 3: Moreness

So far so good. Next, let’s add a few more layers of animated paths. We’ll superimpose them to get different gray-levels. Some of them are time-shifted, to evolve at different rates. Different transfer-modes can liven this up a little. Again: you can animate almost anything, and the kaleidoscope will make it snowflake-like.

Step 4: Imperfect

To make it look just a shade less mechanical, we can have the snowflake grow a little bit irregularly. Now, it is supposed to be a crystal, so we’ll go gently with this. Real snowflakes often show subtle imperfections like this, presumably due to environmental variations during formation.

The above frame is from Step 3, with Time Displaclement applied. Here’s the layer used as the Time Displacement Map. It ranges from black to 50% grey, ensuring that only negative time displacements will be used.

Note. Time Displacement is a render-time hog, since AE needs to render many frames past and present for each final frame. It’s helpful to disable this effect while tinkering, turn it back on for the render.

Step 5: Crystallize

This is the last step, really. We’ll use a few applications of Channel Mapper to convert luma to alpha. And then, this is the exciting part, we use Bevel Alpha with several colored lights to get the colorful crystalline look.

The “light angle” for the bevel effect has been animated with an expression

effect("Bevel Alpha")("Light Angle") + time * 90

just to keep it lively.

Also, Channel Mapper has been used to pull a dark grey in for the snowflake’s color. The different brightnesses earlier were mapped to alpha, for the beveling; the final result looks better with colors darker than white.

And I’m not above adding a bit of Find Edges and Glow, no sir. The secret sauce is ketchup. Whatever it takes!

Here’s the final result:

As ever, here’s the After Effects project. It doesn’t require any assets (the finalMovie comp references the photo and music loops, but pay it no mind).
download snowflakeSteps.aep

And so far, it has not been a cold day in Penang, Malaysia.

We’ll take it as axiomatic that scripting After Effects is pretty keen. But sometimes you can get lost in the nest of properties, property groups, values, attributes, and the subtle differences in nomenclature for properties and attributes used by After Effects and the JavaScript.

It is definitely possible to access and manipulate each vertex on a layer mask. Below is a script which displays some mask points, and then modifies a vertex. It will show this alert:

And this is the script which displays the dialog, and then modifies the 4th vertex of the mask. It demonstrates the recipe to navigate the Masks, Mask, Mask Path, and so on.

One thing first. Many sites and blogs show only very, very short scripts. I take a slightly different approach. I think code should tell a story. I try to build things up as simply as possible. Despite its apparent length, I think you’ll find the script easy to follow. Just go 1 line at a time.

// Utility to find a comp by name.
function findComp(name)
{
	for(var i = 1; i <= app.project.numItems; i++)
	{
		var item = app.project.item(i);
		if(item != null && item.name == name)
			return item;
	}
	return null;
}

// Show the vertices of the first mask of comp1/layer1.
function main()
{
	var comp = findComp("comp1");
	var layer = comp.layer("layer1");
	var masks = layer.Masks;

	var firstMask = masks.property(1);

	if(firstMask == null)
		return;

	var maskPathProperty = firstMask.property("Mask Path");
	var maskPath = maskPathProperty.value; // or valueAtTime(t) if  you like

	var sm = "Vertices\n";
	var vertices = maskPath.vertices; // array of [x,y] pairs
	for(var i = 0; i < vertices.length; i++)
	{
		var p = vertices[i];
		var x = p[0];
		var y = p[1];
		sm += "v[" + i + "] = " + x + "," + y + "\n";
	}

	alert(sm); // Show the vertices.

    // Now, we change a point.
	var p = vertices[3];
	p[1] = 300;

	// Must be "put back" bit by bit.
	maskPath.vertices = vertices;
	maskPathProperty.setValue(maskPath);

}

main();

A few notes:

• To change the vertex, we need to assign it back to the Mask Path and then into the Mask property; changing it "in place" won't alter the actual mask.
• Chris-g notices that you must assign the vertices before assigning closed to true or false; assigning the vertices "automagically" sets closed to true. Thanks Chris!

Hope that's useful.