omino pixel blog

This is a rendering simulating an image projected onto a simple 3d scene. There’s nothing too exotic about it, except that the projection was done in 2d, in After Effects.

And… here’s a moving sequence of the same scene. With old family movies.

The usual way you’d do this is by rendering a movie, taking it into your 3d program, and projecting it onto a scene through a light source.

For my current project, I needed this effect, but wanted a way to work in After Effects. I did this for workflow reasons: my projection content was all AE authored. But it also turns out to save render time. More than a little.

The trick was to render the scene in 3d as a coordinate-map, in Red and Green. I used Modo, and the projection mode is called “Front Projection”, from a spotlight or camera. I projected a red-green ramp onto the scene.

But we’re going to use the red and green values as a projection function. Changing the shader to use unattenuated (“luminous” @ 1W/steradian in modo) accomplishes this. We’re rendering geometry, not optics.

Then, I wrote a simple Pixel Bender kernel to use the red and green pixel values as coordinates to draw from the source projected image.

// by david van brink / poly@omino.com / 2009
// https://omino.com/pixelblog/
kernel rg_projection
<  namespace : "omino";    vendor : "omino";    version : 1;    
   description : "uses red/green from one image to map another"; >
{
    input image4 src; // Red-green mapping image
    input image4 ref; // image to project
    output pixel4 dst;

    parameter float imageXSize;
    parameter float imageYSize;
    parameter float imageXOffset;
    parameter float imageYOffset;

    void
    evaluatePixel() {
        pixel4 a = sampleLinear(src,outCoord());
        float r = a.r * imageXSize + imageXOffset;
        float g = a.g * imageYSize + imageYOffset;

        dst = sampleLinear(ref,float2(r,g));
    }
}

To test the Pixel Bender kernel, I ran a reference grid through the red-green ramp.

The first one shows gamma correction on the ramp image in After Effects. Under “Interpret Footage”, be sure to turn on “Preserve RGB”.

The second one reveals that Photoshop’s default ramp is some sort of s-shaped modified linear function. Creating a custom gradient with more stops along the way helped get the third, which is Good Enough. I went back and redid the scene with the improved red-green gradient.

And of course, use 32-bit color throughout. 16-bit works ok, too. 8-bit showed definite Squarifying in the projection, though.

To reintroduce the optical nature of the projection, back in After Effects it’s multiplied by a normally rendered image and, because I still think it’s pretty neat idea, an ambient occlusion pass as well.

And there you have it! 3d projections done all in 2d.

Download the AE Project

Here’s another variation on additive lighting tricks…

This one is a little experiment using After Effects’ spotlights on still images which have been rendered with similarly-located light sources.

Here’s one of the several still images.

And here’s a short movie showin a little more of the procedure, and some results.

Long, long ago I saw an article or talk by Paul Haeberli, of SGI. Oh, he covered many interesting and clever graphics tricks. One of them was the idea of “synthetic lighting”, or, “doing lighting in post”.

He has a page about it, here.

The idea is pretty simple. If you have images (or animations) of individual lights, you can add them together to get multiple lights. It appears to be a relatively common render feature these days, but I’d never played with it before. So here goes.

+

=

You can color and mask them, too, to get some pretty nifty effects, all from the same source images.

The pictures above, showing my infantile modeling skills, were done in Luxology modo, using full radiosity render. This can take a while! 10 minutes per low res frame, about.

The youtube video below shows some further examples of getting liveliness out of just a few frames, using this idea of additive post-process lighting.

Here’s a little movie that demonstrates a text track being used to “trigger” the opacity of another track.

Controlling Pip Opacity From Text

I needed a blinking pip. (It’s actually for an interesting optical effect… but for now, we’ll just talk about the pip.) Rather than keyframing each blink, I used a Text track where A is a very dim blink, and Z is a very bright blink. The blinks all have the same luminous decay. Every keyframe on the Text track triggers a new blink.

The expression to do this is actually a little program. (Exactly what some artists hate…)

Here’s the expression for controlling the Pip’s opacity. It’s in 3 parts: 1) Find the most recent Text track keyframe, 2) See what letter it was, from A to Z, and 3) Set the opacity, taking into account the brightness and how long it’s been decaying.

// find most recent key
var s = thisComp.layer("controlTrack").text.sourceText; // score layer
var n = s.numKeys;
var kValue = "A"; // default value
var kTime = 0;
for(var i = 1; i <= n; i++)
{
	var k = s.key(i);
	if(k.time <= time)
	{
		// only care if there's a char in our position.
		var a = k.value.charAt(0).toUpperCase();
		if(a >= "A" && a <= "Z")
		{
			kValue = a;
			kTime = k.time;
		}
	}
}

// now, how long ago was that?
var age = time - kTime;

// map character a-z to 0.0-1.0
kValue = kValue.charCodeAt(0);
kValue -= 65;
kValue = kValue / 25.0;

// decay appropriately
var result = kValue * Math.pow(0.3, age) * transform.opacity;

// done.
result

Why do this tricky control business with a text track, instead of just keyframing the flashes more directly? This way, the behavior is very well specified... and even more so for the next movie.

Controlling Five Pips With a Text Track

You can easily see what's going on here. Five characters control five pip's blinks.

The clever bit was to avoid pasting that monster expression from above into five different tracks. Instead, a single slider control track is driven from the text, and shows the first pip's blinking at time=0, the second pip's at time=300, the third's at time=600, and so on. The whole animation is 20 seconds, so there's plenty of room on the timeline for this.

Each pip's opacity is driven from the appropriate section of this slider channel:

pip0 opacity: transform.opacity * thisComp.layer("controlTrack").effect("scoreReader")("Slider").valueAtTime(time + 0*300)

pip1 opacity: transform.opacity * thisComp.layer("controlTrack").effect("scoreReader")("Slider").valueAtTime(time + 1*300)

pip2 opacity: transform.opacity * thisComp.layer("controlTrack").effect("scoreReader")("Slider").valueAtTime(time + 2*300)

... and so on.

So, why pile this cleverness atop an already tricky expression? Because with an expression that tricky, anything is better than duplicating five times across different tracks!

Here's the expression, which extracts brightness from the text track, for each position, at times separated by 300 seconds.

// Allow different sections of this track to extract the brightness for each pip.
// we divide it up into 5 minute segments (300 seconds). To find out your brightness
// at a particular time, examine *this* track at time + 300 * pipNumber.
//
// This is an alternative to duplicating the rather ornate expression into each pip, at
// the expense, of course, of complexity.
//

var segmentDuration = 300;
var segmentIndex = Math.floor(time / segmentDuration);
var segmentTime = time - segmentIndex * segmentDuration + .01; // local time... plus 1/100 for roundoff

// find most recent key
var s = thisComp.layer("controlTrack").text.sourceText; // score layer
var n = s.numKeys;
var kValue = "A"; // default value
var kTime = 0;
for(var i = 1; i <= n; i++)
{
	var k = s.key(i);
	if(k.time <= segmentTime)
	{
		// only care if there's a char in our position.
		var a = k.value.charAt(segmentIndex).toUpperCase();
		if(a >= "A" && a <= "Z")
		{
			kValue = a;
			kTime = k.time;
		}
	}
}

// now, how long ago was that?
var age = segmentTime - kTime;

// map score character a-z to 0.0-1.0
kValue = kValue.charCodeAt(0);
kValue -= 65;
kValue = kValue / 25.0;

// decay appropriately
var result = kValue * Math.pow(0.3, age);

result

And lastly, here's the self-contained After Effects project.