omino pixel blog

Early this month, we visited the San Jose Museum of Quilts and Textiles, featuring patterns and clothing by the Hawaiian designer Alfred Shaheen. Great stuff! Yeah, I have a closet full of Hawaiian shirts.

(Show show goes til August 8, worth visiting. On the first Friday of the month, many of the galleries on First Street in San Jose, CA are open late, definitely a nice night out.)

Since I adore repetition, and halftoning, I got to wondering if I could recreate some of the patterns programmatically. The “Omino Stamper” filter runs in After Effects and attempts to do this.

download ==> omino_stamp.pbk, an After Effects filter.

The idea is pretty simple: draw a “stamp image” in a pattern which can be affected by the source image. It’s a little bit like a particle effect, but more mechanical.

Here’s the controls and few examples, which look nothing like Hawaiian shirts.

A quick rundown:

• The pattern- settings adjust the stamping grid
• The stamp- settings adjust orientation and size of the stampimage image
• The stampOverlaps setting controls how many nearby stamps are computed for each pixel; slows down the render a bit but is necessary for closely-spaced stamping. (If you see missing corners and such, increase this setting. (Those of you familiar with Pixel Bender will immediately apprehend why this is so.
• The affectsStampSizeAmount, affectsStampColorAmount, and affectsStampRotationAmount settings control the halftoning features of the filter.

Here’s a basic stamp:

Next, we adjust the bricking a little, to make it less square-y:

And let the color of the underlying image affect the stamp color:

How about affect the rotation, instead of the color? Kinda wild:

Or back to basics, let the underlying image affect the stamping-size:

Here’s a festive little animation playing with the effect a bit, and does include a Tiki god or two.

download ==> omino_stamp.pbk, an After Effects filter.

Enjoy!

download ==> omino_spindala_32.pbk, an After Effects filter.

Always doing things the hard way. Needed a wheel with some symmetry, like so:

How else to do it than by drawing a circle with some blobs, and then treating it with… a handwritten pixel bender kernel:

I started describing its controls, but wouldn’t it be more fun to just try it out? Below is a slightly reduced version of the Pixel Bender kernel running in Flash.

Give it a spin! Try the parameters, or click Scramble a few times, or Go, to let it choose random parameters.

There’s no shortage of vaguely-trippy image manglers, these days, but that’s no reason not to indulge.

download: ==> omino staragon 32 <== a pixel bender filter

Just recently, Dale Bradshaw posted a rounded-rect pixel bender plugin. Pretty nifty!

It got me thinking, What about other rounded-corner polygons? It was a bit of a puzzle. As with all Pixel Bender plugins, the question is, What color is this pixel? How does a given pixel know if it’s inside a rounded-corner pentagon or not?

I stuck to regular and stellated polygons to start. One solution goes like so:

• Divide the plane into pie-slices, one for each edge of the polygon.
• For each point, decide which slice you’re in.
• Decide if you’re inside or outside the edge.
• For the rounding… decide which corner you’re nearest
• Are you in the arc of that corner? If so, decide if you’re inside the rounding-radius.

Well, that was a bit of obsessive fun. Here’s a screen shot, including the parameter controls:

And here’s a little demo of it.

download: ==> omino staragon 32 <== a pixel bender filter

download: ==> omino glass 32 <== a pixel bender filter

There was a flurry of interest in "chromatic aberration" a while back, like here. I’d actually played with it a bit, simulating lens-effects, but only now have gotten around to weighing in on it.

The implementations I’ve seen work by splitting out your RGB and moving them around a little bit independently.

Here’s my weigh-in: Come on, people! The spectrum has more than three colors!

I find that five to seven is about right. I’ve implemented this combined with refraction into a pixel bender plugin.

Refraction happens when a ray of light changes its direction as it passes from one substance to another, if they have aaah, varying, that is, if they have different, um, refractive indexes. And that’s how aaah magnifying glasses work. See.

It looks like this.

The blue lines are rays of light moving up, or cast-sight-rays going down. It’s all very technical.

But I’ve got this Pixel Bender filter, right?

The filter takes in two images: a bump map (the refracting surface) and an image to view through the refraction. Here’s a bump map:

And here’s a source image:

Those chairs are known as “Chadwick Modular Seating”, from Herman Miller, designed by Don Chadwick, who later designed the Aeron chair, and that room hasn’t been that empty for, like, six years, after my girlfriend moved in and filled it up with shoes and stuff. So anyway.

And here’s the image, through the omino_glass Pixel Bender filter:

Now the fun part. Chromatic aberration happens when different colors are refracted by different amounts. Here’s the same image with just a touch of chromatic aberration applied:

Livens it up a little bit, yes?

The “flat” parts of the image, under the non-sloping parts of the bump map, appear undistored, and uncolored. But the distorted parts of the image have their colors splayed out a bit.

That’s chromatic aberration, comma, simulated.

By the way, you can get this effect in After Effects using CC Glass and overlaying multiple tinted copies, with slightly different settings for Height, and adding them back together. But that’s a lot of layers and settings to juggle; my pixel bender filter does it all in one at 32bpc instead of 8.

Here’s a test grid set with no basic refraction, and lots of chromatic aberration. This leaves the central green portion of the spectrum centered on the original image.

The inset image is the bump map.

We can see how omino_glass breaks the spectrum up into seven regions. It’s arbitrary, but I like the look.

And here’s a little demo showing the filter in motion, animated in After Effects, out on the ‘Tube.

download: ==> omino glass 32 <== a pixel bender filter

Other chromatic aberrations of interest, from some of my favorite blogs:
ae-tuts
maltaannon
satya meka

Here are some super short notes about using Pixel Bender inside of After Effects CS4. This post is just another dicing of the available info culled from the After Effects help, and the Pixel Bender Developer Guide.

But it’s the stuff that I most needed to get started, as well as a handy copy-and-paste source for the parameter control types.

This is about Pixel Bender in After Effects specifically.

Installing

Let’s start at the end. To use a Pixel Bender kernel in After Effects, just drop it in the Plug-Ins folder.

• By default, it shows up in the Pixel Bender effects category.
• It won’t show up until the next time you start After Effects
• If it has any syntax errors, AE will warn you as it starts up, and reject the kernel.
• Any changes to the kernel won’t show up until you quit and re-run After Effects. (Purge-all doesn’t seem to reload it…)
• (And you can put it in a subfolder of Plug-Ins, of course, to keep things organized.)

Authoring

Source Image To work with After Effects, near as I can tell, it must take at least one image4 source. Even if the kernel is a pure generator, it needs to take an image input.

Multiple Source Images If the kernel takes more than one source image, the additional sources can be chosen in AE by a layer control. (Grand!)

Control Types By default, all parameters show up as sliders. But you can request a different control in the parameter description.

Example

Here’s an example of a Pixel Bender kernel that just generates a solid color. It has several parameter controls on it of the different supported types, and accepts a second layer as input. These are all ignored! But we can see how the controls are displayed, which is handy.

Here’s a screen capture, with the About box. (Trivial bug, the display name isn’t used in the About box…)

And here’s the code for the kernel, showing the various descriptors.

<languageVersion : 1.0;>

kernel solid_color
<namespace : "omino";
 vendor : "Omino PixelBlog 2009";
 version : 2;
 description : "Generates a solid color.";
 displayname: "Solid Color";
 category: "omino"; >
{
 input image4 src;
 input image4 src2;
 output pixel4 dst;

 parameter float4 color
  <defaultValue: float4(1,0,0,1);
   aeDisplayName: "Color";
   aeUIControl: "aeColor";>;

 parameter float angle
  <defaultValue: 45.0;
   aeDisplayName: "Angle";
   aeUIControl: "aeAngle";>;

 parameter float2 point
  <aePointRelativeDefaultValue: float2(0.5,0.5);
   aeDisplayName: "Point";
   aeUIControl: "aePoint";>;

 parameter int popup
  <defaultValue: 3;
   aePopupString:"Zero|One|Two|Three|Four";
   aeDisplayName: "Popup";
   aeUIControl: "aePopup";>;

    void evaluatePixel()
    {
        dst = color;
    }
}

Onward

With this amount of flexibility, and the instant cross-platform nature of Pixel Bender kernels, the barriers to entry for effects development have been dropped to almost nothing. Great stuff!

Tweaks and improvements to the previous diced sphere Pixel Bender kernel.

Just click Go. Really.

As before, it uses a sort of orthogonal ray tracing technique. The depth into a particular cell determines the amount of darkness. In this sense it’s a short cut for HDRI and radiosity in that, simply, the further into the “cave”, the darker it is. Seems to work.

This post will show a handful of simple tricks for “drawing” using Pixel Bender. That’s not what it’s for, especially in Flash. But it’s still interesting to see how it’s done.

But first, the demo:

And I’m sure the handy tabs across the top for the source code won’t escape your attention, either!

Pixel Bender Metadata

In the demo above, you can notice a couple of things. Some of the parameters are drawn as sliders, but also there’s a checkbox, some color pickers, and, in the drawing area, some draggable spots. This is done by a collusion between the Pixel Bender kernel, and the Flash viewer code. Here’s the parameters from the Pixel Bender kernel:

    parameter float3 backgroundColor<kind:"color";defaultValue:float3(1,1,1);>;

    parameter float spotRadius<defaultValue:50.0;minValue:1.0;maxValue:800.0;>;
    parameter float3 spotColor<kind:"color";defaultValue:float3(0,0,0);>;
    parameter float spotSquare<kind:"checkbox";defaultValue:0.0;>;

    // This parameter just gets poked with the rendering dimensions, always
    // Doesn't get shown to user. In my custom viewer, that is.
    parameter float2 dims<kind:"dstsize";defaultValue:float2(300,300);>;

    // "point" means it lives somewhere in the image, as a coordinate.
    parameter float2 lineStart<kind:"point";minValue:float2(0,0);maxValue:float2(400,400);defaultValue:float2(10,10);>;
    parameter float2 lineEnd<kind:"point";minValue:float2(0,0);maxValue:float2(400,400);defaultValue:float2(30,100);>;
    parameter float lineWidth<minValue:0.5;maxValue:30.0;defaultValue:2.0;>;

I’ve added a kind property to some of the parameters. Pixel Bender ignores it, but in ActionScript we can access it as part of the ShaderData. For example, it is true that shader.data.spotSquare.kind == "checkbox".

These aren’t special or magic! My Pixel Bender kernel says it, and my Flash viewer reads it. It’s a handy trick, if you happen to be in control of both parts.

How to Draw a Circle

It’s so easy! Given a center and a radius, you check each pixel to see if it’s within that distance of the center. Like so:

        float d = distance(co,center);
        if(d < spotRadius)
            dst = spotColor;

How to Draw a Square

This is ever so slightly trickier. Again, with a center and a radius, we want to see that both the x- and y-distances from the center are within the radius. Like so:

        float d = max(abs(co.x - center.x),abs(co.y - center.y));
        if(d < spotRadius)
            dst = spotColor;

How to Draw a Line

Now this is getting mathy. In an earlier post about Gradient fills, I derive an equation for “position between two points”, like so:

t = dot(uv,xy) / dot(xy,xy)

where a line segment is given by the endpoints (0,0) and (x,y), and some screen spot is given by (u,v). If t is within the range 0 to 1, then (u,v) is somewhere along that line segment if it’s drawn infinitely wide. That’s a short stubby line no matter how you slice it.

To constrain the width of the line, we need to know how far from the center (u,v) is, which is given by:

d = abs(length(xy) * dot(uv,yx) / dot(xy,xy))

(If you followed the Gradient post, it’s the right half — the y-component — of the final 2×2 matrix, times the length of the line, for scaling. But it’s not important.)

So, the combined code to draw a line between two endpoints, with a particular line width, and square-capped ends, is:

       // and now, a bit of fun math, as we plot a line between two points.
        float2 uv = co - lineStart;
        float2 xy = lineEnd - lineStart;
        float xy2 = dot(xy,xy);
        float g = dot(uv,xy) / xy2;
        // if we are between the two points, then g is between 0 and 1.
        float dl = abs(dot(uv,float2(-xy.y,xy.x)) / xy2 * length(xy));
        // dl is now the distance, in pixels, from outCoord to the line

        // if we're between the endpoints and close enough, draw.
        if(g >= 0.0 && g <= 1.0 && dl < lineWidth / 2.0)
            dst = float3(1,0,0); // draw the line in RED.

Ridiculous?

To reiterate: This is a ridiculous use for Pixel Bender. But the ideas may come in handy. Doing something with a squarish area-of-influence isn’t so outlandash… and while a solid-colored line is a bit unsubtle, a shaded field or beam area begins to seem plausible. Pixel Bender is so powerful and flexible, I know we’re going to continue to see unexpected and novel results from it.

Back to the cubes. This Pixel Bender generator shows a volume subdivided into cubes, and constrained to a spherical limit.

And here’s the source code.

Can’t wait to start incorporating this stuff into After Effects CS4… but haven’t uncovered any details on exactly how Pixel Bender integrates to it. Presumably some magic parameters may tell you things like the source or dest extant, and maybe you can add “typing” information to parameters. Looking for links… since it may be a little while before I can afford to upgrade.

I was trying to do something else. But, as you well know, sometimes bugs are really cool. Here’s my bug of the night. Click Go, once, and Scramble a few times.

Geometrical interpretation: a perfectly nonreflective sphere is in each cubical cell, getting cross-sected just like the cubes.

And here’s the source code. It’s based on “cubes” from a few posts back.

Featuring mix(), clamp(), and dot().

Very often I need a simple linear-gradient in After Effects, and end up sing the four-color gradient because that’s all I can find. And so, here, for your perusal, is a Pixel Bender implementation of a Linear Gradient Generator.

After that, we’ll see the code, and after that, a quick review of the math.

Here is the .swf. (With some improvements on the Pixel Bender viewer — you can drag any float2 parameters as points, now.)

Isn’t that fun? Well, I’m easily amused.

The Code

Here’s the Pixel Bender code.

kernel Gradient
{
    output pixel4 dst;
    parameter float2 p1<minValue:float2(0,0);maxValue:float2(400,400);defaultValue:float2(10,10);>;
    parameter float2 p2<minValue:float2(0,0);maxValue:float2(400,400);defaultValue:float2(130,130);>;

    parameter float3 color1<defaultValue:float3(0,0,0);>;
    parameter float3 color2<defaultValue:float3(1,1,1);>;

    void evaluatePixel()
    {
        float2 co = outCoord();

        // shift everything relative to p1.
        float2 uv = co - p1;
        float2 xy = p2 - p1;

        // the math.
        float g = dot(uv,xy) / dot(xy,xy);
        g = clamp(g,0.0,1.0);

        // the color.
        dst.a = 1.0;
        dst.rgb = mix(color1,color2,g);
    }
}

The Math

I’m ok at math, but not always fluent in it. I can remember what a matrix is, but not necessarily exactly what a cross-product is. Maybe you’re like me. In which case this quick run through the math and logic behind the gradient fill will be quite useful in your own Pixel Bender bending!

And be assured, a few pages of high-school-algebra-style scribblings, and the occasional google (How do I invert a rotation again? What’s the formula for perspective?) really can lead to workable results. Good clean fun!

Let’s go.

Consider a trivial gradient, where our two control points are at (0,0) and (1,0). That would be easy! Just take the x-value of any pixel, and that’s your color. (That is, the position on the ramp between your two gradient colors.)

By the way, this picture is from Ron Avitzur’s “Graphing Calculator”. The story of its development is legendary; less well known is that it’s actually a very powerful and useful tool.

I found the easiest way to think about a general two-point gradient was to ask, “How do we rotate and scale our control points back to this trivial gradient?” And let’s assume that the first control point is always at (0,0).

Here is matrix for a basic rotation, which moves any input point by a rotation around (0,0):

[1]

A rotate-and-scale matrix looks like that, times a constant on each element.

We want to think of our general gradient as a transformation on the trivial gradient. Let’s say our second control point is (X,Y), and our first control point is (0,0) as mentioned above. The rotate-and-scale matrix being applied, conceptually, is:

[2]

To figure out the 0-to-1 trivial position of any of our input points, we need to invert it. A matrix inversion looks like:

[3]

So, we invert the second equation [2] by plugging it in to [3] and get:

[4]

Now, let’s call each pixel position of our output (U,V). For our gradient, we actually don’t care about the resulting Y position; just X gives us our gradient value. So we get:

[5]

Now, the definition of the dot product of (A1,A2,A3,…) and (B1,B2,B3,…) is (A1B1 + A2B2 + A3B3 + …). So, from the above, UV+XY is, conveniently, (U,V) dot (X,Y). Similarly, X² + Y² is (X,Y) dot (X,Y).

And now, when you read this part of the code a second time, it should be much clearer what’s going on:

    void evaluatePixel() {
        float2 co = outCoord();

        // shift everything relative to p1.
        float2 uv = co - p1;
        float2 xy = p2 - p1;

        // the math.
        float g = dot(uv,xy) / dot(xy,xy);
        g = clamp(g,0.0,1.0);

        // the color.
        dst.a = 1.0;
        dst.rgb = mix(color1,color2,g);
    }

Simple and useful!

Next up, soon, more cubes.