Generative sketches

Some sketches made with openFrameworks, trying out some generative shapes
and random color generators.

Actually I can't wait till I make this creature in the image
below interactive/responsive to external forces (wind?)

openFrameworks freakday S03E06 - entoforms

Again an inspiring creative coding meeting in Amsterdam today! Got to talk with the enthusiastic Dolf Veenvliet. He's working on a project in a field I really love: generated "things" based on nature.... In his case: animal like creatures. Checkout his website to find more interesting information about this entoform project  He explained me how his codes works and how he uses Blender to generate the bodies. His program give birth to this creatures in a very smart way! I really like this project and got me finally started with coding something which I want to code for a long long time "sentinels" from the matrix :)

I got a rough version working which allows me to generated shapes and extrude these with openFrameworks. 

Flash and openFrameworks

Finally I can remove this form my todo list.  I got several questions about installing ofxFlashCommunication. ofxFlashCommunication is a fast, tiny and clean flash server which you can embed into your openFrameworks application. The previous version was a bit difficult to setup if you were not used to adding libraries, include paths, link to libraries etc... Therefore I rewrote the addon so it uses the latest Poco networking code which is part of the github version of openFrameworks and which will be part of the new openFrameworks release (007) that will be released any day now.  You can download the ofxFlashCommuncation addon from github or download it directly using this link.

Here is a small presentation which explains how to use it:

Physically based sketches

Using position based dynamics, particles and springs to generate 3D meshes then I export them to the Ply file format which is great for these kinds of things! I really love the way the shapes deform while running my application and applying forces to it! 

I.... Need....To....Spend...Time...On...1MLN...Faces...But...

...but I wanted to test the Wave Generators for Computer Graphics as described in this article. These generators are fast (!) and easy to use. Their just plain 1D which makes them actually really handy for lots of generative art/visualisations. These are just some sketches I made while working on these wave generators.  The first image just shows the three different generators which are described.

Testing light ray shader

While testing several visualisations for the 1.000.000 faces project I got taken away with this light ray shader. A couple of screenshots:

WTF

While going through FlipPhotos on my ipad I stumpled upon a picture about this amazing cool new hype which I heard about a couple of months ago called planking... oh yeah and also saw this other photo from uhhhh... oh yeah Sebastiao Salgado

Color histogram

I just sat down to take another look at my 1 million faces project.  At some point I start using these images to create some sort of visualisation based on color, texture roughness, face position, pose etc... I just created some code which calculates a color histogram and has two ways to compare them: Chi Square and the correlation method, kindly borrowed from openCV.  At this moment I've got the basics which will get me started to create some basic structureing of the images. 

Orientation histogram

There are numerous ways one can describe an image. You can use color histograms for example to describe the general colors of an image. You create a histogram with a X-number of bins and sample the colors of each pixel, calculate into which bin it belongs and count how many times a color is stored in a particular bin. This works amazingly well and is used by i.e. Google Image search when you want to fetch images with a particular color. Another solution is to create orientation histograms. An orientation histogram calculates the gradient for each pixel in the image and and stores the angle of the gradient into bins.  Recently I launched my 1.000.000 faces project. I'm collecting/indexing 1.000.000 faces with the help of "you".  So what can you do with these faces? .... exactly! You can sort them by color or orientation histogram :)  I spent some time this evening to figure out how to create these orientation histograms and how these look like for particular images.  I noticed that this can be a good solution for detecting clean images with just a face and a regular background (w/o any texture or noise).

Test application created using openFrameworks

This is still work in progress... I found some wierd round-off error when creating the histograms. I'll share my code when it works as it's supposed to.

Faces of twitter

Created an application which downloads avatars from Twitter and detects faces in them. Nice to see how extremely well openCV face detection, using Haar features works.  I used the default "frontal face" haar xml file.  In a couple of hours I downloaded 100.000 avatars and in the screenshot below you see a generated image named "Faces of Twitter"

Click here to see a bigger version

Update 2011.06.11, some specs I can use:

• Sort by location / geo
• Sort by number of tweets, friends and statuses
• Sort by number of followers
• Sort by profile color
• Sort by number of favourites
• Sort by timezone
• Sort by real name

Facebook like button with dynamic content, AJAX

For a project I had to add Facebook like buttons to a page which uses AJAX a lot.  After going through the Facebook documentation it became clear they did not focus on like buttons which are used on pages that are loaded with AJAX.  In most cases when you click on a like button, you send a request to the facebook servers. Then the facebook servers reads in the HTML of the page you want to like. It parses the HTML and uses the Open Graph meta tags if it finds those; else it uses some content it finds on that page.

After doing some tests I figured out how to create like buttons on the fly using the XFBML markup language and especially their parse function.  The only thing was, that I did not wanted to create a page for each of the possible like items I've got. Therefore I created some code which creates a empty HTML page that contains all the necessary Open Graph tags it needs. Together with a small Javascript class (using Mootools, but easily portable to some other lib), I created a handy tool to create custom like button on the fly with support for custom images, descriptions, etc..

Download the Facebook Like button generator

openFrameworks freakDay S03E04

Again spending a wonderful day with some bright creative people during the openFrameworks freakDay! Thanks everyone for being there and sharing the good vibes!  We did a quick intro to git ... which I hope wasn't to vague. 

Position based dynamics

Instead of working on a regular particle system, I tried to get one working which is based on the paper "position based dynamics" (PBD). With the help from Micky Kelager who wrote a paper on a bending constraint which can be used with PBD I was able to program a very nice smooth cloth application.  PBD is the way I'll go in the future when working on particle systems; it's fast and stable.

Ecoline as input for generative art

At Apollo, we're researching the relation between offline and online. For an experiment on generative visualisations we had the opportunity to work with the great designer TBizzal.  We recorded the effect of ecoline (ink) which flows through water and wet paper as input for generative applications.  Martin imported the recorded video and used it as a texture on a bump/height-mapped sphere. The effects are amazing. Using this kind of offline-generated input gives a nice natural effect.

1$, protractor and protractor3D gesture recognition

For a demo project I'm implementing a couple of gesture recognizers. The simplest to implement, accompanied by a well written and clear paper is the so called "One Dollar Gesture Recognizer". This 2D-gesture (although there are 3D versions of it) recognizer is exteremelly fast and has like a 99% recognition rate.  After implementing this recognizer as a gesture recognition addon for openFrameworks I was  hooked and searched for a 3D gesture recognizer called Protractor3D.  Blown away by the math in this paper, I just started implementing it as most of the techniques used are similar to the $1 recognizer.  

Fog shader - scratchable surfaces

For a long time I've wanted to make a interactive scratchable surface and it seems that I'm almost there.  The nice people of the Alexandra Institute posted an article on a technique to create steam/fog which you can scratch away.  Based on this article I started coding in openFrameworks and together with that post I created a "scratchable" surface app.  The app. shows an image and slowly fades away using a gradient image. This gradient is used by the shader as a "value" map.  The values are used to check how fast the fog should grow on a certain x/y coordinate. 

EYE-SEE-YOU (generated eye)

After I stumbled upon this amazing eye animation created using Blender3D I was thinking how immersive such an eye would be when it would react on people when walking in front of a screen? The dilating of the pupil, makes this eye more humanly than I could have thought.  So I started thinking on how to create such a eye with code so I can control the dilation of the pupil and the movement of the eye based on some interaction.  To make it a bit more easy I'm creating two meshes; one for the pupil and one sphere for the eye ball. The eye is almost finished and the next step will be hooking up ofxOpenNI (kinect, NITE) and do some simple image analysis to track a user, get his distance and base the dilating of the pupil on these values. 

I started coding a sphere based on spherical coordinates as described by Wikipedia.  I could have used the (deprecated gluSphere, though I wanted to be able to tweak the sphere myself.. for example an eye is not completely round. When generating the vertex positions myself I was able to tweak the values to give the eye a shape I want.  I had to rewrite the sphere class a bit because I had to use quads for texturing and needed to find the correct vertex indices.  The pupil mesh is basically just a tiny part of the sphere. I control the dilation by changing the theta value of the spherical coordinates. By incrementing this theta value I change the step size of each stack. 

After I got the sphere and pupil meshes working I started with the vertex and geometry shaders. I had done some simple Lambertian (diffuse) shading before but now I had to use normalmapping to give the eye some sort of rough structure and to make the veins pop-out. Also I use a cubemap to give the eye a "glossy" effect. 

I still have to add the user-tracking code which I'll do as soon as I've got some more time.

openFrameworks freakDay workshop

Today we had our second openFrameworks freakDay meeting of 2011. Different from last year, we're a bit more "organized". The workshop of today was the actual first meeting where a couple of participants created some amazing presentations on topics related to the field of creative coding.  Martin Bartels gave a talk about a bunch of fantastic practical addons he created for a couple of projects he did including fast communication between openFrameworks and AIR (Flash), a state manager, and natural language processing.  Tim Olden from blendid gave us some insights on a great project called "Computational Candles" where he uses a kinect to detect motion and uses this information to toggle the color/brightness of 75 lights! Rick Companje, lecturer at the university of Leiden talked about bump-mapping/displacement shaders which are a ways to add depth effects on 3D surfaces. He also showed us a bit of the insights of his project called Globe4D especially how he uses multiple trackball mouse to calculate a precise position of the "ball" (see link). I gave some information on C++ and how to use it in a generic way.  I also explained the basic physics laws of Newton and how to simulate clothes on a computer. Rick removed the C++ template code and put it online for download here

Computational Candles

Playing with openFrameworks

A couple of screenshots of quick-tests I made while playing and testing with openFrameworks.

openFrameworks Developers Conference Pittsburg

Last week has been amazing! Working together with these talented researchers/artists/developers was a "i-want-more" experience. Here a couple of pictures of a week full of coding and fun in the Studio for Creative Inquiry of the Carnegie Mellon University lead by Golan Levin.