Testing a camera controlled golddust particle/interaction system for upcoming music video.

Programmed in C++ with openFrameworks/openCV/openGL running realtime at 1080 HD, 30fps with 20K particles, optical flow, contour analysis, fluid dynamics, FBOs and VBOs.

All particle interaction is controlled entirely by motion on camera.

Most of the particles and fine detail is lost in the video compression, you can see much more in the original quicktime at vimeo.com/2281077

I've just uploaded v1.2 of ofxMSAPhysics, a traer-like particle/physics library for openFrameworks. Download can be found here and also needs the ObjC Pointer addon.

Demo of v1.0 features at memo.tv/ofxmsaphysics_traer_like_physics_library_for_c_openframeworks

My Secret Heart is a music and film installation & performance commissioned by Streetwise Opera with music composed by Mira Calix. Working with video artists Flat-e, we created visuals for the 48 minute performance, as well as versions for an installation and short film.

More information can be found at http://www.memo.tv/projects/my_secret_heart.

The visuals were created primarily with custom software written with C++/openFrameworks, with some Quartz Composer elements, rendered AfterEffects sequences and live action footage. The custom C++ app is audio-reactive and user-interactive, allowing the visuals to be 'performed' live with full control over the behaviour of the virtual inhabitants of the cylindrical aquarium-like rig.

Excerpts from the visuals (I recommend the HD version at http://vimeo.com/2131989):

More stills and photos from the piece and performance can be seen at www.flickr.com/photos/tags/mysecretheart

Stills from the piece:
(the short-wide strip along the bottom of the images is the full width of the visuals wrapped around the 16m circumference of the cylinder, while the top is closeup detail)

The Switzerland preview showings of My Secret Heart have come and gone and now we are left with but a pleasant memory, and the images below - and high hopes for the premiere at the Royal Festival Hall and subsequent galleries & festivals.

More images, videos and info can be found at the official site www.mysecretheart.co.uk and
memo.tv/projects/my_secret_heart.

Photos from Lucerne, Switzerland:

This is one of the random things I put together at the OF Lab, Ars Electronica 2008. Members of the public submit 5 words, and we have a pretty short amount of time (ranging from a couple of hours to a day) to knock something up (preferably with openFrameworks) inspired by the 5 words.

The 5 words I had here was 'one second before big bang'. Visuals all realtime and purely controlled by motion.

(watch the HD version here)

Well I finally caved in and bought an iPhone - and my favorite feature (and main reason for buying it) is of course the multi-touch capabilities. So currently OSCemote is my favourite app. Apart from having a few sliders and knobs which transmit OSC (similar to TouchOSC), it also has a multitouch pad which sends out TUIO messages, so any app which responds to TUIO (E.g. anything written with reactivision api) will respond. So I had to try out my visualisation for the Roots Project! Up and running in 5 minutes! Awesome! (I had to rotate the coordinates in my processing code though to have the long end of the iphone screen map to the long end of my desktop screen, slightly annoying that this isn't an option in the app... hopefully soon :P).

This is a demo for a traer like physics library for C++/openFrameworks.
I wrote the lib for a little project with Todd Vanderlin while in Linz, Austria at Ars Electronic 2008 (vimeo.com/1707467). I tried to keep the same API as Traer (so processing code using traer can easily be ported to oF) and it is basically a particle system with springs, attractions, gravity etc (but uses verlet integration instead of RK4).

In this demo I am interacting with the app using the keys:

This is an 'early current state of app' demo for a multi-discipline event I'm working on with Streetwise Opera, Mira Calix and fellow visualists Flat-e, to be showcased at the Royal Festival Hall later this year with quite a few more venues lined up.

The app was written in Processing 0135 and is running realtime at 60fps, though if I add another couple hundred eels it does drop, so I may switch to OpenFrameworks if performance does become an issue (which it probably will). There are occasional freezes in the video which happened while capturing the screen so that is a bit annoying.

I'm controlling the eels using the mouse, keyboard and Quartz Composer (just simple sliders sending OSC to vary some parameters - similar to the 'magnetic force fields' video - I'm quite into this technique now, very quick and easy to setup, and you can have loads of sliders with descriptive names at your disposal to play with, and adjust your internal variables in realtime for tweaking heaven).

The final show will have many many more features, both in the digital realm, and physical... more info coming soon...

I strongly recommend watching the HD version at http://www.vimeo.com/1582196

This is a very early version of a GPU based depth-of-field GLSL shader and sample Processing code. Adjust some parameters and it can also be used to give the scene a nice backlight/glow effect.

This is a demo of creating and visualizing magnetic (kind of) fields in Processing and controlling with a tangible multitouch table and Quartz Composer. It gets more interesting after the 1 minute mark :P

I recommend watching the video in HD at http://www.vimeo.com/1569676

The demo came about as a digression off the Roots project I'm working on with Jordan & Owen - makers of the Bricktable (http://bricktable.wordpress.com/). You can read more about the Roots project at http://www.memo.tv/roots_creating_and_visualising_generative_music_on_a_... and http://bricktable.wordpress.com/about/what-is-roots/ .

Seeing as a lot of what I'm doing is based on open-source software and the good intention of others, I think its fair that I release some source code too... So I've tried to clean and comment a bit of the code I used on the psychedelic interactive fluid and particles demo (also used on the Glastonbury 2008 PI Installation.

This is not the full code for the glastonbury app as that is quite complex, multi-threaded and I couldn't release it all anyway. Instead I've simplified it a bit (removed some messy multithreaded stuff and the osc/midi triggering based on camera - you can find quartz composer and processing.org source code for that on this site anyway).

Thanks to the windy ways of the web, I've found myself working with some truly talented musicians/techies/electronics experts over on the otherside of the pond in California, on a very exciting interactive, generative audio/visual project. The number of traditional instruments they have and play wasn't enough for them, so they decided to build their own, as one does when in that situation - one of which is the bricktable, a tangible and multi-touch table - and instrument.

I've worked on a number of interesting interactive audio projects, but the approach in this one is quite different and i"m very excitied to be working with the bricktable guys on it.

In one line: You control parameters of a chaotic environment - which affect the behaviour of its inhabitants - which create and control music. 

To breakdown very briefly without going into much detail:

So I started playing with the House of Cards data in OpenFrameworks, but looks like I'm not gonna have time to finish it for a while. So I thought I'd post a skeleton if anyone else wants to play.

**UPDATE: 20/07/08 **
Just added a Processing source file as well using the BIN data (P5_HoC_bin_v1.zip). Demo video below (all interactions are controlled with the mouse in realtime - pulling/pushing etc.

First video from the Glastonbury 2008 PI installation is below, and photos can be seen here.

Some info on the installation below:

This is a little teaser for an audio visual interactive installation I'm working on for Glastonbury 2008. It'll be projected around the entire (almost) 65ft interior of a 50ft round tent with multiple channels of audio. Everyone inside will be contributing to the audio/visual experience. Located behind the Laundrettas' crashed plane / laundrette in Trash City, more info at the official site pi.seeper.com

Realtime interactive psychedelic fluid simulation with processing for an upcoming installation. Watch the video below to see it in action, or click here for the interactive version (you will need Java). P.S. it takes a while to load so please be patient!

Here is the source code for the previous example of 3D Sprite (billboard) particle system Starry Trails. The source code is by no means generic enough to be considered an engine or library, but is work in progress. I will extend the Node3D, SpringyNode3D classes to use Papervision3D and the Billboards for the rendering, and will definitely carry on working on the particle systems. I've also been converting a lot of noise functions to AS3 and will post them soon... its all getting quite exciting in the flash world! :P

This app is made of quite a few classes so Ill start from the top (application) downwards.

/***********************************************************************
MAIN DOCUMENT CLASS
make sure you have 2 sprites in your library exported for actionscript with linkage identifiers set to RedStar and BlueStar
 
***********************************************************************/	
 
package {
	import flash.display.*;
	import msa.sprite3d.*;
 
	public class StarryMouse3Dapp extends Sprite {
 
		public function StarryMouse3Dapp() {

Another example of an AS3 3D Sprite based particle system. This one is using my 3D sprite engine but will port this to papervision asap. Can post the source if anyone is interested.

** UPDATE **
I posted the source code here:
http://www.memo.tv/3d_particle_system_actionscript_3_source_code

I've been using papervision for a short while now and have been very impressed so far (hopefully soon I"ll upload some demos and code), but in the meantime I wanted to look at the performance of the new Billboard rendering and see how much (if any) overhead there was.

After examining the code at http://www.sebleedelisle.com/?p=157 I ported my AS3 3D Sprite Flies demo to Papervision using the builtin Papervision3D Billboard rendering and the results were very encouraging. The Papervision version doesn't behave exactly like my 3D Sprite engine version, I think this is due to difference in units and needs number tweaking, but I didn't bother with that for now, just adjusted it a tiny bit to make it look roughly the same in terms of pixel coverage (also couldn't figure out how to rotate the papervision billboards, dont think you can). But most importantly the performance seems identical, which means the
Papervision Billboard system is very efficient and does not have any
overheads.

A 3D particles system using Action Script 3. This does not use Papervision 3D or any other 3D engine but just a simple custom 3D sprite based engine. Not sure whether I should spend time enhancing this or just use Papervision for this kind of stuff. I've used it a bit and it looks great (will post some stuff soon), but I don't know if there is any unnessecary overheads for just 3D sprite stuff. When I get a chance I"ll recreate this demo in Papervision and compare the results.

P.S. They are my babies!

Click attachment below to view.

Continuation of the particle system evolved into branch-like structures. Few parameters tweakable in runtime. Click here to view (you will need a java enabled browser to run this applet).

/************************************* CONSTANTS ****************************************/
int CLEAR_MODE   = 0;
int CLEAR_NONE   = CLEAR_MODE++;
int CLEAR_CLEAR  = CLEAR_MODE++;
int CLEAR_FADE   = CLEAR_MODE++;
//char[][] ClearStates = ["Clear:None", "Clear: Clear", "Clear:Fade"]
 
 
int BRANCH_MODE   = 0;
int BRANCH_NONE   = BRANCH_MODE++;
int BRANCH_BRANCH = BRANCH_MODE++;
int BRANCH_DRIP   = BRANCH_MODE++;
 
int BGCOLOR = 255;
int MAX_CIRCLE_SIZE = 15;
 
 
/************************************* VARS ****************************************/
boolean bMouseMode = false;
float fHeadSpeed = 2;
float fNoiseSpeed = 1;
int iClearMode   = CLEAR_CLEAR;
int iBranchMode   = BRANCH_BRANCH;
 
int numBranches;
int numCircles;
BRANCH[] branches;
VECTORFIELD VectorField = new VECTORFIELD(2, 0.5, 1, 1);

First test of the Processing language. Pretty basic particle system. Click here to view (you will need a java enabled browser to run this applet).
The code is quite object oriented so should be pretty straight forward to follow, any questions just ask...

import noc.*;
 
 
 
/************************************* VECTORFIELD ****************************************/
class VECTORFIELD {
  private float fNoiseMin, fNoiseMax;    // used for scaling values to white and black
  private float fScaleMult, fSpeedMult;
  private int iOctaves;
  private float fFallOff;
 
  VECTORFIELD(int to, float tf, float ts1, float ts2) {
    init( to, tf, ts1, ts2);
  }
 
  void init(int to, float tf, float ts1, float ts2) {
    float w = 500, h = 500;
    iOctaves = to;
    fFallOff = tf;