visualaborate

visual collaboration — that was my theme for the final ‘New Media & New Technology‘ course within the ‘Media Technology’ programme.

Final Result

With 8 participants after 5 hours exhibition time.

Concept

I took the underlaying idea from a previous project: controlling a virtual instance with an iOS device and be part of something big. Instead of a game I wanted to do something more permanent and stable, so I decided to do a visualisation. The “big picture” is a collection of peoples own preferences and attitudes. Everybody can contribute a personal touch without artistic skills, as the visualisation does not rely on dexterity, but pure fun and movement of the device.

Implementation

Instead of HTML and JavaScript I created my own iPhone App “GyroTransmitter” which sends accelerometer- and gyroscope-data with some additional custom values like colour and line size/number to the network as UDP broadcast.

On the other end I used Quartz Composer again, as it has very powerful visualisation engines. However, I did not have the skills in OpenGL and OpenCL for personalised visuals, so I had to rely on the ready-made patches from Apple. Some things couldn’t be handled without a little bit of scripting, for example the device queue, which holds all actual values for all devices without any duplicates. This was rather a challenge, as it is quite confusing to work with so many structures:

`function (__structure device, __structure outputQueue, __string debug) main (__structure inputStructure, __structure inputQueue) { var result = new Object(); var device = new Object(); var qObj = new Object(); var queue = new Array(); device.attributes = new Object(); if(inputStructure != undefined) { device.id = inputStructure[0]; device.attributes["ax"] = inputStructure[1]; device.attributes["ay"] = inputStructure[2]; device.attributes["az"] = inputStructure[3]; if (!_testMode) { if(inputQueue != undefined) { var found = false; for(i=0; i < inputQueue.length; i++) { qObj = inputQueue[i]; if(String(qObj["id"]) == String(device.id)) { queue[i] = device; found = true; } else { active = qObj["active"]; queue[i] = qObj; newObj = queue[i]; } } if(!found) { queue.push(device); } } } } result.device = device; result.outputQueue = queue; return result; }`

The base patch is pretty simple: get the values from the network, split them into the right attributes, queue them and display them. As mentioned above, there is some JavaScript processing required to create the queue and an iterator to display one “Line Family” for each device.

Problems with Quartz Composer

Quartz Composer can be tricky sometimes and it took me dozens of hours to debug and get the patch running. Some information which is worth sharing about Quartz Composer:

No Integration or Random in Iterations

By integration I’m talking about the mathematical term. Adding values up/down, which is a pretty simple task. However, once you’re in an iterator-patch, the output of the Integrator-patch is always the same for all instances, so even if one device sends to command to go down and the other one the command to go up, they’re both doing the same. Random is also the same value for all instances, no matter how many random generators you use. This is also valid for any kind of trickiness you might get into your mind. I tried JavaScript, Math, Mathematical Expression, Integrator and a complex Sample & Hold combination, nothing worked.

No JavaScript Debugging

This is actually the worst thing in Quartz Composer: there is no debugging. If there is a logical mistake (which is not recognised by the syntax-check), the output of the patch is simply ‘virtual’, there is no clue on where the mistake actually is. And as JavaScript is an untyped language, you can just write attrbute instead of attribute and search for the error for a while.

Conclusion

Overall I’m quite happy with the results, even if I will probably not use Quartz Composer again and try to use the Quartz Engine with Objective-C next time. To figure things like the iteration problem out just costs too much time and usually works in every other language. The output of the patch and the collaboration on a visualisation is fun, even if the network broadcasting is still not perfect and there could be more options in the graphical device interface instead of random values, but I wanted to avoid too much focus on the interface, but controlling the visualisation with gyro and accelerometer data. For future work I would display the visualisation also on the device and get rid of interface input completely, but use simpler visualisations. These ‘private‘ visualisations could then be added up in a crowd-visualisation, without users seeing the end result while they’re adding their part.

Video

visualaborate from Patrick Heneise on Vimeo.