Home > english, Programming, Technologie > Flash talks to Max MSP via OSC

Flash talks to Max MSP via OSC

August 16th, 2011 Leave a comment Go to comments

As I already mentioned  in an earlier blog post, for reimplementing my "Sound drawing" project to Flash I had to create a communication layer between Max/Msp and Flash. In this blog post I will report about my results and some technical information. If you want to jump directly to the technical part, please skip the part "Why am I doing this".

Why am I doing this?

I want to have access to more performance in general, I want to have access to more technical advanced stuff, I want an easy way for using external hardware devices, and last point, I want to collaborate with other media artists (without changin the working environment).

Flash is a very nice for working with graphics, videos and other media based stuff. That is the total strength of Flash, but it fails in some performance issues (compared to C++) and some algorithmic libraries (compared to libraries like IanniX, Math, OpenCV, and Sphinx4). One motivation for me in this project lies to combine the advantages of these worlds. But I don't want to change the usage of these libraries. Some libraries have to be executes as an exe Application and they just perform great in C++, and not in a Flash environment (in some cases Alchemy could be a solution).

Next point of performance is creating a scalable approach for media installation. Some media installation can't  be run by only one machine.  You need more computers, which talks to each other. For this reason, I put my focus on the Open Sound Control network protocol. As the OSC team drescribed, this protocol is made for communication between computers and media devices. It is often used in very robust environments and the documentation is very well, too. Almost every creative coding software supports the OSC communication, which is very good for collaborating with other media artists (everyone can still work in the preferred software environment).

Some hardware can already talk OSC or it uses an communication software like TUIO, iOSC, AndroidOSC. For Midi controllers you can use graphical environments like Max/Msp or PureData. These softwares have directly access to the midi hardware and can configure the most things. Afterwards you send the midi data from Max/Msp (or PureData) to Flash via OSC. In this way Flash gets access to a new level of external devices...

Flash talks OSC

First I was a little bit scared, because I thought I have to dive into the nerdie byte array socket programming part in Flash. Luckily, I found information about the FlOSC project. FlOSC is Java based application, which receives the OSC messages and routing them to the Flash XML socket. There are some great tutorials available by Eric Socolofsky and Tim Poulin (part 1, part 2, part 3). After watching all of these tutorials, I was a little bit scared of the reliable performance of this approach. The Flash XML socket is nice for chat applications or some round-based multiplayer games, but for realtime data transmission I heard bad things...

Luckily again, some time passed since FlOSC were created and Adobe provided a new binary socket API for the Flash Player. Things became even better. Since Version 2 of Adobe Air, Air supports the network UDP layer for really fast data transmission. I have not to mention that the Flash open-source is great ;-), so I could find easily a nice library.  The tuio flash library solved almost everything I needed. I had to write my own OSCListener, which is great explained in the tuio guide and in its comments. During development I had sometimes problems with the binding of the DatagramSocket. It was obvious that the (binded) port was never closed. I had to wait until Windows shutted it down or just do it manually. Unfortunately, the UDPConnector class does not provide a close function. Therefore I added a close function by myself at the end of the class. Extending or creating a custom UDP Connector was not possible, because all members of the class are private.

Max/MSP talks OSC

After this minor coding everything worked fine. So I could jump to Max/MSP. Using OSC in Max/Msp is not very difficult. The documentation is pretty good and the provided example file helps a lot. I just extended the example with one function. If the max patch receives successfully a message from the Flash Client (Adobe Air app), then send a message back to the Flash client.  For the second example you have to install the fiddle plugIn (for Windows) for Max/Msp.

 

Max/Msp and Flash in love

I created two mini examples. Example one just demonstrates sending messages between Flash and Max/Msp. The second example is little bit more complicated and shoud give you an idea of a real life project. The max patch analyzes an audio file and sends the pitch tracking + volume tracking data to the Flash client. The flash client reads the data and draw it on the canvas. As you can see on my screen capture video above. The performance of this test application is pretty solid and robust. Now imagine instead of the tracking data we send midi data or just some image processing data. Or we do some heavy image processing on one machine and send the data to another machine, which only renders the Flash output... I guess there are many new things possible 🙂

Download Code

The source code is available on my google project SVN or download the zip file. For the examples you must have installed Max/Msp 5, fiddle plugIn (for Windows), Flex SDK 4.5, Flash CS5 or Flash Builder 4. The code examples also requires the additional ActionScript libraries MinimalComps, MonsterDebugger, and TUIO Flash.

  1. August 26th, 2012 at 02:19 | #1

    So, I am trying to recreate your project using your source code, and wanted to know if you had any errors implementing the IOSCListener from the tuio package. I cant seem to get it to implement properly in the SimpleOSCListener for the life of me.

  2. August 26th, 2012 at 04:51 | #2

    @Matthew
    Figured it out, haha! Thanks for the excellent resource. You are going to make my “video game for 2 trombones” composition a blast :)))

  3. admin
    November 14th, 2012 at 15:40 | #3

    Really nice tutorial about working with Flash and VVVV in a projection based environment.

    http://ig.hfg-gmuend.de/Members/mark_lukas/meine-projekte/objekt-mapping-swf-unter-vvvv

  4. eco_bach
    January 23rd, 2013 at 22:12 | #4

    FANTASTIC POST!
    My understanding was that even the latest release of AIR cannot process OSC directly and some 3rd party server (FLOSC, RED5) is necessary.

    But you seem to imply that it IS possible thru the TuioFlash library. Am I correct?

  5. admin
    January 23rd, 2013 at 22:50 | #5

    Hi,

    yes, you are right! It is possible without a 3rd party server. Air supports now direct messaging via UDP. Good luck!

  6. Oli Mason
    February 21st, 2013 at 18:29 | #6

    Hi,

    Thanks fro a great post, I have been using max/msp for a while now, but i am very new to action script and flash.

    I was wondering if you had a basic example flash patch with the action script loaded so i can have a look and try to figure it out?

    Kinds Regards,

    Oli.

  7. admin
    February 22nd, 2013 at 09:50 | #7

    Hi Oli,

    thx for the feedback! You can look at the Flash Patch or better known as Flash Document Class.
    http://code.google.com/p/derhess/source/browse/trunk/AS3/de/derhess/projects/air_flashToMaxMSP/as3_flashToMaxMsp.as

    hope that helps

    Flo

  8. Oli Mason
    February 25th, 2013 at 18:52 | #8

    Great thanks ill try and get it working 🙂

    Regards,

    Oli

  9. admin
    February 25th, 2013 at 22:55 | #9

    Duration Timeline is a nice creative coding tool, as well. In some cases maybe a simple alternative to Max/MSP

  10. April 5th, 2013 at 09:43 | #10

    hallo,

    i am trying to get this project working. i am not a Flash expert and made a empty flash file and added the class in the document. I get no errors, but the swf is totally emtpy.
    Maybe there is a possibility that you explain what I have to do, or can you email me a flash file which works?

    Regards, Martin

  11. admin
    October 31st, 2016 at 16:16 | #11

    Since Flash is getting less and less attention and JavaScript has got the Socket feature, it is possible to speak from your Browser to MaxMsp and vice versa: Two approaches are possible.

    The OSC solutions via direct socket connection, very similiar to my Flash approach described in this text.
    https://github.com/taboada/MaxClientJS

    Or you run a webserver within MaxMSP, which communicates with your Browser.
    https://github.com/olilarkin/wsserver

  12. November 19th, 2016 at 14:32 | #12

    Another cool tool for audiovisual programming and interactions (also with OSC) is i-score. A FLOSS intermedia sequencer for the precise and flexible scripting of interactive scenarios

    http://i-score.org/

  13. November 25th, 2016 at 23:07 | #13

    Amazing project: Music Designed for an Oscilloscope http://sploid.gizmodo.com/music-designed-for-an-oscilloscope-looks-and-sounds-coo-1789345209 | https://asdfg.me/oscistudio/ | https://asdfg.me/osci/ #sound #art –> translating Blender 3D objects to sound

  1. August 16th, 2011 at 15:11 | #1
  2. February 8th, 2012 at 00:02 | #2