A couple of posts ago, I posted the “10 Steps to your Own Virtual Sweatshop” video. This was a collaboration between Annie Ok, Stephanie and I. Our original goal was to make an application for Prix Ars Electronica. But we ended up putting a lot of time and love into the video – so much that Stephanie sent it around to a few film festivals and we were accepted into the DC Film Festival back in April.
While I am very pleased with how it came out and excited about how much people have liked it, it was, of course, never meant to act as a sincere step-by-step guide to anything, much less the complicated process of making a mixed reality project. So I thought I would write a little about some of the strategies we used in Invisible Threads and tell rest of the story – the stuff that was too boring or complicated to include in the video. Great intro, right?
The Approach
The more I work with installations and performances, the more I realize that people (at least the people who see my work) aren’t interested in technology; they are interested in an experience. Seems simple, but as a programmer, it is sometimes hard for me to remember. When I first began thinking about how to create a factory in Second Life, I immediately started planning out all of the systems and widgets that would work together to make the factory possible. These systems would be bigger than the project – they would be abstract enough to be useful to the rest of the world. But I quickly realized that this would be a pretty boring path to take. I would have definitely gotten bogged down in all of these separate systems, and I would have lost sight of the original goal. So what I ended up with was a pretty specific system with little that is going to be immediately useful to others. So none of this is going to save the world, but it gets the job done efficiently and I was able to concentrate on achieving exactly what I wanted, rather than being pulled in 5 other directions by the technological requirements I imposed on myself.
Anyway, on with the show. If you haven’t seen the Double Happiness factory yet, I recommend visiting it before reading on so that you have some idea of what I am talking about. <SLURL HERE>
The entire system is made up of 4 parts: (you will see by the end of this post that I enjoy spitting things up into logical parts)
the Second Life factory
the API/database service/Web Interface
the ExtendScript
the assets
1. The Second Life Factory
The first step is to have an input system in SL. For Invisible Threads, the system consisted of 2 parts: the machines and the template. For the sake of simplicity, we can say that the template is instantiated when one of the workers activates the loom (a specialized machine) by feeding it a prim that looks like a bale of cotton. The template looks like a flat rectangle with a jeans texture on it. When it is instantiated, a “start order” request is sent to the remote API (more about this later) and it gets back an order number. The template is scripted to move around the assembly line more or less on its own using this script:
Each of the 9 machines is activated when a template passes over the machine’s “sensor”, an invisible prim that sits right in the middle of the machine. When this happens, the sensor tells the template to stop, tells the machine to turn on, and starts a timer. The sensor contains this script:
When the machine is activated by the sensor, the operator has N seconds to set the machine to the appropriate state based on the order from the customer. The operator at the machine clicks on the machine to change from one state to another. On each click, the machine enters a different state, which changes the front panel texture, and shouts out a message, like “size 34″ or “ragged hem”, or “Casual Friday syle” depending on the machine and the order.
When the machine shouts out it’s message, the API script hears it, and when the timer ends, the machine makes a simple REST call using the llHTTPRequest() function. The call includes the order number (which was assigned by the API when the “start order” request was made), the name of the machine, and the last message that was shouted by the machine. The REST request might look something like this:
And that is an over-simplified account of what happens in the factory! All of the Second Life scripting was done by my wonderful intern, Andrew Mahon, which is why the descriptions are a little vague and incomplete, but if you have any questions, please email me and I will try to answer them.
2. The Order Database Service/Web Interface
The web interface is made of 2 parts: the API endpoint and the Web Interface.
The API Endpoint
I tried to put as much of the business logic as possible on the server side because, frankly, Second Life scripting is a pain in the ass. The API endpoint is set up to distinguish calls from 3 different User Agents, ‘Second Life’, ‘ExtendScript’, and everything else. If it receives a call from anything but ‘Second Life’ or ‘ExtendScript’ (ie: a web browser), it will print a helpful HTML documentation page. If it receives a call from ‘Second Life’, it knows that the request is updating an order. We will get to the ExtendScript stuff later.
The important PHP classes are pretty straightforward: Order and Attribute. When the API receives a call from the machine called ‘loom’, it knows to create a new order. When it receives an order from the Quality Control machine, it knows that the order has made it all the way around the assembly line and it should be dealt with somehow. All other machines just send messages that get stored by the Order class in the order_attribute_lookup table. An Order object is really nothing more than an array of Attributes, an ID, a ‘status’, and a bunch of logic that tells it how to save itself and retrieve itself from the database. Later on, I started adding names to them, but that is just icing.
So when a request comes into the API that says “the dye vat just said that No Pants Left Behind was selected for order number 225″, the API will construct order number 225, add a new attribute, and save it back to the database. Of course, there is some sanity checks and there are the special cases that I mentioned above, but that is essentially all it does.
If the API is called by ExtendScript, it will check to see if there are any orders in the database whose status is set to ‘ready to print’. If there are, it returns a JSON encoded version of the order and then proceeds to put together the order, which I will talk about in a minute.
On literally the last day of coding, I realized that my mental model for this system was just a bit off. It would have been smarter to model the logic of the factory on what it actually was: a proxy for Photoshop. Instead of sending information about pairs of jeans, it should have been sending commands for Photoshop, like “Open Size34.pdf”, “Hide Layer ‘Roadkill’”, or “Print now”. I think this would have made the code more robust and easy to understand, and could have actually been used by other people. I don’t imagine I will ever want to take the time to re-write this server code, but if anyone else out there wants to make a factory in Second Life, that is what I would suggest
The Web Interface
While explaining Invisible Threads hundreds of times at Sundance and other venues, it occurred to the pragmatic scientist in me that the Double Happiness factory could be considered “the most inefficient way to fill out a web form ever invented.” That is, we went through all of the trouble building this factory, scripting these machines, hiring and training workers, buying an island, when really, what the whole thing is doing amounts to clicking a drop menu on an HTML form. Of course, the project was meant to provide an experience and to make people think, so the pragmatism of the system is kind of moot. But if the factory is the least efficient way to get this data into the database, the Web Interface is the most efficient.
It basically consists of one big table that contains all of the orders and an ‘edit’ button for each order so that you can change the attributes, status, and name of any order easily. This became an essential part of the project when, after weeks of training, I realized that the workers simply weren’t going to be able to consistently get orders right, even on the second or third try, no matter how much I trained them. So I paid a premium salary and the password to this interface to one worker whose job it was to monitor the orders and magically correct the orders if things weren’t going smoothly in the factory.
When we first started Invisible Threads, I was planning on doing all of the graphics generation with The Gimp and Script Foo. But since I am not a glutton for punishment, and not quite so strong in my Open Source principles that I would subject myself to that kind of a headache, after a few days of searching for the right Script-Foo interface and trying to pick up the arcane syntax, I gave up and discovered ExtendScript.
ExtendScript is the API provided by Adobe to control most Adobe products, such as Photoshop, InDesign and Illustrator. It comes in 3 flavors: JavaScript, VisualBasic, and AppleScript. It has a simple HTTP library which allowed me to query the database for unprinted orders and since I am pretty good with JavaScript, it was easy to get the job done. So basically, all the script does is run an infinite loop on a little Mac Mini that we hide away near the printer, polling the server for orders whose status is ‘ready to print’. When it finds one, it asks the server for a JSON version of the order, which it then uses to open the correct PSD, show and hide the appropriate layers, print the size information on a tag, and export the resulting graphic as a PDF file. The PDF file is then opened in Illustrator, and a message is sent back to the server to set the status of the order as ‘printed’.
The dirty secret of the project is that I never did figure out how to make Illustrator automatically print the generated PDF. I still have to go over to the Mac Mini and hit Apple+P. The settings to get the PDF to print properly on the large-format printer are so complex that I didn’t think it would be worth it to puzzle them all out. I *think* I could saved a preset and just told Illustrator to use it to print the document, but I kind of ran out of time.
4. The Assets
We decided to organize the jeans assets into 12 different PSDs by size. There’s not much to tell about them – they are pretty self-explanatory, so here they are. They’re huge.
Finally, one bit of icing that I was particularly fond of was the giant, 1984-style screen that was installed on one wall of the factory. I rented a QuickTime server from a very kind and courteous entrepreneur called Jamie Otis. His company, SLServer, rents audio and video streaming servers for $30/month, which is a very good price. Using QuickTime Broadcaster and an old iSight that I had, we were able to start a Quicktime video stream, and then it was just a matter of setting the media URL on the factory parcel to the URL of the stream, and that’s it. Instant live audio stream. This also made the employees happier, since they were able to see what was going on in the RL storefront.
Behind the Scenes of Invisible Threads
A couple of posts ago, I posted the “10 Steps to your Own Virtual Sweatshop” video. This was a collaboration between Annie Ok, Stephanie and I. Our original goal was to make an application for Prix Ars Electronica. But we ended up putting a lot of time and love into the video – so much that Stephanie sent it around to a few film festivals and we were accepted into the DC Film Festival back in April.
While I am very pleased with how it came out and excited about how much people have liked it, it was, of course, never meant to act as a sincere step-by-step guide to anything, much less the complicated process of making a mixed reality project. So I thought I would write a little about some of the strategies we used in Invisible Threads and tell rest of the story – the stuff that was too boring or complicated to include in the video. Great intro, right?
The Approach
The more I work with installations and performances, the more I realize that people (at least the people who see my work) aren’t interested in technology; they are interested in an experience. Seems simple, but as a programmer, it is sometimes hard for me to remember. When I first began thinking about how to create a factory in Second Life, I immediately started planning out all of the systems and widgets that would work together to make the factory possible. These systems would be bigger than the project – they would be abstract enough to be useful to the rest of the world. But I quickly realized that this would be a pretty boring path to take. I would have definitely gotten bogged down in all of these separate systems, and I would have lost sight of the original goal. So what I ended up with was a pretty specific system with little that is going to be immediately useful to others. So none of this is going to save the world, but it gets the job done efficiently and I was able to concentrate on achieving exactly what I wanted, rather than being pulled in 5 other directions by the technological requirements I imposed on myself.
Anyway, on with the show. If you haven’t seen the Double Happiness factory yet, I recommend visiting it before reading on so that you have some idea of what I am talking about. <SLURL HERE>
The entire system is made up of 4 parts: (you will see by the end of this post that I enjoy spitting things up into logical parts)
1. The Second Life Factory
The first step is to have an input system in SL. For Invisible Threads, the system consisted of 2 parts: the machines and the template. For the sake of simplicity, we can say that the template is instantiated when one of the workers activates the loom (a specialized machine) by feeding it a prim that looks like a bale of cotton. The template looks like a flat rectangle with a jeans texture on it. When it is instantiated, a “start order” request is sent to the remote API (more about this later) and it gets back an order number. The template is scripted to move around the assembly line more or less on its own using this script:
Jean Template Movement Script
Each of the 9 machines is activated when a template passes over the machine’s “sensor”, an invisible prim that sits right in the middle of the machine. When this happens, the sensor tells the template to stop, tells the machine to turn on, and starts a timer. The sensor contains this script:
Machine Sensor Code
When the machine is activated by the sensor, the operator has N seconds to set the machine to the appropriate state based on the order from the customer. The operator at the machine clicks on the machine to change from one state to another. On each click, the machine enters a different state, which changes the front panel texture, and shouts out a message, like “size 34″ or “ragged hem”, or “Casual Friday syle” depending on the machine and the order.
Machine State Code
When the machine shouts out it’s message, the API script hears it, and when the timer ends, the machine makes a simple REST call using the llHTTPRequest() function. The call includes the order number (which was assigned by the API when the “start order” request was made), the name of the machine, and the last message that was shouted by the machine. The REST request might look something like this:
Machine API Client Code
And that is an over-simplified account of what happens in the factory! All of the Second Life scripting was done by my wonderful intern, Andrew Mahon, which is why the descriptions are a little vague and incomplete, but if you have any questions, please email me and I will try to answer them.
2. The Order Database Service/Web Interface
The web interface is made of 2 parts: the API endpoint and the Web Interface.
The API Endpoint
I tried to put as much of the business logic as possible on the server side because, frankly, Second Life scripting is a pain in the ass. The API endpoint is set up to distinguish calls from 3 different User Agents, ‘Second Life’, ‘ExtendScript’, and everything else. If it receives a call from anything but ‘Second Life’ or ‘ExtendScript’ (ie: a web browser), it will print a helpful HTML documentation page. If it receives a call from ‘Second Life’, it knows that the request is updating an order. We will get to the ExtendScript stuff later.
The important PHP classes are pretty straightforward: Order and Attribute. When the API receives a call from the machine called ‘loom’, it knows to create a new order. When it receives an order from the Quality Control machine, it knows that the order has made it all the way around the assembly line and it should be dealt with somehow. All other machines just send messages that get stored by the Order class in the order_attribute_lookup table. An Order object is really nothing more than an array of Attributes, an ID, a ‘status’, and a bunch of logic that tells it how to save itself and retrieve itself from the database. Later on, I started adding names to them, but that is just icing.
So when a request comes into the API that says “the dye vat just said that No Pants Left Behind was selected for order number 225″, the API will construct order number 225, add a new attribute, and save it back to the database. Of course, there is some sanity checks and there are the special cases that I mentioned above, but that is essentially all it does.
If the API is called by ExtendScript, it will check to see if there are any orders in the database whose status is set to ‘ready to print’. If there are, it returns a JSON encoded version of the order and then proceeds to put together the order, which I will talk about in a minute.
On literally the last day of coding, I realized that my mental model for this system was just a bit off. It would have been smarter to model the logic of the factory on what it actually was: a proxy for Photoshop. Instead of sending information about pairs of jeans, it should have been sending commands for Photoshop, like “Open Size34.pdf”, “Hide Layer ‘Roadkill’”, or “Print now”. I think this would have made the code more robust and easy to understand, and could have actually been used by other people. I don’t imagine I will ever want to take the time to re-write this server code, but if anyone else out there wants to make a factory in Second Life, that is what I would suggest
The Web Interface
It basically consists of one big table that contains all of the orders and an ‘edit’ button for each order so that you can change the attributes, status, and name of any order easily. This became an essential part of the project when, after weeks of training, I realized that the workers simply weren’t going to be able to consistently get orders right, even on the second or third try, no matter how much I trained them. So I paid a premium salary and the password to this interface to one worker whose job it was to monitor the orders and magically correct the orders if things weren’t going smoothly in the factory.
And that’s about it for the server-side stuff. Here is an archive of the code: Double Happiness Server Side Stuff
3. The ExtendScript
When we first started Invisible Threads, I was planning on doing all of the graphics generation with The Gimp and Script Foo. But since I am not a glutton for punishment, and not quite so strong in my Open Source principles that I would subject myself to that kind of a headache, after a few days of searching for the right Script-Foo interface and trying to pick up the arcane syntax, I gave up and discovered ExtendScript.
ExtendScript is the API provided by Adobe to control most Adobe products, such as Photoshop, InDesign and Illustrator. It comes in 3 flavors: JavaScript, VisualBasic, and AppleScript. It has a simple HTTP library which allowed me to query the database for unprinted orders and since I am pretty good with JavaScript, it was easy to get the job done. So basically, all the script does is run an infinite loop on a little Mac Mini that we hide away near the printer, polling the server for orders whose status is ‘ready to print’. When it finds one, it asks the server for a JSON version of the order, which it then uses to open the correct PSD, show and hide the appropriate layers, print the size information on a tag, and export the resulting graphic as a PDF file. The PDF file is then opened in Illustrator, and a message is sent back to the server to set the status of the order as ‘printed’.
Factory ExtendScript
The dirty secret of the project is that I never did figure out how to make Illustrator automatically print the generated PDF. I still have to go over to the Mac Mini and hit Apple+P. The settings to get the PDF to print properly on the large-format printer are so complex that I didn’t think it would be worth it to puzzle them all out. I *think* I could saved a preset and just told Illustrator to use it to print the document, but I kind of ran out of time.
4. The Assets
We decided to organize the jeans assets into 12 different PSDs by size. There’s not much to tell about them – they are pretty self-explanatory, so here they are. They’re huge.
Finally, one bit of icing that I was particularly fond of was the giant, 1984-style screen that was installed on one wall of the factory. I rented a QuickTime server from a very kind and courteous entrepreneur called Jamie Otis. His company, SLServer, rents audio and video streaming servers for $30/month, which is a very good price. Using QuickTime Broadcaster and an old iSight that I had, we were able to start a Quicktime video stream, and then it was just a matter of setting the media URL on the factory parcel to the URL of the stream, and that’s it. Instant live audio stream. This also made the employees happier, since they were able to see what was going on in the RL storefront.