Cross-platform compatibility

I was preparing to author a DVD containing the source .dir files, the Projector, Xtras and Soundscape audio file, when it suddenly dawned on me that this project will ONLY work on a PC, it is not compatible with a Mac.

This is due to the use of the WebXtra component that spawns an internet browser window. It only works with Windows. The group would like to apologise for any inconvenience this may cause.

In writing to a DVD I would also like to point out that the program will not run directly off the DVD disc, it needs to be copied to a working directory on the local hard drive (due to the File I/O writing capability).

Back To The Future

Where could this piece go in the future? How could it develop? How could it translate into a different context?

For a start popularity and intrigue demands that we repeat the performance for many more audience - not in terms of more than one at a time, but for a longer period of time so that it can run more times. We are currently looking at not only options for repeating it at Dartington, but also at the Submerge festival 2006.

It has been suggested that we could install it on a number of machines and run more than one performance at a time, but to me taking it out of context into a computer suite or IT department just wouldn't have the same performative quality to it.

This goes for distributing the work over the Internet (via BitTorrents or as a Shockwave website), or via DVD-ROM. Although technologically possible, it would then become all about the production values and design of the interface, not immersing the User within an interactive environment.

The ongoing problems of webcam integration could indeed be solved over time, both through automated webcam components (Xtras, Hardware combinations) and more lo-fi Networked laptop solutions. - I personally am not sure I now want to take part in these modifications; it seems I've become attached to the piece as it stands and would find it a wrench to change it in any way now.

Please feel free to add comments as to where you think the work should progress to...

Desktop Screenshots

Some screenshots of the Desktop portion of the experience.

Beach Screenshots

Some screenshots of the Beach portion of the experience
(courtesy of Marina Savva).

Forest Screenshots

Some screenshots of the Forest portion of the experience
(courtesy of Ruari Floyd).

Promotional Email

This is an extract from an email sent by Tim to the whole college advertising our piece and those by the rest of the group.

Digital Arts Practices & Technologies in Performance Collaborations project - DP/TP301 - Dartington College of Arts

Project showings - Thursday 9th Feb - Showings run throughout the day from 1000 - 1600

Open to all

1000 - 1330 - Theatre Studio 2 - sbazmeno mihani
1100 - Music Studio 3 - Pendulum
1400 - 1500 - Seminar 1 - 5 stages to becoming a Cyborg
1500 - 1600 - Music Studio 3 - Boxes

sbazmeno mihani 1000 - 1330 - Theatre Studio 2

A black room awaits, it is you that bends like a retromingent electric sheep in a blender.
Hark! For the route you take is phantasmagorical and lucid and the road back is hidden.
Splatter your soul on the seat of the widow and envelop yourself.
The tap at the window, the tap at the window is deafening...

It's all about the Users...Josh Stopford

Last but not least to enter the space was Josh. Having had to perform in Pendulum earlier we managed to squeeze him in just before we had to start packing away.

Josh interacted well with Snivitz, and although he didn't quite get to the end either, he went further than many did (another benefit of being in the point-and-click generation).

Josh's feedback at the conclusion of the experience was very beneficial to us too. He found much of the dream imagery to be very "hypnotic" and said "if you made it more comfortable for people to sit, then I could have sat there for hours" which is very interesting considering that is the precice opposite to how it should have been.

Josh also managed to show self-censorship and modesty to his new found "friend" Snivitz.

Snivitz: "What was the last dream you remember?"
Josh: "It was dirty"

It's all about the Users...Duncan Speakman

Duncan, being a digital artist and along with Tim serving as both conceptual and technical advisor for this project had quite a good grasp of the project before he went in.
However, his enthusiasm when he came out of the experience was certainly good for morale within the three of us who designed the piece.

Duncan engaged well with the experience, and even though he knew quite clearly the direction the piece was going in, he was pleased and surprised by the path he took through the Beach and (having taken the time after the experience to give us a short debrief) said how "I still felt like there was so much to explore, that my choices unlocked certain paths but stopped me from taking other ones".

Duncan managed to reach the very end of the experience, and like Tim before him, joined in the performative aspect by coming out and saying "(Snivitz) says my progress can't be saved and that I should see an Administrator".

And not surprisingly:

Snivitz: "What is your favourite colour?"
Duncan: "Blue"

It's all about the Users...Paul Clarke

As the Module Coordinator for this collaboration, I was surprised we hadn't seen an awful lot of Paul during the conception and design phase of the project, but this had it's own benefits as he hadn't experienced much of the piece at all. Fresh meat he he...

Paul was a very quick clicker, so he was out of the experience quite quickly compared to some Users. This was good in itself because he didn't get much of a chance to fully think about the choices he made during the experience. Again he didn't get to the very end of the piece, but this may have been a lack of patience more than anything; after leaving the experience he had to rush off to get lunch before an afternoon meeting.

Interestingly, when Snivitz asks what the User's favourite colour is, the desktop background is predominantly blue. Every User before Paul stated that their favourite colour is blue, however

Snivitz: "What is your favourite colour?"
Paul: "Red"

It's all about the Users...Alison

With our first and only completely new User to Digital Arts (the mother of another student presenting in the Boxes group), we have Alison.

Her reaction to the experience was unique in that she was not a lecturer or student of this subject, so that she had no expectations when going in. And the sketchy information I provided her with before she went in only increased this feeling.

She responded well to the experience, giving honest answers and interfacing well with the 'consciousness' of Snivitz. She didn't get to the very end of the piece, but as there are design elements in place to see if the "point and click" response to computers is a generational trend, this was not that surprising.

Snivitz: "How do you feel?"
Alison: "A little afraid"

It's all about the Users...Sean Eisenstein

First of all an apology should really come from the "Pendulum" group (only teasing guys);
In setting up their performance in the studio next to ours, they inadvertently set off the building's fire alarms (numerous times) whilst using a smoke machine! This led to Sean's repeated evacuation of our studio (not to mention the world's most polite, talking fire alarm). This may have had some bearing on his experience, in that he was repeatedly taken out of the immersion we had created, and can be seen by some of his answers:

Snivitz: "What is your favourite colour?"
Sean: "Why do you want to know?"

This may just have been his inquisitive mind reacting to the questioning, but of course it immediately broke the illusion of conversation as Snivitz' next response would have been

Snivitz: ""Why do you want to know?" That is a nice colour..."

It's all about the Users...Paula Crutchlow

Hmm, all the way through this project I personally thought that Paula, coming from a background of live performative theatre would have trouble appreciating the concept of the piece. Her comments during tutorials and lectures seemed to put us at odds over stylistic choices and unfortunately this attitude continued into her own experience. Or that's initially what I thought...

From the off Paula did not use the headphones (even after being specifically asked to). She only used them once the soundscape was loud enough to bleed into the studio from the headphones' position on the table. This automatically took her out of the experience in that her interaction was different from everybody else in the audience.

She also spent most of the time writing notes on the experience in her notebook. Despite being a lecturer and having to assign marks, I felt that there was no chance of her having a true understanding of the immersive environment we were attempting to create if she wasn't willing to fully interface with it.

However, once out of the studio (she didn't reach the finale) we began to hear from other students she had been talking to that the experience had allegedly "freaked her out" and she thought it was "terrifying". This was interesting because I had had a similar reaction to it when testing it at 5 in the morning!

It seems that the experience actually had a dramatic effect on Paula despite the unorthodox method of her interaction with it.

This can also be seen in the answers to some of her questions; although initially her reluctance can be clearly seen

Snivitz: “How do you feel?”
Paula: “Like I should be somewhere else.”

eventually she responded positively, even telling this 'person' what appears to be some very personal details

Snivitz: “What was the last dream you remember?”
Paula: “I was with my ex walking along the beach.” (paraphrased)

I believe that Paula actually showed a very profound interaction with the application, of course her interpretation of that is as yet unknown...

It's all about the Users...Tim Dollimore

First into the Studio was Tim Dollimore, at 1030am. It wasn't our fault we started late - we had everything set up, just no audience present until then!

Tim seemed to go into this with an open mind, laughing to himself when Snivitz would regurgitate his answers, then respond to his answers with more questions, looking around frenetically when the soundscape began, and even managed to progress to the very end of the piece.

Snivitz: “How do you feel?”
Tim: “I feel fine, you’re nice.”

(It appears we successfully portrayed the application as a real person)

sbazmeno mihani

The studio is set, the lights are dimmed, headphones are the sound source of choice, and we are ready to begin the series of individual experiences that is sbazmeno mihani.

I will be on the door, and at first I considered making some sort of performative character and backstory to the workstation being in the room, the group decided that this artifice was not neccesary.
I will be there to let each User in and to tell them "A workstation has been set up for you to interface with any way you want, please put on the headphones provided",
and once they come out for the studio: "Each experience is unique to the individual, there is so much to learn yet left unlearned, please do not divulge details of your experience to anybody else".

Ruari is responsible for the poster and register on the door; being limited by time (we only have the Studio until 2pm), we have set up a sign-in sheet so that we are fair to the Users (plus we have to work lecturers' experiences around other pieces being performed today).

Marina is responsible for being our 'mole' within the inner corridor. She not only starts the soundscape once the User has reached the beach, but is responsible for taking covert photographs of the Users, and ensuring their general wellbeing.

Unknown to the Users, the false-ELIZA style window not only regurgitates their answers back to them to simulate conversation and intelligence, but using the File I/O Xtra, is able to save their answers onto my hard drive, so that we as "System Administrators" can view and record their answers for later evaluation, or possible blackmail...

ARGH!

Some bugger has dumped a load of staging, seating, and assorted crap in our lovely studio! It wasn't like this yesterday when we set up the lighting.

Well, there's not a lot we can do, we can't move this much stuff especially as we have a performance looming, it's just frustrating how some people can be so bloody inconsiderate!

A night to remember

Ok so, having collaborated together as to the structural and conceptual whole of the piece, yet designing and coding our own individual parts, there came a time when we would have to put all the different parts of the piece together. And what a night it was...

Yesterday (Wed 8th Feb) I met up with Marina and Ruari and we went back to their place and set my computer up next to Ruari's in his bedroom. Still attempting to integrate webcam capabilities by using a hidden networked laptop, we attempted to set up a LAN between both PCs, however, the machines decided not to play fair.

Whilst at their Totnes house, I finished the design of the Desktop, complete with faux-ELIZA questioning and left it to lead into Marina's beach scene. Marina spent much of the day and evening having technical problems which prevented her from making any real progress with the design of the beach section. Slow-running computers and corrupted source files severely hampered her progress, until I finished my work and let her finish her design on my (faster) PC.

During this Ruari was completing design and coding on the forest finale, creating surreal images and intense error-ridden passages.

An interesting aside; in programming the Director projector (Desktop) to load up Marina's Beach section on conclusion, I also set Marina's to play the soundscape Ruari had written. Unfortunately, once the Beach director file finishes and loads up the Forest, the sound stops. As the Forest is too large to import into the Beach Stage, we're going to play the soundscape on Ruari's stereo in the studio, starting it by remote control once the user begins the Beach section.

However, I tested the application whilst listening to the soundscape through headphones and had a very eerie, subjectively terrifying experience, so when we have the space set up we're going to see if having the speakers really loud, or having the user wearing headphones is the best way to experience the sound.

Ruari eventually fell asleep (at about 4am this morning) after finishing the forest segment. Marina and I continued to test the timing and progression between sections until we were happy with it at approximately 6.30am!

We slept until 7.30am, now we're at Dartington waiting for SPU - sorry, Sound and Vision to open so we can begin to set up the studio for the performance.

Setting up

We arrived in Studio 2 for our allotted 60 mins booking time (it was all we could get) today, and Marina set up a nice downlight into the centre of the studio. We decided against having the mirror open, so the User really felt isolated within the space. It is a great setting, the walls are black as is the floor, so having a small table set up in the centre of the room really does seem like there is nothing else around you.

We also found an opening through from the corridor outside that was blocked by a large wooden screen. In creating a small niche through which one could see the User during the experience, not only will we be able to check their progress / make sure they are ok, but Marina plans to take some photographs at the same time.

We found a small, low table outside the studio that will be perfect for placing the monitor, keyboard, and mouse on. Initially we planned to have a standard office table and chair present, giving the impression of an entirely ordinary computer workstation, but having the low table, with the CPU tower on the floor and forcing the User to sit on the floor too is perfect to make an impression that this computer is something special indeed.

We blacked out the windows in the door with gaffa tape, and also covered the red lights on the alarm boxes, so that other than the feint downlight, the only lighting will be from the monitor.

Everything is going well, now it's off to Marina and Ruari's house to finish the designing of the piece.

File I/O Xtra makes conversing simple!

The ELIZA-style webpage now expires after 7200 ticks, and after a stylish deconstruction animation (ok, I admit the screen flashes black a few times), goes to an exact reconstruction of the ELIZA webpage interface (except instead of pressing the Enter key to submit the answers, you press the “Accept” button.

This first occurs before the ELIZA JavaScript runs, where the computer asks for your name. This is then saved using the File I/O Xtra onto a location of my choice on the hard drive (which means the program will not run entirely off a CD).



A (reasonably) simple bit of coding later reads this (.txt) file and spits the file contents back as a Cast Member which can be used like any other asset. This gives the illusion of the conversation continuing as it did before, but now we control it’s progression.



This also means that the file contents can be read by a different Director file (such as the Beach) and used within that file too, giving seamless integration between the files.

Working Desktop Clock

We now have a fully functional and accurate desktop clock, using the worlds most simple coding. Just make a Text member called "clock" and make it look like the system clock (white text, blue background, font matched etc).

Then plop it in the right place on the toolbar (making sure it's on top of the other sprites), and any frame where you want the clock to update (I did it on every frame hold) just add the correct scripting. Consequently on the frame holds we have:

And Bob's your Auntie, you have something like this:

that updates as often as you want it to!

I'l be there in 2 ticks... (or 7200 really)

Now that the ELIZA style conversation is almost completely coded, I need to look at ways to go from the fluid, JavaScript keyword recognition, to a linear conversation that Ruari has composed.

To do so I'm looking at starting a timer once the User is "conversing" with Snivitz, and after 2 minutes (7200 ticks) move the experience along to a staged version which has the exact look and feel of the previous version, but has carefully written dialogue to seague into the dream sequence.

Information on coding time-based interfaces can be found here:

http://webzone.k3.mah.se/k3jolo/Lingo/lingo5.htm

JavaScript Security

This is an interesting one for you; now I have the ELIZA-style web page running over the desktop (thanks to webXtra), and I have coded the "name save" dialog box, you'd think that it would be easy for the chat program to read the name file and begin the conversation with "Hello (name)".

WRONG!

JavaScript has incredibly tight security rules that do not allow any reading or writing of text files by design. Bugger.

Latest web-based research

Had another conference with Marina and Ruari again today; not only is Marina progressing nicely with the dream sequence, but Ruari has provided me with a copy of the ELIZA-style question and keyword sequence so I can code it over the weekend.

Some of the latest web resources I'll be trying:

http://www.phord.com/keyblock.html - blocks a number of keys for kiosk-based apps which include:

• Ctrl-Alt-Del
• Ctrl-Esc
• Alt-Esc,
• Alt-Shift-Esc
• Alt-Tab,
• Alt-Shift-Tab
• Ctrl-Q
• Ctrl-W
• Alt-F4
• Right Windows key
• Left Windows key
• Applications key

providing it is XP-compatible that will enhance the user's experience nicely!

http://xtras.tabuleiro.com/products/web/index.tdb - enables web-browsing capabilities within Director, should help with the ELIZA-based JavaScript. (Tested and works with the project - although you need to purchase and register it first!)

http://webcamxtra.sourceforge.net/download.shtml - webcam integration within Director.

Email to Smoothware re: trackthemcolors

Here is the email that I sent to info@smoothware.com
(complete with accompanying screen grabs). Hopefully they may be able to provide some answers...

Good Afternoon,

I recently downloaded the demo trackthemcolors Xtra for Windows and I am having problems installing it.

After putting it into the Director Xtras folder

the Director program itself reports that it cannot find it!

In trying to open trackthemcolors8.dir it states the following error message:

I'm trying to integrate an Aiptek VGA PenCam into a project and I have
yet to even attempt to get it to work as the Xtra appears not present. Your
advice on this matter would be very much appreciated.

Yours faithfully

Steve Beaty

(Plymouth, UK)

--UPDATE--

As of Wednesday 1st March Smoothware have yet to reply to my email...

Meeting with Tim

This morning heralded another meeting with Tim. After explaining some of the more frustrating technical problems (Eliza JavaScript, trackthemcolors Xtra) he came up with the following technical suggestions:

• Seek advice from Director Forums as to how to integrate Eliza JavaScript into Director form.
• If unsuccessful use the XP illusion to perhaps open Internet Explorer or other applications to run the java.
• Contact Smoothware to seek technical support on the elusive "missing" Xtra.
• Incorporate web-cam accessibility through the use of a second computer networked to the interface that can capture and provide the images.

He had some very good suggestions as to how to incorporate the live performative elements to the piece, and also some advice for documenting the reactions of the participants.

The most useful piece of advice was that next week's "final" showings will be more of a prototype showing due to time constraints within the course, and that the 3 weeks between the showing and the final submission could hold more improvements and another finalised presentation.

Watch your Cyclic Redundancy Checks!

I had a good meeting with Marina and Ruari today, we finessed the look and progression of certain elements within the programme, and I also received 4 CDs of source Director Projects from Marina depicting the actual content of the 'dream'.

However, when I arrived home I found that only disc 3 (CD-R from W H Smith) would actually transfer the data to my hard drive! With the remaining 3 "Mr. Data" discs I kept receiving a "cyclic redundancy check" error (I'll have to remember that for the denoument of the experience). This basically boils down to either not incredible quality CD-Rs, or writing with massive buffer underrun. Either way, the discs were only food for coasters until I did a bit of digging regarding CRCs.

In looking at CRCs more closely I found a freeware application that actually checks the file, then restores it if indeed possible. At the time of writing this entry I'm 22% (10 mins) through the first of 3 discs to be checked!

More information about CRCs and the CDCheck application can be found at

http://www.softwarepatch.com/tips/cyclic-redundancy.html

Buddy API

Thanks to Tim providing me with a number of useful Director projects, and an Xtra known as Buddy API, I'm now looking into DisableSwitching, where I can give the impression of a restricted kiosk-style interface (although it won't actually be a restricted OS).

http://www.mods.com.au/frameset.htm

Java integration with Director?

I admit that where my knowledge of Director and Lingo is limited, my knowledge of Java is non-existent! To that end integrating some ELIZA programming into the interface when all I have to work with are some Java scripts would be really difficult - that is, until I found this particular gem of an Xtra:

http://www.mediamacros.com/item/item-1006687212/

Called Moka Xtra, it promises to fully integrate Java within the Director Stage! I shall do some tests and see how well it fares, with an update later...

UPDATE: Seems more of a tool for checking JavaScript that works within the Director MX 2004 build. Don't think it's going to be of much use unfortunately...


(For more on ELIZA, check the links bar to the right of this post)

Interactive Desktop complete

After a day full of hard work, I got the desktop interface designed and coded.

The most successful test came when I told my sister that I'd made some changes to her User Account and that she should see if it was any easier to use than the last one. She got very frustrated and believed wholeheartedly that it was in fact a genuine XP desktop.

If I have time considering the magnitude of the project, I might insert some Lingo that will pull real-time data into the clock display - if I have the time...

Interactive Desktop initial testing

Believe it or not, but one of the most useful bits of Lingo I've discovered today has been how to insert a carriage return into an error message! I found the coding here: http://livedocs.macromedia.com/director/mx2004/ and it basically boils down to

alert "Some alert text" & RETURN & "a bit more alert text"

Funky huh?

Huge technical difficulty already...

After much testing and messing around with the Xtras folder, I look back on the Smoothware website (http://www.smoothware.com/compatibility.html) to find their "compatibility table". And lo and behold in black and red there is...

The Bad News:

DV FireWire Cameras

so our finely laid plans to run one USB webcam and one FireWire MiniDV camera on a single machine have gone up in smoke.

So either we use two separate machines networked together, or I find a clever way to run two USB webcams on the same machine at the same time...?

Technical meeting with Tim and Duncan

We had a tutorial with Tim (Dollimore) and Duncan (Speakman) today, first expanding and elaborating on the initial Project Proposal (providing more in-depth description of the performance space and the interface intelf), and then going into detail regarding some of the more technical issues.

Duncan suggested some extended research from http://www.boredomresearch.net 's "Ghost" project, in addition to further manipulation of Weizenbaum's "ELIZA" programming.

Tim suggested solutions to technical problems such as trackthemcolors, an Xtra for Director that allows real time integration of live video into a Director project. Available from http://www.smoothware.com/track.html, I have a horrible feeling that it is version 1.2 that allows video and not version 1.0, which is the free demo. Argh!

Without this functionality it looks like a lot of the donkey work will have to be done on-line, with the performance machine networked to another that will capture and transfer the recorded video.

More updates to follow soon...

Project Proposal (in full)

Project Proposal and Technical Requirements - complete

CONCEPT
We will be examining the symbiotic relationship between man and machine, the idea of lucid dreaming, conversational 'rules of interaction', (Andy Lippman/ Allcquere Rosanne Stone), Turing's test, (and the nine arguments against), arguments for and against artificial intelligence, Weizenbaum and the Eliza program, constructing interfaces, subverting the ideas of traditional interactive 'programs' within systems, non-linear story-lines, (speculative fiction), solipsism, chaos, and the idea of blurring borders between installation performance and real time interaction.

FORM
The piece will take the form of an interactive digital application that will enable the user to converse with the computer in a similar way to the Eliza program. This will also combine pre-designed programming with elements of real-time image and data capture (web-cam, user-inputted text) that will be inexorably linked to the content of the application.

CONTENT
An interactive non linear story, experienced through a series of photographic-image-montage-stuff.
There will be the opportunity to converse using Eliza-styled programming.
Some of the material will be generated exclusively through user input, for example the entering of user name and photographs which will influence the experience by appearing later on and affecting how the program is run. eventually the experience will appear to completely degrade into a confused and chaotic series of 'errors' based upon the user's previous actions, which are recorded within the program(in a time-line).

SITE
We will work in studio 2 from 9am-2pm.
The experience that the user will have will be an individual one, therefore only one participant will be conversing with the computer at any one time. They will enter the room on their own and be presented with a solitary computer at a desk, down-lit from above in an otherwise darkened room.

SKILLS TO DEVELOP
Steve
Develop the use of Director and/or third party plug-ins to control a web-cam, then using that material within the 'movie'.
Taking user-inputted text and using it within the narrative framework of the piece.
Successfully replicating the operating system to present the illusion of consciousness form the computer.
Using the 'time-line' to facilitate the journey of the user through the experience.
Ruari
Research and develop non-linear story writing skills.
Continue to examine concepts and integrate them into the piece.
Develop the use of Adobe Audition to create an accompanying soundscape.
Marina
Develop the use of photoshop for the manipulation of captured image
Research the concept of dreams to develop the narrative.

TECHNICAL REQUIREMENTS
1; Desktop computer with monitor
2; Extension cables
3; Gaffa tape
4; one table
5; one office chair
6; one or two lights to be rigged
7; lighting desk
8; lighting gels
9; 2x web-cam or video camera
10; promotional materials

Welcome to the Narrative Threads project.

Welcome to the blog of the Narrative Threads project!

This is a Digital Arts collaboration between Ruari Floyd (Writing), Steve Beaty (Music), and Marina Savva (Theatre).

With the title Narrative Threads, we are looking into ways to explore exactly how human beings interact with computers and where the lines between us and the machine are blurred. Expect an update with extracts from the Proposal soon.