IRL Experience Design

Eureka Room User Testing Feedback

Over the last couple weeks I’ve had some friends come over and test new programs. They’re not so much programs, as ideas I’ve cobbled together to test the mechanics. In fact, I call it mechanic testing rather than beta testing.

Test groups were:

  • Leila, Brian, Alix, and Andrea
  • Wendy, Ross, and Melissa
  • Rebecca, Ryan, and their two friends
  • Kevin and Heather

I took a lot of notes but for this post I’ll just focus on the bigger items. I’m also going to talk about it like they were all there at the same time, but it was three different nights.

Program Name: Caw Caw

This program is a one trick pony: it switches back and forth from high energy crazy to calm and relaxing. Neither of those parts is particularly interesting, just the surprising transition.

Verdict: Shelve it.

Program Name: Hello Howdy

The idea on this one is that you are assigned a color and you have to follow it around the room with your eyes. When it changes into a certain shape, you say “howdy” to the person the color is behind.

Observations

  • People bump into walls too much. (Deal breaker)
  • People aren’t looking at each other, just the color. This might be worked around if you had a more personable thing you had to do with that person (shake hands, nod, make eye contact) but it wasn’t working

Verdict: Shelve it until I have a bigger and sturdier room.

Program Name: Laugh Game

This program was a recent reincarnation of the laughs and smiles game from mindfultainment. I didn’t think it would be good even at this stage, but thought maybe something would come out of the testing that helped me get it to a better place. Nope.

Observations

  • It’s just kind of boring. The vocalizations are fun but I took those out and put them in the harmonizer program, so this is left with funny faces and “ha HA” type call and response. Not enough payoff.

Verdict: Shelve it. Stop going back to this one.

Program Name: Harmonize

“Harmonize” is the working title. The idea is that colors move around the screen and you change your tone based on what you see. Each person has a different color. Though with bigger groups I might have people double-up.

Observations:

  • People love synchronicity. Not just doing the same thing either. They enjoy a weird back-and-forth and also pattern making. Rhythm and Mastery win over and over
  • I thought funnier sounds were: “uh” and kind of whiny sounds. Or like scooby-doo when he is confused. People liked being able to choose the sound (and change it).
  • People who like the stage like this. People who fear the stage do not like this at all. Putting people on the spot is fun and funny to some. Damn near traumatic to others. Ryan said he sat in fear that that moment was going to come and he’d be left making is tone solo. If I do on the spot make it quick and over before they can feel self-conscious. This was Rebecca’s favorite program. I think also Heather’s. This program is made for performer-types.
  • No one seemed to mind the lack of special effects. I think the quiet darkness was nice.
  • Arpeggios went well and felt special in a way that the others didn’t. A nice calm meditative feel.
  • At one point in their harmoning, I blurted out “Live Music Captiol of the World!” Maybe that might be a theme I could use. Like be a part of.. or join… or something like that.
  • People liked the names of the sections even though it was just for my notes. They liked that it wasn’t exactly helpful to what they were about to do.
  • When I did the Volume shapes, this got them louder and more into it. Maybe use this one after quieter or slower sections.
  • Experiment next time with 2-syllable sounds.

Verdict: Keep this one and develop a way to accommodate (or weed out) the people who will not like this. Keep adding more synchronicities.

Program Name: Chant The Thing

I show you pictures and you say what they are. Nothing hard. The joy comes from saying them as a group and being sync’ed with the music.

Observations:

  • This leans on the joy of mastery. I felt this got the quieter people out of their shells because it had more of a “purpose”.
  • As always, synchronicity is king. People love it when it happens and it builds and gets louder the more they are all on the same page. The split screen offered some of that as well, but it was too varied. I need to get more of it built into the split screen part.
  • I had added in some special images (like hay) that people had to say or do something special (like say “hey!” to someone). Turns out this was not just too hard to do but took them out of the rhythm and focus they were enjoying. There are nearly infinite things I could show them. Based on a few of the more interesting things I showed, I think I can stick to just pictures and make it a fun show.
  • The 3D graphics going down the sides of the room were distracting. 2D graphics were not.
  • Ryan said he had trouble with “potato” because it was a longer word. Upon my further examination I think it might be hard because it has the second syllable stressed and the words I had it with were often first syllable (Idaho and buffalo are harder to say with potato than tomato or gazebo)
  • People liked the blurry images and the transitions on screen, so more of that next round.
  • People liked the music and thought it fit well.
  • Certain words are just funnier than others and certain words work better with others. Obviously, but also to the extent that it’s a big deal.
  • Putting things on the side walls, even a big letter E or O or M continues to be difficult. I’m going to drop that.
  • When everyone gets it right except one person, this is very funny.
  • When there’s something on the screen that stumps everyone, it can be funny. Don’t overdo it.

Verdict: This is a keeper for sure. Stick to putting images only on the front wall and 2D backdrop. Make it a nearly constant stream of mastery and synchronicity.

Blogging Update

Note: This post was written on 3/1 and finalized today.

I believe many people fail in their attemps to make new habits and accomplish things because they set up rules and strategies at the beginning of a project but then don’t go back to review and update them frequently enough.

I want to share an example of how I review my strategies.

Ask The Right Question

Be sure you know what you are reviewing. Reviewing your strategy is not the same as reviewing your interest in the strategy (or the mission or project, for that matter). “Do I want to put in the effort to do this?” might be a good question when checking in with yourself, but it’s not a good question to check in with your strategy.

A question like “Is this strategy a good one?” is really asking for a judgment. How will you arrive at that judgment? If it’s a personal project you might not have any metrics or clear vision and as a result base your answer on how you are feeling in the moment. If the work has been exhausting you or if you are hitting lots of walls you might be tempted to say it’s a bad strategy. But sometimes the work you need to do IS exhausting or DOES require hitting lots of walls. Don’t let your current emotional state make the decision.

Another situation the “Is this strategy good?” question can be tricked is Shiny New Things. Some bright shiny new thing is seen on the horizon which looks more fun, exciting, and promising (at a distance) and the whole enterprise reorients toward that. Until the next Shiny New Thing appears on the horizon. This can keep happening and delay or completely prevent you from getting where you want to be.

One more thing to be on the lookout for is: Feelings can trick you into thinking they are intuition. Intuition is real, but before you change the whole strategy based on your “gut”, make sure you aren’t in any kind of heightened emotional state. One of the best tests for real intuition is if it happens during moments of emotional calm.

The question to ask is “Is this strategy still a good one.. to accomplish the goal?”

For this question, you could use metrics and have rules that determine if it is working. But I’m talking about goals that are more casual and don’t exist in measured environment.

An Example of My Strategy Review

Currently I draft blog posts 5 days a week and the other 2 days a week I update drafts and put them on the schedule. I’m currently scheduling 4 weeks out.

Scheduling that far out causes this problem:

I draft something today and it doesn’t get finalized for another month and then by the time it hits the scheduled, it’s posting yet another month out. By the time a reader see it, it’s way old news. Instead, I’d like to schedule it and have it go live in just a few days.

So instead of asking “Do I like this?” or “Do I want to do this?” (these questions are for a higher level analysis), I investigated the strategy. Is this strategy still a good one to get me to the goal?

Well the goal was that I’d have posts twice a week. It’s done that. In fact, the problem is that I have *too much* content and things are backed up in the pipeline. But really, the goal was that I’d have two TIMELY posts. I didn’t really state anything about timeliness in the beginning. So the answer is really.. no it didn’t get me to the goal.

Then I dig in with some why’s.

Why do I have so many posts in the pipeline?

Answer: I was worried I’d run low on posts so I made a buffer.

Why do I think I would fall behind?

Answer: I was afraid that not everything would be post-worthy or that I’d miss some days of writing.

So here’s where we see that obviously this has not come to pass. I blogged 175 days in a row. Most of the content is good enough (IMO) to publish. In fact, I have 6 weeks of publishable posts and another 24 weeks of posts that need final edit. I have LOADS of posts. The odds of me not producing anything for months is practically zero.

Now I can see that I can change the strategy to adjust for reality.

Since every day is easier than some days, I’m not going to stop blogging every day. Which means I’ll have more content. One of the goals of the mission is to have a blog with lots of good content, so why not push more of the content out? I’m going to keep my 4 Scheduled and 8 Final Draft, but as long as I have more than 24 items in First-Draft form, I’m going to schedule a 3 post each week and see how that goes.

Qualify Visitors to Get Quality Visitors

Note: This post was drafted on 2/22/2021 and finalized today.

When you go to a business of a known type you have a general idea of what to expect. You know what to expect at a restaurant, a bar, a department store, an escape room, a baseball game, etc. 

But at the Eureka Room you have no idea what is going to happen or what you will be asked to do. This is by design, because it adds to the anticipation and surprise. But it presents a big problem for the visitor and also for the Eureka Room.

How do people know if they will enjoy the Eureka Room if the Eureka Room won’t tell them what it is?

How does the Eureka Room make sure that the people who visit will enjoy it?

I’ve had over 1200 people come through the Eureka Room and most seem to have enjoyed it. But I’m sure some didn’t. Some probably thought it was terrible but just didn’t want to tell me. 

This might sound crazy but I hope some people thought it was terrible. Or, at least, didn’t like it because it wasn’t for them.

Because when you make something for “everyone” you’ve made it so generic that it probably won’t resonate strongly with anyone. When “everyone” is your customer, nobody is your customer. To paraphrase (or possibly even misunderstand) Priya Parker, meaningful things have disputable value.

Things are meaningful to us when they resonate with those parts of us that have a unique interest in them.  If I’m going to make something awesome, I’m going to make something that isn’t awesome at the same time. Because awesome is a individual judgment made by one person.

Brass Tacks

So this philosophy sheds a light on what I’m trying to do but it doesn’t do the work.  Now that I’ve accepted that some people won’t like it (and that’s a good indicator), how do I sort out the good-fit people from the bad-fit people?

Here’s some practical methods I came up with:

Tell them what will happen. 

The blunt solution. This will ruin some surprises but not all of them. I consider this the option of last resort.

Offer Spoilers.

Keep the website the mysterious thing that it is, but include links to more details about what will happen. Not full spoilers, but more than the ideal visitor would ever want to be told ahead of time.

Pre-Qualify Them With a Quiz.

Before you allow them to buy tickets, make them take a short quiz to pre-qualify them. I don’t view it as a pass-fail test where only those that pass are allowed in. Instead I’d like it to tell them things like, “There’s  95% certainty you will enjoy this.” Or even recommend specific programs based on their comfort needs and interests. I want to get the right people to the right programs.

I like the this idea because:

  1. The obvious: it helps them know if (or how much) they might enjoy visiting and helps me attract more of the best-fit visitors.
  2. It extends their experience at a low cost to me. How? Instead of the experience starting when they walk in the front door, the experience starts days or weeks ahead of time for them. 
  3. It helps them (I hope) feel special when they qualify. They can get the “people like me do things like this” feeling.
  4. The idea can be leveraged for word-of-mouth marketing. What kind of crazy business asks potential customers to take a test to see if they can buy their product? 

I can already see some obvious challenges to this: A group leader buys the tickets and not everyone in the group has filled out the form before arrival. People not understanding the process. People getting in a huff about this added step. 

And of course, me totally miscalculating that people will do this. 

Framing this pre-qualification process is critical here. This has to be a fun thing for them. They get to see how they rank – people love seeing what their “score” is on tests. They should feel really special that they get to take the test, that they get to have their results (for free!). It should feel like they are trying to get into an exclusive club.  It should not feel bureaucratic. It should feel like a game. A mystery. Fun. Absurd. Charming. 

Update 5/8/2021

I attempted the spoilers solution first. I wrote a long and (I think) humorous “spoilers” page, but ultimately I thought it was too big an ask for someone that might have just landed on the page from out of nowhere. I didn’t want to scare potential fans off by asking them to read a lot of crazy talk. I shelved it.

I attempted the Qualification Quiz. While I like the idea of the pre-qualification quiz, I decided to shelve it because:

  1. I was afraid that people might not want to do work to see if they could visit something they don’t know anything about.
  2. It was going to be more work for me and I might be solving a problem I don’t have yet.
  3. I think there’s good reason to believe that the secret nature of the website will self-sort people who are up for novel experiences from the people who want something predictable.

Instead I leaned on the visit page. I put a little copy on the “Visit” page of the website that I hope allow people to qualify themselves quickly. It’s not as much an experience but it’s also not as much work for someone just being introduced to the Eureka Room.

And I decided to make the site mirror the experience in the Eureka Room.. a little. I came up with the idea to make the main page normal professional and then the “Visit” page be really odd. That would mimic the experience of the room and might help the self-sorting of visitors. Currently I just have a hand drawn weird turkey, which sort of indicates that, but I hope to add more.

Maybe as I evolve the room I will go back to a bigger end-to-end experience and do some pre-qualifying, but for now I’m putting it on the shelf.

The Most Important Story

Narrative vs Story in The Eureka Room

For the purposes of this post, “Narrative” will mean the actual objective facts and “Story” will mean an individual’s subjective interpretation of what they have perceived.

When I design a Eureka Room program, I’m not just considering the narrative. I’m thinking about the most important story – the one that a visitor will be telling themselves.

The visitor’s story can include what the visitor pays attention to on the screen, what happens in their head, as well as their interpretation of what happens before and after the experience. It can include everything from the first time they heard about the Eureka Room until the last thought they ever have of it.   

Some of the highlights of a visitor’s story might be:

  1. “The Eureka Room is at some dude’s house!”
  2. “Joining us in the experience was this couple from Holland.”
  3. “The program was this meditation thing but you had to caw like a crow a lot. It was like singing a song sort of. 
  4. It kept getting stranger and stranger.”
  5. “At one point I and the Holland guy were cawing right into each other’s faces for what felt a minute.”
  6. “We had been stressing earlier in the day, but when the program was over it was like the sun came out.”
  7. “Mike invited us back to test some other programs next month.”

A visitor’s story might include thoughts like “wow. this is weird/fun/crazy” or “what’s going to happen next” or “I’m so glad we came to this”.  

If you’re not careful, it’s easy to see the creation as being the writer’s domain and the reaction being the participant’s domain: I, the writer, focus on what I am creating. You, the visitor, focus on taking it in and reacting.  Let’s hope it works out for both of us pretty good, huh? 

Um. Instead maybe let’s see how we can help the visitor write their story.

Help The Visitor Write Their Story

Instead of me focusing on my work and my narrative, I try to focus on the visitor’s story. Because the story that they leave with is way more important to both the visitor and the success of the Eureka Room than any narrative I could ever write.

I would rather have a jumbled mess of a narrative that creates a great story for visitors than to have a great narrative that visitors hate.  The latter might impress other creators like myself because they can appreciate a great narrative, but other creators are not my target audience.

My target audience doesn’t care if there are plot holes in the narrative as long they as get to leave with an awesome story.

Here’s four techniques of my techniques that I use to co-write the visitor’s story:

  1. I write the narrative while standing in the participants shoes, not the narrator’s shoes or the Eureka Room owner’s shoes. I ask “what would I want to feel now? And now? And now?” I don’t think about what I want to see as a creator (applause! praise! power! etc!), but what I would love love love to see as a visitor.
  2. I remember that the lead up to the visit, the location, and the welcoming to the space are all components of the story. I can use these to help guide their story. For example, I want to create surprise and wonder. So I purposely don’t tell them much, creating gaps in their story which heighten curiosity and mystery. But at the same time I’ll present some strange images or colorful backgrounds that I hope will trigger some associations in their head and maybe lead them to write a certain type of story.
  3. I give them status so they feel some power to write their story in what is mostly an agency-less experience. To do this, my go-to is to use dumb and/or crazy narrators in the programs. Using narrators that are a bit off their rocker can make the visitor feel superior to the narrator while keeping a sense of mystery. Both of these can be very memorable parts of a story. Visitors write lines like “The guy in the video giving us instructions was crazy. I was never sure what he was going to ask us to do next!”
  4. I create moments that I believe will be felt generally the same by most people. For example, I’ll create a moment (or beat) that temporarily pauses the action in the middle of the craziest part of the program. This gives them (I hope) just a brief moment to reflect on their experience, which I hope is something like: “this is crazy/nuts” or “I love coming to Austin” or “(friend’s name) has gotten me into another weird situation!”.

One very important note: I never ever ask “What do I want them to do now?” I am here to serve the visitors, not be their puppet master. Asking what I want to see them do is not putting me in their shoes.

Those are just a few of the ways I try to create a great story for the visitor, but ultimately the story is one they write.

What’s The Cost Of Testing In The Real World? Of Not?

Note: This post was written on 2/3/2021 and posted today.

The Messy Homes Tour was last night.  I think it went well and multiple people wrote to tell me it was fun. Despite testing nearly everything ahead of time, we had some tech hiccups.

Tech issues in meetings seem to be commonplace, both online an IRL, so I want to examine this phenomena.

The MHT was, essentially, a Zoom meeting that was streamed to Facebook Live. To test this, my co-host Rebecca made a fake Facebook event in a Facebook group and streamed a test Zoom meeting to it. It worked. We had already tested all the various Zoom/Manycam/Snapchat configurations we would use as well. Every scenario we could think of was tested. 

But when we tried it live it didn’t work.

Why? 

Because we didn’t test her streaming on the exact same page it would be streaming to live. (The Big Weekend Facebook page). It took a while but eventually we found that we had some permissions issues.

But we shouldn’t have permission issues, I thought. Earlier in the week I realized she would need permission so I sent her an invite to be an admin of my Big Weekend Facebook page.

It turns out that you don’t just make someone an admin on Facebook. You send them an invite and they have to *accept* it.

The invitation was mixed in with the slew of notifications Rebecca sees every day. Most of which are not worth reading. She missed the invite.

I could have checked to see if she had accepted the invite, but since I’m used to things like Google docs, where you send an invite and that’s it, I didn’t know to check. Also, finding the spot to check is nonobvious.

So when we kicked off the show and she tried to stream on the live page, it sent me a notification (buried, of course) that I had to approve her stream. (Because as far as FB was concerned she was not an admin yet). Of course I didn’t know this was an issue let alone to look for how to fix it. 

A typical reaction to this problem might be: “Well, now we’ve learned what to do so we know what to do next time.”

But I think a better response is to ask: “What’s the cost of doing an actual true live soup-to-nuts test?”

The “next time we’ll know” reaction might prevent future instances of this exact problem. But the help it provides doesn’t transfer from this scenario.

The proactive response is far more useful. It gives us a tool we can use elsewhere. Testing soup-to-nuts will find “unknown unknowns”.

We See This All The Time

Not testing the real deal costs us all the time when we take our setup out into the real world:

Their projector doesn’t work with my laptop. 

There’s not enough outlets in this room.

We should have brought that adapter.

Sure, it’s potentially embarrassing to show a livestream test on your Facebook page. But how does that compare to having it not work at all when you do it for real? Weigh the pros and cons of a real-life test vs. controlled environment test.