Attending the World Science Fiction convention on the other side of the world by remote telepresence robot

I spent the weekend of August 14th in London, though I personally was visiting Coeur D'Alene Idaho. In particular, I spent most of the weekend piloting, or you might say "inhabiting" a telepresence robot at the World Science Fiction Convention at the ExCeL convention centre in the London Docklands.

Here you'll read a story of the future of travel and meeting. A story with costumes, excitement and anti-robot discrimination. Of technological realities and potential and an understanding of how and where virtual presence works.

In particular I will explain how I came to feel I really spent the weekend in London in some sense -- not fully, but still in a real way.

I've been to this convention -- called the Worldcon -- several times before in the flesh. This time, I would be speaking at a small robotics conference in Idaho and thus would not be going. It's also quite a long trek, of course, and I travel a great deal already. But on a physical visit to London in June, another plan emerged.

In the past few years, no fewer than four different friends of mine have started telepresence robot companies, and a few more have experimented. That's a sign there will be too many of these devices coming out. Scott Hassan, who created the Willow Garage robotics lab, spun off the "Beam" project. The Beam is the best of the devices, and also the most expensive, the reasons for which I will outline below.

(Scott is famously bothered when people refer to the Beam as a robot, because among other things, robots need to have some sort of processing on board that makes decisions, and the Beam is more correctly a remote controlled machine like an RC car or a Waldo. However, the reality is that almost everybody calls it a robot, and it's too much work to try to spend time correcting that with everybody you interact with.)

We have several of the Beams in our offices at Singularity University, and another friend, Sanford, who used to work for Suitable (Beam's maker) now lives in London and is establishing a business to rent them out to people. While in London I asked if he might be willing to let me use one of his at the WorldCon.

The experience was remarkable and highly worthwhile. We spent close to 10 hours a day in the Beam over 3 days. The Worldcon is half conference, half giant party, and while we attended a few panels and the Hugo awards ceremony, the bulk of the time was just spent wandering around and socializing.

A Scottish woman cosplaying as Lara Croft chats with Kathryn and myself during evening parties at the bar. Photo: Mez Naam

This convention was perfect for this experience because the convention centre is modern and (mostly) accessible to wheeled people, with a decent but very expensive premium wifi network. Also particularly important was that almost all the convention parties -- which go on past 2 in the morning -- were held in one of the large convention halls. Traditionally they are in crowded suites at a "party hotel" some distance from the convention centre, and scattered over many floors, though each event is different. The large space meant the parties would not be quite as densely packed with people as they often are. It was a rare opportunity.

You drive the Beam from a desktop/laptop client. Driving can be done with arrow keys, the mouse or a game controller and is reasonably easy. The Beam shows a wide angle view from its top camera, and another view of your base and wheels so you can drive and not run into things. The Beam features a large monitor a little over 5' high on two poles, and a very heavy base. It also features reasonably powerful speakers and an array of microphones -- features that are particularly important to doing this well.

Having done a lot of videoconferencing, I knew it was also important to have a good camera at my end, and to light myself well. I also always used a headset, and when Kathryn joined me, I got a nice free program called "Voicemeeter" which can let you merge two headsets. As good as the Beam's speakerphone tools are, nothing comes close to using a headset for video conferencing or driving a telepresence robot. The nice clear view of me and clear sound helped a lot in socializing in large, noisy rooms.

On the other end, the Beam also sports a decent camera and its microphone array. You can put it in "party" mode and that makes your own voice louder, and also attempts to use beamforming (no relation) sound technology to isolate the voices of the people directly in front of the machine. Without the party mode, you hear too much background noise in a loud room. In the party mode you hear much less background noise, and in fact you barely hear people behind you or far away. This is great for a party, but it also means people can be right behind you and you don't know it. Sadly, the party mode distorts voices a bit, though this should improve with time.

When driving around in quieter areas you want to switch out of the party mode so you can hear people call out to you, or even hear what they are saying around you. You also need it to listen to presentations, as otherwise sound is too distorted.

Controlling your own sound is an area where the Beam needs improvement. You need to manually control your volume and you have few clues as to how to do it based on the remote noise level. Humans do this naturally, and the devices need to measure the ambient noise levels and distance to other parties and set a more automatic voice level.

It was particularly rewarding to be the first to use a device like the Beam at a convention like this. Science Fiction readers were of course largely fascinated by it and a thousand photos were posed for and many tweets were sent out. Almost everybody was extremely friendly. Too friendly perhaps, as it became very difficult to travel with purpose, as every few feet people wanted to stop me and chat. People were super helpful because of the novelty. Others who have done this have noted that they go to events and have no trouble getting famous and important people to talk to them, people who would not normally do so if you were there in the flesh. If you want to meet people this is a good way to do it.

Many were super helpful. Not just opening doors and pushing elevator buttons, but being my hands (thanks Gillian) when I went into a charity casino, or setting a place for me (thanks Moshe) to watch the awards ceremony. (Because the Beam is 5' tall it must go where it will not block other people's view.) They were not just willing to help but eager, in order to enjoy the novelty.

The operations manager for the convention even asked me to do a volunteer ops shift, answering people's questions, which I did, and got "Staff" status in exchange. With a bit of preparation, such as a link to the staff radio network and databases, there is no reason one could not do staff work piloting such a robot. Indeed, Suitable Tech believes many jobs can be done entirely remotely, and even staffs their experimental retail store with nothing but remote workers -- there are no flesh humans in the store at all.

In a couple of the sessions I attended, I was always called on when I "raised my hand" to ask a question -- I suspect I may have jumped the queue in one case, though I could not so readily see the rest of the room without physically spinning in a way that would be distracting.

In a few years' time, this will wear off. By the time there are 100 of these at a convention, I suspect people will actually be slightly more negative towards them, and view them as the lazy or cheap folks who could not be bothered to come in person. Today it is mostly the opposite.

Inhabiting the Beam gives you a small taste of what disabled people experience living on wheels. In that sense it's worth doing for everybody. In addition, many folks in wheelchairs were quite interested in the idea of using this technology in the future. Even though they might be reasonably able to get around an accessible convention centre, the act of flying a long distance and getting to such a building in person can be a major burden for those of various disabilities. Several wheelchair-bound members of the convention were quite keen on the idea. (See more below.)

Use of the Beam is quite different from just calling somewhere with Skype. You are in charge of yourself, able to move and turn your view where you like. You can seek people and things out. You don't need to make an appointment, or even get somebody to accept your call.

Suitable Tech promotes this as the big value for companies that have branch offices. It's better to be able to connect to a Beam in a remote office, and it can substitute for flying there on a business trip. That's no small thing if the office is far away, or even requires visas and immigration. A factory in Shenzen has a unit to let customers connect and see how their products are being manufactured.

The Beam uses a charging dock that you just drive on to. While the battery is rated for 8 hours, with age, my unit was doing about 6. The time zones presented an interesting challenge. We were also spending the weekend in rural Idaho and didn't want to miss it entirely, and on the Friday we spoke at the conference. We found ourselves waking up at about 6am, but that was already 2pm in London. To do it properly we would have converted ourselves to London time. That turns out to be more difficult than you would hope -- the needs of the physical world call out, and the sun keeps telling you what the the real "flesh" time is.

After waking and putting on some clothes, I would connect to the robot, which was docked in the social area. It would always surprise the people hanging around the dock, because for them the robot sits quiet for the first several hours of the day. Suddenly a face appears and it drives off, and this always caused exclamations.

I found it amusing to say "Good Morning" to everybody as I passed, though for them it was mid-afternoon. That was just for fun, because in other ways, I acted like I was in London.

In emulation of Scott Hassan, who created the Beam, when people would ask where I was -- and they all asked even though I had a sign on the robot saying I was in Idaho -- I would say, "I'm at the Worldcon in London, where are you?" Some took this to mean I was saying I was driving from London, others found it appropriately amusing. I would then say, "Oh, you mean my flesh body. I guess everybody here is obsessed by the flesh. That's in Idaho." But in truth I was in both. I felt like I was attending the convention, but at the same time I did things like pop down to the lake for a swim during a charging break, and going out at 6pm/2am after the parties shut down to photograph the scenery in sunset light and have dinner.

Also during the weekend, I attended a meeting back in California at Singularity University. Amusingly, only the local staff showed up in person, the two faculty came in on Beams. This had me in California, Idaho and London over a short period.

Ant-robot discrimination?

When I first made mention I wanted to do this -- just a few weeks before the convention -- I was a bit surprised at a negative reaction that came from a few folks working it. (SF Conventions are not professional affairs, they are all volunteer efforts, and are suitably loaded with personalities and opinions.)

While most of those involved were thrilled with the idea, a few were bothered. The convention has an official policy that you must not record people if they ask you not to. The Beam does not have recording ability, though of course one can install screen recorders on laptops or simply point a camcorder at their screens. As a well known privacy advocate, I felt I could be trusted in my promise that I would not and could not record, but this was not sufficient in some views. Video would be transmitted at all times.

They actually issued a ruling forbidding the machine from a variety of sessions, including a reading by George R.R. Martin (of Game of Thrones) and the rooms where videos were being shown. This was seen as similar to the efforts to stop people from bringing camcorders into video rooms, and I did not protest, as I had no desire to go to those rooms.

Much more controversial was a ban on going to two parties I very much wanted to attend around the Hugo awards. There is a reception for nominees before the awards, and several of the nominees are my friends and were willing to designate me as their guest. There is also the highly popular, invitation- only "Hugo Loser's Party" after the ceremony.

The word eventually came down I would not get to go to the reception, but the Loser's party was more likely. When I got to the door, though, the manager of the party barred me from entry. He believed that simply transmitting from a private party without express consent of all there would violate the law, and a large amount of arguing, from myself and others (including those who know the law better) did not sway the day. There was a 15 minute argument -- with all around shaking their heads in incredulity. Finally I just suggested that he go in and ask the room if anybody minded. Of course, rather than minding, they all cheered and in I went for a great party experience until my batteries started running low.

This question will come up again as more people try this, and get even more pointed when the disabled start using this as a way to get access. The SF Fan community is extremely supportive of disabled access rights, and will be quite conflicted if this debate is not settled.

There were a few other issues: One of the conference halls did not have wifi for me, and so I could attend nothing there. I was even invited to host a discussion in one of those rooms but that was not possible.

There was also an interesting issue about convention membership. I bought a convention membership for myself, though by some arguments a remote attendee puts less burden on the event. Sanford was given a free "helper" membership of the sort normally given to those who help a disabled member. I would have gotten him another membership if needed.

I didn't buy a 2nd membership for Kathryn, though. For about half the time, she came and sat with me and joined in. We were in one device but both of us were socializing. It's an interesting question if that demands two memberships. For further complication, I was asked by somebody from Suitable if hey could use the robot for a while. Should such time-sharing of the device require another membership? (People time-share convention badges all the time at some events of course, but most forbid it and even use Photo ID to enforce that.) None of this was done in secret and the question did not come up. Had it done so I would have been happy to do whatever was requested.

Watching the awards. Livestream on the left, Beam's view on the right

The awards ceremony also offered some interesting insights. The view of the robot was limited due to the high dynamic range of the scene. In general it is very difficult to get a camera to show you monitors or screens which are much brighter than their surroundings. An HD livestream was available so we also watched that. It had a 10 second delay, so we had to avert our eyes from the whole-room view to avoid knowing the future. We dimmed our camera so we would not be a distracting light in a dark room though an actual function for that would be useful. The livestream did not show clips from the nominated movies due to copyright paranoia, so we were able to watch those via the Beam. Nothing was stuff that needed such protection anyway, particularly from our low-res view.

In the Beam you can turn around to look about, but we decided in an audience that this might be disruptive, so we could not check out what the folks behind us were doing.

Technical Issues

To use such a robot requires lots of quality bandwidth and much better wifi than you find in most places. People who buy Beams are encouraged to put in a wifi network just for them using the less-used 5ghz channels. I suspect that if I had attempted to use the convention's free wifi, even in the 5ghz band, it would not have been a great experience.

The convention centre offers a premium wifi but at a steep price. It gave me one megabit each direction that was mostly reliable, and even worked in the main elevators.

The price was high because even the normal high price of 30 pounds/day plus VAT was not enough. In order to work well, the Beam uses two wifi radios, allowing it to do "handoffs" like a cell phone does as you move. It only uses one at a time most of the time, but the convention required me to buy two access passwords, making the total price closer to $125 per day, or $500 for the 4 days of the convention -- that's seriously expensive wifi.

On the other hand, it did provide good quality video for most of the time. About 5 or 6 times, I encountered odd one-way outages where there would be massive packet loss only on the feed from the Beam to my laptop (being relayed through Suitable's servers.) Because the loss was one way, it probably was not in the wifi. This meant my image got super blurry and impossible to use -- and as a result the Beam wisely shuts down your steering ability.

In one case, while on a balcony over the Thames after a nice chat with Charlie Stross (whose books I recommend highly,) we had a very long outage and I had to sit there until somebody came to me. They could see me fine so I could explain that though I could not see them, they might roll me back in.

In some cases, as I have seen elsewhere, the wifi goes out in an elevator. So you don't want to enter an elevator without a helper who can push buttons, and roll you out again manually when you get to your floor.

In theory, if a location has many sharable telepresence robots, you don't bother with elevators or stairs at all -- you just disconnect from one unit and zip instantly to the other one in the location you want. Alas, you can't do that if you are one machine attending a convention, and this also creates issues of allocation and permission.

All these technical challenges put more burden on Sanford (who was loaning me the Beam) than I wanted, but he also enjoyed the experience. At least for now, having somebody skilled at the other end is highly useful. One evening, the dock was unplugged and so when I returned it would not charge. Fortunately, in the London morning, Sanford was able to connect on the remaining power and ask local folks to check the wiring so that it could charge again before 6am rolled around in Idaho.

What doesn't work

While you can feel present in a space with the Beam, it is still limited. Your vision, while decently wide-angle, is still limited, and you don't see people to your sides or behind you. Apparently people were frequently running around behind and to the side of me of whom I was entirely unaware. This lack of peripheral vision is a bit like wearing a blinder collar on your head so you can only see forward and must turn your body rather than your head to see other directions.

This results in a limited situational awareness. You aren't always really aware where you are, or what direction you are facing. It's common to ask people for directions in a way you would not have to do in the flesh.

It's also not very useful for touring things like exhibits and art shows, though we tried to do that. When moving down an aisle, you can't really see what is on either side. You have to stop and turn and zoom the camera to read signs or get a decent look at things. Looking at art is not a particularly rewarding experience, the resolution is just not sufficient, even with the zoom-in feature the Beam offers.

There are plans to use telepresence robots to attend trade shows, but they need some upgrades if the goal is to stroll the aisles looking at booths. They will be fine for going to specific booths or meetings and various social settings. Convention centres and trade shows tend to monopolize anything commercial on the floor, so they will probably not allow foreign machines in, and charge people by the hour -- including convention attendance fees -- to do this.

The lack of resolution -- the Beam transmits at 640x480 though someday soon it will be HD and who knows, perhaps 4K as bandwidth becomes cheaper -- limits many things. You can't read people's badges without zooming in -- and they can see you doing this. You can't read room numbers as you are rolling down a hall.

Having a low-res peripheral view would be very handy. In the future I expect people to even drive these with VR goggles and be able to expect to turn their heads and look to the side without turning the robot. Latency presents an issue but more can be done. Even a view to the rear would be handy in spite of the fact humans don't have it. Our ears give us a better sense of where sounds are coming from that I did not sense here.

Aside from increasing video resolution, a few tricks could also help. For example, the camera software could detect things like badges, signs and other printing and send them at higher resolution, or isolate them in boxes shown to the side of the main video view. Peripheral cameras located to the left and right need not even send their video if bandwidth is not enough. Rather, they could spot potential faces in the field of view that are close enough, and send low frame rate images of those faces for display, just so you can see who is around you. This would let you greet people like a flesh human. (They don't need to recognize the faces, just see that there are faces, something for which there are now many quick algorithms and even hardware to do.)

Telepresence devices could also help you in recognizing faces, though. If the system detects a face, it could let you tag it -- one nice thing about people not seeing your hands -- and then if that face reappears later it could tag it, or show you the set of tagged faces most like that face off to the side, letting you finish the job and giving you a superhuman memory.

VR Goggles are an interesting idea, but you actually do need some awareness of your meat-body. You need to be able to see your keyboard and hands or people who come to talk to you in meatspace. VR goggles would need cameras to put those things in your field of view (using background subtraction) or you could just have a nice wrap-around screen setup instead and probably feel more comfortable. One big challenge with VR goggles is that people can't see your eyes if you wear most models, so things like Oculus Rift are probably out. An interesting option would be goggles that software can edit off of your face, by having an internal camera to see your eyes and images of the covered part of your face from every angle.

Who gets to ride the robot?

Earlier I wrote that while using these devices is novel and fun, the world looks a bit different once they are common. Their use will be tempting, especially for those who only want to spend a modest amount of time at an event. Even at high prices, it's easy for them to beat the cost of hotels, flights, meals and most of all the time cost of attending a remote event.

The cost of the robots will also drop. Most meetings have a large population of locals who didn't have high costs to go, and fewer people who came in from overseas. if it's as easy to come from overseas as to come locally, these demographics will change quite a bit. As an unintended consequence, the fact that remote events require a burden to attend is a feature of sorts, in that it makes the attendees more serious about the event, and this bar goes away.

We see this all the time in meetings that include remote participants by phone or video-call. The remotes are much more easily distracted, able to work on their computers and do e-mail without anybody seeing it. They are also treated as 2nd class citizens in the meeting unless they are the boss. One reason the Beam works well is it lets the remote pilot take responsibility for their view and position and audio. With stationary conferencing, it tends to suck because the people in the room don't really care a lot about making a good experience for the remotes.

At conventions with hundreds of remotes, they may be viewed as those who were not that committed to the event. Eventually, they will be treated worse, not better in the way I was. This will not be true of speakers, special guests and celebrities. They will be very welcome to attend by telepresence, especially if it was the only way to get them, or a way to get them for much less money.

People coming in remotely will truly be less engaged. We've all felt it. When you go to a conference, you are physically there so you give it much more of your attention. You attend sessions because you are there and an interesting session is going on. Often those sessions are available on video, live or after the fact, but we hardly ever watch these videos. A hundred conferences go on a day with interesting sessions that I would attend if I were there, but which I will never watch the video of. In a telepresence robot, you will feel the same, because now the choice is going to the session or doing something you need to do in meatspace -- or in another place in another robot.

The Disabled

I touched upon use by the disabled above. For them, this technology is highly interesting. Those who attend events with disabilities already face the challenges of getting around remote facilities. A person in a wheelchair can go anywhere a robot can, but the use of multiple devices with the ability to switch can allow a mobility not possible in a chair.

The bigger burden for many is that of travel. For many, long plane flights are just not practical or even possible, and so remote presence is the only option.

In addition, disabled people are generally in a much lower income class when their disabilities impede various classes of employment, and so they can't do expensive travel even if it is physically possible. While these robots are expensive, they will become quite cheap over time, and are already cheaper than flights and other costs of travel.

Everybody is in favour of helping the disabled this way. As a result, I think that the oppositions that came from the risk of video transmission and potential recording would largely melt away in the face of the trump card of access. Though the copyright police might still bar the machines from movie screenings even then. More in question will be how the devices are treated socially. While I predict that remote visitors will eventually become 2nd class citizens in the meeting world, we will want to avoid that happening for the disabled, and so we may develop a convention, such as the traditional blue wheelchair icon on the device, to let people know what's up.

Minor Nits and Fun

Here are some features that could make an experience more like being there, and a few amusing notes:

• Overtly taking pictures: Some times you do want to take them, but you also want to promise to not abuse this. So a button could take a picture and bring up the image of a camera on the screen and make a shutter noise. It might even offer a button to let remote people abort it to protect their privacy. Video could also work this way.
• Exchanging business cards: The above picture feature could let somebody hold their card to the camera, with a white LED for illumination. At the same time, the driver's card could be put on the screen for other people to photograph.
• Physical contact: Capacitance could tell you if somebody touches various parts of the machine, and convey a noise or vibration at the driver's end, to let them know they are being touched. One could even wear a vibrating wristband and feel it, or use the vibrator found in most phones.
• Steering: While I've gotten good at it, it can remain a challenge. An interesting option would be to let people lead you places by having a token mounted on the machine body which a leader can pick up and hold out, and the machine would then track the token with its camera and follow it. The leader would be responsible for avoiding obstacles and other people. The new Beam has a sensor to help you not hit people, but it makes it very difficult to move in crowded spaces as it is too paranoid.
• I used a special tool so I could put still photos and videos in place of my image. A popular one was HAL-9000 (A "HAL" costume) which won me a Hall Costume award. Also popular were a Dalek, a Terminator and a video of Edward Snowden, who famously used the Beam robot to appear at TED. The Hall Costume Award hung from the monitor, like my badge.
• People loved a bit of robot roleplaying. I got in line for a drink at the bar, but said I was turned away because my body was only 1 year old.
• When people helped, I often said, "You are a good human. We will kill you last."
• If the crowd was too thick I would bellow "Out of my way, humans!" and then say, "wow, that works" (because it does.) It was also fun to complain about the anti-robot discrimination or note that the event seemed very biased towards humans.

These devices face a marketing challenge. They are bought by the place they will live, but driven by remote people. The people getting the primary value are not the ones paying for the device, unless you're talking about a multi-branch company. As a user, I want the machine to be in the various remote places I want to go, not my office. To do that, a vibrant rental business is needed which lets you get machines sent quickly to destinations you want at a reasonable price, or has them already there.

Even the very small learning curve makes them less practical as a way for people to visit you with one that you own. If all you want is a video call, a standard video conference tool is simplest. You generally don't want to give outsiders the ability to wander your building (and possibly break your expensive machine by driving it onto the stairs, or into somebody.)

Moving Forward

The remote presence device is real and here to stay. It's going to get better -- more bandwidth and better resolution, more field of view, better audio. It already works so these improvements and others will just make it get even more usable.

Expect to see more of these at meetings, parties and even tourist locations in time. I would love to have had one for various school reunions I didn't attend. I missed them because they weren't worth 10 hours of flying and days of travel, but they would have easily been worth a few hours in a robot. And you would not have to worry about how fat your body has gotten.

There is also a green aspect to this. By far the vast majority of my carbon footprint is coming from my air travel. I write about cars all day long, but two avoided overseas flights would save more fuel than I burn all year in my car.

Is it as good as being there? No. But can it be a reasonable substitute some of the time? Yes.