MALE SPEAKER: This is Jeremy Frank He’s going to [INAUDIBLE] It’s turns out he went to school with Laszlo Bock JEREMY FRANK: Almost MALE SPEAKER: Almost You were off by a couple years, at Pomona College, which I think is kind of funny So you send things to space He sends his Googlers into work So, yeah So I don’t want to waste too much time But I wanted to draw attention to his shirt He used to work for Yoyodyne Propulsion Systems, which is a small company in [INAUDIBLE], right? Where they made an advance B-1 bomber subsystem for the defense department before they were shut down JEREMY FRANK: Yeah I can neither confirm nor deny that aliens were involved MALE SPEAKER: Yeah So you worked for John something Well, anyway the long and short of it is I don’t waste any more time So please [INAUDIBLE] JEREMY FRANK: That’s plausible OK So thank you very much So I wanted to talk to you today about some work that NASA is doing to figure out how to send people to far distant planets, possibly Mars, but possibly other things that are not quite planet-like, near-Earth asteroids So what I want to do is tell you a little bit about what some of those destinations are, and how far away they are, and in particular how long it would take to talk to a crew of astronauts who might be at those destinations And the reason I want to talk about this is because when we think about how space missions are operated today, in particular human space mission operations, time delay is going to be a real problem But to give you a sense of what all that means, I need to tell you a little about operations And in particular, some of the things that can go wrong during operations and why it’s really important to think about those things, especially in the presence of these large time delays And what this really boils down to is a matter of communication So the communication between the spacecraft and the flight control team, what happens during that communication, why people need to communicate, and why this is going to be a little bit different if we do these things tomorrow, than it is today And after I do that, I can tell you about some experiments that NASA has done in order to really try and quantify the impact of time delay on mission operations, explain how we try and quantify what can go wrong, what things are going to go right And we have some technologies that we’ve been investigating, some things that you are very familiar with and some things that you may not be so familiar with it, that we think are going to actually mitigate the impact of time delay And so I want to describe this experiment that we did and tell you what we learned And I have a flow of this presentation, but please feel free to interrupt with questions at any time I should also warn you there are some audience participation questions So be prepared I will be calling for people to help me out here in the middle of my talk So without further delay, that is a picture of Apollo 11 on the lunar surface Are you surprised at the color of the dirt? Because the Moon’s gray, right? Well, no It turns out not all the Moon is gray Sometimes it looks a little reddish-orange But more interestingly to us for this talk, the Moon is a paltry 238,000 miles away from Earth So light travels from the Earth to the Moon in only 1.2 seconds Well, so in a day and age of instant communication, 1.2 seconds may seem like a long time But really from the point of view of mission operations, that’s not so bad So we’ve done that We’ve experienced that before But there are some places that are interesting that NASA would like to potentially send people that are a little bit further away One thing that you may have been hearing a little bit about in the news recently is asteroids Certain classes of asteroids have been termed near-Earth asteroids So what is a near-Earth asteroid? So this is one This asteroid is called 1998 QE2 What a great sexy name But what should be slightly more interesting perhaps is that in May of this year, this asteroid passed a paltry 3.6 million miles from the Earth Well, OK, so 3.6 million miles, that sounds like a long time It’s 14 light seconds from Earth So it’s actually quite a bit further away from the Moon It’s not the closest thing that has passed to the Earth and there are some others that are further away But the point is these are things that are very, very interesting They’re very old They’re very scientifically interesting There’s a lot of very, very cool exploration science that people want to do Some of that science we think we want to do by sending people there And so the question is where are they? How far away are they? Well, so here’s a close-ish one And here’s a slightly more interesting one

So this one’s interesting because if you take a look at it, you see these kind of smooth surfaces and all this bumpy stuff So your picture of asteroids may be a great big, solid hunk of rock Well, not all of them are like that Something like this may resemble more like a great big pile of dirt in space, loosely held together So how many of them are like that? Well, we don’t really know This one’s a little bit far away though It is 26 million miles, 2.3 minute light time delay to Earth So that one’s pretty far away And yet, there are some asteroids like that that we’re considering sending people to So pretty far away, yes And then, of course there’s Mars So for those of you who don’t necessarily recognize this picture This is Mount Sharp This is the shadow of Curiosity Ooh, the shadow of Curiosity is a little bit more shaded than I thought it would be And so everybody knows how far away Mars is, right? Anybody know? No When Mars is close to Earth, it’s 35 million miles Light takes three minutes to get to Mars from Earth Three minutes, that’s when it’s close? What about when it’s far away? Well, when it’s far away, it’s pretty far, 250 million miles, 24 minutes one way So these numbers should give you some pause when we think casually about sending people to these places And that’s the whole point So these are a range of places NASA’s considered sending people So when we talk about spaceflight mission operations, future human destinations, this is the range of places that people are interested in, the kinds of things that we would like to do So what does it take today to actually operate a human space flight mission? What do people do? Well, they sit at these banks of computers, many, many of them Each one of them has a job And that job is to keep their crew and their spacecraft safe That’s primarily what they do But what does that actually mean? So to an astronaut, mission control is the power company They’re the plumber They’re the air conditioner repairman They are the doctor They’re the phone company They’re geek squad They’re everything to those people Because any time anything goes wrong on their spacecraft, any time the crew needs help doing something on board their spacecraft, if they have a problem, these are the people who they have to call Very much like we do, only it’s not like you can drive to the doctor’s office It’s not like you can get UPS to deliver you a new hard drive for your computer You pretty much have to take care of it yourself, with the help of those people on the other end of the phone So far so good? So the people who sit in the mission control center who assist the astronauts in doing all of this work, they have a large profusion of different kinds of tools that they use in order to help operate that spacecraft So what I want to do is show you a couple of those tools and talk a little bit about what they do So one thing that they do, for example, this is a set of what are called displays for, in this case, it’s a little hard to read, but you might be able to see that this is ECLSS, Environmental Control and Life Support Systems So if you’re a flight controller who’s responsibility is ECLSS, you care about what the atmosphere inside the spacecraft is composed of Is it to hot? Is it to warm? Is it too cold? Does it have the right gas composition? Is there CO2 collecting any place inside your spacecraft, which could be very dangerous for your astronauts? But that’s what you’re doing So you have these displays which are basically watching data which is coming from the spacecraft and telling you whether things are doing what they are supposed to do or not So there are a variety of these tools and their whole purpose in life is to give those flight controllers insight into the spacecraft So here’s another one So instead of just getting the numbers and watching those numbers updating You might need to see plots for trends So it’s a different view of the same class of data So another thing that you might need to do as a flight controller is you might actually need to tell your vehicle to do something So you might need to say, turn a piece of equipment on, turn a piece of equipment off, gather information about things So what you now have is you have a set of commanding displays, very similar to your data displays But instead of being used for monitoring your vehicle, these are used to command your vehicle So a slightly different form of a tool

Sometimes you don’t want to send one command You may need to send a large number of commands And if you do this enough times, you may want to write a simple script or essentially a program in order to send that sequence of commands and just watch it go So in addition to single commanding tools, you may have some sort of script execution monitoring tools like this So similar in spirit to sending a set of commands is the idea of a procedure So a step by step set of instructions that someone, either a flight controller or a crew person, is going to use in order to accomplish some task So in addition to just the set of commands, you may need some explanatory text You may have some sort of flow control boxes, something a little bit like a flow chart that helps you understand what it is that you’re doing, what tools you need, what pieces of information are supposed to say what in order for you to go to the next step So these procedures are very, very standardized and regularized And so there are tools to help you see what those procedures say and to help you track where they are So now it is strategic level, procedure steps are things to take tens of seconds to maybe a minute to execute At a much more strategic level, on the course of a day, you have tools that help you understand what your daily plan is So in this case, we see a plan for a subset of the International Space Station astronauts over the course of a couple of hours You’re seeing information like is it day or night apparently, because of where the spacecraft is relative to the Sun You’re seeing what kind of communication coverage you have You’re seeing things like what’s happening with respect to communication coverage on the ground Is the ground tracking activities that the crew is doing? And so you have an information display like this that’s sort of showing you what your plans are So who has lost something in your house? You don’t know where it is It’s in the garage It’s in the closet And it’s really important because you need it Because you’re going to the beach and your kids are yelling at you because you can’t find the sand toys OK, so astronauts don’t have sand toys But similarly, a lot of the activities that they do require something, say batteries, a pair of scissors, a piece of tubing And this is somewhere in your space craft Well, where is it? So in addition to all of these activities and procedures, you may have information that’s basically logistics, stowage, and tracking So that you can efficiently go find those pieces of equipment that you actually need in order to do whatever you need to do on your day So this is also very important So as an aside, any web developers in the audience? Since the year 2000, the International Space Station has flown a variety of tools based on web technology that help them do all of these things And they’ve also been in use in the flight control room as well So from what I’ve been able to gather, the first of these tools was basically used to monitor the state of air to ground file transfers so that the crew knew what they had to do But since then, it’s been extended to both procedure display and plan display and execution tracking So technology you built, may be on board ISS today How about that? So a small subset of the technology that’s being used is listed here OK So you’re all experts on operations now, right? So let’s talk a little bit about communications So I think it’s pretty clear at this point that communications is pretty critical to both delivering all of this information to these tools, as well as to people So voice and video communication to the control center is quite important I’m not going to dwell on that It’s not a big part of what I want to talk about today And yet we’ve talked about how time delay can make things worse So here is the first of our little audience participation games So I need a volunteer You’re my volunteer, great You don’t need to go anywhere You can stay there So what we’re going to do is we’re going to simulate something that might happen on board a hypothetical future space mission with time delay So some blue text is going to appear on the screen, which I will ask you to read when it appears So perhaps you’ve made a phone call So hi, this is ACE Air Conditioner How can I help you? Well, so five seconds after I say that– AUDIENCE: Our A/C hasn’t been working for the last day JEREMY FRANK: You hear it And then you get to say your piece And five seconds after that, I hear it When was the last time that your air

conditioner was serviced? Oh, this is really frustrating We’ve haven’t even gotten to the interesting stuff and its frustrating AUDIENCE: When you last serviced it three years ago JEREMY FRANK: I’m twiddling my thumbs Oh, the tension is mounting Have there been any interruptions in power? OK, so you get the idea So there’s one more text box But the point is that if you were say on your way to QE 1998, 14 light seconds away, you’re a third the way there, five light seconds, and the air conditioner on your spacecraft goes out And you’re not entirely sure what to do about it as an astronaut, you might have that very experience And it’s just going to get worse as you go on to Mars So ever since I started looking into this time delay business, I’ve been thinking about how we communicate with each other And so I have a very, very brief and incomplete history of how people have communicated, not going all the way back to the Dark Ages But think about if you were alive in 1860 before the Civil War, and you wanted to send a message to someone, and you had to put your message on a stagecoach that was going roughly in the direction of wherever your friends lived, back in the East coast It could take you weeks in order to send a message, never mind if you wanted to get a message across the ocean So revolutions in communication like the Pony Express in 1861 and the telegraph and especially the transatlantic telegraph, this just profoundly changed the way that we talk to each other And, of course, we’ve lived through the revolution in SMS and email We now have devices in our pockets, that people could not have imagined having in their homes, which allow us to pretty much communicate anything, voice, video, text, data, anything we want, at an instant We’re not tied to our houses anymore So this idea of instantaneous communication is really something quite new And so to some extent, we’re going back to the past if we think about what happens when we send people to very, very distant planets We’re going to be stuck back in the dark ages of communication again And so of course the human factors issues of texting from the surface of the Moon again aside, things are going to be quite different So that’s enough about communications But really we’ve been operating human spacecraft, human-occupied spacecraft for over 50 years now There have been ISS crew people on board ISS continuously for over a decade Is there really anything to learn? So I want to reinforce that space is a dangerous place, even now Being in space is dangerous Bad things can happen on board your spacecraft all the time And part of the reason why we care about this is so that we can respond if something bad happens So almost everybody remembers that Apollo 13 was a big problem during the Apollo era and that that mission was saved by ground It is not widely remembered that of the other Apollo missions, many of them were saved by timely intervention from the ground when something really bad happened So Apollo 11 I think is one of the more interesting ones So what this 1202 alarm during the lunar module descent was– basically the Apollo guidance computer on the lunar module was overworked And as a result of its overwork, it basically started issuing these alerts messages And nobody but the engineers who built it really understood what was going on So the crew didn’t know if they were going to have to abort this landing or not until ground told them everything’s fine You’re still safe to land But they didn’t know And the others are equally interesting and I invite you to learn more about them later But unlike in the Apollo era, flight controllers sitting in the control center fly the ISS The astronauts are up there to do science The vehicle is flown from the ground So we don’t even have the experience that they had in terms of astronauts owning and understanding how to fly the vehicle So when, for example, you have a problem on board the International Space Station like a big old tear in the solar arrays, ground has to understand what the implications of that and figure out how to fix it When, for example, a micrometeorite punches a hole in a thermal panel causing it to blow out, ground has to understand how to reroute the thermal control system and keep the vehicle safe When, for instance– you won’t see it here–

there is an ammonia leak in the thermal control system, which basically forces you to shut down a whole power channel because you can’t keep it cool anymore, what do you do when your station is now at one eighth less power? Ground is taking care of business Crew can’t do that, not today That’s why all this time delay stuff really matters So hopefully I’ve hammered all that home So as a result, we understand that something has to change We can’t operate the way that we usually do in the presence of time delay But what exactly can’t we do? What kind of responsibility can we shift to the crew? What does the ground have to do differently in the presence of time delay for these future missions? So that’s really the question that the work I’m now going to talk about is meant to answer So let me pause and ask anybody, you got any questions? Anybody got anything they want to know about, any observations before we get into the next bit? Yeah? AUDIENCE: The near-Earth asteroids, are there any motivations to gather resources from an asteroid? And can you even tell like what kinds of resources are available remotely? JEREMY FRANK: Right Great question So the short answer is yes There are a lot of hypotheses about what kinds of resources could be on those asteroids I think the notion of heavy metals, like gold, and iron, and nickel, there are some asteroids that have those But they tend to be very far away, at least as far as I understand it And so other kinds of resources people are interested in are things like radioactives People think that, for instance, there are radioactives is on the Moon that might be mined But it’s very difficult to get a good sense of what’s underneath an asteroid If there’s something that’s on the surface, you might be able to observe its spectra with a telescope from the ground or a space-based telescope But hard to find out what’s inside And I think that nobody should get too excited about us being able to mine asteroids in the near term It’s a very exciting idea But we can hardly even get there in the first place, let alone bring stuff back Yeah? AUDIENCE: Do you think crew obviously also mean the on-board computers, which given what you said so far, seem to be needing to take on a large part of the responsibility JEREMY FRANK: That’s right So we distinguish between vehicle automation and crew autonomy So things like computers taking action and being able to control systems, we would tend to refer to that as automation in some way, shape, or form And you’re absolutely right One of the things that we think we’re going to have to do is make the computers able to do more or different tasks than they do today There are questions about how effective that can be Can it be trusted? Is it complex? Is it too expensive? But many of those same questions apply to shifting responsibility to the crew Nobody else? OK So far, so good So we’re NASA So we can do all sorts of very, very interesting things So in order to answer this question, we decided that we were going to build ourselves a spacecraft And we were going to send it someplace really far away, like Texas And we were then going to operate it under a variety of these conditions in order to simulate the time delays, simulate the kinds of activities that we think would happen on such mission, try and introduce some of these unexpected events, and produce a set of sort of impact assessment of what would happen And then we have some of these mitigation technologies that people have been thinking about So we wanted to experiment with those under basically the same experimental conditions and see what worked and what didn’t work before we do it for real So here you are You don’t want to see that when your on board your spacecraft So I’m not entirely joking when I said that NASA built the spacecraft We didn’t build it to fly in space, unfortunately But the Deep Space Habitat spacecraft test facility that we used here was built at Johnson Space Center And its whole purpose in life was to do this class of experiment, to basically serve as an analog so that you could do these kinds of operational tests, try out a variety of advanced technologies, in order to see what does and does not work for these future missions And so in addition to that, mission control in Johnson Space Center has a similar facility, the operations technology facility, where they test operations technologies for use on future and also on current spacecraft So they kindly lent us some space that we could use to create a mini-control center in order to operate the Deep Space Hab And so that’s our test harness And so what I’m going to do now is I’ll tell you a little bit about the sort of miniature mission that we created And I’ll go through its mission timeline a little bit And then I’ll describe the mitigation technologies that we used to try and see what happened with time delay

So what we had was a crew of four quote, quote “astronauts” in our spacecraft, one of whom actually was an astronaut And then we had a set of flight control positions, with responsibilities like those things that I described So we had a power systems flight controller, thermal systems, chief medical officer, operations planning, and so on and so forth And these people basically conducted these simulated two-hour mission timelines And here is our mission timeline So what I’ll do is go through a subset of these activities and describe what they are But this is the kind of thing that you would expect to see during a part of the mission where there’s no vehicle docking, there’s no vehicle undocking, there’s nobody going outside on an extravehicular activity People are basically staying inside the spacecraft, living and working, taking care of business Not exactly doing the laundry, but doing things that are sort of regular, recurring tasks that are, if you like, important, but not sort of highly dynamic, energetic events And so let’s walk through the timeline and see what some of these activities are in a little bit more detail So one thing that happens a fair amount with your mechanical equipment on board the spacecraft is you need to inspect it frequently to make sure that it’s still working properly This is a piece of exercise equipment It’s very important for the crew exercise, to maintain bone and muscle mass inside zero gravity But all that mechanical equipment can wear out You definitely don’t want it breaking while a crew person is exercising with it So you conduct inspections periodically to make sure it’s working right You send pictures back to the ground Ground does an inspection, makes sure everything is working OK Part of the hypothesis for these long duration missions of the future is that perhaps we can use plants, both as sources of food and also environmental control So we pretended that we had plants on board So there’s a regular soil sampling activity to make sure that the plants are staying healthy Just like at home, if your air conditioner filter doesn’t get replaced, your A/C doesn’t work very well So we had a simulated air conditioner filter replacement activity In addition, we assumed that we were on our way back from some planetary destination where we collected a number of samples, rock and soil samples So there’s the sample inventory activity to make sure that those samples’ condition and placement is well understood So you remember the micrometeorite strike on ISS? Well, so every so often you may need to conduct some sort of external inspection of part of your vehicle with a camera that’s mounted outside, that can take pictures to make sure that the outside of your spacecraft is looking all right And then the last one is a fluid transfer activity So there are various repositories of fluid It can either be fuel, sometimes it could be water, waste perhaps And so every so often, you may need to move fluid from one place to another And so that’s another common activity that people had So we had about a dozen of these activities And we had a full timeline So really in that two hours, there wasn’t a whole lot of time for anything to go wrong There wasn’t a whole lot of time for activities to take longer than expected either And so what we wanted to do also was to disrupt the timeline with a variety of unexpected events So we had a pretty rich power system inside the Deep Space Hab and we were able to selectively fail components So as you’ll see later, there’s power distribution, power conversion, and a variety of other pieces of the power system which we could decide weren’t working properly Sometimes they decided they weren’t working properly all by themselves, which was kind of exciting I’m sorry Did you have a question? AUDIENCE: Were either the crew or the simulated mission control notified in advance what failures you’re going to run? JEREMY FRANK: No No There’s an extensive experiment design that I’m not going to really get into But no, we wanted very much to keep that secret So we had a set of people working those failures up, making sure that we had good, reasonable failures that actually could disrupt the timeline, but we’re going to totally blow us out of the water And in fact, there’s lots of good insight about that that we can discuss a little bit later So the fluid system was a software fluid simulation We thought about using a hardware simulation, decided not to do that So with the software fluid-based simulation, we do just about anything we wanted So that was very good And we were exceedingly lucky We were able to work with some people at Johnson Space Center in space medicine, both certified ultrasound technicians and also some very, very cool medical autonomous operations technology So they partnered with us And we were able to do crew medical emergencies, in which case either urinary retention or kidney stone actually

present with very similar symptoms and both of them are diagnosed using ultrasounds And so that was a great ambiguous crew down, medical emergency situation We were able to perform the ultrasound We weren’t able to do anything after the ultrasound But even that was pretty cool Some of our flight controllers got to do ultrasounds for the first time That was pretty exciting So that gives us a pretty interesting set of things that we can introduce in order to disrupt our timeline So here we are Here’s our poor, hapless flight controller doing his ultrasound OK So we had four distinct crews And it was very important to us to have distinct crews so that basically crews couldn’t sort of gain experience in the experiment design And that was really tremendous We got volunteer astronauts in order to serve as our commanders But all of our crews and flight controllers were either certified flight controllers from the International Space Station or shuttle program So when you hear the results of what these experiments are telling you, keep this in mind Because these are the experts, the people who actually do this every single day, telling us what did and didn’t work And that was very, very valuable to us So I’m just going to skip this because I think I want to make up a little bit of time here Oh, I’m actually doing great So we essentially took all of those situations, either nothing happens, we disrupt things with system failures, we disrupt them with crew medical emergencies, these time delays that we wanted to experiment with And we said, let’s pretend we were doing basically the way ISS is operated today, see what happens, try and get sort of a baseline what’s the impact of time delay under these circumstances? And then let’s take some things that we think are going to make delayed operations a little bit less painful and compare them point by point and see if we can get a sense of whether things are made better and if so how and why So let’s talk a little bit about what some of these are So you remember, we talked a little bit our procedures, the step by step instructions that tell people what to do, and where to go, and what pieces of equipment they’re going to use Well, today those are mostly electronic documents presented as PDFs But if you’re going to send a command, and the procedure tells you to send a command, you got to go to some new tool to send the command If you need to refer to a piece of data and decide whether to do the next step or not, you have to go to another tool in order to see that So an advanced electronic procedure display allows you to send commands, renders your telemetry, and tells you whether or not you’ve got a problem with the next step So we see here this piece of telemetry in this yellow box Well, that’s telling you, if you get to that step and that box is still yellow, you can’t do that command yet You can see which procedures are active You can see we’ve procedures are done If other pieces of technology are recommending procedures, you see those You have step tracking and all kinds of other very cool advanced features So this is one thing that’s sort of a step above today’s practice in terms of operations technology Both crews and flight controllers can make use of this Primarily, this was for the crew’s use in our experiments though So you have the great question about our software going to make us smarter? Is it going to do more? Well, one of the things it can do is it can tell us whether we have a problem It can diagnose problems It can tell us what the consequences of problems are It can tell us well, I know there’s a problem Maybe there’s an ambiguity I can recommend something to disambiguate what the problem is So here’s a somewhat hard to see, schematic of the power system But what we’re seeing here is that a 24-volt DC converter has failed So that’s what the solid red box indicates As a consequence, all these things that are kind of dimly visible in this outlined red box, those are components that have been taken down as a result of that failure So this is a combined schematic of your system, with a piece of software living behind it that is trying to do that advance caution and warning, fault detection, isolation, and recovery technology task So again, this is the kind of thing that could be available on board if the computational resources are available It could also be used inside the flight control room So in addition to those things, something a little simpler Why don’t the astronauts chat? Why not give them chat? So that was a pretty obvious one In fact, there should be an experiment on board ISS with chat now, partially driven by what we did during this experiment a couple years ago So in order to create these mitigation technologies, we put together a gigantic list of components under the covers One of the things that I want to draw attention to is over here on the protocol side, not only are these protocols for basically moving information around, both within our

simulated spacecraft and also inside our simulated flight control room, this was also what we used in order to simulate the time delay Voice, video, data, audio, all of them were actually put into simulated time delay loops And this was part of the way that we did it So you can make me look dumb by asking me questions about this since this is not really my expertise area But But this is just a small subset of the things that we used So any questions about that? I kind of flew through it pretty quickly, but before we get on into what we learned? Yeah? AUDIENCE: You mentioned kind of the advance electronic document sort of thing, so PDFs One of the side effects of that, though– I mean is that the only control mechanism that someone would have or is that in addition to that extra tool that you described? JEREMY FRANK: Right Good question So even today, procedures are always backed up by single command displays where you can issue individual commands Because no battle plan ever survives contact with the enemy No procedure survives contact with the real situation So in the event that you sort of wandered outside the bounds of what you know how to do, you always need another way to command the system and to get the insight So in terms of measurement, how do we actually measure what happened? So there are a variety of questionnaires that people have developed over the years in order to assess things like what is a person’s workload, what is the difficulty of communication coordination, and things like that We could also take metric measurements of things like how often did people talk, how long did people talk And those are the kinds of things that we were able to use in order to figure out whether things became harder or easier You might ask well, could you count the number of tasks performed? Could you count the number of procedure steps and their delays? We did that and we saw no impact So I’m not to say anything more about that But audience participation opportunity number two So what do you think? Based on this experiment, based on what you’ve heard, do you think the time delay is going to make things harder or easier, just curious? Show of hands if you think it’s going to be harder Almost everybody thinks it’s going to be harder So without knowing anything more than I’ve said about these mitigation technologies, do you think they’re going to make things better? Show of hands if they’re going to be better Some people, a little bit AUDIENCE: Harder to identify JEREMY FRANK: Anybody think it’s going to be worse? Is this going to cause damage? Are we going to mess this up? No, OK All right AUDIENCE: What did you think? JEREMY FRANK: Pardon? AUDIENCE: What did you think? JEREMY FRANK: We really didn’t know We formulated this hypothesis that things were going to be different Things did not turn out quite the way we expected though We didn’t really have a good sense of what was going to happen So I apologize There’s some quantitative data here We ran a total of 10 experiments at each time delay setting, 10 at 50, 10 at 300 So this is what the crew members said they thought how workload changed with time delay So the crew thought workload was higher at the higher time delay The crew thought workload went down when we added those mitigation technologies We did not expect that We expected something like well, it’ll increase with time delay, but it’ll be less We didn’t expect workload to actually decrease That was a surprise We didn’t understand that This is more like what we expected So this is what the flight control team said So the flight controller said well, time delay is making things harder But the mitigations are making things easier But it’s making it easier across the board So that was interesting So we got some surprises right away OK, so this in words is pretty much what the charts are showing So that was interesting So we have statistical significance tests And yeah, most of this looks like it’s actually real The variances are a little bit high, but it’s actually a real effect So just looking at that though, that’s not quite enough Because that’s only telling us sort of grossly speaking what people’s experience was in sort of metric terms But in addition to these sort of numerical measurements, we also got hem or to answer questions And what they told us in words about what did and didn’t work and why it didn’t work was really interesting So here’s one example And I have a lot of these in the charts And I don’t know if I’m going to go through them all But what are they saying? So this second quote is especially interesting To keep from falling behind, while I was waiting for the ground to tell me the answer to a question, I started

something else So what are they doing? They’re getting ahead But in doing so, they’re imposing workload on themselves to keep track of what tasks they’re currently doing So they’re trying to multitask They’re trying to do multiple things at once And it’s making it harder What else are they doing? Oh, when they ask a question and they’re not getting an immediate response, they’re creating work because now they have to monitor the auditory channel for the come-back So even if they had a timer, they would still have to be listening Oh, it’s about time for them for them to get back to me now So they’re creating work This one was also very interesting I had to write things down because I wasn’t sure I was going to remember what question I asked If I got to “say again,” which could happen, communications get garbled all the time So these things are things that they’re doing This is what they’re experiencing in a time delay that’s creating work So this is great This is solid gold This is exactly what you need to know So luckily we had very clever people who had questionnaires prepared in order to assess the coordination difficulty, which is one of the things that’s creating this extra work So things like tracking tasks that I’ve started at the same time, that’s one thing But this– how hard is it to talk to people? Is it hard to talk to people between crew and ground? So we had some questionnaires for that So what’s interesting here is you’re seeing here’s how hard crew felt it was to communicate with the ground And here’s how hard ground felt it was to communicate with the crew So similar, but not identical And again, you can see that the mitigations are making things better So why are the mitigations making things better? We had four classes of thing that we did We had the procedure display We advance the caution and warning We had chat And I had forgotten to mention that the procedure display basically is shared between the vehicle and the ground So anything the crew is doing, it’s basically mirrored on the ground So the ground can basically say oh, that crew person is in the middle of that procedure, or was five minutes ago So show of hands, who thinks chat made things better? Most people think chat made things better Procedure display and technology, anybody think that make things better? Less so than chat Advance caution and warning for the systems failures, that make things better? A couple nods How about spying on the crew? I know exactly what step of the procedure you’re on Did that make things better? Probably, all of them a little bit Anybody think that made things harder in any way? Maybe A couple of thoughtful looks, nobody willing to raise their hand Aah, raise your hand OK So once again, the comments are gold So this is great Ground says, WebPD made it really easy to follow along, even with the time delay So I don’t even need to call up I don’t need to ask what procedure step you’re on It just tells me what procedure step you’re on So today, you’ll even hear flight controllers calling up to the crew, saying what procedure step are you in? How far did you get on that step? Are you done? I’m seeing this Where am I? No more No more need for that But look at this This one here is very interesting too Now, flight controllers have to watch the web procedure display and listen to the audio So they now have more things to watch So some of what was being made better is also being made a tiny bit worse So that was interesting We didn’t quite expect that That was good So the advance caution and warning, this one is a pretty interesting one because several people said that the failures that we had were fairly straightforward and simple, given the simplicity of the vehicle But in a real vehicle where you’ve got two, three, four, five failures deep, something like this could really shine But even here, even in these simple cases, people said, gosh, I knew exactly what to do I worked these procedures Ground told me what to do But I was done already So this was pretty much a win So that was good All right So last question, which is good because we’re almost done So even without something like chat to basically make it possible for people to talk less, do you think people would talk more or less with increased time delay? Anybody think more talking with increased time delay?

Less talking with increased delay? Anybody think it’s about the same? No idea And then obviously, we would expect them to talk less But let’s see what the numbers say Well, surprise, surprise, time delay didn’t seem to impact how much people talked They talked about the same So that was a surprise So what we’re seeing here is total talk time as measured by basically clicking on and clicking off of the voice loop mics So if you count the total number of seconds, what you’re seeing is that in both cases, both for the mitigation case and for the baseline, people were talking roughly the same amount So that was interesting Yeah? AUDIENCE: About the technology, when there’s no delay, do you still have to hold duplex, hold the duplex and talk at the same time? Or if there’s no delay, do you still push Talk? JEREMY FRANK: There is full duplex And actually one of the things that you sometimes find is people are stomping on each other So I’m trying to talk But I’ve mistimed when your voice is coming So I’m basically stepping on you And that was something that some other people are taking our data and analyzing, trying to understand how did that sort of come into play But yeah, so you could have– I don’t know exactly I don’t remember how many channels we had And our Kennedy Space Center participants actually managed all the coms for us And I’m not sure exactly how they did it Yeah? AUDIENCE: It seems to me that voice is just completely wrong for this type of– when delay than a few seconds, it’s just not reasonable to– I mean voice is great when it’s instantaneous or almost instantaneous because then you can have a conversation But when can’t have a conservation, what you said, chat is basically email Or back in the old days, you sent a letter Never started talking and then have someone relay your voice in the same thing JEREMY FRANK: That’s a really good point Except that a huge amount of information is conveyed in voice, even if it’s delayed, and so voice timbre, choice of wording And one of the things that other people are doing, which I’ll get to in a little bit, well, voice, email, sure You could do that But other things that people started to ask was what was chat used for as opposed to voice? When people had freedom of choice, what did they choose to do? And it turned out that chat was pretty good for quantitative data like numbers Voice was good for non-numbers So there are other kinds of interesting modality questions that I think don’t totally preclude using voice AUDIENCE: But I mean if you managed voice like email in the sense that its all recorded You can always go back You can always relisten And then basically when the length of is by an order of magnitude or even sorter than the delay, it makes sense to construct a message and send it I mean you can optimize and start to sending the message before its finished But on management, you don’t start playing the message until you’re ready to listen to it JEREMY FRANK: That’s true too But as you’ll see from some of the chat context question and answers, why do people like chat, there’s some other advantages that we’re not quite there on for voice But you’re absolutely right But this is all about how do you think differently about what people do and how they do it Yeah? AUDIENCE: This might be your next slide I mean did you learn anything from this set of exercises that you immediately wanted to put into practice with the ISS? JEREMY FRANK: There were a number of things that people wanted to at least consider trying AUDIENCE: [INAUDIBLE] it seems like a huge win JEREMY FRANK: Absolutely AUDIENCE: –lowered orbit JEREMY FRANK: So chat’s one of them Chat had been something people wanted to do for a little while The sharing of data with the WebPD is another one that people were very excited about using There are some barriers to putting things on ISS that make it harder So this result that we found here where basically the time delay didn’t influence how much talking, that was a real surprise Because we all sort of expected that it would be different But what is very striking and quite obvious, in mitigation, people talked something like 20% to 33% less Or to put it more accurately, 80% of the time that people spend talking, they spent a lot less So the question is why? Was it all chat or was there something else going on? And so we did a– whoops We did a badly constructed slide, which I failed to QA before the talk Oh, well So had something else had to go wrong So what we can do is we could take a look at the chat messages We could pretend, unlike me, that a chat message would have been spoken by either a crew person or a member of the flight control team at roughly two words per second And then we can say, if you add that to the time people actually spent talking, what do you get? And what you see is that you still get reductions in the amount of time between baseline and communication So something else in that set of tools, which we could not

pinpoint, is also contributing to a reduction in how much time people are talking Some of it’s chat Some of it’s something else AUDIENCE: [INAUDIBLE] procedure display [INAUDIBLE]? JEREMY FRANK: Probably Yeah? AUDIENCE: I had a question I didn’t quite understand the complaint that a procedure display increased monitoring [INAUDIBLE]? It was a few slides back Because I would expect that if they have a display they’d have to answer it some other way, whether keeping in their head or writing it down on a piece of paper, what step the remote astronaut was on JEREMY FRANK: So this I think gets down into the intricacies of when the crew needs to tell someone, I’m in the middle of this procedure, I’m on this step and I have an issue, how is that done? Or if the ground wants to know where someone is and they can’t otherwise find out, what other displays are they looking at? And we didn’t have the analysis and we didn’t have quite the amount of information to get down to that level of detail AUDIENCE: It seems to me then if some of the time is spent checking, asking the astronaut what step are you on? Oh, I’m on this step Can you repeat? And if you had a procedural display, then wouldn’t some of that talk be eliminated by just referring to the procedural display And it might explain some of the– JEREMY FRANK: Absolutely But the fact that it’s a nuanced story is the interesting thing It wasn’t a knock it out of the park There were no issues with this It was all very natural Flight controllers were saying, I’ve got a piece of information I didn’t have before I felt like I was doing things that were contributing to my workload that I didn’t do So when we talked to them more about this, it was less a matter of this is a problem with the technology It’s an awareness of something we would need to train people to do How is it that you shift your attention focus from your monitor displays to your procedure display, if you had such a thing? Yeah? AUDIENCE: [INAUDIBLE] But I assume you also have to think of the content of the discussion I think this is something [INAUDIBLE] JEREMY FRANK: So some other people are analyzing the content a little more to try and understand and sort of come up with, if you like, sort of a formal model of what people were and weren’t saying AUDIENCE: I mean obviously mission control could be more easily manned by adding more mission controllers They’re significantly cheaper than the technical crew on the shift AUDIENCE: [INAUDIBLE] JEREMY FRANK: Pardon AUDIENCE: We hope you get paid better [INAUDIBLE] JEREMY FRANK: Well, so full disclosure, I’m not a control center person I just try and give them stuff that they can use AUDIENCE: [INAUDIBLE] JEREMY FRANK: And it’s very true But the pressure has been to try and understand what you must do in order to control Can you do it with fewer people? Do you really need people there 24 by 7? And the answer is it’s not entirely clear if we can But the costs of a control center are very high, data, voice, the building, all the com Communication itself is very expensive If you look at how much it costs to maintain deep space network communications to all of this remote places, it’s a lot of money So controlling the cost of the control center is still an important thing to consider how to do All right So let’s see So once again, we got this sort of nuanced story So chat removed the need to do a lot of things You could roll back and read the chat So if you had good voice to text, well that would be a good justification for saying, well, you just speak into the mic and send an email But we don’t have good voice to text yet But again, there’s now this other piece of information Some of this is just they’re not accustomed to monitoring these pieces of information, which I think is to overcomeable compared to the obvious benefits of it But it’s all little new for them So they’re still coming to terms with it OK I’m almost done So I didn’t fully understand at the time that we did this, about a year ago, a year and a half ago, that if you look at some of these things that we’re employing in order to mitigate a time delay, it’s this mix of old and new or it’s this mix of sort of high end NASA stuff plus things that really we’re using all the time and every day now And there are other people who are much more sophisticated about how communication takes place But deep space missions are going to be managed in an era of asynchronous communication in a way that today’s missions really are not And thinking about that and coming to terms with it, it’s not a big surprise Mars missions are managed like this all the time Every single Mars mission that we conduct is managed this way Missions to places like Mercury, Mercury Messenger being the prime example, they’re managed this way today So the human space flight world is going to ultimately learn how to make use of these things It’s just a matter of time

And if that’s the place where we’re going to send people, then I think we have a pretty good idea of how to at least get started So I want to linger on this slide filled with names because this was a giant, cross-NASA achievement that we actually managed to pull off, ridiculously in about a year I still don’t quite know how we did it But props to every single person on this list Everybody worked really, really hard to accomplish this It was absolutely fantastic, a wonderful group of people I was very privileged to work with them And it was great and I’m done So thank you very much