An Interview with: A Naval F-18 Pilot and Trainer

In All, Blog by Robert Pool6 Comments

Lt. Steven Shaw trains naval aviators to fly the F-18, the main jet fighter of the Navy and Marine Corps, at a base in Virginia Beach, Virginia. That’s his official job. But unofficially he is working to introduce the principles of deliberate practice into naval flight training, with the ultimate goal of changing the entire training culture for naval aviators. (A brief disclaimer: The thoughts and ideas described  in this article are Steven Shaw’s alone and in no way represent those of the United States Navy.)

Long before he was in the Navy, Lt. Shaw told me, he had become familiar with deliberate practice. He spent many years, some of them quite intense, taking lessons on and practicing the piano—something he still does today. He took years of violin lessons and trumpet lessons. He took ballet lessons. He learned to play chess. He did gymnastics for all four years of high school. And he also played football at his small Texas high school.

Lt. Steven Shaw, U.S. Navy

“The level of focus on the details a piano player has on the performance of a piece far outweighed anything we did on the gridiron,” he told me. “Even a beginner piano player learns to break things down, focus on one hand and then the other, slow at first, and then gradually increase tempo, incorporate both hands, focus on dynamics, etc. In my opinion, this is much more in line with deliberate practice than the beginner football or basketball player.”

The result, he said, was that he came into military aviation with a fair amount of experience in many of the activities in which deliberate practice has been most highly developed. “I can assure you,” he wrote me, “I have yet to encounter another fighter pilot that has a fair amount of experience with piano, violin, gymnastics, and ballet all at once (especially ballet). So I feel as though I entered flight school with a pretty developed understanding of practice, how it works, and how to do it.”

But when he got to flight school he found that the training there was quite different from—and, in his mind, far inferior to—the sorts of deliberate practice that he had been accustomed to employing in other areas.

Part of the problem, he told, me is the Navy’s focus on knowledge in its training rather than on doing. That is, the focus is on learning through lectures, computer slideshows, and reading, with the assumption that once a pilot knows what to do, he will be able to do it. This attitude is very similar to the attitude in medicine that Anders Ericsson and I wrote about in Peak: much of traditional medical education has been based on the assumption that once a doctor, even a surgeon, has been told what to do, he or she will be able to do it.

To Lt. Shaw, having been trained on the piano and violin and in ballet and gymnastics, this seemed obviously wrong. No one would teach piano by explaining the patterns of how your fingers hit the keys and then expect you to play a Mozart concerto—or even chopsticks. No one would teach gymnastics by diagramming exactly what your body is supposed to do in a backflip and then have you give it a shot. So why was so much of the instruction on flying a $70 million jet that reaches Mach 1.8 and can reach 7.5 G’s based on knowing what to do rather than doing it?

Which brings us to the other part of the problem that Lt. Shaw recognized: it is essentially impossible to really “practice” with a $70 million plane that can fly at 1.8 times the speed of sound. The costs of making a mistake are too high.

Pretty much every type of effective practice involves making mistakes. You push yourself outside your comfort zone, you see where you screw up, you come up with a practice technique specifically designed to fix your mistakes, and then you practice until you’re not making that mistake anymore. But you can’t do this with flying the F-18.

As Lt. Shaw put it to me, probably 80 percent or more of the pre-flight briefings are spent on avoiding catastrophe in one form or another on that flight. “If that’s what you’re talking about before you do it, you’re probably not practicing.” If your entire focus is on avoiding making mistakes, you’re not going to be thinking too much about what you need to do to get better.

Or, as Lt. Shaw phrased it somewhat more colorfully, “If you can reasonably expect you might die from something, it’s probably not practice.”

F-18 landing on a carrier.

So how do you get better flying an F-18? There clearly is a certain amount of improvement just from going on and doing it day after day, just as happens when you first learn to drive a car. But in both cases you reach a point where you are performing acceptably, and then you tend to stop improving. You reach a plateau. And once you’re on that plateau, you tend to keep doing things the same way, comfortable in the level of performance that you have reached. But you don’t get much better after that point because you are no longer being challenged.

This was the situation that Lt. Shaw faced, and he looked around for other ways to improve than flying his F-18s at the edge of his comfort zone. As he described it to me, he realized that there are other professions—such as surgery—where you don’t have the opportunity to practice in real life because the consequences of mistakes are too great. And he ended up settling on an approach to practice that is very similar to one that Anders has been recommending to surgeons and other doctors who wish to improve their own performance.

Lt. Shaw settled on an approach to practice with two main components. The first is based on carefully analyzing actions during actual flights to identify areas in which to improve.

The jets are equipped with cockpit cameras, and every flight is videotaped. Both the instrument displays and the radio communications are recorded, and together they provide an objective history of pretty much everything that happened during the flight. The original purpose of the cameras, Lt. Shaw explained, was to record the use of the plane’s guns, missiles, and bombs and to study their effects on the targets, so most pilots turn the tapes on fifteen or twenty minutes after take-off in order to capture the combat aspects of the flight and turn them off once those are past. But Lt. Shaw records everything from the time he turns his engines on until he turns them off at the end of the flight.

In terms of improving one’s flying, the debriefing period, where pilots talk about what went wrong (and what went right) on a flight is generally considered the most important part of a flight, Lt. Shaw told me. So it is a standard part of training. But he takes it to an extreme that is unusual, if not unheard of.

As soon as he can after he has finished a flight, Lt. Shaw will sit down with the videotape from his flight and watch it from start to finish, making note of everything that was not perfect, even it didn’t cause any problems. For his analysis he developed a list of five categories that cover everything a pilot does in a cockpit: knowledge, decision making, perception (or the passive receiving of information), physicality (active processes), and communication.

Every error or imperfection will fall into one of these categories. It might be that he didn’t notice something on his instruments—that’s a perception problem. Or he didn’t handle the controls exactly right—a physicality problem. Or he misheard a radio call—a communications problem.

He has a pre-made form on which he lists all of the issues as he identifies them, along with their category, and on the back of the form he writes down a training technique to use to work on the issue. His goal is to practice on each area in which he fell short.

He often does not meet that goal, he noted, simply because of a lack of time. A typical flight lasts about an hour and fifteen minutes. He’ll generally spend three hours on the analysis afterword. And it generally takes a couple of days to work through all of the exercises for the problems he identified on one flight. But since he generally has one flight a day and military obligations unrelated to flying, it can be difficult to get to everything.

He does the same thing with his students after their flights, assigning them exercises to work on the areas with imperfections.

Lt. Shaw conceptualizes all of this in terms of a four-step process:

 

(1) Identify what the problem is.

(2) Figure out why it is happening.

(3) Figure out a way to fix it.

(4) Actually do what you came up with in step 3.

 

This is, of course, the classic deliberate practice approach, the same thing that piano teachers or ballet teachers or gymnastics teachers do to help their students improve: What went wrong? Why? How can we fix it? Okay, now practice!

The approach has proved extremely successful, Lt. Shaw says, both for him and his students. Because of his career timing, he told me, he probably has fewer hours of flying than any of the other Navy pilots at the same stage in their careers. “But I’m proud of that because there are many things that I do well because of all the practice time even though I don’t have the same amount of flight time.”

As for his students, they tell him that his approach makes a major difference in their flying. He has developed a reputation for providing effective instruction, and often students who have failed a particular task will pass after working with him. It is difficult to determine the effects of the deliberate practice precisely, however, because there are no objective measures of student performance, only the subjective assessments of the instructors.

Over the past three-and-a-half years, Lt. Shaw has assembled a videotape collection of nearly 800 of his flights and has also collected videotapes from other pilots’ flights. He uses these as teaching tools.

For example, if his students are working on a particular air-to-air tactic that requires them to make a decision, he can pull out a tape where a pilot went through the same situation, run it up to the decision point, and then ask them what they would do. After they answer, he can continue on with the tape and show them what the original pilot did. In addition, Lt. Shaw told me, he will have the students move their hands as they watch the video just as they would if they were flying an aircraft. “I can watch people’s hands and see what they’re doing wrong.”

It is even possible to learn from tapes of flights where the pilot did not do everything right. Lt. Shaw described watching a tape from a flight where a pilot was forced into a situation where his fuel had gotten very low. He watched the whole tape and asked himself what he would do in the same situation. Later, it happened that he was on a flight when a similar situation started to unfold. “I knew what to say, do, and think, and I responded correctly,” he said. “I knew what to do because of someone else’s experience.”

Most pilots, Lt. Shaw said, will tell you that experience is getting plenty of hours flying. “But experience is having been in situations before and having figured out what to do.” You don’t have to have experienced it yourself to learn from it. “I can acquire experience from other pilots’ flight time at no cost and no danger to me. Almost all of my skill development as a pilot has happened on the ground in front of a video.”

The second component of Lt. Shaw’s approach to deliberate practice involves the use of simulators—another tool he uses in a way that is different from how most naval aviators use it. He mentioned a time in 2015 when there had been two different F-18 crashes within a week, both of which involved not being able to recover from a dive. “I went into the sim and put myself into the same position,” he said. “I hit the ground probably about half the time, then I stopped hitting the ground.” He had taught himself how to deal effectively with a dangerous situation.

However, he said, the basic way that the Navy uses simulators with its pilot trainees is as a testing mechanism. The pilot will “fly” an entire mission on the simulator and be graded on it. That makes flying the simulators not much different from flying a jet—the pilot’s main goal will still be to avoid mistakes, which makes it difficult to take full advantage of the simulator’s possibilities.

Lt. Shaw uses the simulators to teach, not grade. When students are going to come across a particular situation in an upcoming flight, he will develop an exercise for them in the simulator to prepare them for it. He will sit in the simulator with them, tell them what they’re doing wrong, and show them how to do it right. Sometimes he will slow down the simulation to let them practice something they’ve been having trouble with—much like a music teacher will have a student play a difficult passage slowly at first and then speed up. He will instruct the students not to worry about half of the things going on in the simulator and focus on a particular area—again, reminiscent of a piano teacher instructing a student to focus on just one hand at first. He will stop the simulation and give the student time to think through a particular decision.

These are all familiar deliberate practice techniques in other fields, but they are not commonly used in teaching Navy pilots. “I’m probably the only pilot at the base who sits next to students in a sim,” he said. “I go in, look at their hands, look at where they’re looking, that kind of stuff. That’s one of the reasons I’m able to get quicker results.”

Lt. Shaw’s approach to using the simulator has proved particularly valuable in teaching students to do carrier landings—one of the trickiest things that Navy pilots must do. Typically about 20 percent of the pilot trainees fail their carrier landing test, Lt. Shaw said. But of the students who have done his carrier landing simulation exercise at least ten times—the amount he identifies as the necessary minimum—not a single one has failed.

His ultimate goal, he said, is to make deliberate practice the norm in training Navy pilots. He has been training pilots for nearly a year and has had about 100 students. Many of them are already thinking in terms of deliberate practice and using its terms when speaking about training. Not everyone really gets it, though. These pilots have already had two years of flight training before they get to Lt. Shaw, and some of them have already settled into the usual Navy approach.

By the end of this three-year tour, Lt. Shaw figures he will have taught maybe 350 flyers. And in five to ten years, when these pilots have moved up the ranks and are in positions of greater authority, maybe they will be able to transform deliberate practice from a minority practice to standard procedure in the Navy.

That’s the hope, anyway. And it needs to happen, Lt. Shaw said, because the next generation of fighters—the controversial F-35 family—will require even more creative training methods. “The F-35 is the one of the most complex and expensive military fighter planes ever created,” he said. “We have to find a more efficient way to train with that.”

Actually, there already is a way to train with that. Deliberate practice. It just has to be recognized and put into practice. Stay tuned.

Comments

  1. This kid is an narcissistic self-promoter. Thinks he invented the debrief? Naval aviators have been debriefing for decades. And we don’t just debrief to confirm weapons effects. We debrief what we did and how me maneuvered the aircraft. There are a lot of things that you can’t garner from the tapes, e.g. all the things he does from start up to take off. I’ve been stationed at this base and I know of people who sit next to people in the simulator. But guess what, after a few sim, you know where things are and know where to get the information you need. This kid is a pariah in naval aviation who thinks he’s gods gift to aviation. Naval aviators are of the best in the world… additionally you can’t teach people to fly like you teach people the piano. One you can sit and practice for hours with no risk in your living room. The other costs 90 million dollars to build and 10’s of thousands of dollars to operate. So you can’t have the luxury of holding someone’s head through chopsticks. I’m glad Steve is still in the business of thinking he’s better than everyone. Bravo Zulu Minajah

    1. Author

      A comment from the Admin: I’ve received several comments on this post, all apparently from F-18 pilots in the U.S. Navy. The comments have been uniformly negative. This is the only one I have approved to this point because the others had no substantive arguments or observations but were simply expressions of disapproval. I have also been engaged in a more thoughtful conversation with a Navy F-18 pilot who does not wish to speak on the record. As I see it, the gist of the complaints about the article fall into two major categories: (1) Lt. Shaw is doing nothing that the Navy wasn’t already doing, and (2) Lt. Shaw is a self-promoter who is trying to make himself look good by denigrating his fellow pilots. I am interested in hearing more about this perspective, and I welcome any substantive comments and am willing to e-mail or speak with any of these other pilots to learn more about what they have to say. As I see it, there are two basic possibilities: First, I have been hoodwinked by a self-promoter who really isn’t doing anything different from other pilots but wants to seem like a big thinker, or, second, that Lt. Shaw actually is taking an approach to training that is substantially different from what the Navy has traditionally done and this insistence that the Navy is doing it wrong — perhaps combined with something about Lt. Shaw’s personality — rubs a lot of pilot the wrong way, even if there is a lot of value in what Lt. Shaw is doing. I admit that I do not know enough about flying Navy fighter planes to be sure which of the two possibilities is closest to reality, and I am willing to look at more evidence to try to form a clearer opinion. However, I have spoken with Lt. Shaw extensively, and I can say this much with some certainty: He is very thoughtful and focused on finding ways to improve pilot training. He understands the basic ideas behind deliberate practice and is looking for ways to implement them in pilot training. He dreams big and someday would like to see his approach to training — or something like it — be the accepted approach in the Navy. He does not strike me so much as a self-promoter but rather as a “true believer” — someone who is convinced he has found the truth, or at least a better way, and wants very much to spread the message. History is full of such true believers who have revolutionized some part of science or medicine or industry or some other aspect of human endeavor. The doctors who believed that stomach ulcers are caused by a bacterial infection rather than stress come to mind, as does Steve Jobs, who was convinced that the personal computer could be extremely important at a time when the conventional wisdom in the computing industry was that they would be little more than a novelty item. Of course, history is also full of true believers who were ultimately proved wrong. When you stand up and announce — rightly or wrongly — that the conventional wisdom is incorrect, you will inevitably have people oppose you, so the fact that I am hearing from Navy pilots who don’t believe Lt. Shaw has anything to offer does not convince me that he does not have an important message. It does convince me, however, that this is a subject worth pursuing, and I urge any Navy fighter pilots with something thoughtful to say about this issue — in support of Lt. Shaw’s ideas, opposed to them, or perhaps somewhere in the middle — to contact me through this site.

      1. Hoodwinked? That would be kind. You’ve been scammed.

        I am a retired USAF fighter pilot; I retired 25 years ago. I can’t speak for today’s USAF or Navy, but back in my day, we did pretty much exactly what Tommy is describing. Building block approach that starts with fundamentals, skills-based training to go from basic skill to application; post flight debrief in three steps: Reconstruct, Analyze, Critique. All that was religion. I’d bet the house that the USAF still goes through this process today. Go look at the syllabus of any USAF Weapons School course; they all take this approach.

        I only know what the Naval fighter pilot did back in my day by flying with them, briefing with them, and debriefing with them. Some of that occurred via squadron visits, some occurred when Navy (and Marine) fighter pilots were assigned to my USAF fighter squadron. Their process was as identical to the USAF process as two processes from rival services could possibly be. So while I did not go through Naval training per se, I have seen the Navy’s fighter pilots train first hand.

        This guy, Tommy, is no doubt a good writer, has a good presentation, and is no doubt charming. He has also fooled you. I don’t know who your audience is, but this article is making the rounds on Facebook, and no doubt via other electronic media. To those who know about USAF/USN fighter aviation, you and your website are being laughed at.

        Just my opinion.

        1. Author

          I’m fascinated by the replies I’ve been getting to this article. I have no skin in the game, as they say. If I was indeed hoodwinked by Lt. Shaw, that’s a great story, and I’m happy to write about that. I’m interested in getting as close to the truth as I can. What I’ve learned is that there are a lot of military pilots who are convinced that they have been given great training and that they are the “best in the world.” What I’m not seeing is anything approaching proof. As Lt. Shaw pointed out in one comment, there are no objective measures of performance here and thus no evidence-based sense of what effective flying reallyis. Indeed, if we are to believe Flash, today’s training is pretty much the same as it was 25 years ago. Flash seems to think that’s a good thing, but to me it seems evidence of a culture that is self-satisfied and complacent, happy to keep doing things the way they’ve always been done because they’re convinced — without any evidence — that they’re doing things the right way.

          The point of my original article was that Lt. Shaw believes he has found a better way to teach pilots with an approach that is built on the principles of deliberate practice. Maybe he is right, maybe he is wrong. We will never know until we have an objective measure of performance and use it to compare pilots trained in Lt. Shaw’s way versus those trained in the traditional way. One of the things I appreciate about Lt. Shaw is that he wants to have the Navy bring back pilot competitions that pit pilot against pilot in testing various skills, and he is willing to use this sort of competition to test his methods versus other methods of training. He is willing, in short. to put his money where his mouth is. All of the negative comments I’ve gotten about the article rely on some variation of the argument, “All of the pilots I know think that this Lt. Shaw is a joke, and people are laughing at him and at this article.” That carries very little weight with me. Just because lots and lots of people believe something without evidence doesn’t make it true.

          Thus I ask the pilots out there who are reading this article and thinking it’s a joke: Convince me it is a joke using some sort of evidence, not by telling me that everyone you know thinks it is a joke. Where is the evidence that your approach to training is the best one? Have other methods been tried and found wanting? Is your approach to training based on any sort of scientific understanding of how the human brain works? Give me something, please.

          Here is where I am coming from: In every area that I know of, the most effective training follows the principles of deliberate practice. At this point, there is little debate about deliberate practice — and related sorts of training — being the most effective sort of practice. Yet I see very little sign that the training of Navy pilots has much of deliberate practice in it. Lt. Shaw wants to change that and ground pilot training in the principles of deliberate practice — which, if we are to believe the evidence from other skills, is likely to produce more skilled pilots than the traditional approach. So, without any evidence to the contrary, my tendency is to believe that he is right — that introducing the principles of deliberate practice into the training of Navy pilots will lead to better pilots.

          But I am first and foremost a scientist, and I follow the evidence. At this point my sense that introducing Lt. Shaw’s methods into Navy fighter pilot training would produce better pilots is no more than a hypothesis that needs testing, and I am eager to hear as much evidence as I can from pilots who do not agree with Lt. Shaw. Again, please contact me or post here if you have something to add. This is an important debate, and I will post anything from anyone who has something substantive to offer.

        2. Author

          One more thing: At this point, in addition to all the negative comments, I have heard from a Navy fighter pilot who says that Lt. Shaw’s comments are spot on. He does not want to speak on the record, but he supports Lt. Shaw. This gives me some faith that there is an issue here that does indeed need addressing. We will, of course, not resolve anything here on this website, but perhaps by bringing the subject to light, we can trigger some conversations in the Navy and Air Force about whether pilot training really is the best that it can be.

  2. Tommy’s unsubstantiated claim that American Naval Aviators are the “best in the world” is a perfect example of the kinds of subjective assessments that are at the root of our current problems. With no clear metrics or criteria for objective evaluation, he’s making a very dangerous assumption – both for the safety of our aircrew, and the security of our nation.

Leave a Comment