h1

Off Topic: Steve Fossett Search

September 20, 2007

I’ve been away from this blog for a week because I had the opportunity to participate in the Steve Fossett search in Nevada. I’ve been a member of the Civil Air Patrol for 5 years, and this has been the largest search I’ve been part of. I was impressed by the professionalism and skill exhibited by the CAP members participating in the search, and it made me proud to be a member.

What prompted me to go off topic, besides making an excuse for being absent from the site for way too long, was something I noted while checking out Amazon’s “Mechanical Turk” (http://www.mturk.com), and the effort to locate Mr. Fossett. This effort lets anyone participate by looking at satellite photographs of sections of the search and saying whether or not they think the section they’re looking at contains something that warrants further investigation. Each section is shown to many people, and any image that is flagged consistently is reviewed by experts and possibly passed on to search teams.

I think the idea of pursuing the search with many eyes is fantastic, and I hope it turns up useful leads. However, I was surprised to see this activity compared with the techniques outlined in James Surowiecki’s “The Wisdom of Crowds.” According to Surowiecki, as quoted in Wikipedia (The Wisdom of Crowds), for a crowd to be wise, it needs to have the following characteristics:

  • Diversity of opinion: Each person should have private information even if it’s just an eccentric interpretation of the known facts.
  • Independence: People’s opinions aren’t determined by the opinions of those around them.
  • Decentralization: People are able to specialize and draw on local knowledge.
  • Aggregation: Some mechanism exists for turning private judgments into a collective decision.

The MTurk process is independent and diverse, but it doesn’t provide a way to turn a private judgment into a collective decision, and it isn’t really decentralized, at least in the sense that localized knowledge can be applied by individuals. It really is asking for people’s observations, not their wisdom. In fairness, I don’t see any claim by Amazon that this is the technique in use, but references pop up repeatedly in other sources.

I’m not saying that what Amazon is doing is flawed; on the contrary, it is valid and important. However, I think it could be supplemented by using the ideas in the Wisdom of Crowds.

Suppose everyone was given a description of the situation (i.e., basic facts like the capability of the plane, maps of the area, confirmed and possible sightings of the plane in flight, etc.), and asked to make his or her best independent estimate of where the plane will be found. Then, take that information and aggregate it to identify a potential search area. This would be similar to the search for a Navy submarine that is described in Surowiecki’s book. In that case, even though some guesses were way off, the aggregate was very close to the actual location.

I suspect the toughest thing about using this method would be to keep participants from being influenced by the potential hits that have been reported by people participating in Amazon’s visual effort. That said, such an activity might yield useful results that certainly wouldn’t be any worse than the leads coming from people who claim to be psychics.

Advertisements

4 comments

  1. Technology is redefining the way search and recovery is being done. It’s a wonderful thing.


  2. As someone who has been participating in the Mechanical Turk search for Steve Fosset since Sept. 10th, I found your comments and observations as someone who also participated in the CAP side of the search to be very interesting.

    I’m not sure I like the idea of letting people search where they may. It might have merit, but there is something to be said about methodically covering every square meter of a search area repeatedly. If people just “surfed” satellite photos as they felt like it, I would be concerned that “uninteresting” areas might get overlooked, which could lead to a down plane being missed.

    I do wonder, however, how many times is enough times to cover a given search area, especially if the CAP already covered said area several times via planes? Should the Mechanical Turk search area be expanded beyond the original search area if the CAP already covered that area comprehensively? I will continue to search via Mechanical Turk whatever satellite photos those in charge feel need to be searched, because I have to assume they know best.

    One thing that they have finally added, but I would like to see more of, are sample photos of real plane crashes and not 20 – 40 year old crashes, but recent ones where the landscape has not had time to “heal”. To be effective, untrained eyes like mine need some tutorials as to what we might be looking for.

    There is a great potential for this technology to revolutionize search and rescue, but it will need to evolve to reach its full potential. Who knows, maybe there will be a virtual “arm chair” wing of the CAP where slippers and PJs are the “official” uniform. It would sure beat sleeping in a cold tent on some mountain air field.

    Cheers.


  3. Ken,

    Thanks for the note. I agree that having people surf through the photos as they choose wouldn’t be very productive. When you’re talking about a visual search, the MTurk method makes sense to me, and the fact that it hasn’t born fruit, yet, doesn’t invalidate the approach.

    Regarding the question of how many times you need to cover a given search area, there are ways of estimating the effectiveness of a search, and CAP uses them. The surprising thing to me about those estimates is how low the probability of detection is after one, or even a few, passes. That’s one reason why the MTurk search is so valuable; you get lots of eyes on the same area. And, from what I can tell, you’re right that the MTurk folks know what they’re doing.

    What I’m suggesting is a complementary method of trying to locate good places to search. You ask people to set aside the photos and have each give his or her single best guess as to where the plane is based on the known information, i.e., the aircraft, the pilot, the terrain, the weather, etc. You then aggregate those guesses into a single combined guess.

    Surowiecki’s idea in “The Wisdom of Crowds” is that this combined guess will be as good, or better, than any individual guess.

    You would, of course, follow up by examining the photos at the aggregate location and if promising sending a search crew.

    Having spent time looking for folks in uncomfortable weather, bouncing around in a small plane, the idea of searching in your PJs with a cool drink at hand is attractive. But, there is nothing like being there to convey the sheer size of the search; it makes you really appreciate how hard the search is.


  4. I would agree, a combination of methods would probably bear the most fruit. Your explanation about probability does help explain why search areas get covered so many times.

    What will probably make MTurk searches more effective in the future is when we have access to higher resolution satellite photos (one such satellite was launched this past week).

    I have lived in a few Rocky Mountain states as well as Alaska, I even used to drive tour buses through the Yukon Territory of Canada, so I can fully appreciate what you mean by appreciating the sheer size of search areas and how rugged some areas like this region of Nevada can be. You are right that it is impossible to convey the vastness and ruggedness of such areas without first hand experience. Where Steve Fossett could have went down no human may have set foot for a hundred years if ever.



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: