Friday, August 28

Fake Numbers

In the last few days, I've noticed a few instances of fake numbers. Take any statistic, any measurement, any anything, and you're almost guaranteed to get a non-round number. So, when you see a number that is particularly round, you should get curious. The number on my mind today is 10,000. Evidently that is how many hours it takes to become an expert at something, according to Malcolm Gladwell. (Disclaimer: I've never read anything he's written, other than a lame piece on ESPN.com that was NOT worth my time.) Now, I have no idea how he arrived at that number, but it doesn't make sense. Why would there be something magical about 10,000 hours?

Now, you might be thinking that his point isn't the exact number of hours it takes to master something, he's just driving at the idea that it takes a long time. Well, if that were his point, he should have said, "it takes a long time". Or "it takes years and years". But the one thing I'm almost certain of is that it doesn't take 10,000 hours to become an expert. And when you think about it, you know that number can't really be calculated the way it should be, unless Gladwell managed to somehow measure how much time Bill Gates spent working on computers decades ago, or how many hours John Lennon spent fiddling with his guitar and writing bad songs. In the end, what I think Gladwell did is determine that it takes a really long time to become an expert at something (surprise, surprise) and then picked a nice, round number that matched up with that.

11 comments:

Ben said...

Having read it, I'm afraid your guess at his characterization isn't what he was saying.
He does readily point out that it's an approximation (which is why it's roung). He also does in fact provide a very solid estimation of exactly how many hours Bill Gates spent working on computers in his youth before writing DOS. Several other people as well. It's not just the point that it takes a long time but an actual claim that it takes approximately 10k hours.
While, I've found about everything of his that gets published online to be a waste of time, both Blink and Outliers were interesting reads for me. Can't remember the name of the one about epidemics but it wasn't nearly as good.

Clark said...

1: I suppose that what I should have said is that the way that others present his number of 10,000 hours is poor, rather than insinuating that the problem comes directly from him. Heavens knows that the internet often misinterprets what people say.

2: My point is that putting a number in print gives credibility and authority to an idea. And publishing a number without placing uncertainty around it doesn't provide the whole story. "It takes 10,000 hours to become an expert" sounds very different than "In 90% of cases, it takes somewhere between 6,500 and 12,000 hours to become an expert."

3: Actually, my REAL point when I started all this was to point out that people have a fixation with round numbers. If the whole world understood significant figures, we could just report the number as 1.0e4 and I'd be happy!

Melissa said...

I'm thinking that I like the more concrete way 10,000 feels. While I admit that the number itself seems rather arbitrary, I think that it implies that you have to actively be working during those hours. If you spend "years and years" on something, it doesn't really tell how much actual time you've spent. Tinkering on weekends? As a "second job" in the evenings? Full time? I've been doing a lot of things for "years and years" but have not put in the time over those years to be an expert. But, if you are actively working on something, you could put in 10,000 hours in less than three years, even allowing for eating and sleeping and an occasional birthday party. In summary, sure, the number seems round and convenient, but I think it implies a focused, earnest effort over time (actually working for those hours) as opposed to casual exposure over an equally long or longer time.

(ps, I haven't read what he's written, but I think a husband of mine has. Also, not such a good writer. Hope you understood my point.)

Melissa said...

Also ps, I was typing while you were typing, so I didn't ready your comment before I typed mine. Not that it would have changed much . . .

Sabrina said...

I liked both Blink and Outliers too. I actually think you might as well if you give them a try.

Adam Lowe said...

I was going to say what Melissa said about the meaning conveyed by using hours instead of years.

But I also think using round numbers like 10,000 is kind of a signal to the reader that it is a very rough estimation. The "90% between 6500 and 12000" that you suggest conveys much more precision, and implies underlying statistical research that I don't think exists. His 10,000 number suggests that it's probably based on anecdotal evidence at best, and is intended to give people more of an order-of-magnitude idea (is it closer to a hundred hours or a million?) than an actual mathematical average.

I have a much bigger problem with fake numbers when their not nice and round, because then they're pretending to be real.

Nathan said...

Your first post in 57683.27 minutes. Very scientific. Very credible.

Clark said...

Maybe I'll have to go read whichever book this stat came from, to get the info from the horses mouth, so to speak.

Ben has read the book, and indicates that "he also does in fact provide a very solid estimation of exactly how many hours Bill Gates spent working on computers in his youth before writing DOS."

Melissa then indicated that Adam has read the book, too, ("I think a husband of mine has [read it]") (she only has the one husband to my knowledge) but Adam says "underlying statistical research that I don't think exists. His 10,000 number suggests that it's probably based on anecdotal evidence at best"

Adam Lowe said...

I'm not sure if you're saying that Ben and I drew different conclusions, but I think we're saying the same thing.

I read the book a while ago, and I can't remember if Gladwell cites an exhaustive study or not, but my recollection is that he draws his conclusion based on a few examples like Gates. Enough to convince him that the 10,000 hour thing was persistent, but not nearly enough to support any kind of statistical accuracy like margin of error or confidence intervals.

All that being said, I'm not at all comfortable in the role of Gladwell apologist. I ate his stuff up for a while, but since then it's kind of been debunked for me as unsubstantial. He comes across as a really smart guy with interesting ideas, but I think your instinct that he speaks with an unjustified air of authority is probably right on.

Clark said...

Yes, I did mean to say that Adam and Ben drew different conclusions. To me "a very solid estimation of exactly how many hours" and "probably based on anecdotal evidence at best" don't describe the same thing.

And don't worry that I'll mistake you for a Gladwell apologist, as your own blog posts on his writings have strongly influenced my opinion of him. :)

Finally, I sort of wish that I had come up with a better non-Gladwell example of my original point. I wanted this whole thing to be more focused on round numbers that are pulled out of the air which are then treated with some special reverence. I had a perfect example that I encountered at work recently, but as it is a number dealing with work and a customer, I didn't want to throw it out on the internet, and it wouldn't really make any sense with more details than I'm willing to give to the world. So, that's how we ended up on this discussion.

carrie said...

I hope you meant that John Lennon spent 10,000 hours writing bad songs before he started writing good songs. Honestly, that's the only issue that jumped out at me from your original post. Maybe I am too sleep deprived.