+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
COMMENTARY:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
+- Warts and All
On returning from ICT2005 I recalled a specialist conference I attended nearly 25 years ago. A gentleman I met there, who must then have been about my age now, put science in a context I have never forgotten. He told me with great enthusiasm what a wonderful career this 'science' is. You make guesses and you try them out. Sometimes you are right and sometimes you are not, but you report what you find out 'warts and all'. And they pay you to do this. What more are you looking for out of life?
I was reminded of this view of science because it seems people no longer report the warts. Here's what I mean. The majority of the papers at ICTs are really pretty good, or I would stop going. But there are some 'regulars' reporting really high ZT values (would they were true), or violations of Onsager's relation or some such craziness. I can think of a good half dozen suspicious reports off the top of my head. It is not that I'm so sure they are wrong. It is just that the burden of proof is on them, not me. And for each of this particular group, someone has in fact tried to reproduce the results and failed. But you hear these things only in the hallways or over beer. They aren't getting written down and publicly reported. I can't tell you why the warts aren't reported (presumably each for their own reasons), but I can tell you the result is damaging everyone in the field.
I suspect what has changed is the 'and they pay you to do this' part. Doing careful work on negative results won't attract many sponsors.
Causes aside, here is what I ask. When you see a suspicious claim crop up over and over (or are asked to review one), climb all over them to get an independent verification. Climb over them in public, with questions after their talk. Buy them a beer later, but be brutal on the data. This isn't personal, it is about good science. Good data grows stronger when you beat on it, bad data falls apart.
When you are asked to do an external verification, do it only on condition that you will publish the results "warts and all". Make this clear before you start. But most of all, if you've made 400 samples and only 3 show the effect you are describing: say so in the abstract. Don't make me ask you about it. Whether it is your own work or not, publish "warts and all." Anything less is just a useless waste.
End of rant. Return to your normal routine. Or not.