Don’t think it’s exactly Dunning Kruger. We all think about the curve of gathered knowledge and perceived knowledge.
But they didn’t even start to gather knowledge, they just respond with something that sounds truthful and fits their world view in order to feel better without doing anything.
But hey maybe that’s just my Dunning Kruger talking.
If after all that preparation, your pride can be pierced and wounded by one of myriad neckbeards or Karens on twatter, you might need to let go a little bit.
This, but to some degree, unironically. If studies aren’t reproducible (or deemed worthy of reproduction) then there’s definitely a disconnect between the folks handing out research assignments and the folks engineering applicable solutions to scientific problems.
That goes two ways. You could be a guy who successfully formulates a mathematical model to support the existence of Neutrinos and face a funding board that has no interest in building a LHC. That’s arguably a problem of malinvestment within the scientific community. Or you could be a guy who successfully formulates a mathematical model for a new kind of mouse trap that’s 10% less efficient than traditional mouse traps. That’s more of a university research assignment problem. Or you could have a researcher who claims he’s the only one who can do a particular thing, because he’s got the magic touch. If the research is unfalsifiable by design, that’s an entirely new kind of problem.
i think you bring up valid instances where this is fair.
but i think i’m speaking to the very obvious and important ones that are worthy of reproduction. like i’ve seen articles be like “these corporations are responsible for 99% of climate change” or something
and the comments will be like “duh we knew that”
which true, but not empirically. being able to cite data from actual research from professionals is so valuable and far better than anecdotes or guesses. edit: and also informs meaningful policy.
that said, is there some way for a layperson like me to identify when research is not deemed worthy of reproduction? or is it a lost cause
which true, but not empirically. being able to cite data from actual research from professionals is so valuable and far better than anecdotes or guesses.
While its certainly helpful to get the raw numbers down on paper, you don’t need a filing cabinet full of documents to recognize that fossil fuel consuming electricity producers and airliners and manufacturing centers the but-for cause of climate change. Fossil Fuel goes in. Carbon emissions come out.
We can definitely use a more meticulous bit of R&D to find exactly where and when these emissions peak, in order to reduce total emissions without sacrificing an abundance of economic productivity. But “did you know burning the fuel makes the pollution?” isn’t a shocking conclusion.
Where things get annoying (and where in-depth research genuinely comes in handy) is in the functional policy that follows this recognition. Once you know a widget factory in China is 10x less efficient than its counterpart in the US, you can formulate a trade law to limit imports contingent on reform. But as soon as you start impacting some retailer’s bottom line, you get some screamer ad “Congressman Greenpeace Wants To Make Your Widgets 10x as Expensive to Save The Stupid Spotted Owl! In Truth it is the Spotted Owl that produces all the emissions! Kill the Spotted Owl!” financed by the worst people you know.
And that’s when you get some facebook troll group (or marketing team or bot army) spamming “Spotted Owl Farts Killed The Environment While Joe Brandon Clapped!!!” And then it becomes orthodoxy in the denialist community such that you’ve got Sunday Morning talk shows with people arguing over Spotted Owl emissions rather than trade law.
is there some way for a layperson like me to identify when research is not deemed worthy of reproduction?
Not practically, no. As soon as you’ve got that kind of info, you’re no longer a lay person.
At some level, you need a network of trust with someone who does know and does have a serious take on this. And that network is going to be informed by who you already trust and listen to. And that’s going to be informed by who they trust and listen to.
That’s the real terror of the modern mass media system. We’ve corrupted so much of our information stream that its genuinely hard to find a serious media venue that’s not been gobbled up by a for-profit marketing firm.
So what’s the harm of doing research on subjects with “obvious” no-surprise conclusions? The basic reality that it provides foundation for meaningful policy should be enough to justify it, no?
You kind of lost me with your spotted owl hypothetical? Not disagreeing I just genuinely got lost there was a lot if layers to it lol.
And thanks for the details on identifying problematic research as a layperson. Good to know, even if it’s depressing.
Anything could have enough significance to warrant further study. If it has societal implications or environmental concerns, it could be deemed worthy. I’ve read some guidelines on how to read scientific papers, but don’t have the link on me. The scientists are supposed to list their biases, but it’s kind of on the honor system, I think.
Facebook and WhatsApp were separate apps until Facebook bought WhatsApp. That’s why there’s so much difference in the design language, functionality, etc.
To be fair, journal articles and scientific research in general have gotten to be pretty bullshit. Haven’t they studied this and proven the vast majority of published journal papers probably shouldn’t have been?
A couple easily Google examples of discussion regarding scientific publications likely being bullshit.
I saw a clip on how kids out of uni don’t believe anything not peer reviewed; even intuitive observations in nature that otherwise undocumented or site specific observations that went against the grain.
Science is a way of thinking and observing, rather than papers, but papers are a good way to refine your thinking
In theory, a paper gives you a methodology that you can use to reproduce the findings. And a refusal to use papers to repeat findings (because shit costs money and nobody wants to publish iterative studies) means you end up with a bunch of novel findings that are never confirmed through repetition.
But the fact that nobody is bothering to repeat these studies also raises a question of what exactly is being researched. Certainly, the more useful scientific research efforts are about formulating applicable techniques. So they would need to be reproducible to have any functional value.
The fact that we’re not seeking to replicate studies suggests that we’re investing a ton of time in niche under-utilized fields. And that may be a problem of investigative research (we’re so focused on publishing that we don’t care what we’re actually studying) or a problem of applied sciences (we’re so focused on scaling up older methods to industrial scale that we’re leaving better methods of production on the cutting room floor).
TBF I’ve lost count of the number of times someone has cited some paper as a reference for the point they are trying to make and when I inspect the paper it has shitty “n”, the paper is written for an agenda (not sure what that’s called where I.e. a paper saying smoking is good for you/not harmful is paid for by the tobacco industry and written by tobacco industry scientists), or it might even just be straight up bullshit written to look like a legit paper.
Peer Review at least offers some barriers to the problems with papers, but it’s definitely not a panacea.
I’m guessing not all hypotheses receive the same interest or funding to begin with. Definitely seems to be a selection bias on what actually gets funded/studied. Even worse, when they withhold results they don’t like from being published.
Well, sometimes there’s another step missing just before the Bullshit: “Use the small, narrow findings to inform a greater narrative beyond the data’s scope”
Well I’d like to think I’m not! I wanted to point to an actually dubious thing where we might call into question a study, so we could still respect the work being done while validating the importance of standards in research.
You’re right though that it’s disappointing how many responses seem to address only the flaws in modern science and not acknowledge the strength of the scientific process. I think a big part of it does come down to how scientific findings are interpreted and reported to the public, and even further an all-too-human misunderstanding of epistemic limitations. Our cultures should spend more time educating people about the limits of knowledge and fact, how they are constrained by other flawed systems, etc. That would be a half-decent start, if we could only fix the entire reporting problem too.
slrpnk.net
Oldest