I talk with researcher Erica Thompson about her new book, "Escape from Model Land: How Mathematical Models Can Lead Us Astray and What We Can Do About It." She explain how models are often misused, how we can make better use of them, and how her critiques apply to climate models.
I love Dave Roberts's work in general, but this episode had me tearing my little remaining hair out. Fortunately, the guest was mostly sensible and kept pulling DR back from the most extreme nihilism about models and human understanding in general. While I have no doubt that many of the stated criticisms of economic and social models are valid in a great many cases, tarring all models, especially the basical physical models of the atmosphere, with the same brush is grossly unfair and is simply giving aid and comfort to climate deniers. DR repeatedly seemed not to understand that model predictions are in most cases not absolute but conditional predictions--not what will happen, but what will happen if a particular course of action is taken. I really think that after this poor showing, Dave owes his listeners an in-depth conversation with someone who does top-flight physical climate modeling and can speak rationally about these models' uncertainties and imperfections, but also their very real capabilities. Otherwise, the audience is left with the impression that models and by extension science are mostly bullshit, that we'll get different answers to any given question depending on the race and gender of the modeler, and that none of it should be taken very seriously. I know and esteem DR's past work well enough to be confident that this was not his intention, but it leaves a highly unpleasant aftertaste nonetheless.
As a creator and user of moderately simple business/financial models, and in a previous life in geology, I really appreciate this deeper look into the challenges of modeling a complicated system like the earth's climate.
One important aspect not mentioned in the discussion of models with gradual incremental changes is the concept of tipping points. Gradual change seems very benign which is comforting but very misleading.
Physical systems like the climate have complex inputs that have results in a certain range which we call the weather. These systems have quite a bit of resilience as inputs change, that is the other inputs constrain the results to a common usually familiar range. Until they don't. We don't always get gradual change when you change the inputs; there are phase shifts and interactions that can cause fast changes. Once some inputs are pushed far enough, the range of results will move dramatically to a different range and could again prove resilient to change meaning it will be hard to get the climate system back to the original range we currently experience. I haven't found the discussion of climate change really face the challenge of what the world might look like if we push the climate into a new set of different values and how hard is might be to get back to what we know. Another possibility is that a tipping point will push our climate into a less stable condition with wide ranges of results.
To get more concrete, we are now melting the Arctic polar ice cap. It will take a long time before we can the world's climate back to a place where we rebuild that ice and get all the impacts back that the ice gave our climate. Another example - what happens if we change large parts of our agricultural land today into deserts.
One view of the devastating Syrian civil war holds that it started with the climate changing to drought which pushed impoverished farmers into the cites, triggered greater political instability. The war drove millions of refugees into Europe with a resulting rise in anti-immigrant authoritarian politics.
Our climate is less stable than we think. Viewing it as mild warming is a mistake.
For a model to be accurate, its complexity must equal, or exceed, the complexity of the modeled system. However, models which are less complex than the systems they model may be adequate for some purposes.
If a simple model appears to accurately predict the behavior of an apparently more complex system, it is possible, although not certain, that the modeled system's actual complexity is less than its apparent complexity.
SO much to respond to in this one. I think the bottom line response though is this -- she said just a few seconds before the end that what we really need are leaders who are better at representing "our values" -- and I just about fell over because we absolutely already HAVE leaders who represent our deepest values. Our dire ecological state and diminishing prospects for survival of organized civilization flow directly from that fact, because our values are precisely those we developed when we were small bands of scrappy primates in an essentially empty, extremely calorie-scarce world and the only real wealth was social status within our small band, and "greed, for want of a better word, [was] good."
Essentially all our environmental problems flow from our victories, and continuing to operate from our bedrock values in a very different context that makes those values (greed for social status) so powerfully destructive.
Arrhenius did essentially all the climate modeling we have ever needed more than a century ago -- to an astounding degree, he gave us a very good approximation and since then, "all else is commentary."
But, nonetheless, we busy ourselves polishing up climate models like Democrats in the Senate busy themselves polishing up policy proposals in the wistful hope that there's some magic degree of complexity or wordsmithing or something that will bring the GOP around enough to allow policy proposals that do not comport with their prime directive (serve the high-status rich) to go through. That just never gets old, apparently, because it's all we know how to do.
It's fine to critique climate models for being developed by WEIRD types, but don't think for a minute that if we had a whole suite of truly diverse climate models where we really welcomed diverse strategies for trying to illuminate possible futures it would make a damn bit of difference, because the models really don't matter, the making of ever more sophisticated models is simply the status seeking avenue for quants who like to do puzzles with computers. Our models don't matter because, as noted above, we've known the big picture outline of this story since Arrhenius and haven't really added much since then. And models will never serve to reason people out of positions they didn't reason themselves into, and by "people" I mean the planetary elites, the people who, by their actions, clearly communicate that that their only goal is to feed their greed for status/wealth, to which they are slaves . . . just as we non-elites would be in their positions.
One point that shouldn't be overlooked about non-quantitative modelling -- i.e., "let's read science fiction to know what the future holds" -- is that at least it's possible to isolate and identify the parameters being assumed in quantitative physical models and make them accessible through a universal language of math. Fiction is a much more powerful mechanism of selling models to a mass audience, especially video fiction. When you were talking about "let's read fiction" as a form of candidate models, I kept thinking about Woodrow Wilson showing the very first movie ever screened in the White House -- "Birth of a Nation." Now THAT was some powerful modeling on screen, and exceedingly effective, and totally non-quantitative. And it shaped policy for decades and decades.
Hugely appreciated. My job involved "building a model", which took 20 years.
( http://brander.ca/#watermodel - only interesting if you replace water mains for a living, and want to guess which ones are the worst, to be replaced next year, with only the limited data you have about things ten feet underground.)
All these issues came up, though it was such a simple thing compared to the million-variable problems in economics and climate. Are you just demanding it echo back your assumptions? Your prejudices? Is the model for getting at the truth, admitting past failures - or just justifying the guesses you've already taken? Is it for giving your boss an excuse to spend more this year, or (more often) for spending less this year?
And, having built it variable by variable, tested ranges for everything, I could "play" that model like a violin, have it tell you anything, just name the conclusion you want. And, indeed, "reasonable" ranges of assumptions were wide enough to increase, or decrease the water-main budget.
It was obvious to me, from contacts with my managers, that they weren't the ones to morally trust with such responsibility, nor did they want it. They would subject any assumptions that recommended budget increase to the strictest of "prove it" standards, but the opposite would just make me a trusted SME ("Subject Matter Experts", whose word stands behind most model 'assumptions'.)
So, I just stuck to the best numbers I could really justify to myself. And I was the lowest guy on the tree. A "senior engineer" with no staff reporting, in Management world, is a nobody.
So, when the discussion got to be about how those assumptions that go into the model ARE our values, our standards, our morals, I was cheering - because I was the guy that had to have them.
We had a whole exercise, went on for years, about picking our morals: how much money was a "day without water" worth to people? How much should we spend to avoid a 15 minute traffic delay? (I picked $250/house and $10/car, respectively.)
It might be a valuable follow-up to interview the people at EPA, the highways designers, the airplane regulators - how, how exactly, when the number will be public and top people are questioned about it, do you pick a monetary value on human life?
Holy cow, I'd heard of this meeting, I think Slashdot mentioned it at the time. But the full story is jaw-dropping, at least when I got to the shock collars and only-I-know-the-food-locker combination. Like the prison in Andor. Damn.
I loved this interview so much. The interviewee did a good job but DR’s framing is what really stood out to me on this. Roberts shepherding us to the very subtle point about epistemology and action act the end. Chef’s kiss. Want to assign it to my poli sci seminar students.
The critique I have is with the interviewee’s conceptual framework. Do we really need more diversity of models or a way to have a democratic discussion about values as we are building climate models and making policy recommendations. I liked the more abstract philosophical point that any human representation is a model, but I thought that she lost the thread a bit when she starts to conflate the basic moral epistemological models we carry around with actual scientific tools we rely on to cope with climate change.These phenomena are interrelated, but they are not the same. It’s like saying we’re all kind of dying all the time is the same as somebody bleeding to death from a gunshot wound. Everyone can—should— how to say in what makes a life valuable in the short time we have. Not everyone is qualified to perform surgery.
Basically we need a three phase process: deliberation on values, good faith attempts by scientists to model those, which spurs another round of deliberation. It’s not that we need more models We need more inclusive processes for building/critiquing it.
The import of this discussion for policy makers seems abundantly clear, and, as fate would have it, I've got a nephew, or at least my wife does (so I guess I do too...) who's a high level staffer for a cabinet secretary - not DOE, but still one with climate related activity and I've sent him the link.
I'm going to see about getting it to my Senators and Congress-critter as well.
It's great for us to hear this, but those are the folks who need to listen to it.
This pod is a useful reminder that the technological aspects of confronting climate change are but one dimension of a complex sociopolitical problem. Models have built-in limitations and are not immune to bias (I love this term "conviction narrative"). I would have liked to hear more about the evolution of climate modeling from the its early days to the machine-learning techniques now emerging.
Since we're around 420ppm and climbing and the pre-industrial level was 280ppm could we all agree that we're playing with dynamite. With or without models were heading for a world full of hurt very quickly.
FWIW: The Grist Article you shared is a gem, otters and all. I think we need a new Climate discount rate to capture some of those non-linear growth assumptions and ethical calls…but somehow make this as approachable as possible. And, what would markets do with that information?
I love Dave Roberts's work in general, but this episode had me tearing my little remaining hair out. Fortunately, the guest was mostly sensible and kept pulling DR back from the most extreme nihilism about models and human understanding in general. While I have no doubt that many of the stated criticisms of economic and social models are valid in a great many cases, tarring all models, especially the basical physical models of the atmosphere, with the same brush is grossly unfair and is simply giving aid and comfort to climate deniers. DR repeatedly seemed not to understand that model predictions are in most cases not absolute but conditional predictions--not what will happen, but what will happen if a particular course of action is taken. I really think that after this poor showing, Dave owes his listeners an in-depth conversation with someone who does top-flight physical climate modeling and can speak rationally about these models' uncertainties and imperfections, but also their very real capabilities. Otherwise, the audience is left with the impression that models and by extension science are mostly bullshit, that we'll get different answers to any given question depending on the race and gender of the modeler, and that none of it should be taken very seriously. I know and esteem DR's past work well enough to be confident that this was not his intention, but it leaves a highly unpleasant aftertaste nonetheless.
As a creator and user of moderately simple business/financial models, and in a previous life in geology, I really appreciate this deeper look into the challenges of modeling a complicated system like the earth's climate.
One important aspect not mentioned in the discussion of models with gradual incremental changes is the concept of tipping points. Gradual change seems very benign which is comforting but very misleading.
Physical systems like the climate have complex inputs that have results in a certain range which we call the weather. These systems have quite a bit of resilience as inputs change, that is the other inputs constrain the results to a common usually familiar range. Until they don't. We don't always get gradual change when you change the inputs; there are phase shifts and interactions that can cause fast changes. Once some inputs are pushed far enough, the range of results will move dramatically to a different range and could again prove resilient to change meaning it will be hard to get the climate system back to the original range we currently experience. I haven't found the discussion of climate change really face the challenge of what the world might look like if we push the climate into a new set of different values and how hard is might be to get back to what we know. Another possibility is that a tipping point will push our climate into a less stable condition with wide ranges of results.
To get more concrete, we are now melting the Arctic polar ice cap. It will take a long time before we can the world's climate back to a place where we rebuild that ice and get all the impacts back that the ice gave our climate. Another example - what happens if we change large parts of our agricultural land today into deserts.
One view of the devastating Syrian civil war holds that it started with the climate changing to drought which pushed impoverished farmers into the cites, triggered greater political instability. The war drove millions of refugees into Europe with a resulting rise in anti-immigrant authoritarian politics.
Our climate is less stable than we think. Viewing it as mild warming is a mistake.
For a model to be accurate, its complexity must equal, or exceed, the complexity of the modeled system. However, models which are less complex than the systems they model may be adequate for some purposes.
If a simple model appears to accurately predict the behavior of an apparently more complex system, it is possible, although not certain, that the modeled system's actual complexity is less than its apparent complexity.
SO much to respond to in this one. I think the bottom line response though is this -- she said just a few seconds before the end that what we really need are leaders who are better at representing "our values" -- and I just about fell over because we absolutely already HAVE leaders who represent our deepest values. Our dire ecological state and diminishing prospects for survival of organized civilization flow directly from that fact, because our values are precisely those we developed when we were small bands of scrappy primates in an essentially empty, extremely calorie-scarce world and the only real wealth was social status within our small band, and "greed, for want of a better word, [was] good."
Essentially all our environmental problems flow from our victories, and continuing to operate from our bedrock values in a very different context that makes those values (greed for social status) so powerfully destructive.
Arrhenius did essentially all the climate modeling we have ever needed more than a century ago -- to an astounding degree, he gave us a very good approximation and since then, "all else is commentary."
But, nonetheless, we busy ourselves polishing up climate models like Democrats in the Senate busy themselves polishing up policy proposals in the wistful hope that there's some magic degree of complexity or wordsmithing or something that will bring the GOP around enough to allow policy proposals that do not comport with their prime directive (serve the high-status rich) to go through. That just never gets old, apparently, because it's all we know how to do.
It's fine to critique climate models for being developed by WEIRD types, but don't think for a minute that if we had a whole suite of truly diverse climate models where we really welcomed diverse strategies for trying to illuminate possible futures it would make a damn bit of difference, because the models really don't matter, the making of ever more sophisticated models is simply the status seeking avenue for quants who like to do puzzles with computers. Our models don't matter because, as noted above, we've known the big picture outline of this story since Arrhenius and haven't really added much since then. And models will never serve to reason people out of positions they didn't reason themselves into, and by "people" I mean the planetary elites, the people who, by their actions, clearly communicate that that their only goal is to feed their greed for status/wealth, to which they are slaves . . . just as we non-elites would be in their positions.
One point that shouldn't be overlooked about non-quantitative modelling -- i.e., "let's read science fiction to know what the future holds" -- is that at least it's possible to isolate and identify the parameters being assumed in quantitative physical models and make them accessible through a universal language of math. Fiction is a much more powerful mechanism of selling models to a mass audience, especially video fiction. When you were talking about "let's read fiction" as a form of candidate models, I kept thinking about Woodrow Wilson showing the very first movie ever screened in the White House -- "Birth of a Nation." Now THAT was some powerful modeling on screen, and exceedingly effective, and totally non-quantitative. And it shaped policy for decades and decades.
Hugely appreciated. My job involved "building a model", which took 20 years.
( http://brander.ca/#watermodel - only interesting if you replace water mains for a living, and want to guess which ones are the worst, to be replaced next year, with only the limited data you have about things ten feet underground.)
All these issues came up, though it was such a simple thing compared to the million-variable problems in economics and climate. Are you just demanding it echo back your assumptions? Your prejudices? Is the model for getting at the truth, admitting past failures - or just justifying the guesses you've already taken? Is it for giving your boss an excuse to spend more this year, or (more often) for spending less this year?
And, having built it variable by variable, tested ranges for everything, I could "play" that model like a violin, have it tell you anything, just name the conclusion you want. And, indeed, "reasonable" ranges of assumptions were wide enough to increase, or decrease the water-main budget.
It was obvious to me, from contacts with my managers, that they weren't the ones to morally trust with such responsibility, nor did they want it. They would subject any assumptions that recommended budget increase to the strictest of "prove it" standards, but the opposite would just make me a trusted SME ("Subject Matter Experts", whose word stands behind most model 'assumptions'.)
So, I just stuck to the best numbers I could really justify to myself. And I was the lowest guy on the tree. A "senior engineer" with no staff reporting, in Management world, is a nobody.
So, when the discussion got to be about how those assumptions that go into the model ARE our values, our standards, our morals, I was cheering - because I was the guy that had to have them.
We had a whole exercise, went on for years, about picking our morals: how much money was a "day without water" worth to people? How much should we spend to avoid a 15 minute traffic delay? (I picked $250/house and $10/car, respectively.)
It might be a valuable follow-up to interview the people at EPA, the highways designers, the airplane regulators - how, how exactly, when the number will be public and top people are questioned about it, do you pick a monetary value on human life?
Can anybody find me a link to David Rothkopf being interviewed about billionaire bunkers?
I biffed it on the pod, Roy. It's Douglas Rushkoff.
https://www.theguardian.com/news/2022/sep/04/super-rich-prepper-bunkers-apocalypse-survival-richest-rushkoff
Many thanks.
Holy cow, I'd heard of this meeting, I think Slashdot mentioned it at the time. But the full story is jaw-dropping, at least when I got to the shock collars and only-I-know-the-food-locker combination. Like the prison in Andor. Damn.
I loved this interview so much. The interviewee did a good job but DR’s framing is what really stood out to me on this. Roberts shepherding us to the very subtle point about epistemology and action act the end. Chef’s kiss. Want to assign it to my poli sci seminar students.
The critique I have is with the interviewee’s conceptual framework. Do we really need more diversity of models or a way to have a democratic discussion about values as we are building climate models and making policy recommendations. I liked the more abstract philosophical point that any human representation is a model, but I thought that she lost the thread a bit when she starts to conflate the basic moral epistemological models we carry around with actual scientific tools we rely on to cope with climate change.These phenomena are interrelated, but they are not the same. It’s like saying we’re all kind of dying all the time is the same as somebody bleeding to death from a gunshot wound. Everyone can—should— how to say in what makes a life valuable in the short time we have. Not everyone is qualified to perform surgery.
Basically we need a three phase process: deliberation on values, good faith attempts by scientists to model those, which spurs another round of deliberation. It’s not that we need more models We need more inclusive processes for building/critiquing it.
The import of this discussion for policy makers seems abundantly clear, and, as fate would have it, I've got a nephew, or at least my wife does (so I guess I do too...) who's a high level staffer for a cabinet secretary - not DOE, but still one with climate related activity and I've sent him the link.
I'm going to see about getting it to my Senators and Congress-critter as well.
It's great for us to hear this, but those are the folks who need to listen to it.
Delightful conversation. I like the idea of being a better epistemic citizen. Much appreciated.
:^D
This pod is a useful reminder that the technological aspects of confronting climate change are but one dimension of a complex sociopolitical problem. Models have built-in limitations and are not immune to bias (I love this term "conviction narrative"). I would have liked to hear more about the evolution of climate modeling from the its early days to the machine-learning techniques now emerging.
Since we're around 420ppm and climbing and the pre-industrial level was 280ppm could we all agree that we're playing with dynamite. With or without models were heading for a world full of hurt very quickly.
FWIW: The Grist Article you shared is a gem, otters and all. I think we need a new Climate discount rate to capture some of those non-linear growth assumptions and ethical calls…but somehow make this as approachable as possible. And, what would markets do with that information?