Derivative formula

Recipe for Disaster: The Formula That Killed Wall Street

Felix Salmon

Wired February 2009

A year ago, it was hardly unthinkable that a math wizard like David X. Li might someday earn a Nobel Prize. After all, financial economists—even Wall Street quants—have received the Nobel in economics before, and Li's work on measuring risk has had more impact, more quickly, than previous Nobel Prize-winning contributions to the field. Today, though, as dazed bankers, politicians, regulators, and investors survey the wreckage of the biggest financial meltdown since the Great Depression, Li is probably thankful he still has a job in finance at all. Not that his achievement should be dismissed. He took a notoriously tough nut—determining correlation, or how seemingly disparate events are related—and cracked it wide open with a simple and elegant mathematical formula, one that would become ubiquitous in finance worldwide.

For five years, Li's formula, known as a Gaussian copula function, looked like an unambiguously positive breakthrough, a piece of financial technology that allowed hugely complex risks to be modeled with more ease and accuracy than ever before. With his brilliant spark of mathematical legerdemain, Li made it possible for traders to sell vast quantities of new securities, expanding financial markets to unimaginable levels.

His method was adopted by everybody from bond investors and Wall Street banks to ratings agencies and regulators. And it became so deeply entrenched—and was making people so much money—that warnings about its limitations were largely ignored.

Then the model fell apart. Cracks started appearing early on, when financial markets began behaving in ways that users of Li's formula hadn't expected. The cracks became full-fledged canyons in 2008—when ruptures in the financial system's foundation swallowed up trillions of dollars and put the survival of the global banking system in serious peril.

David X. Li, it's safe to say, won't be getting that Nobel anytime soon. One result of the collapse has been the end of financial economics as something to be celebrated rather than feared. And Li's Gaussian copula formula will go down in history as instrumental in causing the unfathomable losses that brought the world financial system to its knees.

How could one formula pack such a devastating punch? The answer lies in the bond market, the multitrillion-dollar system that allows pension funds, insurance companies, and hedge funds to lend trillions of dollars to companies, countries, and home buyers.

A bond, of course, is just an IOU, a promise to pay back money with interest by certain dates. If a company—say, IBM—borrows money by issuing a bond, investors will look very closely over its accounts to make sure it has the wherewithal to repay them. The higher the perceived risk—and there's always some risk—the higher the interest rate the bond must carry.

Bond investors are very comfortable with the concept of probability. If there's a 1 percent chance of default but they get an extra two percentage points in interest, they're ahead of the game overall—like a casino, which is happy to lose big sums every so often in return for profits most of the time.
Bond investors also invest in pools of hundreds or even thousands of mortgages. The potential sums involved are staggering: Americans now owe more than $11 trillion on their homes. But mortgage pools are messier than most bonds. There's no guaranteed interest rate, since the amount of money homeowners collectively pay back every month is a function of how many have refinanced and how many have defaulted. There's certainly no fixed maturity date: Money shows up in irregular chunks as people pay down their mortgages at unpredictable times—for instance, when they decide to sell their house. And most problematic, there's no easy way to assign a single probability to the chance of default.

Wall Street solved many of these problems through a process called tranching, which divides a pool and allows for the creation of safe bonds with a risk-free triple-A credit rating. Investors in the first tranche, or slice, are first in line to be paid off. Those next in line might get only a double-A credit rating on their tranche of bonds but will be able to charge a higher interest rate for bearing the slightly higher chance of default. And so on.

The reason that ratings agencies and investors felt so safe with the triple-A tranches was that they believed there was no way hundreds of homeowners would all default on their loans at the same time. One person might lose his job, another might fall ill. But those are individual calamities that don't affect the mortgage pool much as a whole: Everybody else is still making their payments on time.

But not all calamities are individual, and tranching still hadn't solved all the problems of mortgage-pool risk. Some things, like falling house prices, affect a large number of people at once. If home values in your neighborhood decline and you lose some of your equity, there's a good chance your neighbors will lose theirs as well. If, as a result, you default on your mortgage, there's a higher probability they will default, too. That's called correlation—the degree to which one variable moves in line with another—and measuring it is an important part of determining how risky mortgage bonds are.

Investors like risk, as long as they can price it. What they hate is uncertainty—not knowing how big the risk is. As a result, bond investors and mortgage lenders desperately want to be able to measure, model, and price correlation. Before quantitative models came along, the only time investors were comfortable putting their money in mortgage pools was when there was no risk whatsoever—in other words, when the bonds were guaranteed implicitly by the federal government through Fannie Mae or Freddie Mac.

Yet during the '90s, as global markets expanded, there were trillions of new dollars waiting to be put to use lending to borrowers around the world—not just mortgage seekers but also corporations and car buyers and anybody running a balance on their credit card—if only investors could put a number on the correlations between them. The problem is excruciatingly hard, especially when you're talking about thousands of moving parts. Whoever solved it would earn the eternal gratitude of Wall Street and quite possibly the attention of the Nobel committee as well.

To understand the mathematics of correlation better, consider something simple, like a kid in an elementary school: Let's call her Alice. The probability that her parents will get divorced this year is about 5 percent, the risk of her getting head lice is about 5 percent, the chance of her seeing a teacher slip on a banana peel is about 5 percent, and the likelihood of her winning the class spelling bee is about 5 percent. If investors were trading securities based on the chances of those things happening only to Alice, they would all trade at more or less the same price.
But something important happens when we start looking at two kids rather than one—not just Alice but also the girl she sits next to, Britney. If Britney's parents get divorced, what are the chances that Alice's parents will get divorced, too? Still about 5 percent: The correlation there is close to zero. But if Britney gets head lice, the chance that Alice will get head lice is much higher, about 50 percent—which means the correlation is probably up in the 0.5 range. If Britney sees a teacher slip on a banana peel, what is the chance that Alice will see it, too? Very high indeed, since they sit next to each other: It could be as much as 95 percent, which means the correlation is close to 1. And if Britney wins the class spelling bee, the chance of Alice winning it is zero, which means the correlation is negative: -1.

If investors were trading securities based on the chances of these things happening to both Alice and Britney, the prices would be all over the place, because the correlations vary so much.
But it's a very inexact science. Just measuring those initial 5 percent probabilities involves collecting lots of disparate data points and subjecting them to all manner of statistical and error analysis. Trying to assess the conditional probabilities—the chance that Alice will get head lice if Britney gets head lice—is an order of magnitude harder, since those data points are much rarer. As a result of the scarcity of historical data, the errors there are likely to be much greater.
In the world of mortgages, it's harder still. What is the chance that any given home will decline in value? You can look at the past history of housing prices to give you an idea, but surely the nation's macroeconomic situation also plays an important role. And what is the chance that if a home in one state falls in value, a similar home in another state will fall in value as well?

Here's what killed your 401(k) David X. Li's Gaussian copula function as first published in 2000. Investors exploited it as a quick—and fatally flawed—way to assess risk. A shorter version appears on this month's cover of Wired.

Pr[Ta<1,tb<1]=Φ2(Φ-1(Fa(1)),Φ-1(Fb(1)),γ)

ProbabilitySpecifically, this is a joint default probability—the likelihood that any two members of the pool (A and B) will both default. It's what investors are looking for, and the rest of the formula provides the answer.
Survival times
The amount of time between now and when A and B can be expected to default. Li took the idea from a concept in actuarial science that charts what happens to someone's life expectancy when their spouse dies.
Equality
A dangerously precise concept, since it leaves no room for error. Clean equations help both quants and their managers forget that the real world contains a surprising amount of uncertainty, fuzziness, and precariousness.
Copula
This couples (hence the Latinate term copula) the individual probabilities associated with A and B to come up with a single number. Errors here massively increase the risk of the whole equation blowing up.
Distribution functions
The probabilities of how long A and B are likely to survive. Since these are not certainties, they can be dangerous: Small miscalculations may leave you facing much more risk than the formula indicates.
Gamma
The all-powerful correlation parameter, which reduces correlation to a single constant—something that should be highly improbable, if not impossible. This is the magic number that made Li's copula function irresistible.

Enter Li, a star mathematician who grew up in rural China in the 1960s. He excelled in school and eventually got a master's degree in economics from Nankai University before leaving the country to get an MBA from Laval University in Quebec. That was followed by two more degrees: a master's in actuarial science and a PhD in statistics, both from Ontario's University of Waterloo. In 1997 he landed at Canadian Imperial Bank of Commerce, where his financial career began in earnest; he later moved to Barclays Capital and by 2004 was charged with rebuilding its quantitative analytics team.

Li's trajectory is typical of the quant era, which began in the mid-1980s. Academia could never compete with the enormous salaries that banks and hedge funds were offering. At the same time, legions of math and physics PhDs were required to create, price, and arbitrage Wall Street's ever more complex investment structures.

In 2000, while working at JPMorgan Chase, Li published a paper in The Journal of Fixed Income titled "On Default Correlation: A Copula Function Approach." (In statistics, a copula is used to couple the behavior of two or more variables.) Using some relatively simple math—by Wall Street standards, anyway—Li came up with an ingenious way to model default correlation without even looking at historical default data. Instead, he used market data about the prices of instruments known as credit default swaps.

If you're an investor, you have a choice these days: You can either lend directly to borrowers or sell investors credit default swaps, insurance against those same borrowers defaulting. Either way, you get a regular income stream—interest payments or insurance payments—and either way, if the borrower defaults, you lose a lot of money. The returns on both strategies are nearly identical, but because an unlimited number of credit default swaps can be sold against each borrower, the supply of swaps isn't constrained the way the supply of bonds is, so the CDS market managed to grow extremely rapidly. Though credit default swaps were relatively new when Li's paper came out, they soon became a bigger and more liquid market than the bonds on which they were based.
When the price of a credit default swap goes up, that indicates that default risk has risen. Li's breakthrough was that instead of waiting to assemble enough historical data about actual defaults, which are rare in the real world, he used historical prices from the CDS market. It's hard to build a historical model to predict Alice's or Britney's behavior, but anybody could see whether the price of credit default swaps on Britney tended to move in the same direction as that on Alice. If it did, then there was a strong correlation between Alice's and Britney's default risks, as priced by the market. Li wrote a model that used price rather than real-world default data as a shortcut (making an implicit assumption that financial markets in general, and CDS markets in particular, can price default risk correctly).

It was a brilliant simplification of an intractable problem. And Li didn't just radically dumb down the difficulty of working out correlations; he decided not to even bother trying to map and calculate all the nearly infinite relationships between the various loans that made up a pool. What happens when the number of pool members increases or when you mix negative correlations with positive ones? Never mind all that, he said. The only thing that matters is the final correlation number—one clean, simple, all-sufficient figure that sums up everything.
The effect on the securitization market was electric. Armed with Li's formula, Wall Street's quants saw a new world of possibilities. And the first thing they did was start creating a huge number of brand-new triple-A securities. Using Li's copula approach meant that ratings agencies like Moody's—or anybody wanting to model the risk of a tranche—no longer needed to puzzle over the underlying securities. All they needed was that correlation number, and out would come a rating telling them how safe or risky the tranche was.

As a result, just about anything could be bundled and turned into a triple-A bond—corporate bonds, bank loans, mortgage-backed securities, whatever you liked. The consequent pools were often known as collateralized debt obligations, or CDOs. You could tranche that pool and create a triple-A security even if none of the components were themselves triple-A. You could even take lower-rated tranches of other CDOs, put them in a pool, and tranche them—an instrument known as a CDO-squared, which at that point was so far removed from any actual underlying bond or loan or mortgage that no one really had a clue what it included. But it didn't matter. All you needed was Li's copula function.

The CDS and CDO markets grew together, feeding on each other. At the end of 2001, there was $920 billion in credit default swaps outstanding. By the end of 2007, that number had skyrocketed to more than $62 trillion. The CDO market, which stood at $275 billion in 2000, grew to $4.7 trillion by 2006.

At the heart of it all was Li's formula. When you talk to market participants, they use words like beautiful, simple, and, most commonly, tractable. It could be applied anywhere, for anything, and was quickly adopted not only by banks packaging new bonds but also by traders and hedge funds dreaming up complex trades between those bonds.

"The corporate CDO world relied almost exclusively on this copula-based correlation model," says Darrell Duffie, a Stanford University finance professor who served on Moody's Academic Advisory Research Committee. The Gaussian copula soon became such a universally accepted part of the world's financial vocabulary that brokers started quoting prices for bond tranches based on their correlations. "Correlation trading has spread through the psyche of the financial markets like a highly infectious thought virus," wrote derivatives guru Janet Tavakoli in 2006.

The damage was foreseeable and, in fact, foreseen. In 1998, before Li had even invented his copula function, Paul Wilmott wrote that "the correlations between financial quantities are notoriously unstable." Wilmott, a quantitative-finance consultant and lecturer, argued that no theory should be built on such unpredictable parameters. And he wasn't alone. During the boom years, everybody could reel off reasons why the Gaussian copula function wasn't perfect. Li's approach made no allowance for unpredictability: It assumed that correlation was a constant rather than something mercurial. Investment banks would regularly phone Stanford's Duffie and ask him to come in and talk to them about exactly what Li's copula was. Every time, he would warn them that it was not suitable for use in risk management or valuation.

In hindsight, ignoring those warnings looks foolhardy. But at the time, it was easy. Banks dismissed them, partly because the managers empowered to apply the brakes didn't understand the arguments between various arms of the quant universe. Besides, they were making too much money to stop.

In finance, you can never reduce risk outright; you can only try to set up a market in which people who don't want risk sell it to those who do. But in the CDO market, people used the Gaussian copula model to convince themselves they didn't have any risk at all, when in fact they just didn't have any risk 99 percent of the time. The other 1 percent of the time they blew up. Those explosions may have been rare, but they could destroy all previous gains, and then some.
Li's copula function was used to price hundreds of billions of dollars' worth of CDOs filled with mortgages. And because the copula function used CDS prices to calculate correlation, it was forced to confine itself to looking at the period of time when those credit default swaps had been in existence: less than a decade, a period when house prices soared. Naturally, default correlations were very low in those years. But when the mortgage boom ended abruptly and home values started falling across the country, correlations soared.

Bankers securitizing mortgages knew that their models were highly sensitive to house-price appreciation. If it ever turned negative on a national scale, a lot of bonds that had been rated triple-A, or risk-free, by copula-powered computer models would blow up. But no one was willing to stop the creation of CDOs, and the big investment banks happily kept on building more, drawing their correlation data from a period when real estate only went up.

"Everyone was pinning their hopes on house prices continuing to rise," says Kai Gilkes of the credit research firm CreditSights, who spent 10 years working at ratings agencies. "When they stopped rising, pretty much everyone was caught on the wrong side, because the sensitivity to house prices was huge. And there was just no getting around it. Why didn't rating agencies build in some cushion for this sensitivity to a house-price-depreciation scenario? Because if they had, they would have never rated a single mortgage-backed CDO."

Bankers should have noted that very small changes in their underlying assumptions could result in very large changes in the correlation number. They also should have noticed that the results they were seeing were much less volatile than they should have been—which implied that the risk was being moved elsewhere. Where had the risk gone?

They didn't know, or didn't ask. One reason was that the outputs came from "black box" computer models and were hard to subject to a commonsense smell test. Another was that the quants, who should have been more aware of the copula's weaknesses, weren't the ones making the big asset-allocation decisions. Their managers, who made the actual calls, lacked the math skills to understand what the models were doing or how they worked. They could, however, understand something as simple as a single correlation number. That was the problem.
"The relationship between two assets can never be captured by a single scalar quantity," Wilmott says. For instance, consider the share prices of two sneaker manufacturers: When the market for sneakers is growing, both companies do well and the correlation between them is high. But when one company gets a lot of celebrity endorsements and starts stealing market share from the other, the stock prices diverge and the correlation between them turns negative. And when the nation morphs into a land of flip-flop-wearing couch potatoes, both companies decline and the correlation becomes positive again. It's impossible to sum up such a history in one correlation number, but CDOs were invariably sold on the premise that correlation was more of a constant than a variable.
No one knew all of this better than David X. Li: "Very few people understand the essence of the model," he told The Wall Street Journal way back in fall 2005.
"Li can't be blamed," says Gilkes of CreditSights. After all, he just invented the model. Instead, we should blame the bankers who misinterpreted it. And even then, the real danger was created not because any given trader adopted it but because every trader did. In financial markets, everybody doing the same thing is the classic recipe for a bubble and inevitable bust.
Nassim Nicholas Taleb, hedge fund manager and author of The Black Swan, is particularly harsh when it comes to the copula. "People got very excited about the Gaussian copula because of its mathematical elegance, but the thing never worked," he says. "Co-association between securities is not measurable using correlation," because past history can never prepare you for that one day when everything goes south. "Anything that relies on correlation is charlatanism."
Li has been notably absent from the current debate over the causes of the crash. In fact, he is no longer even in the US. Last year, he moved to Beijing to head up the risk-management department of China International Capital Corporation. In a recent conversation, he seemed reluctant to discuss his paper and said he couldn't talk without permission from the PR department. In response to a subsequent request, CICC's press office sent an email saying that Li was no longer doing the kind of work he did in his previous job and, therefore, would not be speaking to the media.
In the world of finance, too many quants see only the numbers before them and forget about the concrete reality the figures are supposed to represent. They think they can model just a few years' worth of data and come up with probabilities for things that may happen only once every 10,000 years. Then people invest on the basis of those probabilities, without stopping to wonder whether the numbers make any sense at all.

As Li himself said of his own model: "The most dangerous part is when people believe everything coming out of it."

The Pope Versus the Vatican
GEORGE WEIGEL

Standpoint April 2009

Pope John Paul II was buried on 8 April 2005. In the nine days between his funeral and the sealing of the conclave to elect his successor, the critics and enemies of Cardinal Joseph Ratzinger worked feverishly to prevent his being chosen pope. The Sunday Times went trolling for stories contrasting Ratzinger-the-Hitler-Youth with heroic Karol Wojtyla, Polish resistance fighter against Nazism and communism. La Repubblica, the flagship daily of the Italian Left, spun fanciful tales about a "German-American coalition" capable of blocking Ratzinger's election, not least by appealing to Third World cardinals whose dioceses depended on German financial support. These at times risible media efforts at electoral preemption had at least the tacit support, and in some cases the encouragement, of progressive Catholic activists, intellectuals and prelates for whom the idea of "God's Rottweiler" as pope was the nightmare that dared not speak its name.

Ironically, Joseph Ratzinger spent part of those same nine days in a parallel effort to forestall his own election. A modest man, he nonetheless knew that his brilliant performance as Dean of the College of Cardinals - leading the cardinals in their deliberations after John Paul's death and leading the world in prayer at his funeral - had made him the odds-on favourite to be the 264th successor to St Peter. And he wanted none of it. He had planned to submit his resignation to the new pope and to demand its acceptance. Thrice before he had retreated when he had tried to resign and John Paul had asked him to stay. Now he was determined to return to his native Bavaria to take up housekeeping with his older brother, a priest and distinguished choirmaster. He would turn 78 two days before the conclave was immured. It was time to go home and pick up the threads of the scholarly life he had sacrificed on becoming archbishop of Munich and Freising in 1977.

How does a papal frontrunner work against his own election, particularly if he actually means it and is not simply making an outward show of humility or diffidence? Ratzinger's case against Ratzinger was simply put: "I am not a man of governo, of governance," he said, in each of the half-dozen languages he speaks fluently. Don't do this to me; don't do this to yourselves. Those who were forwarding his candidacy - men like George Pell of Sydney, Christoph Schönborn of Vienna and Angelo Scola of Venice, three of the Catholic Church's most impressive younger leaders - replied, in so many words, "Why don't you let God have his say? Don't prejudge the work of the Holy Spirit." As things turned out, the actual voting was close to a formality, as Ratzinger was elected on the fourth ballot in one of the shortest conclaves in history. Yet there were, evidently, lingering questions among both his supporters and opponents as to whether he would accept the burden he had tried to avoid: in their post-conclave press conference, the German cardinals (who included both supporters and opponents) told reporters that there was a "great collective sigh of relief" when Cardinal Ratzinger accepted his election. That the newly-chosen Benedict XVI remained acutely aware of his own limitations, however, was made unmistakably clear on the morning after the election. Celebrating Mass with the College of Cardinals in the Sistine Chapel, Benedict asked, in his homily, "Do not deny me your counsel."
In the four years since the 265th Bishop of Rome stepped out on to the loggia of St Peter's to be presented urbi et orbi, "to the city and the world", Pope Benedict XVI has systematically dismantled the media cartoon of "God's Rottweiler" that had dogged him for decades. A few months after his election, while vacationing at the papal summer villa, Castel Gandolfo, he invited his old theological adversary, the Swiss dissident Hans Küng, to stop by for a conversation and a beer or two. Confounding the critics who claimed he could never compete with "John Paul Superstar" for the affections of the young, he has presided over two successful World Youth Days, in Cologne and Sydney. He displayed a well-honed pastoral sensibility in his May 2008 Washington meeting with the victims of clerical sexual abuse, as he did in reaching out to the relatives of those who had died on 9/11 after his silent prayer over the ruins of the Twin Towers in New York.

His September 2006 Regensburg Lecture on faith and reason is still deplored by uncomprehending reporters and soi-disant Vatican insiders as a major diplomatic "gaffe". The truth of the matter is that Benedict sent such salutary shock waves throughout the worlds-within-world of Islam that more robust patterns of interreligious dialogue are slowly emerging. Issues once considered untouchable - religious freedom as a fundamental human right that can be known by reason, and the necessary separation of religious and political authority in a just state - are now, at Benedict's insistence, at the forefront of the dialogue between Catholicism and Islam. His address to the United Nations General Assembly in April 2008 was a powerful and compelling argument that the exercise of freedom must be guided by moral truths, and that those moral truths are accessible to men and women of good will who take the risk of thinking seriously. Regularly defying the nay-sayers who argued that Ratzinger would be simply unpresentable in public, Benedict's weekly audiences in Rome continue to draw large crowds, many of them larger than those drawn by John Paul II.

In his first two encyclicals, Deus Caritas Est [God Is Love] and Spe Salvi [Saved in Hope], he has displayed a striking ability to elucidate the basics of Christian faith in a way that takes full account of postmodern scepticism. In these letters, the once-dreaded "enforcer of orthodoxy" responds to unbelief or weak belief in a spirit of conversation, not condemnation. His international bestseller, Jesus of Nazareth, sympathetically explored an American rabbi's imaginary conversation with Jesus, even as Benedict reminded all Christians of the debt that Christianity owes to its parent, Judaism.

And the thumbscrews and rack have remained locked in the basement of the office Ratzinger once headed, the Congregation for the Doctrine of the Faith, which (as the world media never cease to remind us) "was once known as the Inquisition".
Yet for all these impressive accomplishments, the fifth year of his pontificate is opening under storm clouds of crisis. The Pope's January effort to extend a hand of reconciliation to the ultra-traditionalist followers of the late Archbishop Marcel Lefebvre ignited a worldwide uproar. One of the four Lefebvrist bishops whose excommunications were lifted, an ex-Anglican named Richard Williamson, turned out to be a Holocaust denier - a point of which the Pope and his senior advisers were evidently unaware, although bloggers and other internet literates from the Antipodes to Zimbabwe had the full, nasty story. The Lefebvrist fiasco and the chaos it caused in Catholic-Jewish relations were just settling down when Austria erupted. The issue in this case was the appointment of a new auxiliary bishop of Linz, who turned out to have interesting ideas about the relationship between divine providence and meteorology: Hurricane Katrina's ravaging of New Orleans, the nominee once claimed, was God's punishment for decades of debauchery in the Big Easy. The bishop-elect eventually asked the Pope to withdraw his nomination, and Benedict agreed. Some wondered whether a new round of Josephinism - the Enlightenment-era Austrian resistance to papal authority in the nomination of bishops - was at hand. As this Alpine ecclesiastical earthquake rumbled across Europe, the Roman Curia proved itself incapable of frankly and expeditiously handling another disaster - the revelations that Father Marcial Maciel, founder of the priestly order of the Legion of Christ and the lay movement Regnum Christi, had led a life of sexual dissolution and financial irregularity, even as the order and the lay movement provided some of the most vibrant young priests and lay activists in the Church. While that slow-motion train-wreck headed towards the abyss, the best of the Vaticanisti, Sandro Magister of the weekly L'espresso, reported that the Vatican Secretariat of State may have badly misread the character and qualifications of Joseph Li Shan, the new bishop of Beijing, who has, Magister argued, been far too cosy with the Chinese communist regime and the regime-supported Patriotic Catholic Association.

Benedict XVI did take advantage of a February meeting demanded by the pro-abortion Catholic Speaker of the US House of Representatives, Nancy Pelosi, to deliver a firm reminder of one of those moral truths that can be known by reason - that innocent human life deserves the protection of the laws in any just society. Speaker Pelosi was also denied the photo-op she obviously craved, a sign that the Vatican had not completely lost its capacity to control its own agenda and the Pope's role in advancing that agenda. Yet, for Benedict's supporters, the Pelosi reprimand was a brief moment of sun through the darkening clouds. Had Ratzinger been right in April 2005? Was he really not a man of governo? And what did that portend for the future of his papacy which, despite his age, could stretch well into the next decade?

The Lefebvrist fiasco was a microcosm of the complex set of administrative and managerial problems that Benedict must confront and resolve, if his intellectual lucidity and pastoral good sense are not to be obscured by the incapacities and incompetence of the Curia, the reform of which he was expected to undertake by those who elected him in 2005.

The Curia exists for one reason: to give effect to the will of the Bishop of Rome, who is the source of both legislative authority and policy initiative in the universal Church. As Canon 360 of the Code of Canon Law puts it, "The Supreme Pontiff usually conducts the business of the universal Church through the Roman Curia, which acts in his name and with his authority for the good and for the service of the [local] Churches." As in all governmental bureaucracies, of course, stated rationale and actual performance are not always aligned. For the Curia not infrequently mimics the behaviour of every other bureaucratised power structure on the planet. It staples a salute to the leader to its collective forehead even as it pursues its own interests and intrigues, all the while attempting to bring the leader around to "the way we do things here". It is often thought that popes have a unique freedom of action. The fact is that the exercise of papal governance is deeply affected, for good or ill, by the competence of the Curia and its senior officials. Contemporary popes can and do go over or around the Curia to shape the international debate, as John Paul II and Benedict XVI have shown. Yet there is no governing the Catholic Church over or around the Curia. So a great deal depends on how successful a given pope is in selecting the Curia's leaders and guiding their work.

Most of what the world thinks it knows about the Curia is, in fact, mistaken. It is, for example, a remarkably small operation, given that it is the administrative centre of a human community with 1.2 billion members living in every corner of the globe. Among its 3,000 or so employees, perhaps 40 at the most have real operational or decision-making roles. The rest are worker-bees - often very able, polyglot worker-bees with advanced degrees - whose task is to serve what are known in Curia-speak as "the superiors": the two or three heads of each of the curial "dicasteries", which are the rough equivalents of cabinet departments.

Despite its international character, Italian remains the Curia's lingua franca, and an inability to speak the language well is usually an insuperable impediment to serious influence - unlike the inability to speak English, the now-universal language of commerce, science and diplomacy. The Curia also remains Italianate, which is to say laid-back, in its work culture: while the Curia does not acknowledge that staple of modern temporality known as the "weekend" and its offices are open for business on Saturday mornings, those same offices are only open in the afternoon on Tuesdays and Fridays (the days, Curial wags note, that Catholic piety traditionally assigns to the Sorrowful Mysteries of the rosary). An Italianate approach to crisis management - or, better, crisis non-management - is also pervasive, rooted in the sense that "we've seen it all before, so there's no reason to get into a flap". That "thinking in centuries" approach has its undeniable advantages in a world in which everyone is supposed to have an opinion on, or answer to, everything, 24/7: it allows for situations to mature, for calm to be restored and for rational decision-making to take place. It can also result in a pope getting blindsided by events to the detriment of his authority, as Benedict XVI has learned to his regret and John Paul II learned before him. In April 2002, for example, John Paul was learning things about the American crisis of clerical sexual abuse that he should have known four months earlier. Once he knew he acted, and acted decisively. But he should have known earlier, and the Curia's entrenched scepticism about media-driven crises was one reason he didn't.

John Paul II was acutely aware that the election of the first non-Italian pope in 455 years, as well as his own free-wheeling personal style, unsettled the traditional managers of popes. Moreover, he was not a man who took much satisfaction from shuffling and reshuffling the boxes on an organisational chart. So rather than undertaking a wholesale re-examination of how the Curia ought to function in the 21st century, he left the basic Curial structure created by Pope Paul VI in 1967 intact, while running his own foreign policy out of the papal apartment - much to the chagrin of the papal Secretariat of State, some of whose senior members imagined that they knew more about, say, Poland, than the Polish pope. John Paul's most important Curial innovation was to jettison the tradition of the pope's spokesman being a Curial priest by installing the Spanish layman Joaquin Navarro-Valls, an accomplished journalist, as head of the Holy See Press Office. It was often said that Dr Navarro-Valls (who liked to joke that his early professional experience as a psychiatrist had prepared him well for dealing with the Vatican press corps) brought the Holy See's press operation "into the 20th century". To which the proper response was, "Yes, the first half of the 20th century." That was no mean accomplishment, given Curial resentments over a layman who was spokesman, confidant and private diplomatic agent of the pope. But Navarro-Valls's personal accomplishments were mistaken by many as a sign that the Curia had entered the world of 21st-century communications. It hasn't, as the first day of the Lefebvrist crisis made painfully clear: Fr Federico Lombardi SJ, Navarro-Valls's successor, was sadly unprepared for the informal press briefing he gave the day the story broke, because he hadn't been brought into whatever deliberations there had been about lifting the Lefebvrist excommunications. Thus the false impression was immediately created, and just as immediately hardened into "fact", that the Lefebvrist bishops had been restored to the full communion of the Church, which hadn't happened. That misimpression, the result of inept communications and bureaucratic blundering, intensified the outrage over Bishop Williamson's Holocaust denial. American Catholic black humour at this incompetence turned to memories of the legendary baseball manager Casey Stengel, who once asked of his woefully inept 1962 New York Mets, "Can't anybody here play this game?"

Every ecumenical council in the history of the Church was prompted by turmoil, conducted in turmoil or resulted in turmoil. The Second Vatican Council, which brought two young central Europeans named Karol Wojtyla and Joseph Ratzinger to prominence on the world Catholic stage, was no exception. The post-conciliar turmoil following Vatican II is generally thought to have taken place on the port side of Catholic life, with liberal and progressive theologians publicly dissenting from authoritative Catholic teaching while attempting to stretch the boundaries of acceptable Catholic thought and practice with the aid of acquiescent bishops and religious superiors. Yet the only schism following Vatican II - the only formal, legal break in the unity of the Church - did not come on the Catholic Left (which knew that its magnetic attraction for the world media required staying formally inside the tent). It came on the far reaches of the Catholic Right. Its protagonist was a French archbishop with extensive missionary experience in Africa, Marcel Lefebvre.

Throughout the world, Lefebvrists and other traditionalist Catholics are known for their preference for the older forms of Catholic worship, particularly the Mass celebrated in Latin according to the missal established by the Council of Trent and revised by Pope John XXIII in 1962. Yet the gravamen of the hardcore Lefebvrists' rejection of Vatican II involved, not liturgy, but politics, and specifically Catholic church-state theory. Vatican II's definition of religious freedom as an inalienable human right - carrying the implication that religious establishments of the ancien régime type were no longer the preferred arrangement - was, in fact, a development of Catholic social doctrine. To the Lefebvrists, however, it was heresy, and the opening wedge to a fatal Catholic accommodation with modernity. Archbishop Lefebvre's war, in other words, was not simply against modern liturgy - it was against modernity. To those who took the trouble to look, there was no surprise here, for the ideological sensibilities of the Lefebvrist movement sprang from the same French political-cultural sources that gave rise to the anti-Dreyfusards of the late 19th century and the Petainists of the mid-20th century. Paul VI, a Francophile who knew Lefebvre's politics and detested them, was nonetheless patient with the French intransigent, fearing a formal schism like that of the Old Catholics after the First Vatican Council. Yet the often indecisive Paul finally suspended Lefebvre from the public exercise of his priestly and episcopal functions in 1976, although he and his followers remained in tenuous communion with the Church.

Popes are duty-bound to try to prevent schisms and to repair breaches in the unity of the Church. Thus John Paul II and then-Cardinal Joseph Ratzinger made every effort, over a long decade, to reconcile Archbishop Lefebvre and his followers - which would have meant allowing them the use of the pre-conciliar liturgy while securing their agreement that Vatican II had been an authentic expression of Catholic faith. But Lefebvre, as Ratzinger once said, was a "very difficult man", and he eventually reneged on a deal he had struck with Ratzinger, who was acting as John Paul II's agent. Moreover, having reneged, he proceeded to commit what is perhaps the ultimate offence for a Catholic bishop - in 1988, he ordained other bishops without the mandate of the pope. Those bishops (including Richard Williamson) and Lefebvre himself were immediately excommunicated. The Lefebvrist movement, the Society of St Pius X (named after the pope who instigated the "anti-Modernist oath" as a precondition to priestly ordination), went into formal schism. From Rome's point of view, Lefebvre's ordinations triggered the worst of worst-case scenarios - for the Church considered the episcopal orders of the men Lefebvre had ordained to be sacramentally valid (although illicit), which meant that the Society of St Pius X now had the capacity to perpetuate its schism indefinitely, through further illicit but valid ordinations of bishops. In secular terms, this was treason, sedition and rebellion in one lethal package.

On the same day that Lefebvre and his four co-conspirators were excommunicated, John Paul II established a new Curial office, the Ecclesia Dei Commission, to bring back into the full communion of the Church those Lefebvrist priests and lay people who could not stomach a formal break with Rome. Over time, Ecclesia Dei became the Curial interlocutor, not only with Lefebvrists who wanted to return to Rome, but with the Lefebvrist movement (typically known as the SSPX) itself. The problem, which became glaringly apparent in January, was that the Ecclesia Dei Commission was a Curial free agent - a loose cannon, rolling around the deck of the Barque of Peter, accountable to no other Curial office.

The current head of the commission is a Colombian, Cardinal Darío Castrillón Hoyos, a brave man who once confronted the cocaine kingpin, Pablo Escobar, by posing as a milkman. On gaining entry to Escobar's home, the doughty Castrillón demanded that the notorious drug lord confess his sins. Cardinal Castillón is also, alas, remembered for having given the worst Curia press conference in living memory. It was 2002, and his task was to present John Paul II's annual Holy Thursday letter to the world's priests. When reporters raised the inevitable questions about the clerical sexual abuse scandal in the US, Castrillón brushed them aside and said that the pope had far more important things to worry about, like peace in the Middle East. It was not a persuasive argument, and it ill-served the pope. Castrillón will turn 80 in July, and it seems that he had become determined to pull off a career-capping spectacular: the reconciliation of the Lefebvrist schism. That cause was also close to the pastoral heart of Benedict XVI, who knew that the majority of Lefebvrist faithful cared little for church-state theory but simply wanted to worship according to the old Latin rite. Thus Benedict, again defying media stereotypes of the papal rottweiler, was prepared to make the opening, public gesture by lifting the excommunications of the four bishops whom Lefebvre (who died in 1991) had illicitly ordained, the assumption presumably being that a similar graceful gesture would come from the Lefebvrist leadership.

It did not. On the contrary, Bishop Bernard Fellay, Lefebvre's successor as head of the SSPX, issued a letter to the faithful, in which he crowed that "the Tradition is no longer excommunicate" - an astonishingly arrogant formulation that seemed to imply that it was the rest of the Catholic Church, not the minuscule sect of Lefebvrists, that was in schism. Moreover, Fellay indicated that the SSPX still had grave difficulties reconciling the teaching of Vatican II with "the Tradition" (always capitalised), as the SSPX understood it. Whatever negotiations Cardinal Castrillón had conducted had not, evidently, received agreement on that crucial point. And that, rather than the media circus over the Holocaust denial of an internationally-known crank like Richard Williamson, cut to the heart of the matter. For it opened up the possibility that, just as the "cafeteria Catholicism" of the progressives was dying of its own intellectual and pastoral sterility, a new cafeteria was being opened in the fever swamps of the Catholic Right. As the media uproar faded, Jewish groups outraged over Williamson's antics were reassured of what they already knew: Benedict XVI was a philo-Semite who would defend living Judaism with his life, and would do so for the strongest of reasons-because he believes, as Vatican II affirmed, that God does not break his covenantal promises. That understanding, and the Church's commitment to religious freedom, was what was being put in jeopardy by a dangerously ill-prepared and likely premature reconciliation with the SSPX.

Cardinal Castrillón was not, of course, the only senior Curia official who did not Google "Richard Williamson" and who did not check the fine print of the negotiations with the SSPX - who did not protect the pope he lived to serve. It is no secret in Roman clerical, diplomatic and media circles that Benedict XVI's choice as Cardinal Secretary of State, the Salesian Tarcisio Bertone (who had been Ratzinger's deputy at the Congregation for the Doctrine of the Faith), has not developed a firm grip on the machinery of Curia governance. Bertone, whose personal loyalty to Benedict XVI is unquestioned, is much given to travelling. Some of his travels have become embarrassments, as in his fulsome embrace of Cuba's Raúl Castro as a reformer, or his glowing description of Belarussian dictator Alexander Lukashenko as a possible bridge between East and West. Happily for Cardinal Bertone, those gaffes (far worse than anything Benedict XVI was alleged to have done at Regensburg) drew little press attention outside the Italian hothouse. Yet to Vatican veterans, they reflected the mistakes of a man who does not do sufficient homework, who does not know how to use the Curial bureaucracy and who has not bent the Curia to the pope's will, but in fact has allowed it to reset its institutional default positions to those in place at the time of John Paul II's election. In the Curial system devised by Paul VI, the Cardinal Secretary of State functions like a prime minister in political life. If he is not firmly in charge, and known to be in charge, the system will relapse into forms of stasis that ill-serve the pope and his purposes. Thus senior American churchmen complain, privately, that much of today's Curia is dysfunctional, including such key dicasteries as the Congregation for the Clergy, the Congregation for Catholic Education and the Congregation for Institutes of Religious Life.

The administrative challenges of this pontificate do not stop with the Curia. Benedict XVI has also been poorly served by some of his nuncios, those ambassadors who represent the Holy See to national governments and local churches. Where the Church's legal position is insecure, or there is covert or open persecution, the nuncio can be an invaluable lifeline for hard-pressed Catholics, connecting them to the papal megaphone in Rome and its capacity to focus world attention on the depredations of tyrants. Where the Church's legal position is stable (which is, happily, more often the case), the nuncio's principal task is to recommend the appointment of new bishops to Rome. There, the decision on a nomination is ultimately the pope's. Making such calls is one of the gravest responsibilities of the modern papacy.

Yet in December 2006, despite warnings from Poles and others that potential Polish episcopal appointments ought to be quietly vetted with Poland's Institute of National Memory in order to avoid the nomination of men who had collaborated in egregious ways with the secret police during the communist period, the nuncio in Poland pushed through the nomination as archbishop of Warsaw of an otherwise impressive candidate, Stanislaw Wielgus, who turned out to have had an unfortunate record of collaboration with the secret police at one point in his career. The ensuing firestorm of criticism, which led to Wielgus's withdrawal from the Warsaw appointment, was a deep embarrassment to Benedict XVI, who had already made a successful pilgrimage to Poland and whose stock there was very high. Notwithstanding that fiasco, the nuncio who ought to have known and to have protected the pope but didn't remains in place. In the US, episcopal sees have remained vacant for as long as two years, thanks to inferior work at the nunciature in Washington. The relevant "superiors" in Rome know this, yet the situation remains unaddressed, because of a combination of Curial politics and the regnant Curial inability to rectify mistakes quickly.

What could turn out to be a botched appointment in Beijing has already been mentioned - it, too, was probably the result of Curial foolishness, in this case the willingness of the Curial China Lobby to bend over backwards in order to establish formal diplomatic relations with the Chinese communist regime. Benedict XVI has been generally supportive of the tougher line taken by the feisty cardinal of Hong Kong, Joseph Zen. But it seems likely that the pope was not sufficiently briefed before he made the Beijing episcopal appointment because the Curia hadn't done its homework adequately.

By contrast, when a nuncio knows his job, speaks the local language, consults intelligently and has a strategic vision, Benedict has shown a willingness to initiate bold and effective change. The Anglophone hierarchy of Canada, for example, has been remade according to the John Paul II/Benedict XVI model of dynamic orthodoxy in a relatively brief period of time.
It is unlikely that Joseph Ratzinger accepted his election thinking of himself as another Leo XIII, who created the modern papacy and died in 1903, after a 25-year reign, at the ripe old age of 93. Always conscious of his health, Benedict XVI in all likelihood imagined his as a short pontificate. Thus the question of Curial reform could be deferred until he was gone, for several reasons. He knew that bold administrative restructuring was not his forte. And given the assumption of a short papacy, he probably thought it papal bad manners to saddle his successor with a long bench of youngish, senior Curial officials just getting adjusted to a new system. So he would find someone he had worked with comfortably in the past - Cardinal Bertone - to keep the machinery running, while he would concentrate on the work he knew he did well - the compelling proclamation of Christian truth to both the Church and the world.

Yet as the events of recent months have made painfully clear, Curial incapacity can impede and even damage the evangelical mission of the most intelligent pope. It was nothing short of a tragedy that a world-class Catholic theologian like Ratzinger, who had spent 50 years explaining Christianity's debt to Judaism to his Christian co-believers, should find himself saddled with the charge that he had reconciled a Holocaust denier to the Church. Yet that is what happened, because no one in whom Benedict XVI reposed trust had the sense to find out about Richard Williamson, and because the Curial culture of the day did not encourage those who did know the facts to warn the superiors. The entire Lefebvrist mess was preventable: if the pope had insisted throughout his pontificate on competence and had taken forceful measures to rectify incompetence; if those whose sole purpose is to give effect to the pope's will had done their jobs better; or if Benedict had reached outside the apostolic palace to take private soundings as to the likely effects of his gesture of reconciliation.

The world, not simply the Church, needs a Benedict XVI working at the top of his form and being enabled to do so by his closest associates. Whether the question is challenging Europe to pull out of the demographic death-spiral caused by its debonair nihilism, or inviting Muslim leaders to seek an Islamically - faithful rapprochement with political modernity, or defending the dignity of human life against the dangers of a brave new world of bio-technically manufactured humanity, there is no substitute for the combination of insight and institutional authority that Pope Benedict brings to the world table. Yet he now faces a crisis in his papacy, for the wisdom of his voice is being muted by the decline in his authority attendant on the managerial incompetence of the Curia.

The Rottweiler Brigade has taken advantage of this crisis to rekindle its assault on Ratzinger's character, through stories retailed to credulous journalists about the pope's "distant, regal style". The must ludicrous of these calumnies came from Robert Fisk, who is "beginning to suspect" that Benedict XVI "might be a very nasty piece of work". The seriousness with which Fisk's judgments are to be taken may be measured by one phrase in his screed in the Independent, in which he claims that Benedict had shown his odious colours by his attitude towards "the pro-Palestinian Angelo Cardinal Sidaro [he means Sodano], John XXIII's secretary of state" [he means John Paul II's secretary of state], from whom Benedict has been "distancing himself" [Benedict allowed Sodano to stay in his Curial office until he was long past the normal retirement age, ratified Sodano's election as Dean of the College of Cardinals, and has permitted Sodano to stay in that post beyond age 80, at which point several previous deans have had the courtesy to resign, having lost their votes in any future conclave]. If his article were a paper in Vaticanology 101, Fisk would receive a grade of "F". Such stories are not only ill-informed and cruel; they get the problem backwards. For Benedict XVI has been reluctant to seek help and to enforce discipline, not because of ego or vanity, but because of his shyness, his respect for others and his unwillingness to add to their burdens, and his profound reluctance to cause pain. These are the qualities of the man that account for the administrative and managerial difficulties of his pontificate, not some bizarre notion of himself as a Theologian-Sun King.

The pope's March 10 letter to the world's bishops - explaining his motives in the SSPX affair, deploring the temporary rift in Catholic-Jewish relations, and promising a more effective curial use of modern communications - underscored Benedict XVI's intelligence, decency and humility. But no pope can govern successfully with an ineffectual Curia whose gaffes undercut the papal message and erode its authority. Both pope and senior churchmen must find new ways to work together if the promise of this papacy is to be fulfilled.