It must be quite something to have one of the world’s most famous spy novelists give you the idea for your next book. This is what happened to acclaimed journalist Ben Macintyre (Double-Cross: The True Story of the D-Day Spies, etc.) about eight years ago, when he was walking with a friend on Hampstead Heath in London. Macintyre’s friend happens to be the novelist David Cornwall—better known to most readers as John Le Carré (A Delicate Truth, etc.). While telling old war stories, Macintyre asked Cornwall, “What do you think is the best untold story of the Cold War?”

Without hesitation, Cornwall replied, “It’s the friendship between Nicholas Elliot and Kim Philby. It’s an extraordinary story. You should write it.” And so he did, crafting the story of the world’s most successful double agent in A Spy Among Friends: Kim Philby and the Great Betrayal.

For those not up on their Cold War lore, Philby was an upper-class British bloke who had all the qualifications to become the master spy hunter for MI6, the British intelligence service. Philby ultimately turned out to be a dyed-in-the-wool communist operative who defected to Moscow in 1963 following a dramatic confrontation with his best friend, fellow MI6 officer Nicholas Elliot.

Cornwall knew this because he was there, working alongside men like Elliot, before being forced to leave the intelligence services after his cover was blown during the Philby affair. As Le Carré, he fictionalized the story as the classic 1974 spy novel Tinker, Tailor, Soldier, Spy, and he contributes his own remembrances of the fierce relationship in an inspiring afterword to A Spy Among Friends.

“It seemed to me that the Philby story had never been told as a chronicle of personal betrayal,” says Macintyre on the phone from London. “I wanted to produce a portrait of a very particular, male, post-war friendship, not only with Nicholas Elliot but also with the American CIA man James Jesus Angleton. These are men who were cut from the same cloth, who were as close in their friendship as any upper class heterosexual men could be in the 1940s and 1950s.”

In the book, Macintyre observes that a betrayal on the global level that Philby achieved was ironically due to Philby’s very specific role in the British class system.

“Philby was perfect for MI6 in that he came from the right class and he had the right accent, the right education, the right breeding, and the right interests and hobbies. He was absolutely part of ‘the club,’ and that made him perfectly invisible,” Macintyre says. “What made him such an effective double agent was the fact that his colleagues and close friends at MI6 simply could not believe that someone like Kim Philby—who was someone like them—could possibly be on the other side. He was absolutely part of the brethren and that was what made his great coup so unbelievable.”

Macintyre also gets at the roots of Philby’s apparent fanaticism but also explores the dichotomies that made him such a complex character.

“I think Philby’s conversion to communism was very swift and very complete in its early stages,” he explains. “Over time, I think he became absorbed by the romance of what he was doing. The drug of infidelity was very strong for him. At the same time, he’s a very odd sort of communist. He was an upper class Englishman who professed to believe in a set of ideals but he would no more have mixed with the working class than he would cut off his own hand. He makes for a peculiar, rather elitist revolutionary.”

In fact, says Macintyre, the rarefied air that Philby occupied probably fed into his delusions.

“The whole question of ‘clubs’ is very important here,” says Macintyre. “It is possible to read Philby’s evolution as a desire to belong to more and more exclusive clubs. It’s pretty exclusive to have gone to Westminster. It’s even more exclusive to have gone to Trinity College, Cambridge. MI6 of course is a very exclusive club. But to be the lone KGB agent inside MI6? That’s a club of one.”

In 1962, Nicholas Elliot confronted a drunken, paranoid Kim Philby in Beirut, to tell him the game was finally up. One of the great questions of the Philby affair is whether the double agent outwitted his enemies in order to flee to Moscow, or if MI6 manipulated the old operative into pulling off what spies refer to as a “fade.”

“Which side you come down on in the end comes down to whether you believe MI6 was incredibly incompetent, or was actually operating quite a complex manipulation,” Macintyre says. “Clearly, MI6 did not want Philby back in London but I think they were ambivalent whether he fled to Moscow or stayed in Beirut and drank himself to death. I think either option was fine with them. The clinching element of all this subterfuge is that in the end, Philby believed he was pushed.”

Asked about the comparisons between someone like Philby and the elusive whistle blower Edward Snowden, also ensconced in Moscow, Macintyre says times have changed.

“These are really different sorts of defections,” he says. “However, I do believe that Philby did Anglo-American intelligence services a favor in some ways because the old boy network died the day that he defected. The realization that they had been betrayed created a new era for intelligence work. Today, we have a highly professional intelligence service that does not recruit through the club and we are arguably all safer as a result. You can put that change down to Philby. He would be furious at the compliment but there is a sort of poetic justice to it.”

Though the story is so twisted and cinematic as to seem fictitious, Macintyre believes the divide between spies and novelists is not so great.

“I don’t think it’s an accident that some of the greatest novelist have been spies—Somerset Maugham, Graham Greene, Ian Fleming, or John Le Carré,” he says. “In some ways, what spies do is a form of fiction. They create an alternative reality and try to lure people into it. Philby in some ways was a kind of frustrated novelist, and a man of great theater. Never was a man more acutely aware of his role in his own drama.”

Clayton Moore is a freelance writer, journalist, book critic, and prolific interviewer of other writers. His work appears in numerous newspapers, magazines, websites and other media. He is based in Boulder, Colorado.