Every time I hear somebody mention the word purity, I shiver,” says Guillermo del Toro. “Whether it’s one side or another, whether it’s a fascist ideology, or it’s a dictatorship, it doesn’t matter. That’s why the American dream is such a pernicious thing – or perfect, romantic love.” In the Mexican fabulist’s world of ghosts and monsters, the most frightening thing of all is someone who can only think in absolutes.

That’s how he’s always defined his villains: the Francoist captain in Pan’s Labyrinth, so fervent in his loyalty that he’s blind to the fairy tale world unfolding before his stepdaughter’s eyes, or Michael Shannon’s colonel in del Toro’s Best Picture winner, The Shape of Water, who discovers a humanoid amphibian and never sees beyond the chance for personal glory.

There’s certainly nothing absolute about Del Toro. He is a wondrous cauldron of contradictions. When we speak over Zoom, he’s been momentarily wrenched away from his creative haven (he’s working on a black-and-white version of his latest film, Nightmare Alley, to be released in US theatres) and plopped down in an armchair in some characterless LA hotel suite. There’s a stillness to him – one that suggests inner peace and wisdom, but with a constant flicker of something mischievous behind those gigantic, spherical glasses of his.

In Nightmare Alley, the typical gumshoes and mobsters of film noir have been replaced with carnivals and occultists. It’s still noir, though, in the moral sense – in the way it exposes the dark heart and fractured reality of the American dream. When I ask him about the state of the genre today, and how it may reflect our own disillusionment, his voice kicks in with an extra note of determination. It’s clear he’s been thinking about this.

“I think – if we are to make it past this decade, we will look at this moment in time as a moment of solipsistic anxiety,” he says, with a small, bitter laugh. “We are so busy reacting all the time, because we are aided by machines, some of which we carry in our pocket. We process what people think is information, but it’s really emotion. And there’s no way we can go through 150 emotions in the day, in a sane manner there isn’t.”

And that overload only leads us to draw up our defences. “These little fiefdoms that we curate in everything we look, read, listen to only compel us to what we already know,” he continues. “It just confirms our biases. That gives birth to a lot of charlatans in politics. And people that tell us comforting, very well-constructed lies.”

Nightmare Alley is all about those comforting, very well-constructed lies. An adaptation of William Lindsay Gresham’s 1946 noir novel, it marks Del Toro’s first attempt to tackle someone else’s work, outside of his more mainstream, Hollywood efforts – that is, the Hellboy and Blade movies he directed and the ill-fated Mimic from 1997, cut to shreds by Harvey Weinstein. The film, which he wrote together with film historian and screenwriter Kim Morgan (the pair have since married), is also the first of his films not to feature any kind of supernatural entity.

In fact, its story holds a certain disdain for that realm. Gresham’s novel concerns an ambitious carny, Stanton Carlisle (played by Bradley Cooper in the film), who unlocks the powers of manipulation through learning the tricks of mentalism – that great grift which claims to see people’s innermost thoughts and awaiting futures, or which leads some to claim that they can speak to the dead.

But Del Toro can’t quite bring himself to call Nightmare Alley a departure from his previous work. His chief concern lies not in the monstrous, but in the “monstrosity of what it is to be human” that we so fervently try to ignore. “To try to expunge the bad in order to think that an idealised version of us is possible is one of the cruellest exercises that we can do,” del Toro says. “And yet, we go back at it again and again in civilisation.”

It’s far too limiting to define his work only by its strangeness, by the lusciousness of his style or the streaks of blood-red violence that occasionally splatter across the screen. The care that del Toro puts into his creatures – demons, fairies, fauns and phantoms – not only proves his affection for the outsider figure, but shows an understanding that not all that much separates us from them.

Gresham, on some level, understood this. He lived a terribly sad life, riddled with illness, alcoholism, and deep unhappiness. His first wife, the poet Joy Davidman, left him for CS Lewis. After he was diagnosed with tongue cancer, he checked into the hotel where he wrote much of Nightmare Alley and took his own life. In lieu of a suicide note, he was found with a series of business cards – printed on the back were the words “you would rather die than face the truth”. For Del Toro, “that dictum is basically Stan”. The very first thing he does in the film is bury a body beneath the floorboards, this thing that will warp him and force him into a strange state of duality – desperate to run away from himself but also desperate to be seen, to be found out. He is the monster and the human.

Del Toro’s own life has seen its fair share of contradictions, strange and beautiful and sometimes tragic in nature. The director will talk about seeing dead and mutilated bodies while growing up in Guadalajara. He’ll talk about the large house he lived in after his father won the Mexican national lottery, where he kept rats, snakes and a crow as pets. He’ll talk about growing up in a strict Catholic household, with a tarot-reading mother and a father who seemed baffled by his son’s morbidity. He’ll talk about the time James Cameron, a friend of his, once came to his aid and helped pay the ransom when his father was kidnapped in 1997.

The director will speak about any topic with a sense of soul and clarity that makes you feel like you’ve climbed a mountain, scaled the stone steps, and found the hidden shrine just to hear him share a few of his thoughts. There’s no PR spin to how he talks. He is interested purely in his art. At the mention of Nightmare Alley’s colour palette, he dives into an explanation of how Stan, the psychiatrist that seduces him (Cate Blanchett’s Lilith, every inch a femme fatale), and Enoch, a cyclopean fetus in a jar, are all connected. That’s why, apparently, Cooper himself suggested the film’s scene of full-frontal male nudity – a bathing scene, in which Stan is, like Enoch, suspended in liquid. “He could have done it with the underwear on,” del Toro explains. “But he said, no, I want to be this floating, deformed, little baby in the water.”

Stan, by the end, only finds relief when he stops the pretence – when he accepts that he is, as del Toro describes, “not a sophisticated guy, not a superior intellect, not a lover boy with the heart of a teenager, but this wretch amalgam of all that”. The director, however, has found a simpler path to some quiet in all the chaos and the paradoxes. “I have gotten to the age of 57 to a place where I don’t feel lonely,” he explains. “I can be alone without feeling lonely.”

His career is often defined just as much by the films he hasn’t made, as the films he has: an adaptation of Lovecraft’s At the Mountains of Madness, for example, and a new take on The Haunted Mansion ride at Disneyland (officially dead in the water, Del Toro confirms, now that Disney has greenlit an entirely different version). “Guillermo del Toro’s unrealised projects” currently has its own, not unsubstantial, Wikipedia page. But he seems to have found some peace there, too. “Look, we are like fishermen,” the director says. ‘We always lie and say that the best one is the one that got away. You never know. You are like the fool in the tarot cards – just go motley into the precipice, you know? Followed by the yapping dog of your hopes.”

‘Nightmare Alley’ is out now