Ben Miller was halfway through a PhD on novel quantum effects in quasi-zero-dimensional electron systems when he decided that, on balance, he’d rather be a comedian.

Millennials will know the 55-year-old from The Armstrong and Miller Show, an irreverent yet cerebral sketch show that climbed the ranks to BBC One in the late Noughties. Gen Z will know him as the taciturn patriarch of the Featherington family in the Netflix phenomenon Bridgerton. And baby boomers will know him as an awkward, fish-out-of-water detective in the Caribbean-set crime drama Death in Paradise. Tomorrow he’ll be on British screens again, performing on ITV’s delightfully frothy All Star Musicals wearing a bald cap and monstrous makeup. But if you’d told 20-year-old Ben Miller all that, he’d have snorted and returned to his text book.

“I did undergraduate physics at Cambridge,” he recalls now, from his shed, “and I never did anything other than science. I never even left the library or the lecture theatre.” He was competitive by nature, and being one of only two out of 100 students in his year who went to a state school, he felt he had something to prove. “That was a huge culture shock,” he says of finding himself being surrounded by posh Etonians. “But it was also really great for me, because one of the things you don’t have an enormous supply of, going to a comprehensive school, is confidence. At Cambridge, I found out that the advantages I thought other people had didn’t really exist.”

By the time he was doing his PhD, Miller had more confidence – and more time on his hands. “I thought, ‘Maybe I’ll audition for a play.’” He ended up being cast as Cassio in the Shakespeare tragedy Othello – not exactly a cheery role, and yet, “I’d come on stage playing this straight character in this quite heavy play, and everyone would just start laughing. After a while I thought, ‘Maybe I could do something with this.’” He started writing sketches for the Cambridge comedy troupe Footlights, and refamiliarising himself with the TV comedies he had watched with his dad growing up in Cheshire: The Young Ones, Morecambe and Wise, Monty Python. Through that, he developed “a really forensic knowledge of all British comedy” – spoken like a true scientist. Soon, much to his parents’ horror, he was bidding his PhD goodbye and moving to London to pursue comedy.

It was there that he met Alexander Armstrong. “I was sharing a house with [playwright] Jez Butterworth,” he recalls, “and Jez was in a play with Xander.” Armstrong had also been at Cambridge, studying English, but they hadn’t really crossed paths there. “Jez said, ‘You’ve got to see this guy, he’s fantastic – you two should do a double act together.’ Xander was living on a barge in Chiswick, and Jez and I went round one night. We ended up having a right old time. Xander was playing the piano and we were singing ‘Pennies from Heaven’, absolutely hammered, obviously, at two o’clock in the morning. And it grew from there.”

The pair had a spark; they shared a bumbling deadpan demeanour and a humour that was slightly out of step with the zeitgeist – though they had crucial differences, too. “I’m a complete introvert; Xander’s a complete extrovert. Xander’s very posh; I’m not at all. Something so unique happens when it’s the two of us.”

Comedy club performances in Notting Hill led to a sketch show, which led to Armstrong and Miller, later rebranded The Armstrong & Miller Show when it landed on BBC One. The most outlandish sketches are on YouTube, with a few million views a piece. There’s the heart surgeon with a personal vendetta against his former-PE teacher patient; the train station worker who helps a suicidal would-be jumper lie down in front of the rail replacement bus instead; the obnoxious businessman who travels around the world but doesn’t know what his job is: “I have meetings with other men who look very like me, we josh about golf handicaps and the wives’ breasts… but I really don’t know why.” And then there are the Second World War pilots who speak Noughties slang in old-fashioned RP voices. “Jumbo Wilkins got shot down last week and it was, like, really bad because he died and everything.”

“We had a set of rules,” says Miller. “There mustn’t be any references to the real world; we wouldn’t have any punchlines; we’d never break the logic of the sketch; and our sketches are surreal but they must have a kernel of observed truth.” He pauses. “Sometimes I’d struggle to tell you what the kernel was, but there’s meant to be one in there somewhere.”

Another of those fighter pilot sketches involves the word “Gaylord”, which was a popular insult at the time, but doesn’t exactly play well today. Are there some sketches he thinks have dated badly? “Yeah, I mean, plenty,” he says. “But by and large I’m really proud of it.”

He is proud, too, of his performance on All Star Musicals, which sees Miller and a handful of other celebrities, including presenter Fern Britton and Corrie star Catherine Tyldesley take on showtunes in front of a live audience. It harkens back to the golden days of talent show TV, except everyone is extremely kind. “I did love doing it, I have to say,” says Miller. “I mean, I was absolutely terrified. I was bricking it.”

You wouldn’t have known. Miller gives it his camp, creepy best, and one of the judges describes it as his favourite performance in the show’s history. But rehearsing wasn’t all fun and games. “The world of musicals is quite… ‘hostile’ isn’t the right word… it’s just quite formal compared to acting. With acting, you just sort of chat to somebody about your character or whatever, but they all sit behind desks. There were just rows and rows of desks at one end of this huge dance studio, and you suddenly became aware of the scale of the thing.” In the end, though, “it was very, very emotional. When I got in front of an audience, I realised how much I’d missed that experience over the last two years.”

Like the rest of us, Miller has spent the past two years having a “big internal emotional journey” and watching lots of TV. He particularly loved I Hate Suzie, Lucy Prebble and Billie Piper’s sublime dark comedy. “I just think the best comedy is being written by women,” he says. “There’s nothing to touch I Hate Suzie, Fleabag… I always hear Lucy Prebble’s voice in Succession… That’s where it’s at right now. That’s where the really, really good work is being done.”

When he was first coming up, things were very different for women in comedy. “Certainly there was never any misapprehension as to whether women were funny or not; there just weren’t many of them, that was the difficulty. It was just very hard to find women to work with.” Then again, “there were some great women coming through, like Jessica Hynes, Sue Perkins and Sally Phillips. Smack the Pony is such a great show. They said, ‘OK, we’re going to define a new comedy’ and they created their own rules and did it their own way.’”

Was he aware of the frustration among the women he knew, that they couldn’t get the opportunities men could? “I wouldn’t say there was a real sense of frustration because the way women dealt with it then was very different. You would talk to a stand-up comedian and she would say, ‘These are some of the things they say to me when I’m on stage. These are some of the heckles that I get, so it’s very hard for me to do material. So I do this, and I have this act, and that’s how I deal with it.’ But nobody thought it was possible to change the actual context in which we were performing. Now there’s been such a hunger for change. The audiences have changed. We’ve all changed.”

There was certainly a hunger for a show such as Bridgerton, a Regency-era romp with an anachronistic soundtrack, a defiantly feminist tone and a diverse cast. “It’s cutting edge,” says Miller. “Funnily enough, on the diversity point – Adjoa Andoh, who plays Lady Danbury, said, ‘This is what it looked like in Regency times.’ I went and I got some portraits from the National Portrait Gallery, and it’s absolutely true. London was a very diverse society in 1813. What’s weird is that period dramas were all white.” He shakes his head. “That’s what was weird.”

‘All Star Musicals at Christmas’ airs on Boxing Day at 8pm on ITV and from 7am on ITV Hub