How did macaroni and cheese become elevated to the new sought-after side dish?

Mac and cheese, it seems to me, is the Alexander the Great of food. For in the past 12 months the dish has annexed restaurant menus with the same speed and facility which the King of Macedon cleaved his way through Persia. It is a culinary whirlwind, a phenomenon cast in molten cheese and elbow pasta.

It is now as ubiquitous in restaurants as "small plates" and those exposed filament light bulbs. Visit Byron in Cambridge and you'll find it served as a gold-crusted side; at Goodman, in The City, it reaches some sort of apotheosis coming with black truffle; go south from there to Brixton's Wishbone and witness them throwing angina-worries to the wind as they dip their version in deep-fat fryer. Or, if you prefer, have it as an adornment to your finger-lickin' chicken at Brighton's BBQ Shack. Not your thing? Oh well, you can still visit Mayfair's Automat and order it as a starter or main, plain or with bacon.

The list astounds me because once upon a time in the land of my Nineties and early-noughties childhood it was a thing to be reviled; a culinary freak, the flabbiest of TV dinners – a dish whose preparation was measured out only in microwave bings.

Mrs Beeton may have written two recipes for it in 1861, but a century or so later it seemed only to come in pre-prepared packets, or worse, in tins –glutinous, gloopy, like the contents of some prehistoric swamp liberally doused with Bird's Custard powder.

No restaurant of quality would have had it on its menu. It was as uncool and as Seventies as the Bee Gees. I recall wistfully looking at packets of it in supermarket freezers, taking in the parental dogma that it was "gloppy rubbish" but all the same thinking that melting cheese was good, and pasta was good – so surely this would be double good? At that time what was à la mode in our house was Jamie's Moroccan lamb curry and rocket salads with parmesan shavings.

As I grew older, hungrier, I resented my parents' attitude. I recognise now, however, that they had suffered themselves at the receiving end of the packets of dehydrated pasta and cheese powder. Yes, they could have made the dish from scratch, but they had little desire to make a meal of such little adventurousness; we were after all still in the first years of New Labour, we were optimistic, grasping at sophistication. We made our own guacamole.

Maybe now that optimism has gone, we are looking to simplicity, to homeliness once again. Perhaps we want comfort in discomfiting times. Which explains why we have lustily gobbled up so much down-home Americana of late, a genre of food in which mac and cheese surely sits.

Either way, what is clear is that it is a survivor. Maccheroni, to use its Roman name, has survived centuries of changing fashion largely unscathed. Why? Because it is as adaptable as a chameleon. Google "mac 'n' cheese recipe" and you get 13 million hits.

"There are so many different ways to make it," says Wishbone's William Leigh, the father of the deep-fried version. "You can make it with béchamel, with condensed milk, with ham, leave it gooey or crisp it up and use any number of cheeses."

The cheese, in fact, is a point of fierce debate. You will find proponents of soft French cheese, Italian parmesan, Swiss gruyére and Welsh Caerphilly – on their own, and combined together.

At London's Pitt Cue Co, Tom Adams counsels: "Ogle Shield and Montgomery Cheddar, which has a nice acidity, for the base sauce; then finish with a stilton or stichelton." William Leigh won't even tell me what he uses at Wishbone (though he has generously given us his home recipe, published here).

A hastily convened focus group in my kitchen, however, preferred a mixture of English cheddars. A béchamel base we disapproved of, too. Much better to bind the mix with crème fraîche or cream or cottage cheese. Butter does quite nicely, too.

In fact the exact composition of flavours is secondary to the texture of the finished product. Leigh hits the nail on the head when he says, "the important thing is that it is soft and gooey, with a certain stringiness; the macaroni should have a limited bite and a bit of crunch should be in there somewhere."

The way one can guarantee a happily-textured macaroni is simple: add lots and lots of cheese, whatever it is. If in doubt, add more. If you dare, and you should, the ratio is best at around 2:1, cheese to mac.

Why is it important to use so much? Only at the level when you risk a cheese avalanche do you get a truly indulgent dish. And as Leigh says, that is what mac and cheese is now: "It's a luxury product – and the luxury is the indulgence."

It may not be rarefied, or expensive or require a water bath and tweezers to construct, but what it is, however, is a momentary two fingers to healthy eating, a cocked snook at the make-your-own-guacamole years – and it is all the finer for that.

William Leigh’s Ham Hock and Jalapeño Mac ’n’ Cheese

Serves 4 hungry people

285g elbow macaroni
80g ham hock, flaked
Pickled jalapeños (as many or as few as you'd like, diced)
50g butter
50g plain flour
600ml milk
1 dash hot sauce (Frank's is great)
1 dash of Worcestershire sauce
150g red Leicester grated
75g mozzarella (the grated, bagged stuff – nothing posh – this needs to go really gooey and stringy)
75g Parmesan cheese, grated
1 tablespoon cornflour
75g panko
Salt and pepper

Method

Cook the pasta according to the packet instructions and set aside. Pre-heat your oven to 180C. Melt the butter in a saucepan and add the flour. Cook over a low heat, stirring all the time, until the mixture starts to change colour. I always cook it a little further just to make sure the flour is "cooked out" and loses its raw taste.

Add the milk gradually until the mixture thickens. Add the hot sauce and Worcestershire to the milk. Mix all the cheeses together and reserve about 1/5. Toss the 4/5 with the cornflour (this is to stop it sticking together). Add to the milk/butter/flour mix (your roux) and stir until smooth. This may take a few minutes. Season to taste.

Mix with the pasta, add the flaked ham hock and the jalapeños and pour into a baking dish. Mix the remaining cheese with the panko breadcrumbs and scatter over the top. Bake till golden and bubbling.