Rastro comes from the trail of blood from carcases dragged from the slaughterhouse down the Calle de la Ribera de Curtidores to the tanneries. Rag and bone men selling clothes and gypsies from southern Spain selling antiques to dealers were the original traders down this street.

A visit to El Rastro is a Sunday ritual for Madrilenians. For tourists it is an opportunity to step away from the museums and magnificent edifices, interact with locals or people-watch. For everyone it is a chance to browse, haggle and hangout in the numerous bars and cafes. Unlike the iconic Les Puces in Paris, El Rastro is right in the city, an easy 20-minute walk from our apartment in Atocha. Proximity and food are my bargaining points with my two reluctant travelling mates. But I am on a mission to find the antique and retro section of the market.

I tell them to hurry up at the Plaza de Cascorro, the gateway to the market. They complain it's only 9am. For me that's late. At the top of Calle de la Ribera de Curtidores, I think we're in the wrong place – there's lots of cheap touristy stuff, Asian imports, heavy metal and revolutionary T-shirts, badges, Spanish ceramics, fans and synthetic flamenco frocks. These give way to stalls selling birdcages, bull-fighting posters, bric a brac, and racks of secondhand fur, wool, leather, suede and tweed coats. Laneways shooting off to the side look interesting, one with art materials. But the junk continues.

My travelling mates are clearly not impressed and make a beeline to a lively bar on the corner of Calle Mira el Rio Baja for a second breakfast of tortilla de patatas and a cafe au leche, then another drink with cakes. Many tourists can make the mistake of thinking the market is just the arterial strip down the hill ending with what looks like a car boot sale. One stall has a cluster of men hanging around, looking at the ground, hands in pockets gazing at bits of plumbing, beautiful old tools, and anything else you would find in a dream shed.

I am getting warmer. A vintage comic and magazine stallholder points the way to antiques up the hill along Calle Mira el Rio Baja and intersecting streets. I start scanning the stalls, tiny shops, glitzy showrooms and jumble sales.

A tiny vintage toy shop is full of seriously collectible stuff. Adults look longingly at wind-up tin toys, kids are restrained with firm hands. I ask to look at a Spanish police car from the 1960s. The weary shopkeeper opens the cabinet. It's a cool €500. I'm just another tyre kicker wasting his time.

There are many shops selling mid to late 20th-century furniture and objects that not only include the stars of Scandi design, but desirables from Italy, Germany, France and Spain.

There are plenty of other stalls inside buildings and courtyards selling old photos, door handles, picture frames, doors and 18th-century silverware. Another street has rows of antiquarian booksellers.

A flea market is my idea of shopping when travelling. Even if you don't buy it's fun to look at the culture and history of a place through design, curios and clutter, beyond the curated zones of museums and tourist areas. This part of old working-class Madrid is older than the Prado and its collections of paintings by Goya and Velasquez.

My friends join me and have no desire to backtrack – Calle de la Ribera de Curtidores is gridlocked by 11am. In these small streets around the market they see why I was so insistent on coming here.

I buy myself two presents – a vase from the late '60s by Spanish ceramicist Richard Serra (no relation to the sculptor) and an early '70s Italian chrome lamp with a flexible neck, which I justify saying it will bend into my suitcase.

We follow the crowd to nearby La Latina for tapas and the Madrilinean specialty, a bocadillo de calamares, a crusty bread roll stuffed with deep-fried calamari washed down with a cana, the compact name for a small beer. Around 2pm stallholders pack up and shop owners pull down the shutters and head for the bars to join locals and tourists. The market, which can have up to 3500 stalls, is deserted.

Even if you're not a fan of markets, don't miss this one for atmosphere, great food and a cana or two. El Rastro is an easy name to remember and after experiencing the market, not easy to forget.

TRIP NOTES

GETTING THERE

Qatar airlines flies to Sydney to Madrid; for trains see The Man in Seat 61 seat61.com

SHOPPING THERE

El Rastro is just south of La Latina metro station. It is bound by Plaza de Cascorro and Ribera de Curtidores, between Calle Embajadores and the Ronda de Toledo. Hours Sundays 9-2.

Take plenty of euros as some places don't take cards. Or look for the ATM called the Cajero automatico. Watch out for pickpockets. See madridtourist.info/rastro_market.html

THINGS TO DO ON A SUNDAY IN MADRID

MUSEUMS

The awesome threesome – the Museo del Prado, Museo Nacional Centro de Arte Reina Sofia and the Museo Thyssen-Bornemisza – are open on Sunday.

ANTIQUE SHOPS

Many of these are open during the week around Nuevas Galerias and Galerias Piquer. Shops are generally open 10am-2pm and 5pm-8pm, Monday to Saturday. Not all are open on Sunday during the market. See rastromadrid.es/blog/

RETRO AND SPECIALTY SHOPS

See rastromadrid.es/blog/

To end a Sunday indulgence, it's a short walk to the famous Chocolateria San Gines for an artery-blocking serve of chocolate and chorros.

Pasadizo San Gines, 5, 28013 Madrid

Open 24 hours