One by one the tour buses descended on the blue collar neighborhood in Hong Kong known as To Kwa Wan — literally translated as Potato Bay — unloading throngs of travelers from mainland China outside large restaurants where a quick lunch awaited them inside.

Outfitted in white, red and orange ball caps to denote which tour they belonged to, the visitors crowded the sidewalks, smoked cigarettes under a “No Smoking” sign and bumped into the glass storefront of a real estate office where Nicky Lam, a property agent, was rolling her eyes.

“They’re very loud,” Ms. Lam said, complaining that some of the tourists used her office bathroom and water cooler without asking.

“One tourist came in and asked for restaurant recommendations,” she added. “I stared at him and said, ‘This is a real estate office.’”