In 1940s Shanghai, two petty criminals, Sing and Bone, dream of ascending the ranks of the infamous Axe Gang, headed by the ruthless Brother Sum. Their journey begins as they attempt to intimidate the residents of the decrepit Pigsty Alley, masquerading as members of the gang. Unfortunately, their antics draw the ire of the real Axe Gang, leading to a confrontation that threatens the villagers. Fortunately, three tenants of the alley—Coolie, Tailor, and Donut—who are, unbeknownst to the gang, skilled kung fu masters, effortlessly thwart the gang's first wave of attack.
However, in a bid to protect their sanctuary, the Landlady of Pigsty Alley evicts the trio of fighters to shield them from the impending wrath of Brother Sum. The ruthless gangster seizes Sing and Bone, planning to execute them for their impersonation. Yet, the duo manages a last-minute escape, which surprisingly catches Brother Sum's interest. He proposes a deal: if they want to join the Axe Gang, they must first prove themselves by committing murder.
Haunted by memories of his childhood, Sing shares a tale of being duped by a vagrant into purchasing a martial arts pamphlet, only to face humiliation while trying to defend a mute girl named Fong from bullies. This painful experience instills in him the belief that heroes never prevail; consequently, he resolves to embrace a villainous path.
Determined to complete their task, Sing and Bone make an ill-fated attempt to assassinate the Landlady but are instead sent fleeing. In a moment of desperation, Sing takes refuge in a traffic pulpit, where to his astonishment, he finds his injuries healing instantaneously. Meanwhile, Brother Sum, bent on revenge, hires two Harpists wielding a magical guzheng to take out the kung fu masters of Pigsty Alley. The Harpists prove effective, but they ultimately meet their match against the Landlady and her husband, the Landlord, who reveal surprising combat prowess.
In a twist of fate, Sing confronts an ice cream vendor—whom he recognizes as Fong. Overwhelmed by memories, he reacts destructively and leaves, ashamed. Brother Sum, seeing an opportunity, lures Sing with the promise of gang membership if he can liberate the Beast, a renowned assassin, from a mental institution. Sing succeeds, proudly bringing the Beast back to the Axe Gang fold.
Initially dubious about the Beast's capabilities due to his flippant demeanor, Brother Sum quickly changes his stance when the Beast displays his extraordinary skill by halting a bullet mid-flight. A fierce showdown ensues between the Beast and the Landlady and Landlord at a nearby casino, ending in a deadlock. Realizing the missteps of his journey, Sing rebels against the Beast, who retaliates fiercely. The Landlady and Landlord rescue the unconscious Sing and bring him back to Pigsty Alley.
After a swift recovery, the Landlady recognizes Sing's innate kung fu talent. With newfound abilities, he defeats the remaining members of the Axe Gang and confronts the Beast once more. In a climactic battle, Sing unleashes the Buddhist Palm technique upon the Beast, ultimately forcing him to concede.
In the aftermath, Sing and Bone open a candy shop, featuring Fong's lollipop as their logo. As fate would have it, Fong pays a visit, and they share a tender embrace, while the vagabond who once sold Sing a martial arts pamphlet is seen outside, vending more pamphlets, signaling the cycle of misadventure continues.
