Picking up two to three years after the events of the first French Connection, New York City police detective James R. "Popeye" Doyle (Gene Hackman) is dispatched to Marseilles, France. His mission is to collaborate with the French Sûreté, under the pretense that he is the only officer able to identify Alain Charnier (Fernando Rey), also known as "Frog One," a notorious heroin kingpin who managed to escape New York at the conclusion of the previous film. Upon his arrival, Popeye encounters his French counterpart, Inspector Henri Barthélémy (Bernard Fresson), during a drug operation at the waterfront. Tensions rise as Barthélémy harbors resentment towards Popeye for his reckless, lone-wolf approach that previously led to the deaths of two police officers. A further complication arises from the corruption in the New York Police Department, which has resulted in the disappearance of the heroin Popeye had secured.
Throughout the mission, Popeye's disdain for French customs and language is evident, as he dismisses the competencies of the French police, deeming them lazy. Meanwhile, amidst the bustling tourist scene in Marseilles, Jacques (Philippe Léotard), a member of Charnier's crew, discreetly pays a Japanese captain to smuggle heroin into the city. Charnier, who enjoys the sport of shooting pheasants on his estate alongside U.S. General William Brian (Ed Lauter), is simultaneously orchestrating his nefarious operations.
During a raid, Popeye pursues a dark figure only to discover later that he is an undercover officer, a man who shortly faces a grim fate at the hands of local thugs. As Charnier conducts business discussions with General Brian in a restaurant, he spots Popeye lurking outside.
In a sinister turn of events, Charnier's henchmen capture Popeye and take him to a dilapidated locale known as the Hotel Colonnades. There, the beleaguered detective, partially conscious, recognizes the building's former name, Hotel de Tanger, marked in the tiles beneath him. Charnier, seeking information from Popeye, ensures that he remains drugged and slowly, over three long weeks, injects him with heroin. During this harrowing period, an elderly English woman (Cathleen Nesbitt), driven by addiction, takes advantage of his helpless state and steals his watch.
In a moment of vulnerability, Popeye divulges that his reason for being in Marseilles is his unique knowledge of Charnier's appearance. This confession pleases Charnier, who orders an overdose to eliminate Popeye. Fortunately, a police medical team miraculously revives him, but Popeye is then locked away by Barthélémy for weeks, hidden from view to conceal his drug dependency. Enduring intense withdrawal, Popeye ultimately overcomes his physical addiction.
After being driven around the city by his compatriots to jog his memory, Popeye uncovers the truth: he was bait, strategically placed to lure Charnier into the open. Committed to regaining his strength, he begins a vigorous regimen of exercise, working out in his hotel, jogging through the Marseilles streets, and ultimately discovering the “Hotel de Tanger” sidewalk once more. Fueled by vengeance and clarity, Popeye calls Barthélémy for water and proceeds to set the hotel ablaze.
The authorities arrive just in time as the occupants of the building flee, and amidst the chaos, Popeye extracts crucial information from one of Charnier's thugs regarding a drug shipment scheduled to arrive from Rotterdam. Barthélémy and a contingent of officers race to the waterfront, where they confront Charnier's men in the act of unloading considerable quantities of heroin from a ship. The situation escalates as water is poured into the dry dock to impede the police, but they arrive just in time to prevent any fatalities.
That night, alone in his room, Popeye contemplates the two bags of heroin he seized earlier but resolutely dumps them on the floor. The following days see the police surveilling a ship, leading to critical exchanges between a captain and an unidentified individual, which Popeye identifies as Jacques. As Sûreté agents track Jacques to a heroin-processing factory, an ensuing raid leads to his demise, though Charnier manages to elude capture once more. A relentless pursuit ensues, culminating in a thrilling chase through the marinas as Popeye relentlessly hunts down Charnier and ultimately shoots him down on the yacht's deck.
