Mackenzie Phillips, who had just worked with Coppola on American Graffiti, had a supporting role as Harry Caul's niece that was cut during editing, and Abe Vigoda's role as Caul's lawyer was also cut.

The original cinematographer was Haskell Wexler, but severe creative and personal differences with Coppola led to his firing shortly after production began; Coppola replaced him with Bill Butler, and nearly all of Wexler's footage was reshot except for the complex Union Square surveillance scene.

This was the first of two Oscar-nominated films where Haskell Wexler was fired and replaced by Bill Butler, the second being One Flew Over the Cuckoo's Nest (1975), where Wexler had similar conflicts with director Miloš Forman.

Walter Murch served as supervising editor and sound designer and had considerable creative freedom during editing because Coppola was simultaneously working on The Godfather Part II.

Gene Hackman reportedly had a difficult time adapting to the introverted, socially awkward Harry Caul character because it was so unlike his own outgoing personality, which made him moody and irritable on set; he nonetheless considers it one of his favorite performances and has called the film his personal favorite among his own work.

Coppola has cited Michelangelo Antonioni's Blowup (1966) as a key influence, conceiving the idea to fuse its themes of perception versus reality with the world of audio surveillance.

The script was actually completed in the mid-1960s, before Nixon became president, and filming wrapped months before the most revelatory Watergate stories broke, meaning the film's uncanny resemblance to the Watergate scandal and its use of real surveillance equipment used by the Nixon administration was entirely coincidental.

Surveillance expert Martin Kaiser claims to be the inspiration for Harry Caul and says he served as an uncredited technical consultant; he notes the final scene, where Caul cannot find a bug hidden in his own saxophone, was inspired by Léon Theremin's passive covert listening devices like the Great Seal bug.

Coppola also based Caul on Harry Haller, the reclusive protagonist of Herman Hesse's 1927 novel Steppenwolf, and gave Caul a Catholic confession scene, viewing confession as one of the earliest forms of surveillance.

A private investigator and occasional actor named Karl Schnazer, who appeared in Coppola's early films Tonight for Sure and Dementia 13, told Coppola an anecdote about a man he tailed for months who failed to recognize him at a party, which inspired a sequence in the film.

David Shire's piano score was composed and performed before the film was even shot, and for some cues he used musique concrète techniques, distorting taped piano sounds to create alternative tonalities.

In the 1998 film Enemy of the State, Gene Hackman's character Edward Lyle closely resembles Harry Caul, wearing the same translucent raincoat and using a nearly identical workshop; the photograph used for Lyle's NSA file is actually a photo of Caul, and the film includes a surveillance scene similar to The Conversation's Union Square opening.

A television pilot starring Kyle MacLachlan as Harry Caul was produced for NBC but was not picked up for a full series.