The series was adapted from David Simon's non-fiction book Homicide: A Year on the Killing Streets, based on his year embedded with a real Baltimore Police Department homicide unit in 1988; Simon later became a consultant and producer on the show.

Simon sent the book to Baltimore native director Barry Levinson hoping for a film adaptation, but Levinson felt the material suited television better since the characters and stories could develop over a longer period.

NBC changed the title from Homicide: A Year on the Killing Streets to Homicide: Life on the Street so viewers wouldn't think the show was limited to a single year, and because the network felt 'life' sounded more reaffirming than 'killing streets'.

Although every episode credits Paul Attanasio as creator, writers Eric Overmyer and James Yoshimura disputed this on a DVD audio commentary, claiming the show was actually created by Tom Fontana and Yoshimura.

The show was filmed almost entirely on location in Baltimore using hand-held 16mm cameras, giving it a distinctive gritty look, and the police department scenes were shot at the historic City Recreation Pier in the Fells Point neighborhood.

Despite critical acclaim, the series consistently struggled in the Nielsen ratings; in July 1997 NBC gave producers an ultimatum to beat Nash Bridges in the ratings or face cancellation, and the show was only saved by factors like the loss of Seinfeld and the costly deal to retain ER.

By the start of season 3, NBC ordered that talented but 'unphotogenic' veteran actor Jon Polito be dropped from the cast despite the network otherwise giving writers unusual creative freedom for dark, non-traditional storylines.

Baltimore filmmaker John Waters, who called the show 'the grittiest, best-acted, coolest-looking show on TV', made two cameo appearances, including as a talkative prisoner transferred by Chris Noth's Detective Mike Logan, with Waters and Noth receiving special thanks in that episode's closing credits.

Journalist Tim Russert appeared as himself in a self-referential episode, bickering about birthday presents with his fictional 'cousin' Lieutenant Megan Russert, while director Barry Levinson played himself directing a show-within-a-show in the episode 'The Documentary'.

The show's crossovers with Law & Order began because executive producer Tom Fontana and Law & Order creator Dick Wolf were friends from the 1980s working as writers on St. Elsewhere and Hill Street Blues in the same building; their first crossover featured Chris Noth's Detective Mike Logan delivering a fugitive to Frank Pembleton.

The show's influence extended into reality: after the series depicted the Baltimore Homicide Unit's dry-erase clearance-rate board, the real department temporarily discontinued the practice due to concerns about negative publicity, before reinstating it at detectives' insistence.

A draft shooting script for the series finale, Homicide: The Movie, leaked online under the title Homicide: Life Everlasting before its premiere; although never officially used by NBC, fans have since adopted that title.