The role of Bill Davis was first offered to Glenn Ford, who turned it down, before Brian Keith was cast; Keith also held part ownership of the show.

Kathy Garver was cast as Cissy at the last minute, in the middle of shooting the pilot, after the originally cast actress gained 15 pounds while traveling in Europe; no scenes were ever filmed with her predecessor.

Although Cissy was written as a teenager, Kathy Garver was actually 20 years old at the time of filming.

Brian Keith personally suggested Johnny Whitaker for the role of Jody.

Jody and Buffy were originally meant to be different ages, but after producers saw how well Whitaker looked alongside the already-cast Anissa Jones, they rewrote the characters as twins.

To accommodate movie star Brian Keith's schedule, the show used a 60-day production system where all of his scenes for the season were shot in two 30-day blocks, with co-stars filming their scenes afterward, similar to how My Three Sons handled Fred MacMurray.

Because child actors Johnny Whitaker and Anissa Jones could legally work only eight hours a day, their scenes were always shot first, resulting in the cast and crew sometimes filming as many as four episodes simultaneously.

When Sebastian Cabot fell ill, the show introduced Giles French's brother Niles French (played by John Williams) for nine episodes in 1967, with the storyline explaining that Giles was away touring with the Queen in the Commonwealth countries.

Producer Don Fedderson's son, Gregg Fedderson, played Cissy's boyfriend Gregg Bartlett from seasons two through five, and while working on the show he began dating Kathy Garver in real life.

Due to the show's popularity with young girls, Buffy's doll 'Mrs. Beasley' was turned into a talking toy by Mattel, which also produced two additional Buffy dolls modeled after the character, each including a miniature Mrs. Beasley.

In Police Academy 2: Their First Assignment, gang leader Zed is shown tearfully watching Family Affair in his hideout.

A Mrs. Beasley doll with missing glasses appears in the music video for the song 'California Tuffy' by the Geraldine Fibbers.

In 1982, the L.A. musical group Angel and the Reruns released a song titled 'Buffy Come Back,' referencing actress Anissa Jones' teenage drug overdose.