Yarmy was born in New York in 1933, son of William Yarmy and his wife, Consuelo (née Deiter), who were Jewish and Catholic, respectively. Yarmy was the younger brother of actor and Get Smart star, Don Adams. He also had an elder sister, Gloria Ella Yarmy (later Gloria Burton), a writer who wrote an episode of Get Smart. Dick had served as a corporal, like his brother had in World War II, during the Korean War.

Dick Yarmy graduated from the New York University with a degree in engineering and began his acting career in 1968, appearing in the TV series Get Smart, also That Girl, The Partners, The Partridge Family, and Arnie. He also appeared in several films including Bone, The Kentucky Fried Movie, The Swinging Barmaids, and Carpool among others. Yarmy also appeared in commercials, such as George and Marge for Union Oil.