Grimes was born in New York City. His acting career began at age seven, during the depression, when he helped earn money for his family. He appeared as Jackie Grimes in the Broadway play The Old Maid, which won a Pulitzer Prize and ran for 10 months in New York. It then went on tour for another 11 months.

Grimes worked on radio, beginning with the CBS program Let's Pretend. He was also a regular on The Fred Allen Show, The Philip Morris Playhouse, Second Husband, CBS Radio Myster Theater, and Death Valley Days. By age 12, he was appearing on 35 to 40 radio shows a week.

In 1944, he moved to California to work for Universal. His credits include Fairytale Murder, Lady on a Train, and Week-End at the Waldorf. In the early 1950s, he switched to television. His credits include Alcoa Presents, Love of Life, The Aldrich Family, Tom Corbett, Space Cadet, Maude, On the Rocks, as well as All in the Family.

In 1962, Grimes and Peter Fernandez worked together on a series of records for MGM. Five years later, Fernandez hired him to do the voices of Sparky and Chim Chim on Speed Racer. He retired from acting in 1991.