Kasznar was born in Vienna, Austria as Kurt Servischer. His father left when Kurt was very young, his mother married an Hungarian restaurateur named Ferdinand Kasznar, and Kurt assumed his surname. He emigrated to the United States in the mid-1930s for The Eternal Road in which he played 12 roles. In 1941, he was drafted into the United States Army. He was trained as a cinematographer and later served in the Pacific. He was assigned to an army photographic unit and was part of the team that filmed the signing of the Japanese surrender aboard the USS Missouri.
Kasznar's first major Broadway appearance was in The Happy Time. (He recreated his role for the film version and earned a Golden Globe nomination for his performance.) He also appeared on Broadway in Barefoot in the Park, Waiting for Godot, Six Characters in Search of an Author, and The Sound of Music, for which he was nominated for the Tony Award for Best Performance by a Featured Actor in a Musical.