Arkin was born in Brooklyn, New York City, the son of Beatrice (née Wortis), a teacher, and David I. Arkin, a painter and writer who mostly worked as a teacher.[1] Arkin was raised in a Jewish family with "no emphasis on religion;" his maternal grandfather was an immigrant from Odessa, Ukraine. The family moved from Brooklyn to Los Angeles, California when Arkin was 11 years old, but an eight-month Hollywood strike cost Arkin's father a set designer job he had wanted to take. Arkin's parents were accused during the 1950s Red Scare of being Communists, which led to Arkin's father losing his job after refusing to answer questions regarding his political affiliation. David Arkin challenged the dismissal and ultimately prevailed, but after his death.

Arkin, who had been taking acting lessons since age 10, became a scholarship student at various drama academies, including one run by Stanislavsky student Benjamin Zemach, who taught Arkin a psychological approach to acting.[4] Arkin attended Franklin High School,[5] in Los Angeles, followed by Los Angeles City College from 1951 to 1953. With two friends, he formed the folk music group The Tarriers, in which Arkin sang and played guitar. The band-members co-composed the group's 1956 hit "The Banana Boat Song", a reworking, with some new lyrics, of a traditional, same-name Jamaican calypso folk song combined with another titled "Hill and Gully Rider".[6] It reached #4 on the Billboard magazine chart the same year as Harry Belafonte's better-known hit version.

Arkin is one of only eight actors to receive an Academy Award nomination for Best Actor for his first screen appearance (for The Russians Are Coming, the Russians Are Coming in 1966). Two years later, he was again nominated, for The HeartIs a Lonely Hunter.

Arkin is equally comfortable in comedy and dramatic roles. Among those for which he has garnered the most favorable critical attention are his Oscar-nominated turns above; Wait Until Dark, as the erudite killer stalking Audrey Hepburn director Mike Nichols' Catch-22; The Seven-Per-Cent Solution (where he played Sigmund Freud); writer Jules Feiffer's Little Murders, which Arkin directed; the The In-Laws, co-starring Peter Falk; Glengarry Glen Ross; and Little Miss Sunshine, for which he received his third Oscar nomination, in the category of Best Supporting Actor. On the 11 February 2007 he received a BAFTA Film Award for Best Actor in a Supporting Role for his portrayal of Grandfather Edwin in Little Miss Sunshine. On February 25, 2007, upon winning the Academy Award for Best Actor in a Supporting Role, Arkin, who plays a foul-mouthed grandfather with a taste for heroin said, "More than anything, I'm deeply moved by the open-hearted appreciation our small film has received, which in these fragmented times speaks so openly of the possibility of innocence, growth and connection". At 72 years old, Arkin became the sixth oldest winner of the Best Supporting Actor Oscar. On Broadway, Arkin starred in Enter Laughing, for which he won a Tony Award, and Luv. He also directed The Sunshine Boys, among others.