Born and raised in Connecticut, Willard Smith spent his early childhood in and out of hospitals. He contracted rheumatic fever when he was five years old. He spent several years in a wheelchair and one full year in a children's hospital. Eventually he recovered and joined the Marine Corps on his 17th birthday. Smith spent the next five years with the Marine Corps, serving two years in Vietnam as a scout/sniper in a reconnaissance unit.

In 1969, Smith left the Marine Corps. He spent two years working as a letter carrier for the US Postal Service while attending college full-time at night. He quit the Post Office in 1972 and completed his undergraduate degree at San Diego State University where he graduated with high honors and earned a Bachelor of Arts in sociology and philosophy. He was also a member of Phi Beta Kappa.

On July 4, 1971, Smith met his future wife. They moved in together that night and have been together ever since. He joined the Army in 1975 and served as a race relations specialist until 1981. He applied to law school and graduated from the University of San Diego School of Law in 1984. He and his wife then moved to Washington where he has since had a private practice in the Seattle area. His wife, who has a master's degree in history, has spent the last 10 years working as a teacher's assistant at a private elementary school for gifted children. After leaving active duty in 1981, Willard continued to serve in the Army National Guard until 1992 when he was transferred to the Retired Reserve as a Sergeant First Class.

Smith was a member of the University of San Diego Appellate Moot Court Board and was a member of the National Order of Barristers. He is currently a member of the Washington State Bar Association and the First Marine Division Association.