Better known as Lefty, the Norfolk, Virginia-born Driesell was an All-State basketball player at Granby High School who helped lead his team to a state championship, winning the tournament MVP in the process. He also played at Duke University but instead of going pro after graduation, he worked for Ford Motor Company and briefly played semi-pro on the side before switching to coaching.
In 1960, Driesell assumed the head coach mantle at Davidson. From the start of his tenure until 1969, he led the team to five regular season championships and three Southern Conference tournaments, winning four straight Coach of the Year awards. He then joined the University of Maryland, College Park, where he led the team to eight NCAA tournament appearances and pushed them to a No. 2 ranking by the Associated Press four seasons in a row.