A three-year letterman as a running back in football, Baker finished his career ranked sixth on Cornell's all-time rushing list (1,579 yards). He received Associated Press Division I-AA All-America honorable mention honors and was an All-Ivy first-team selection in 1984 when he was second in the league in rushing. Baker was an All-Ivy honorable mention pick in 1982, when he was fifth overall in the league in rushing. He led the 1984 team in rushing (165 carries for 847 yards) and scoring (eight touchdowns, 48 points) and was second in receptions (18 catches for 133 yards).