The American Athletic Union of the Deaf was founded in 1945 at the first national deaf basketball tournament in Akron, Ohio, sponsored by the Akron Club for the Deaf. The AAUD was the brainchild of several deaf sports fans, particularly deaf sportswriter and booster Art Kruger. Its goal was to increase recognition of deaf sports nationwide and help arrange and promote deaf sporting events. It also provided support for American deaf athletes in international competitions. The AAUD originally had 58 local sports organizations affiliated with it, a number that quickly grew to over 150. In 1957, the group’s name was changed to the American Athletic Association of the Deaf (AAAD) to avoid confusion with the larger American Athletic Union (AAU).

In 1948, the AAAD affiliated with the Comite International des Sports des Sourds (CISS), the predecessor of the International Committee of Sports for the Deaf (ICSD). This led to the AAAD serving as host organization for World Games of the Deaf, later known as Deaflympics, in 1965, 1975, 1985, and 2007.

An AAAD Hall of Fame was established in 1953, with pro baseball’s William “Dummy” Hoy as its first member. In the 1970s and 1980s, the AAAD added sponsored tournaments in more sports, including golf, volleyball, and softball.

In 1997, the group’s name was changed again, this time to the USA Deaf Sports Federation.