The Great 78 Project. Hosted at archive.org, the Great 78 Project has 50,000 78s (3+ minute records that spin at 78 rpms) that were recorded mostly from 1898 to the 1950s. A variety of digital audio formats (mp3, FLACs, OGG, and more) can be downloaded directly from the site.
The Smithsonian — This archive includes more than 3,700 digitized recordings, but you have to search for them specifically (use the phrase “sound recordings”) and look for phonograph-specific sources. Mp3s are available for download.
The Belfer Cylinders Digital Connection. Hosted by Syracuse University, this collection consists of over 1,600 digitized cylinders (a cylinder was an early recording format originally invented by Thomas Edison). Mp3s are available for each.
Edison Sound Recordings — Likewise, the National Park Service hosts over 100 early cylinder recordings from Thomas Edison’s archive in New Jersey.
National Jukebox — This Library of Congress archive was launched in 2011 and hosts over 10,000 digitized recordings most of which came from the Victor Talking Machine Company (that’s right — the same company that was founded in Camden, NJ). However, these records are stream-only and not available to download, so you might have to find the files elsewhere or record them yourself from your computer.