At least as far as Google is concerned, the sitemap file will be helping… but not as much as actual links in pages would. It's covered in this Q&A video with Matt Cutts. He's talking about public facing site maps, which I imagine isn't an option for 5 million pages, but the principle is the same: crawlable links on web pages.
I'd definitely consider some sort of "normal" navigation, even if limited. Breadcrumbs would be particularly helpful, especially if you can get them through all pages.