Agreed. Start with The Arduin Grimoire, vol. 1 (published in 1977).
Further volumes are more of the same sort of thing.
The first three volumes (also called the Arduin Trilogy and published in 1977-8) are easily the most packed with gonzo stuff. The subsequent volumes (published in 1984-1988) are sparser.
One can see the spiritual descent thus:
1974: original D&D rules 1975: Greyhawk and Blackmoor D&D supplements 1976: Eldritch Wizardry and Gods, Demigods & Heroes D&D supplements 1977: The Arduin Grimoire (1st book of the Arduin Trilogy) 1978: Welcome to Skull Tower and The Runes of Doom (2nd and 3rd books of the Arduin Trilogy) 1979: Encounter Critical
Hmm, seems like Dragon Tree Press no longer has Arduin Volumes I-III, and Emperor's Choice no longer sells them individually (they have Volumes I-III packaged together for $45). Volumes IV+ are available for about $11 from either place.
The last time I had checked a few months ago, I thought you could still get individual copies of Volumes I-III.
Sounds like if you have been thinking about getting Arduin, get it now while the getting is good.
You will notice that Emperors Choice has vols 4-9 of the Arduin Grimoire available in both "little brown book" style and in a larger, red book style. I own both. The LBB style is neater in an old-school way, while the red book style is easier to read.
I have all of the Arduin books. I would say, as others have, go with the first three to get a good feel for the system and conventions of Arduin. If you like 'em, the next three books (4-6) are chock full of details, beasties, societies, poisons, magic items, spells, etc. from the world of Arduin, and I have found a lot of good stuff in them to use in many different campaigns over the years.
Also, there was a product out in the early to mid 80's called The Arduin Adventure. It was pretty much the Arduin version of The Basic Set. It came with a nice rulesbook that contained a very stripped-down version of the rules, some monster and magic item cards, and dice (I don't recall if it came with an adventure or not). I don't know if it is still available anywhere, but I'm sure it must come up on Amazon or Ebay now and then.