While I have had bases in the caves they were strictly for practical purposes, either right next to the pseudoscience station for crafting (and a quick teleport jump to fuelweaver which I always find on the same level) or as resting/recuperation spot elsewhere in the ruins. My approach to caves has always been about harvesting thulecite, gems, etc., it was just a road trip for goodies.
A few days ago I stumbled upon a cave base and helped out with outfitting it, in the process I learned quite a bit and also appreciated it much more. It is quiet in there, you only have depth worms to worry about and the supply of food is just endless (base next to huge mushroom tree biomes). I think my next solo project will be a cave base.