What constitutes a "large amount of ammo?" If you own a gun, then you should possess ammunition. What one person considers to be "not enough" could easily be considered a "large amount" by someone else. I want numbers here—don't be vague, news voice. I demand to be impressed!
The first comparison that comes to mind is a car: if you own a car and want to drive it, you need to have gas. "Well, yes, Tom—but you don't need to fill the tank and then buy a 200 gallon tank to keep filled in your garage, do you?" No, you don't need to do that. There are gas stations everywhere, and many of them are open for extended hours. So, in the case of your car, you can get gas relatively easily and conveniently, no matter where you are.
That's not the case with ammunition, regardless of whether or not you shoot an odd or rare caliber. One can't run off to the ammo store at all hours, and any town that has as many ammunition stores as it has gas stations probably only has one gas station. And whereas a gas station generally only needs to carry one or two products—gas and diesel—an ammunition store needs to carry .22 ammunition, multiple shotgun loads, and everything in between. So it makes sense to get not just enough for today and tomorrow when you're buying ammunition, but to get enough for whatever you may need in the future. How far into the future is up to you!
