"How can you even define overpaying when it comes to collectibles."
One clear example is when people rush to buy collectibles for investment purposes, like what happened with stamp collecting in the 1950s through 1980s, or Beanie Babies in the 1990s, and what is happening with Pokémon cards right now. Thank god for
ebay, et. al., which greatly reduced hoarding and price gouging, but unfortunately put an end to any hope of ever recovering financial losses incurred by investing in those things.
As for my response to the OP's question:
If one passes on on an uncommon stamp like a mint $5 Columbian today, there will be another one for sale tomorrow. And the next day. And the day after that. But if you get caught up in the heat of the moment during an auction and go over budget, you my regret it, saying to yourself "I overpaid for that item".
But examples like the document show by the OP fall into a different category than common/uncommon stamps. How do you put a price on something unique like that; something that no catalog could put a price on? I think regret would fall on the side of "I wish I hadn't let that item get away".
So which do I regret more? Both situations suck, but I lean towards regretting those items that got away more.
Brian