
If I had a traditional album with one example of each stamp, I would mount it right side up. (Actually, I'd probably pick a stamp with a subtler cancel in that instance.)
But as someone who doesn't mind multiples of a stamp one bit when looking for cancels, yes, I might leave it inverted. When I'm storing stamps in stock pages, inverting the stamp tells me that the watermark is inverted (or otherwise interesting). So until it made it onto a display page, it would stay right side up.