To add to what has already been said, the reasoning goes back to the very definition of what constitutes a postage stamp. That is of course that it is available to be postally used.
The very first stamps issued in most countries/colonies were only available to be used postally. The collection of revenue and duties etc for other services or taxes was a separate issue and had it's own methods, whether it be stamp, label or handstamp etc.
Almost immediately people started collecting postage stamps. For whatever reason there was little interest in revenue stamp collecting, people were interested in mail and postage stamps, so that's what they collected. Most likely because that was the most readily available and perhaps I could use the term "visible." The mail arrived with the stamp/s attached, ready for collecting.

They were also relatively cheap compared to what most revenues demanded.
With all the different revenues that needed to be collected and the printing involved in having both revenue stamps and postage stamps, some countries/colonies changed their laws to make it that stamps could pay either revenue or postage, there would be no separation. In the colony of Victoria, this happened in 1884. As soon as the Act was passed, all the revenue stamps that were already on issue became available to be used for regular postage. Victoria also issued a new series of stamps with the words "Stamp Duty" inscribed, for use as either revenue or postage.
But people were already firmly collecting "postage" stamps. They weren't interested in a stamp that wasn't used postally. So right from the start, stamps that were fiscally used were not deemed to be "postage" stamps, even if it was the same as a stamp that could be used to mail a letter. This prejudice has just carried on through the years, the majority of collectors think of revenue stamps and postage stamps as two entirely different entities. Most people only collect postage stamps, and even if collecting revenue as well that will be a separate collection. They don't want a revenue stamp sitting amongst their postage stamp collection.

And a stamp with a revenue cancel, whether it was available to be used postally or not, is a revenue stamp and not a postage stamp. So this is why you see the revenue cancels frowned upon.
Most (if not all, I'm not sure) countries later reverted back to separate stamps for postal use.
The common view is if a stamp was available to be used as revenue, the only way to include it in a postage stamp collection is mint or postally used. The price of a stamp is dependant on demand, not rarity. A stamp can be rare but still be cheap if no-one wants it. It's only buyers that drive prices up.
Now I've rambled on for long enough.

Balf