Your question opens a can of worms and you may find it easier to collect what you like and not get worried about this term.
It does not help that "stamp" is both a noun and a verb.
There tend to be two broad camps without much middle ground.
1. Those who adamantly believe it must be an adhesive to be a stamp.
2. Those who adamantly believe virtually anything which represents payment of postage or a revenue tax/fee can be a stamp (adhesives, embossed envelopes, meter stamps, revenue stamped paper, special tax stamps, etc., etc.)
I would note the USPOD referred to these as "stamped envelopes" as seen here on this July 1, 1875 order form:

Also consider the England's variious Stamp Acts of the 1700s paid by a revenue stamp printed onto the paper/document.
As you may guess, I fall strongly toward category 2, and this post will no doubt serve as a lightning rod for those who want to tell me that I am wrong, but I don't care.