Let me chime in here and add my two-cents worth.
The primary basis for any philatelic collection is stamps, whether they are mint, used or on cover (or document for revenues).
So what is ephemera and how does it relate to stamp collecting?
Anything which helps tell the story of the stamp, but is not a stamp itself.
One of the items I try to get for modern stamps are the deck cards, those card covers which are placed on top of a specific number of panes and then sealed in plastic. Here's a couple of examples:


These would be ephemera as they were never intended as collectibles, but are a part of the production story of the stamps.
How about the old post office bulletins from the 1960's and 1970's? Here's a couple of examples of those:


Once upon a time and not so long ago, the USPOD and USPS would send pictures of upcoming stamps to newspapers and the philatelic press so they could tell the public about it. These are ephemera and here's some examples:

The above image DID NOT make it out as a stamp. It was decided the little girl was a wee too evil looking, so she was changed. Here's what the final image for the stamp looks like (and no, that first publicity photo is not considered an essay):

As technology improved, other options became available. There are color transparencies and color publicity photos too. Here are all three types for Scott 2968, the 1995 Texas Statehood Sesquicentennial stamp.



Hopefully, this gives you some visual ideas about what ephemera can mean to a collection.