Is there a list of rare (or scarce) postmarks which resulted from post offices that were only open for a short period time? I know there is a searchable list of post offices on one particular site with opening and closing dates, but it would be time consuming to look up each cover that I suspected was possibly in this category. If anyone knows of a publication or reference (alphabetical would be great) that could help, it would be greatly appreciated.
There are books which speak to the rarity of particular cancels from towns. These book tend to be related to only one state or for the small states, a region. each tends to use its own rarity scale based upon numbers known. But if you ignore the 1, 2, 3, a, b, c and go by the raw number known you can get a comparison.
A good start is knowing how short lived a PO was to begin value determination.
All that said, you have not specified the era or region of your covers. A PO that lasted five years, from 1962-1967 will be in a different league than one which had the five years 1847-1852. Then there are the secret addresses and thus cancels that have a good value due to the "nature of the address" such as WWII nuclear development sites. Other times the common cancels become quite valuable for a particular date, think 12-7-1941 and Hawaii.
Town cancels are a wide ranging area. Rarity does not change much (there is always the possibility of new finds coming to the market). However if your use of rarity is really code for value, town cancel value is very demand driven varying greatly as a result of if the area is or is not currently being actively collected.
You did not say which source you are using but the one by Jim Forte is considered the best offered and offered free. It has been well posted on the SCF.
The simpliest answer to your question is you need specific information for the post office using the postmark. Identify the type of postmark you are looking at. Then go to a reference book or web site for that type of postmark.
Can you post an example of the device(s) you wish to evaluate?
The series published by Helbock (8 volumes) attempts to give a rarity factor to all of the Post Offices in his books. The important thing to consider is the desirability of a particular state or county. I.E. are there active collectors? 19th century California town cancels were hot at one time, now less so and the same holds true to other states. A particular county can be hot. The two counties on Long Island, Nassau and Suffolk, had their moment in the sun when there was an active Long Island Postal History Society. A rating of 4 (scale of 0 to 10) from an active state might sell for more than an 8 from an inactive state.