Usually the clerk number. When the numbers get into the double digits, it usually signifies a larger post office with many clerks. The smaller post offices usually had numbers 1 and 2, etc. The larger post offices had numbers up to the 90s (maybe the postmaster or assistant postmaster used those high numbers). Actually, I started a mini-collection of what numbers I came across in early US postmarks:

The highest number I have come across in my collection:
