I have a number of covers like this that I found in my grandfather's collection, all sent from the same friend in Ottawa to my grandfather. As far as I have been able to find out, there are two possible explainations for these markings. The first is probably the most likely as it is related directly to the post office. When a postal worker added information to an address, they had to reference the location, so "directory 35" would be page 35 of the Toronto postal address directory. I could not find an example of a Toronto one, but here is a link to one for Madison City.
http://www.google.ca/url?sa=t&rct=j...FjAGOAo&url=http://images.library.wisc.edu/WI/EFacs/MadisonLocHist/MCD/MCD18712/reference/wi.mcd18712.i0010.pdf&ei=36aBT6bvM6K02gXy35H0Bg&usg=AFQjCNEQ6RmsvIzrfCw6cBaD9DHEhkK_sg
The second and less likely option is that the "directory" stamps (most of mine say "directory 12" or "station 7") were already on the envelope before shipping, and are inter-office mail directions for a large office building. They work sort of like an extension on a phone system, if you needed to send a letter to someone or to a department in the building it got stamped with their pesonal number from the postal person on your floor, and was delivered via the mailroom service to the correct person. I guess this is the what was done before email. I guess rather than throwing the envelope away, some people recycled them.