No, not a 104. See this photo from another recent thread that shows this stamp (#65) comes in many shades.
https://goscf.com/t/85543#792499See also Siegel Power Search for example for an idea of what the "brown red" shade of 104 looks like.

Also note that not all stamps in this color are going to be #104. Most are just a darker shade of #65, or possibly oxidized.
Yes, the paper on your taller stamp is a little brighter and whiter than the other one, but those differences are also natural with age and the environmental conditions how the stamps were stored over the last 150+ years. Zero chance (in my opinion) that either of your two stamps is a 104.