The sticky substance in those "magic" photo albums will eventually turn brown but it will migrate onto the stamps well before you notice them turning brown. The stamps will already be damaged. Buy some stock books (bound) or some stock pages that you can put into a binder. Use the search button at the top of this page for stock pages.
Yes, indeed, remove those stamps out of the photo albums!
Don't try peeling the stamps off the pages as you would a photograph. Take an album page, remove the clear overlay, then soak the page in a pan of water with a couple drops of hand dishwashing liquid added. The stamps will then eventually fall off of the page.
Soak the loose stamps in a container of clean water for a couple of hours. Then, dry and press the stamps.
Discard and replace the soak and rinse water after each page of stamps.