Metal and glass are recycleable. And if they do get into the environment, they are really just purified rocks and will gradually become sand in the weather. (Not that it’s great to have soda cans and broken glass in the sea, but to some extent it’s not as bad as microplastics).
Paper is recycleable.
Paper, wood, and other plant products (e.g. cotton) are biodegradable and come from plants that can be farmed.
In terms of sustainability it’s something like:
Plant products
Metal and glass
Plastics
But also even more important than that, it’s far better to reuse something many times than to use single use products, regardless of the material they are made of.