OSM is an awesome project I agree it is fun contributing to it but it is not as useful yet due to what you describe, details.
Finding addresses and other specific information is very hit or miss in some areas. Especially in the US where no one really cares about it other than tech corporations using it for quick info like street layouts.
My entire city has address numbers missing and for a long time I used an app on my phone (StreetComplete) to fix address numbers and other various details. But it can be overwhelming very quickly depending on how outdated or not updated your place is. Time you enjoyed wasting is not wasted time by the way.