If you just tell Country B that you do so, like in an oath or something? No, no effects on the old country and they may even still want taxes from you.
If you contact Country A’s government (usually the state department or foreign ministry) and go through whatever process they have and pay a fee then yes. You will no longer have the responsibilities (taxes, military service, etc) or the rights, (voting, travel, education, etc) in regards to the old country.
Should you actually go through that process? Consult a lawyer in your new country to see if it’s necessary. Sometimes it’s better to keep the old passport, like if you’re a western expat. Sometimes you’re trying to get out from under international tax obligations or required military service. Regardless, the new country generally doesn’t care what the old country thinks as long as you follow the laws of your new country. Sometimes it’s a verbal promise to renounce loyalty and sometimes they want that receipt from the old country saying it’s done.