I suspect that the real “extra cost” comes from having the slight amount of extra space it takes to stock the non-milk, ship it, handle it and the extra time it takes the employee at the counter to make a different drink.
Not saying they can’t just “eat the costs”, but companies never do that. Everything is accounted for and has the 10%+taxes profit margin slapped on top.
If the usage of non-milk would increase, I bet prices would come down in coffee shops as well.