I wouldn't discount Harris' innate advantages there too; he was 10 years older than Gambon, aged more poorly (having been an alcoholic hellraiser in his younger years), and his natural delivery - even when he was much younger - had that sort-of wizened wheezing sound to it; "old and physically frail but with incredible magic power" was sort of baked in even before he added any actual acting to the mix.
But I don't know if there's an alternative who would have been better in that regard; the three I'm aware of them talking about were Christopher Lee, Ian McKellen, and Peter O'Toole, but the latter two would have played him very much like Gambon did, and I'm not sure if Lee could have pulled off "frail" either given his voice + physical stature.