I can see where you’re coming from - if you look at Ultimate Doom or Doom II and put Doom 64 next to it, then there is a raft of limitations and shortcomings that make it appear to be an inferior product. I personally prefer Doom II as well, with the banging soundtrack and the iconic levels.
However, I’d argue you’re missing out by approaching it with that mindset. Aubrey Hodges score is less of a soundtrack, and more of an ominous hum in the background - and the overall art style is far more drab and depressing. I’d probably suggest it has more in common with Doom 3 than it does with the first two. I personally prefer the sound effect collection in Doom 64 - the shotgun sounds meatier, the Barons from Hell’s alert sound is a bit more worrisome, and the doors sound a bit more industrial.
Personally I agree with the N64 controller assessment - it feels like it was built by someone who forgot how many hands humans have 😂 but the adaptations in the Unity ports make it more than enjoyable now on Xbox or Dualshock controllers.