At the very beginning in early 90’s Linux adopted X11 implementation that was XFree86. It was obvious and pragmatic move, because Linux was UNIX clone with full POSIX standard compatibility, and X11 was already there for almost a decade. Porting it allowed for having graphical interface very early on (Linux started in 1991, X11 support was added one year later) and allowed all the contemporary UNIX software to be easily ported to Linux.
X11 however was designed with completely different needs in mind, as UNIX machines were mostly mainframes or powerful workstations and not home computers. It was about a lot of features that make no sense in this day and age (like network transparency, drawing primitives, printing capabilities, font rendering etc) and its design aged like milk. Xorg (that was fork of XFree86 started after license change) was implemented in a way that allows keeping compatibility for the time being with many issues being worked around and the old solution being effectively forcefully framed into modern use. It’s basically huge
Wayland started as an idea on how to do graphics on Linux (and other UNIX systems) without X, but it was never meant to be drop-in replacement. That being said, it’s vastly incompatible and the shape towards having Wayland desktops is long process of gradual implementation of new protocols to make it complete eventually.
Making Wayland possible took redesign of the OS itself. In old days, Linux didn’t think much about graphics and it was the monolithic X server that took responsibility of things like loading video drivers, setting screen modes or pushing stuff to video memory. Wayland was all about split of X’s features outside of X to gradually remove the dependency, so now the kernel has native system interfaces like kernel mode setting, direct rendering manager and so on. It’s not only Wayland taking advantage of it, as the same infrastructure is now used under X too.
Your experience wasn’t much different because it wasn’t meant to be. Desktops that are ported to Wayland are very good at abstracting things that are specific to both (otherwise completely different) display systems. You can gradually find about some things being different over time as you dive deeper.
There are certain limitations of X that Wayland doesn’t have:
X cannot handle multiple DPI settings, so it is only possible to set one scaling factor globally for all monitors no matter their size/pixel density
X could never properly handle multiple refresh rates for different monitors
No way for proper HDR support on X
VR is not really a good idea on X
On the other hand, X is very open to the user and applications, providing all sorts of information about opened windows and sniffing input globally by any client (focused or not) is a feature. In 1984 no one really thought cybersecurity will be important factor. So on Wayland:
App can’t keylog keyboard presses or mouse movements unless its window is focused (global shortcuts are still unsolved issue, WIP)
App can’t directly control its window position and size as it is only controlled by compositor (the idea is to introduce protocol for asking compositor on window positions relative to some area, it’s WIP)
App cannot get image of screen or window (this is solved via PipeWire video capabilities and xdg-desktop-portal)
Any GUI automation is compositor-specific, at least for now.
For those and other reasons (like availability of desktop environments and window managers), some still prefer Xserver.