You haven’t actually removed all the components of mate - there is always some parts left over after installing a full desktop like mate and then trying to remove it.
It is always better here to format the server and reinstall from scratch.
Remember ubuntu server networking is different from a regular desktop - so you’ll need to switch from systemd based networking to network manager.
Correct - all the “ubuntu-xyz-desktop” meta packages will install a full desktop system.
In a production system you really don’t want this because it opens up the system to multiple areas of security potential vulnerabilities.
For my production ubuntu servers we don’t have a graphical interface - this greatly improves the security posture.
If you absolutely need a graphical interface then keep it as simple as possible - for example just openbox with a terminal. Don’t use a login manager - startx from a TTY. If you want a wayland minimal setup then stick with labwc or weston.