We analyzed the logs. It looks like the network connection with the server is very unstable for some reason. Just as an experiment and to exclude the port conflict reason, try to use another port, something less used or even default (5655). Don't forget to change it on either side of the connection (Viewer, Server and Host).
We can still connect to them using RDP with VPN without Remote utilities when remote utilities doesn't work.
Sorry, it doesn't make sense to compare direct connection (RDP through VPN) with indirect/mediated connection (RU via a relay server). With Remote Utilities you can connect directly as well - see this tutorial.
That said, although I didn't see the Host logs - only them can give some definitive information about the issue - I can guess that you might have a port conflict. You use port 443 as communication port for the server. This is a widely used port and perhaps, some other service or application also uses it.
The only reason behind using port 443 is that this port is often open by default in many organizations. But that doesn't necessarily make this port "more secure" than other ports. Ports are just numbers, you can use any port because security has nothing to do with the port number. Rather, it's implemented in the application itself and transport protocols.
after 5-10 minutes the viewer will freeze and I will get a connection issue but it always reconnects right away when i re-initiate it.
Is the Viewer auto-reconnecting or just freezing (i.e. erroneous work of the application)?
As a test, I also have also tried a TeamViewer remote session
Such tests can do little in diagnosing connectivity issues. Even if you have a network-related issue, and not a software-related one, it well may be that this network issue manifests itself only with this specific program, rather than with other programs. For example, there are plenty of user reports about when TeamViewer can't connect and yet it doesn't necessarily mean that the problem is with TeamViewer.
It could help if you sent us Host logs from a couple of your Hosts. You can send them to firstname.lastname@example.org.
and a standard RDP
Standard RDP cannot connect over the Internet and bypass firewalls. To compare apples to apples you need to compare Remote Utilities direct connection (i.e. connecting using IP address or DNS name) vs. RDP's connection.
If you use Remote Utilities Internet ID connection then comparing it vs. RDP doesn't make sense - direct connectivity always wins over indirect one. That said, if your remote Hosts are accessible directly, you don't even have to use Internet-ID connection - you set up direct connection.
The hosts and Relay server are all located internally.
It looks like your Hosts cannot connect to your server. In order to verify that, you can check the Host logs located in C:\Program Files\Remote Utilities - Host\Logs\.
I assume that the reason why your Hosts cannot connect is perhaps that you use the external IP address of your server in the Host's Internet-ID connection settings. Try using the internal/local server address so long as the server is accessible to your Hosts via LAN.