Can anyone with my Internet-ID connect to my computer?
No, they cannot. The knowledge of the Internet-ID code alone is not enough to be able to connect to a remote Host. One also needs to know the Host's access password provided Remote Utilities Security is used as the authorization method.
If Windows Security is used as the authorization method, one needs to know the credentials of the Windows user who is permitted to access the Host.
A blank password will not work. It is possible to install the Host without creating the access password during installation. However, no one will be able to connect to that Host (including you) until you create the password or enable another authorization method (e.g. Windows Security) and set access permissions.
All said above about the Host also applies to the Agent.