We've got a relay (RU server) up and running with hosts mostly on version 7.x and viewers also on version 7.x. From the server logs it seems that we have a large number of non-TLS connections from viewers. I can always send an email to everybody to update to the specially created 7.x viewer we have, but it's certain that a large number of users will not update.
For that purpose, is it possible to block old 6.x viewers automatically on the relay server?
Unfortunately, there's no feature that would allow automatically blocking outdated Viewers from connecting to the RU Server. However, you can try setting up the Centralized License Key Management feature on the RU Server's side that allows to manage license keys centrally and assign license keys to connected Viewers. For example, it can be set up so that users would have to use a license key for version 7 in order to connect to the RU Server (and it wouldn't be possible if their Viewer is outdated since license key for version 7 is not compatible with Viewer of version 6). For more information on how to set this up, please see this blog post.