We have exactly the same problem. We are using this product to support our customers, and have recently taken to deploying Windows 8 tablets to them. Remote Utilities gives a completely useless, sideways-on, screen display when you try to connect to them. If the tablet is used in the default (0 degrees rotation) mode then it works fine, but we have to lock these to either 90 or 270 degrees to support the application running on the tablet.
TeamViewer, on the other hand, works perfectly on these tabletss and sdjusts to whicheve angle of rotation the tablet is set to use, displaying it the right way up within the viewer.
This is a major issue as the only reason we purchased Remote Utilities was to enable us to support our customers - we are now looking at having to go and buy TeamViewer instead to enable us to do what we need.
I discovered this post through an Internet search. I ran into the same issue on Windows 10 with the display rotated 270 degrees. The problem clears up when switching to legacy capture mode (or with the screen rotated 0 degrees). So, it would appear that the issue isn't fixed with Windows 10.
You had stated in an earlier post that "Windows 10 looks more promising", so I was basically responding to that.
However, what would be preferable is if Remote Utilities could somehow detect that screen capture won't work properly and implicitly switch to legacy capture mode. When I first encountered this issue, I thought that it was likely that Remote Utilities had a bug. If I hadn't been able to find this post, this would likely have been a showstopper for me.
Interestingly, this same issue, or one that is very similar, also exists for certain VNC servers. Both TightVNC and UltraVNC are affected. RealVNC works fine.
I will note down the suggestion, of course. Perhaps, there is indeed a way to automatically detect and fix the issue, although it wouldn't be a good practice to change any settings without the user knowing. We'll see.
I wasn't suggesting that Remote Utilities change a setting behind the user's back. Instead, it could leave the setting as-is but internally use the legacy approach if it can detect that this is necessary. If that is done, maybe the legacy capture setting can be removed (assuming it isn't useful for other reasons). I would tend to think that automatically working around issues rather than requiring user intervention is preferable.