Host and guest are Windows 7 x64. I've had this problem on multiple PCs now.
If I connect and stay connected, after some amount of time, left clicking no longer seems to work. i.e. In Outlook, I can click the New button, to start a new email. I can see the button highlight and appear to be clicked, but nothing happens. And if I press enter, it minimizes Outlook altogether.
So I've learned that when this happens to just disconnect, restart the viewer and reconnect. Now I can click New and start a new email. Then, as I'm writing the email and just typing normally, Outlook starts popping up things like the Research pane, as if I'm sending a ctrl+something, and just now I had it send an email I was still in the middle of writing (ctrl-enter.) So it seems as if the ctrl key is stuck down, but I've tried toggling the ctrl and alt keys; it doesn't seem to make a difference.
It's easily reproducible; I just have to stay connected for 30 minutes or so and it starts happening. If there's any kind of debugging or log file, or video you want to help the case, I'll be happy to supply whatever.