A common problem I’ve had when working at game studios is that game interpreted mouse movement over remote desktop is unreliable. Usually the effect is that any movement of the mouse causes the camera to fly off to an undesired location without any ability to get the camera back to where you would like it. Luckily I’ve been able to track down this problem (twice now) to make working over remote desktop bearable again. Sadly, I’ve always had problems in finding a resource online to solve the problem.
Both problem cases have come up when parsing WM_INPUT. Although I’ve only had two examples so far, I’ve noticed that both seemed to follow this posted example. For most cases, it seems that the example works as intended but for some reason the values appear as garbage over remote desktop. The problem is that it is assumed the WM_INPUT message is passing delta coordinate values while it is documented as passing either absolute or delta values. Of course, parsing these absolute coordinate values as delta results in unexpected behavior when passed to your application.
The solution is, of course, to check for the type of data being sent to you. That sounds easy enough but I’ve been unable to find documentation stating the coordinate system the absolute values are in. I’ve tried using ClientToScreen/ScreenToClient but neither seem to do what I expect. What I have noticed is that the coordinates seem to come through between 0->65535 (although stored in a LONG). In practice, I’ve used this observation to convert the coordinates after checking if the MOUSE_VIRTUAL_DESKTOP flag was set.
The following is an example code snippit.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
case WM_INPUT: { UINT buffer_size = 48; LPBYTE buffer[48]; GetRawInputData( (HRAWINPUT)lparam, RID_INPUT, buffer, &buffer_size, sizeof(RAWINPUTHEADER) ); RAWINPUT* raw = (RAWINPUT*)buffer; if (raw->header.dwType != RIM_TYPEMOUSE) { break; } const RAWMOUSE & mouse = raw->data.mouse; if ((mouse.usFlags & MOUSE_MOVE_ABSOLUTE) == MOUSE_MOVE_ABSOLUTE) { static Vector3 s_lastPoint = vector3(FLT_MAX, FLT_MAX, FLT_MAX); const bool virtualDesktop = (mouse.usFlags & MOUSE_VIRTUAL_DESKTOP) == MOUSE_VIRTUAL_DESKTOP; const int width = GetSystemMetrics( virtualDesktop ? SM_CXVIRTUALSCREEN : SM_CXSCREEN ); const int height = GetSystemMetrics( virtualDesktop ? SM_CYVIRTUALSCREEN : SM_CYSCREEN ); const Vector3 newPoint = vector3( (mouse.lLastX / 65535.0f) * width, (mouse.lLastY / 65535.0f) * height, 0 ); if (s_lastPoint != vector3(FLT_MAX, FLT_MAX, FLT_MAX)) { // NOTE: Converted absolute to delta codepath OnMouseMove(newPoint - s_lastPoint); } s_lastPoint = newPoint; } else { // NOTE: Normal, delta codepath OnMouseMove(vector3((float)mouse.lLastX, (float)mouse.lLastY, 0)); } } break; |
Hope that helps! At minimum, I hope the next time I google this problem I can discover my old findings. 🙂
YES this helps!
Glad it could help you, let me know if you’ve got any questions!
Thank you for this! I hadn’t seen absolute coordinates until I used my app over Remote Desktop.
Though having to handle both cases makes me think it’d be easier to go back to using WM_MOUSEMOVE instead 🙂
Thanks!