Draggable objects, 2022

Many of my interactive pages have a draggable object. I want the reader to move the object around, and I want the diagram to respond in some way. Here I’ll document the code I use to make this work with both mouse and touch input, using browser features that are widely supported since 2020. Here’s what I want to support:

I’ve run tests on Gecko/Firefox (Mac, Windows, Linux, Android), Blink/Chrome (Mac, Windows, Linux, Android), and WebKit/Safari (Mac, iPhone, iPad). I have not tested on hoverable stylus, hybrid touch+mouse devices, or voice input.

This is the simple model in my head:

However it’s not so simple! Mouses have multiple buttons. Touch events can include multiple fingers. Events can go to multiple destinations.

{{ TODO :: just show the answer!! the rest of the page is explanation and demos of the things that go wrong }}

 1  Mouse events only#

The simplest thing to do is to capture mousedown, mouseup, and mousemove on the circle element. If the move occurs while dragging, then move the circle to the mouse position.

Try the demo with a mouse:

This might seem like it works but it works badly.

• If you move the pointer quickly it is no longer over the circle, it stops receiving events.
• If you release the button while not on the circle, it will get stuck in the “dragging” state.

To fix these problems we can use mousedown on the circle to add mousemove and mouseup on the document. Then on mouseup we remove the mousemove and mouseup from the document.

Try it out. It works better.

There are still a few issues, which we’ll solve one by one.

• This code doesn’t handle touch events.
• The text can get selected while trying to drag the circle.
• If you select all the text on the page, the drag can apply to the text instead of the circle.
• Right clicking the circle will bring up a context menu, which leaves the circle stuck in drag mode.

 2  Touch events#

Touch events automatically capture on touchstart and direct all touchmove events to the original element. This means we don’t have to temporarily put an event handler on document. We can go back to the logic in the first mouse example. If for any reason the browser needs to cancel the touch sequence, it sends touchcancel.

Try the demo with a touch device:

There are a few problems to deal with:

• When dragging the circle around, the page will scroll. That’s because both our drag handler and the browser’s scroll handler are active.
• Long pressing on the text will select it or bring up a context menu. This leaves us stuck in the dragging state.

 3  Pointer events#

To handle both mouse and touch events we end up having lots of different event handlers:

Pointer events attempt to unify mouse and touch events. The pointer capture^[7] feature lets us use the simpler logic that doesn’t require us to add/remove global event handlers to the document like we had to with mouse events.

Much simpler! Try the demo with either a mouse or touch device:

There are several things we still need to fix.

 3.1. Fix: text selection

When dragging the circle, the text inside gets selected sometimes.

Fix: CSS user-select: none on the circle.

If I want to select the text when not dragging the circle (left drag, right click, long press, or keyboard), I can apply that CSS only while dragging.

This problem won’t happen in Canvas and WebGL because that text is unselectable.

TODO: demo with three objects, one that always has the css, one that never has it, and one that only has it while dragging

 3.2. Fix: scrolling with touch

On touch devices, single-finger drag will scroll the page. But single-finger drag also drags the circle. By default, it will do both.

Fix: CSS touch-action: none on the diagram.

But this prevents scrolling anywhere in the diagram. So a better fix is to apply it only to the circle. We can do this for SVG diagrams because each element has its own CSS separate from the containing diagram. But we can’t do the same for Canvas or WebGL diagams, because elements don’t have their own CSS. So another possible fix is to preventDefault() on touchstart. In Canvas and WebGL, check that the pointer is on the circle and conditionally prevent default.

TODO: demo with four separate diagrams side by side

1. default
2. touch-action none on diagram
3. touch-action none on circle
4. touchstart.preventdefault

 3.3. Fix: text drag

Try selecting all text on the page, then drag the circle. This will trigger text dragging on many browser+OS combination, where you can drag some text as an alternative to copy/paste. I want to allow this, except when dragging the circle.

Fix is: preventDefault() on dragstart on the circle. This works in SVG but not in Canvas or WebGL. For those, check that the pointer is on the circle and conditionally prevent default.

TODO: demo

 3.4. Fix: context menu

Context menus are different across platforms, and that makes handling it tricky.

Windows

right click (down+up), Shift + F10 key

Linux

right click, right button down, Shift + F10 key

Mac

right button down, Ctrl + left click

iPhone, iPad

long press on text only

Android

long press on anything

There are more ways to bring up context menus (examples: two finger tap on Mac or Windows; right side click on Windows; press-and-hold on Wacom pens) but I haven’t tested all of them.

Windows, right click, no capture:

Firefox, Chrome, Edge

pointerdown, pointerup, auxclick, contextmenu

Windows, right click, capture:

Firefox

pointerdown, get capture, pointerup, lose capture, auxclick, contextmenu

Chrome, Edge

pointerdown, get capture, pointerup, auxclick, lose capture, contextmenu

Linux right click, no capture:

Firefox

pointerdown, contextmenu, pointermove² while menu is up

Chrome

pointerdown, contextmenu, no pointermove² while menu is up

Linux hold right down, no capture:

Firefox

pointerdown, contextmenu, pointermove² while menu is up

Chrome

pointerdown, contextmenu, no pointermove² while menu is up

Linux right click, capture:

Firefox

pointerdown, contextmenu, get capture, pointermove² while menu is up tells us button released

Chrome

pointerdown, contextmenu, get capture; not until another click do we get pointerup, lose capture

Linux hold right down, capture:

Firefox

pointerdown, contextmenu, got capture, pointermove² while menu is up tells us button released; when releasing button, menu stays up but we get pointerup, lose capture

Chrome

pointerdown, contextmenu, got capture, no pointermove² while menu is up; when releasing button, menu stays up but we don’t get pointerup; not until another click do we get pointerup, click, lose capture

Mac, ctrl + left click:

Firefox

pointermove with buttons≠0, contextmenu (no pointerdown or pointerup)

Chrome

pointerdown with button=left, contextmenu (no pointerup)

Safari

pointerdown with button=left, contextmenu (no pointerup); but subsequent clicks only fire contextmenu

Mac, right button down:

Firefox

pointerdown with button=right, contextmenu (no pointerup)

Chrome

pointerdown with button=right, contextmenu (no pointerup)

Safari

pointerdown with button=right, contextmenu (no pointerup); but subsequent right clicks only fire contextmenu

If we capture events on pointerdown, Firefox and Safari will keep the capture even after the button is released. Chrome will keep capture until you move the mouse, and then it will release capture. [This seems like a Firefox/Safari bug to me, as pointer capture is supposed to be automatically released on mouse up]

It’s frustrating that on Mac, there’s no pointerup or pointercapture when releasing the mouse button. On Linux, the pointerup only shows up if you click to exit the context menu. It doesn’t show up if you press Esc to exit. The workaround is to watch pointermove events to see when no buttons are set. Windows doesn’t seem to have these issues, as both pointerdown and pointerup are delivered before the context menu.

Android, long press:

Firefox

pointerdown, get capture, contextmenu, pointerup, lose capture

Chrome

pointerdown, get capture, contextmenu, pointerup or pointercancel¹, lose capture

¹if the finger moves at all, this starts a scroll event which cancels the captured pointer ²Firefox lets the page see events outside the menu overlay, whereas Chrome doesn’t let the page see any events while the menu is up

--–— tests ---–—

The problem is that if I’m using pointerdown and then pointerup to track the state of the mouse, and I never get pointerup, my code thinks I’m still holding the button down (and the browser’s pointer capture is still active).

Try right clicking the circle. It will bring up a context menu. That itself is fine. Unfortunately the events aren’t consistent across platforms:

What should we do?

• https://www.w3.org/TR/pointerevents/#the-pointerdown-event^[8] says preventDefault() on pointerdown does not stop click or contextmenu events. I can preventDefault() on contextmenu to prevent the menu. But I still want to get pointerup and/or pointercancel! I think I have to treat contextmenu as the up event which means I’ll get multiple up events on Windows.
• https://w3c.github.io/pointerevents/#the-button-property^[9] says button = 0. indicates the primary button. This would let me exclude middle button and right button. But I still get a pointerdown.left on Mac/Chrome and Mac/Safari (not on Mac/Firefox) so this may not be enough.
• on pointermove, if we have pointer capture, we could check the state of the buttons?? TODO: test this! it’s mentioned as a workaround on https://github.com/w3c/pointerevents/issues/408^[10]

TODO: demo of both solutions?

 3.5. Feature: handle drag offset

This is not implementation specific, but a design issue. If you pick up the edge of the circle then you want to keep holding it at that point, not from the center. The solution is to remember where the center is relative to where you started the drag. Then when you move the object, you add that offset back in.

TODO: demo both ways (although it duplicates little-details page) ; could draw something showing the pick up point

 3.6. Feature: handle multitouch

isPrimary vs pointer id; need to test what happens if there are two independent drags going on

 3.7. Feature: handle simultaneous dragging

 3.8. TODO: what about multiple buttons?

So here’s a tricky one. If you are using multiple buttons at the same time, what happens? The Pointer Events spec says that the first button that was pressed leads to a pointerdown event, and the last one that was released leads to a pointerup event. But that means we might get a up event on a different button than the down event.

This doesn’t seem to be what I want, but it’s what Pointer Events do^[11]. The workaround is to check the button state in pointermove. But pointer capture continues until you release all the buttons, unless you explicitly release capture.

Mouse Events behave the way I want but don’t handle touch events.

 3.9. TODO: what about multiple mice?

• mousedown (mouse 1)
• mousedown (mouse 2)
• mouseup (mouse 2)
• I stop dragging but the mouse that started the drag is still dragging

Does PointerEvent.pointerId help here?

What happens when ipad is used to control mouse, or mouse is used to control ipad?

TODO: test middle clicking to drag, like some mice support on Windows and maybe Linux

TODO: test left click drag, then right button down, then left/right up. Since I only have one dragging state it might get confused.

https://www.w3.org/TR/pointerevents/#the-primary-pointer^[12] says

Current operating systems and user agents don’t usually have a concept of multiple mouse inputs. When more than one mouse device is present (for instance, on a laptop with both a trackpad and an external mouse), all mouse devices are generally treated as a single device - movements on any of the devices are translated to movement of a single mouse pointer, and there is no distinction between button presses on different mouse devices. For this reason, there will usually only be a single mouse pointer, and that pointer will be primary.

 3.10. TODO: test nested dragging

 4  Vue version#

I think it’ll look something like this

<template>
  <g
    :transform="`translate(${pos.x},${pos.y})`"
    @pointerdown.left="start" @pointerup="end" 
    @pointermove="dragging ? move($event) : null"
    @pointercancel="end" @lostpointercapture="end"
    @touchstart.prevent="" @dragstart.prevent="">
    v-bind:class="{dragging}"
<slot />
  </g>
</template>

<style>
  g { cursor: grab; }
  g.dragging { user-select: none; cursor: grabbing; }
</style>

<script setup>
// pos is a prop {x: y:}

const dragging = ref(false);

function start(event) {
  if (event.ctrlKey) return;
  let {x, y} = convertPixelToSvgCoord(event);
  dragging.value = {dx: pos.x - x, dy: pos.y - y,
                    pointerId: event.pointerId};
  el.setPointerCapture(event.pointerId);
}

function end(event) {
  dragging.value = null;
}

function move(event) {
  if (!(event.buttons & 1)) return end(event);
  if (event.pointerId !== dragging.value.pointerId) return;
  let {x, y} = convertPixelToSvgCoord(event);
  $emit('move', {
    x: x + dragging.value.dx,
    y: y + dragging.value.dy,
  });
}
</script>

 5  Notes - event log#

eventlog.html

Testing a click:

• Desktop:
  • Firefox/Mac, Chrome/Mac, Safari/Mac, Firefox/Windows, Chrome/Windows, Edge/Windows, Firefox/Linux all produce pointerdown, mousedown, pointerup, mouseup, click
  • Firefox/Mac: if loading a page and the mouse is already over an element, will fire mouseover,mouseenter but not pointerover,pointerenter until the mouse is moved a tiny bit {need to test on Firefox/Windows, Firefox/Linux but probably does the same there}
  • Mac: if you mouse down over the circle and then alt+tab to another window and then release the mouse, the web page still gets pointerup, mouseup, pointerout, pointerleave, mouseout, mouseleave (!). It also gets those if you put the computer to sleep. On Windows, it will go out as soon as you press alt+tab, and not come back when you switch to the same app, whereas on Mac it triggers pointerover etc when you come back to the app (further testing needed)
  • Firefox vs Chrome (both Mac and Windows): if your mouse goes under the element when you scroll the page with the keyboard, Firefox will fire mouseover, mouseenter whereas Chrome will fire pointerover, pointerenter, mouseover, mouseenter. I feel like Chrome is doing the right thing here. [TODO: file a bug^[13]]
• Mobile:
  • Safari/iOS, Firefox/Android, Chrome/Android all produce pointerdown, touchstart, pointerup, touchend, but if quick: also produce [mousedown, mouseup, click]
  • Android: contextmenu event if holding down; need to preventDefault to prevent the menu from showing up
  • Android: if there’s text in the draggable event, need to use user-select:none to prevent text from being selected. If it’s in the middle of text, might be best to apply apply it only during a drag event; otherwise it would prevent text selection when not dragging.

 6  Notes on dragging#

tests.html

• Need touchstart.prevent to prevent scrolling
• Need either pointerdown.prevent or user-select:none to prevent double click from selecting text

TODO: test tablet

 7  Variations#

 8  Notes#

pointerdown + pointerup will trigger click (left mouse button) or auxclick (middle mouse button) or contextmenu (right mouse button)