Many of my interactive pages have a draggable object. I want the reader to move the object around, and I want the diagram to respond in some way. Here I’ll document the code I use to make this work with both mouse and touch input, using browser features that are widely supported since 2020. Here’s what I want to support:
History →
From 2011 to 2014 I used d3-drag[1], but for my non-d3 projects, I ended up developing my own mouse+touch code, which I wrote about in 2018.
By 2012 MS IE had added support for pointer events[2] which unify and simplify mouse+touch handling. Chrome added support in 2017; Firefox in 2018; Safari in 2020[3].
Over the years browsers have changed the rules, including in 2017 when Chrome changed some events to default to passive mode[4] which causes the page to scroll while you’re trying to drag the object. This broke some pages[5]. Safari made this change in 2018[6]. Firefox maintained backwards compatibility.
I’ve run tests on Gecko/Firefox (Mac, Windows, Linux, Android), Blink/Chrome (Mac, Windows, Linux, Android), and WebKit/Safari (Mac, iPhone, iPad). I have not tested on hoverable stylus, hybrid touch+mouse devices, or voice input.
This is the simple model in my head:
However it’s not so simple! Mouses have multiple buttons. Touch events can include multiple fingers. Events can go to multiple destinations.
{{ TODO :: just show the answer!! the rest of the page is explanation and demos of the things that go wrong }}
1 Mouse events only#
The simplest thing to do is to capture mousedown
, mouseup
, and mousemove
on the circle element. If the move occurs while dragging, then move the circle to the mouse position.
Try the demo with a mouse:
This might seem like it works but it works badly.
- If you move the pointer quickly it is no longer over the circle, it stops receiving events.
- If you release the button while not on the circle, it will get stuck in the “dragging” state.
To fix these problems we can use mousedown
on the circle to add mousemove
and mouseup
on the document. Then on mouseup
we remove the mousemove
and mouseup
from the document.
Try it out. It works better.
There are still a few issues, which we’ll solve one by one.
- This code doesn’t handle touch events.
- The text can get selected while trying to drag the circle.
- If you select all the text on the page, the drag can apply to the text instead of the circle.
- Right clicking the circle will bring up a context menu, which leaves the circle stuck in drag mode.
2 Touch events#
Touch events automatically capture on touchstart
and direct all touchmove
events to the original element. This means we don’t have to temporarily put an event handler on document
. We can go back to the logic in the first mouse example. If for any reason the browser needs to cancel the touch sequence, it sends touchcancel
.
Try the demo with a touch device:
There are a few problems to deal with:
- When dragging the circle around, the page will scroll. That’s because both our drag handler and the browser’s scroll handler are active.
- Long pressing on the text will select it or bring up a context menu. This leaves us stuck in the dragging state.
3 Pointer events#
To handle both mouse and touch events we end up having lots of different event handlers:
Pointer events attempt to unify mouse and touch events. The pointer capture[7] feature lets us use the simpler logic that doesn’t require us to add/remove global event handlers to the document like we had to with mouse events.
Much simpler! Try the demo with either a mouse or touch device:
There are several things we still need to fix.
3.1. Fix: text selection
When dragging the circle, the text inside gets selected sometimes.
Fix: CSS user-select: none on the circle.
If I want to select the text when not dragging the circle (left drag, right click, long press, or keyboard), I can apply that CSS only while dragging.
This problem won’t happen in Canvas and WebGL because that text is unselectable.
TODO: demo with three objects, one that always has the css, one that never has it, and one that only has it while dragging
Try this | Watch for | Circle 1 | Circle 2 | Circle 3 |
---|---|---|---|---|
drag circle | text is selected | yes ⛌ | no ✓ | no ✓ |
drag text | text is selected | yes ✓ | no ⛌ | yes ✓ |
3.2. Fix: scrolling with touch
On touch devices, single-finger drag will scroll the page. But single-finger drag also drags the circle. By default, it will do both.
Fix: CSS touch-action: none on the diagram.
But this prevents scrolling anywhere in the diagram. So a better fix is to apply it only to the circle. We can do this for SVG diagrams because each element has its own CSS separate from the containing diagram. But we can’t do the same for Canvas or WebGL diagams, because elements don’t have their own CSS. So another possible fix is to preventDefault()
on touchstart
. In Canvas and WebGL, check that the pointer is on the circle and conditionally prevent default.
TODO: demo with four separate diagrams side by side
- default
- touch-action none on diagram
- touch-action none on circle
- touchstart.preventdefault
Try this | Watch for | Circle 1 | Circle 2 | Circle 3 | Circle 4 |
---|---|---|---|---|---|
drag circle | page scrolls | yes ⛌ | no ✓ | no ✓ | no ✓ |
drag diagram | page scrolls | yes ✓ | no ⛌ | yes ✓ | yes ✓ |
3.3. Fix: text drag
Try selecting all text on the page, then drag the circle. This will trigger text dragging on many browser+OS combination, where you can drag some text as an alternative to copy/paste. I want to allow this, except when dragging the circle.
Fix is: preventDefault()
on dragstart
on the circle. This works in SVG but not in Canvas or WebGL. For those, check that the pointer is on the circle and conditionally prevent default.
TODO: demo
Try this | Watch for | Circle 1 | Circle 2 |
---|---|---|---|
drag circle | page text drags | yes ⛌ | no ✓ |
3.4. Fix: context menu
Context menus are different across platforms, and that makes handling it tricky.
- Windows
- right click (down+up), Shift + F10 key
- Linux
- right click, right button down, Shift + F10 key
- Mac
- right button down, Ctrl + left click
- iPhone, iPad
- long press on text only
- Android
- long press on anything
There are more ways to bring up context menus (examples: two finger tap on Mac or Windows; right side click on Windows; press-and-hold on Wacom pens) but I haven’t tested all of them.
Windows, right click, no capture:
- Firefox, Chrome, Edge
- pointerdown, pointerup, auxclick, contextmenu
Windows, right click, capture:
- Firefox
- pointerdown, get capture, pointerup, lose capture, auxclick, contextmenu
- Chrome, Edge
- pointerdown, get capture, pointerup, auxclick, lose capture, contextmenu
Linux right click, no capture:
- Firefox
- pointerdown, contextmenu, pointermove² while menu is up
- Chrome
- pointerdown, contextmenu, no pointermove² while menu is up
Linux hold right down, no capture:
- Firefox
- pointerdown, contextmenu, pointermove² while menu is up
- Chrome
- pointerdown, contextmenu, no pointermove² while menu is up
Linux right click, capture:
- Firefox
- pointerdown, contextmenu, get capture, pointermove² while menu is up tells us button released
- Chrome
- pointerdown, contextmenu, get capture; not until another click do we get pointerup, lose capture
Linux hold right down, capture:
- Firefox
- pointerdown, contextmenu, got capture, pointermove² while menu is up tells us button released; when releasing button, menu stays up but we get pointerup, lose capture
- Chrome
- pointerdown, contextmenu, got capture, no pointermove² while menu is up; when releasing button, menu stays up but we don’t get pointerup; not until another click do we get pointerup, click, lose capture
Mac, ctrl + left click:
- Firefox
- pointermove with buttons≠0, contextmenu (no pointerdown or pointerup)
- Chrome
- pointerdown with button=left, contextmenu (no pointerup)
- Safari
- pointerdown with button=left, contextmenu (no pointerup); but subsequent clicks only fire contextmenu
Mac, right button down:
- Firefox
- pointerdown with button=right, contextmenu (no pointerup)
- Chrome
- pointerdown with button=right, contextmenu (no pointerup)
- Safari
- pointerdown with button=right, contextmenu (no pointerup); but subsequent right clicks only fire contextmenu
If we capture events on pointerdown
, Firefox and Safari will keep the capture even after the button is released. Chrome will keep capture until you move the mouse, and then it will release capture. [This seems like a Firefox/Safari bug to me, as pointer capture is supposed to be automatically released on mouse up]
It’s frustrating that on Mac, there’s no pointerup
or pointercapture
when releasing the mouse button. On Linux, the pointerup
only shows up if you click to exit the context menu. It doesn’t show up if you press Esc to exit. The workaround is to watch pointermove
events to see when no buttons are set. Windows doesn’t seem to have these issues, as both pointerdown
and pointerup
are delivered before the context menu.
Android, long press:
- Firefox
- pointerdown, get capture, contextmenu, pointerup, lose capture
- Chrome
- pointerdown, get capture, contextmenu, pointerup or pointercancel¹, lose capture
¹if the finger moves at all, this starts a scroll event which cancels the captured pointer ²Firefox lets the page see events outside the menu overlay, whereas Chrome doesn’t let the page see any events while the menu is up
--–— tests ---–—
The problem is that if I’m using pointerdown
and then pointerup
to track the state of the mouse, and I never get pointerup
, my code thinks I’m still holding the button down (and the browser’s pointer capture is still active).
Try right clicking the circle. It will bring up a context menu. That itself is fine. Unfortunately the events aren’t consistent across platforms:
Windows | Mac | Linux |
---|---|---|
pointerdown | pointerdown | pointerdown |
pointerup | ||
auxclick | ||
contextmenu | contextmenu | contextmenu |
pointerup † | ||
auxclick † |
What should we do?
- https://www.w3.org/TR/pointerevents/#the-pointerdown-event[8] says
preventDefault()
onpointerdown
does not stop click orcontextmenu
events. I canpreventDefault()
oncontextmenu
to prevent the menu. But I still want to getpointerup
and/orpointercancel
! I think I have to treatcontextmenu
as the up event which means I’ll get multiple up events on Windows. - https://w3c.github.io/pointerevents/#the-button-property[9] says
button
= 0. indicates the primary button. This would let me exclude middle button and right button. But I still get apointerdown.left
on Mac/Chrome and Mac/Safari (not on Mac/Firefox) so this may not be enough. - on pointermove, if we have pointer capture, we could check the state of the buttons?? TODO: test this! it’s mentioned as a workaround on https://github.com/w3c/pointerevents/issues/408[10]
TODO: demo of both solutions?
Try this | Watch for | Circle 1 | Circle 2 | Circle 3 |
---|---|---|---|---|
right click on circle | circle moves | yes ⛌ | no ? | yes ? |
right drag on circle | circle moves |
3.5. Feature: handle drag offset
This is not implementation specific, but a design issue. If you pick up the edge of the circle then you want to keep holding it at that point, not from the center. The solution is to remember where the center is relative to where you started the drag. Then when you move the object, you add that offset back in.
TODO: demo both ways (although it duplicates little-details page) ; could draw something showing the pick up point
Try this | Watch for | Circle 1 | Circle 2 |
---|---|---|---|
drag from edge of circle | circle jumps | yes ⛌ | no ✓ |
3.6. Feature: handle multitouch
isPrimary vs pointer id; need to test what happens if there are two independent drags going on
3.7. Feature: handle simultaneous dragging
3.8. TODO: what about multiple buttons?
So here’s a tricky one. If you are using multiple buttons at the same time, what happens? The Pointer Events spec says that the first button that was pressed leads to a pointerdown
event, and the last one that was released leads to a pointerup
event. But that means we might get a up event on a different button than the down event.
This doesn’t seem to be what I want, but it’s what Pointer Events do[11]. The workaround is to check the button state in pointermove
. But pointer capture continues until you release all the buttons, unless you explicitly release capture.
Mouse Events behave the way I want but don’t handle touch events.
3.9. TODO: what about multiple mice?
- mousedown (mouse 1)
- mousedown (mouse 2)
- mouseup (mouse 2)
- I stop dragging but the mouse that started the drag is still dragging
Does PointerEvent.pointerId
help here?
What happens when ipad is used to control mouse, or mouse is used to control ipad?
TODO: test middle clicking to drag, like some mice support on Windows and maybe Linux
TODO: test left click drag, then right button down, then left/right up. Since I only have one dragging state it might get confused.
https://www.w3.org/TR/pointerevents/#the-primary-pointer[12] says
Current operating systems and user agents don’t usually have a concept of multiple mouse inputs. When more than one mouse device is present (for instance, on a laptop with both a trackpad and an external mouse), all mouse devices are generally treated as a single device - movements on any of the devices are translated to movement of a single mouse pointer, and there is no distinction between button presses on different mouse devices. For this reason, there will usually only be a single mouse pointer, and that pointer will be primary.
3.10. TODO: test nested dragging
4 Vue version#
I think it’ll look something like this
<template> <g :transform="`translate(${pos.x},${pos.y})`" @pointerdown.left="start" @pointerup="end" @pointermove="dragging ? move($event) : null" @pointercancel="end" @lostpointercapture="end" @touchstart.prevent="" @dragstart.prevent=""> v-bind:class="{dragging}" <slot /> </g> </template> <style> g { cursor: grab; } g.dragging { user-select: none; cursor: grabbing; } </style> <script setup> // pos is a prop {x: y:} const dragging = ref(false); function start(event) { if (event.ctrlKey) return; let {x, y} = convertPixelToSvgCoord(event); dragging.value = {dx: pos.x - x, dy: pos.y - y, pointerId: event.pointerId}; el.setPointerCapture(event.pointerId); } function end(event) { dragging.value = null; } function move(event) { if (!(event.buttons & 1)) return end(event); if (event.pointerId !== dragging.value.pointerId) return; let {x, y} = convertPixelToSvgCoord(event); $emit('move', { x: x + dragging.value.dx, y: y + dragging.value.dy, }); } </script>
5 Notes - event log#
Testing a click:
- Desktop:
- Firefox/Mac, Chrome/Mac, Safari/Mac, Firefox/Windows, Chrome/Windows, Edge/Windows, Firefox/Linux all produce pointerdown, mousedown, pointerup, mouseup, click
- Firefox/Mac: if loading a page and the mouse is already over an element, will fire mouseover,mouseenter but not pointerover,pointerenter until the mouse is moved a tiny bit {need to test on Firefox/Windows, Firefox/Linux but probably does the same there}
- Mac: if you mouse down over the circle and then alt+tab to another window and then release the mouse, the web page still gets pointerup, mouseup, pointerout, pointerleave, mouseout, mouseleave (!). It also gets those if you put the computer to sleep. On Windows, it will go out as soon as you press alt+tab, and not come back when you switch to the same app, whereas on Mac it triggers pointerover etc when you come back to the app (further testing needed)
- Firefox vs Chrome (both Mac and Windows): if your mouse goes under the element when you scroll the page with the keyboard, Firefox will fire mouseover, mouseenter whereas Chrome will fire pointerover, pointerenter, mouseover, mouseenter. I feel like Chrome is doing the right thing here. [TODO: file a bug[13]]
- Mobile:
- Safari/iOS, Firefox/Android, Chrome/Android all produce pointerdown, touchstart, pointerup, touchend, but if quick: also produce [mousedown, mouseup, click]
- Android: contextmenu event if holding down; need to preventDefault to prevent the menu from showing up
- Android: if there’s text in the draggable event, need to use user-select:none to prevent text from being selected. If it’s in the middle of text, might be best to apply apply it only during a drag event; otherwise it would prevent text selection when not dragging.
6 Notes on dragging#
- Need touchstart.prevent to prevent scrolling
- Need either pointerdown.prevent or user-select:none to prevent double click from selecting text
TODO: test tablet
7 Variations#
nil8 Notes#
pointerdown
+ pointerup
will trigger click
(left mouse button) or auxclick
(middle mouse button) or contextmenu
(right mouse button)