Add SDL Pinch events

[1bsyl]

Some update:

Added SDL Pinch event (begin/update/end), with the delta scale factor.

We could add more data then the scale factor value if needed afterward (position center, diameter, rotation, global scale factor, velocity), but not sure all back-end can provide it.

I've removed the Swipe event, because it seems to be really different across platform (number fingers, swipe vs scroll definition).

Currently implemented:

• back-ends: x11/wayland/macosx/ios/android
• cmake detection
• updated testgeometry to test pinch gesture (add --info event_motion in command line for verbosity)

Todo:

• ios: is sending the global scale factor, whereas other backend are sending the delta scale ...
• never tested on macosx (no hardware)
• overall needs better testing

keeping the old message, because it has some info:

Add native Pinch and Swipe events for SDL.

This is DRAFT, it starts to show this can work, but clearly not functional.
Currently, I've only tried on x11 and wayland, but will also do macosx since there was some old commit over-there, and android/ios.

What can be noticed:

• on x11, it needs xinput2 protocol 2.4 (before 2.2). that sounds ok to me.

• on wayland, it needs "pointer-gestures-unstable-v1-client-protocol.h", since we already have added some unstable protocol, why not adding this one.

• x11/wayland: it only detects the swipe/pinch on a touchpad (eg laptop or some device tablet). but nothing with a real touch screen. that's a restriction.

• SDL pinch/swipe events are sent as "begin", many update, "end" sequences. seems what is done with x11 and wayland android. (at least

• Not sure, what we should publicly expose in SDL_Event.h for the structure.
  I guess, that the big thing to determine. Also to have something coherent between desktop and smartphone/tablets.

we can just map to wayland:
https://wayland.app/protocols/pointer-gestures-unstable-v1#zwp_pointer_gesture_swipe_v1:event:begin
or x11: https://lists.x.org/archives/xorg-commit/2021-May/043325.html (see datas: GesturePinchEvent GestureSwipeEvent )

android:
https://developer.android.com/reference/android/view/ScaleGestureDetector.OnScaleGestureListener
https://developer.android.com/reference/android/view/ScaleGestureDetector

[Kontrabant]

• on wayland, it needs "pointer-gestures-unstable-v1-client-protocol.h", since we already have added some unstable protocol, why not adding this one.

Don't worry about this; 'unstable' is a legacy categorization at this point, as they realized that requiring backwards-incompatible changes to promote protocols to stable was a bad idea. Unstable protocols will basically be unstable forever at this point.

[1bsyl]

Original ticket was #7706

I've added some code for Macosx, but this is incomplete and untested (I have no trackpad on mac mini, and cannot get the ipad to work as a trackpad).

For MACOSX, it seems:

• no notion of begin/end for swipe/pinch.
  • we can always encapsulate the event with a begin/end anyway.
• pinch is actually "magnify". there is another "rotate" method.
  • should add/separate rotate to/from pinch ?
• swipe is 3 fingers. but there is also scroll (not sure where it is), that should but a 2 fingers gesture.
  • wantsScrollEventsForSwipeTrackingOnAxis to map scroll to swipe

[1bsyl]

Android has begin/update/end callbacks for pinch events.
but for swipe, nothing yet really clear for me. if you move 1 finger, you get a swipe.
in somecase, users would prefer usual touch event than swipe event.
( https://stackoverflow.com/questions/4139288/android-how-to-handle-right-to-left-swipe-gestures )

[1bsyl]

On IOS, added callbacks to handle pinch gestures.
(tvos seems to already use the swipe gestures, and doesn't compile with pinch)

maybe, we shouldn't try to expose the swipe gesture, because, it's seems to be very different across platforms, 1 or several fingers (even the ability to report the number of fingers) and also behavior (scroll vs swipe -finger goes up at the end-)

[1bsyl]

@libsdl-org/a-team hey, It compiles and has been cleaned up ! needs some feedback !

[ssouzawallace]

Merge This Please!

[slouken]

@1bsyl, sorry I missed that you needed feedback on this. Let me review and get back to you.

[slouken]

I think this is good to go as-is, it looks like your questions are about swipe gestures?

I think enabling gesture support on macOS and iOS delays touch event delivery. @icculus, what do you think about implementing pinch/zoom/rotate/swipe on top of touch events to have consistent cross-platform behavior? We could include the number of fingers in the swipe across all platforms, etc.?

[icculus]

Making our own pinch gestures is only going to make iOS users say "this doesn't feel right for some reason." I'd say either the OS provides them or the app doesn't get those events.

[slime73]

Would it make sense to have them opt-in with some documentation about the consequences of doing so?

[1bsyl]

@slouken no problem for the delay. actually, the PR is just a prototype to see where it goes.

yes, swipe gesture are going to be really different across platform if there are implement with different os back-end (number of finger/speed are not always provided i guess). also swipe gesture is different of scroll gesture (one expect the fingers up at the border of the screen).

but here, it's only about pinch. to simplify. but probably all back-end wont provide the number of finger for pinch, the rotation angle, the center, speed.

It makes sense that this introduce a delay for touch event. Probably that internally it starts to buffer touch event, and try to detect gestures: when the gesture is detected and sent, the touch event are discarded.

A question here is: should we send the touch event if a gesture is detected ?

I personally won´t really use the gesture things, but that sounds much easier to implement on top of touch event, than per back-end. I don't see how different ios people are used to pinch/scroll/etc that others, but maybe we could add some customization: like setting how many event before detecting a pinch/ how many finger for a scroll on ios, etc. I mean some customization so that the generic layer feels like it was a real platform back-end.

[ssouzawallace]

Can I have write access to this repository can merge pull request?

[icculus]

Can I have write access to this repository can merge pull request?

Uh, no, but we should probably merge this soon.

Where I stand on this:

• I don't think Swipe is worth holding this PR up for; we can add it later, or never; both options are fine with me.
• I don't think we should emulate pinch on platforms that don't provide it at the system level, or try to build a generic version on top of multitouch that some/all platforms use. Any app that relies on pinch gestures better be exclusively a smartphone app or offer other input techniques to fall back to. For pinch gestures specifically, I don't think that's an unreasonable demand.
• While the most important thing at the moment is the public ABI (which in this case, is just the new event types)...someone probably should test this on iOS/macOS before we merge it.
• Other than that, I'm fine with this PR merging as-is.

[icculus]

Also, this needs to be updated to the latest in main, with an extremely minor conflict resolved.

[slouken]

Zoom events? It wasn't intuitive for me that a pinch event would also mean the opposite of pinch. :)

[slouken]

Although looking through the implementations, at least half of them refer to this as pinch, so this is probably fine. The event is well documented.

[1bsyl]

hum not sure I got it, but yes pinch and zoom are the same thing.
and this can be "pinch in" or "pinch out" for "zoom in/out"

[1bsyl]

I have documented more the events enum started/updated/end

[slouken]

We could potentially add a rotation here, as some of the implementations support it.

[1bsyl]

I think, we could have more, but all back-end won't have them:
number of finger user for pinch,
the rotation angle,
the center of fingers used,
speed

[1bsyl]

Updated to latest head.

macosx: need some testing.
ios: needs to be fixed I think. the zoom here is accumulated according to documentation (whereas we only want the delta).

After thinking, implementing internally the gesture seems quite difficult: like buffering events, how many threshold before deciding whether this is a pinch/swipe/something else and so on. I mean it sounds it would be a lots of UI experience to solve and discussion to tell what a gesture should be.
And so another option, maybe, would to include something already existing and generally approved where we could inject the SDL touch event, and get the gestures (libinput ?).

Another question: should we have at some point a SDL PINCH_CANCEL event ? when it starts detecting a pinch but that turns to be something else ? maybe not much relevant with pinch events.

[AntTheAlchemist]

Does there need to any delay for pinch / rotate? Can't it come through immediately after the first touch of more than 1 finger? If there was any delay, I'd be bypassing SDL and calculating it myself to get the UX I want.

[1bsyl]

I guess :

• it would add delay with more than 1 finger. because it should buffer a few event internally and threshold to detect a pinch/etc.
• no delay if only 1 finger. (unless there are 1 finger gesture...like swipe/scroll 1 finger)

(only simple guess)

[1bsyl]

@Kontrabant thanks for the feedback ! I've updated and hasn't squashed the commit yet, please have a look

[Kontrabant]

One small destruction order nitpick, but other than that, LGTM.

[StrangePan]

Is this PR still alive?

[1bsyl]

I've sync'ed the branch... not sure if it still compiles ...

[1bsyl]

@madebr Hey, I try to compile (quickly) a single sdl test apk (testgeometry).

but that fails after it builds, any idea ?

not sure but:

• it say "compiler skipped"
• it compiles nage, psp, direct3d...
• ulibc

Thanks!

here's my commands:

cd build

cmake .. -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_HOME/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DSDL_ANDROID_HOME=$ANDROID_HOME -DANDROID_PLATFORM=25 -DSDL_TESTS=ON

cmake --build . --target SDL3-jar

cmake --build . --target testgeometry-apk

adb install -r ./test/testgeometry.apk

edit: it's working now thanks @madebr

[1bsyl]

hey @madebr ! sorry to bother, any idea to this error ?
I remember there used to be a simple command line to build a test-apk on android.

[madebr]

hey @madebr ! sorry to bother, any idea to this error ? I remember there used to be a simple command line to build a test-apk on android.

That code is still present and mostly unchanged, and exercised daily on ci.

SDL/CMakeLists.txt

Lines 1563 to 1616 in a1672f2

	if(SDL_ANDROID_JAR)
	find_package(Java)
	find_package(SdlAndroidPlatform MODULE)
	if(Java_FOUND AND SdlAndroidPlatform_FOUND AND NOT CMAKE_VERSION VERSION_LESS "3.19")
	include(UseJava)
	set(path_android_jar "${SDL_ANDROID_PLATFORM_ROOT}/android.jar")
	set(android_java_sources_root "${SDL3_SOURCE_DIR}/android-project/app/src/main/java")
	file(GLOB SDL_JAVA_SOURCES "${android_java_sources_root}/org/libsdl/app/*.java")
	set(CMAKE_JAVA_COMPILE_FLAGS "-encoding;utf-8")
	add_jar(SDL3-jar
	SOURCES ${SDL_JAVA_SOURCES}
	INCLUDE_JARS "${path_android_jar}"
	OUTPUT_NAME "SDL3"
	VERSION "${SDL3_VERSION}"
	)
	set_property(TARGET SDL3-jar PROPERTY OUTPUT "${SDL3_BINARY_DIR}/SDL3-${SDL3_VERSION}.jar")
	add_library(SDL3__Jar INTERFACE)
	add_library(SDL3::Jar ALIAS SDL3__Jar)
	get_property(sdl3_jar_location TARGET SDL3-jar PROPERTY JAR_FILE)
	set_property(TARGET SDL3__Jar PROPERTY JAR_FILE "${sdl3_jar_location}")
	set(javasourcesjar "${SDL3_BINARY_DIR}/SDL3-${SDL3_VERSION}-sources.jar")
	string(REGEX REPLACE "${android_java_sources_root}/" "" sdl_relative_java_sources "${SDL_JAVA_SOURCES}")
	add_custom_command(
	OUTPUT "${javasourcesjar}"
	COMMAND ${Java_JAR_EXECUTABLE} cf "${javasourcesjar}" ${sdl_relative_java_sources}
	WORKING_DIRECTORY "${android_java_sources_root}"
	DEPENDS ${SDL_JAVA_SOURCES}
	)
	add_custom_target(SDL3-javasources ALL DEPENDS "${javasourcesjar}")
	if(SDL_INSTALL_DOCS)
	set(javadocdir "${SDL3_BINARY_DIR}/docs/javadoc")
	set(javadocjar "${SDL3_BINARY_DIR}/SDL3-${SDL3_VERSION}-javadoc.jar")
	set(javadoc_index_html "${javadocdir}/index.html")
	add_custom_command(
	OUTPUT "${javadoc_index_html}"
	COMMAND ${CMAKE_COMMAND} -E rm -rf "${javadocdir}" "${javadocjar}"
	COMMAND ${Java_JAVADOC_EXECUTABLE} -encoding utf8 -d "${javadocdir}"
	-classpath "${path_android_jar}"
	-author -use -version ${SDL_JAVA_SOURCES}
	DEPENDS ${SDL_JAVA_SOURCES} "${path_android_jar}"
	)
	add_custom_command(
	OUTPUT "${javadocjar}"
	COMMAND ${Java_JAR_EXECUTABLE} -c -f "${javadocjar}"
	-C "${javadocdir}" *
	WORKING_DIRECTORY "${javadocdir}"
	DEPENDS ${javadoc_index_html}
	)
	add_custom_target(SDL3-javadoc ALL DEPENDS "${javadoc_index_html}" "${javadocjar}")
	set_property(TARGET SDL3-javadoc PROPERTY OUTPUT_DIR "${javadocdir}")
	endif()
	endif()
	endif()

The SDL3-jar target should create a SDL3 Java archive.

Clean you build directory and try again?
Try to find out using cmake --trace-expand why this target is not created?

[1bsyl]

ok, this is working now, I forget to call the SDL3-jar target
thanks @madebr

[1bsyl]

though, I am still not sure why it tries to bulid ulicm, direct3d, vita, ngage, etc.

[  0%] Building C object CMakeFiles/SDL3-shared.dir/src/render/direct3d12/SDL_render_d3d12.c.o
[  0%] Building C object CMakeFiles/SDL3-shared.dir/src/render/direct3d12/SDL_shaders_d3d12.c.o
[  0%] Building C object CMakeFiles/SDL3-shared.dir/src/render/ps2/SDL_render_ps2.c.o
[  0%] Building C object CMakeFiles/SDL3-shared.dir/src/render/psp/SDL_render_psp.c.o
[  0%] Building C object CMakeFiles/SDL3-shared.dir/src/render/vitagxm/SDL_render_vita_gxm_tools.c.o