Dragging LinearRegion limits with mouse

[lfreites]

I am quite new with the library and i am struggling to get some LinearRegion to be updated in real time with mouse drag events. Based on the LinearRegion example:

import sys
import numpy as np
from vispy import app, scene

# vertex positions of data to draw
N = 200
pos = np.zeros((N, 2), dtype=np.float32)
x_lim = [50., 750.]
y_lim = [-2., 2.]
pos[:, 0] = np.linspace(x_lim[0], x_lim[1], N)
pos[:, 1] = np.random.normal(size=N)

# color array
color = np.ones((N, 4), dtype=np.float32)
color[:, 0] = np.linspace(0, 1, N)
color[:, 1] = color[::-1, 0]

canvas = scene.SceneCanvas(keys='interactive', show=True)
grid = canvas.central_widget.add_grid(spacing=0)

viewbox = grid.add_view(row=0, col=1, camera='panzoom')

# add some axes
x_axis = scene.AxisWidget(orientation='bottom')
x_axis.stretch = (1, 0.1)
grid.add_widget(x_axis, row=1, col=1)
x_axis.link_view(viewbox)
y_axis = scene.AxisWidget(orientation='left')
y_axis.stretch = (0.1, 1)
grid.add_widget(y_axis, row=0, col=0)
y_axis.link_view(viewbox)

# add a line plot inside the viewbox
line = scene.Line(pos, color, parent=viewbox.scene)

# add vertical lines
color = np.array([[1.0, 0.0, 0.0, 1.0],
                  [0.0, 1.0, 0.0, 1.0],
                  [0.0, 0.0, 1.0, 1.0],
                  [0.0, 1.0, 0.0, 1.0],
                  [1.0, 0.0, 0.0, 1.0],
                  [0.0, 1.0, 0.0, 1.0]])
#pos = np.array([100, 120, 140, 160, 180, 200], dtype=np.float32)
#vert_region1 = scene.LinearRegion(pos, color,
#                                  parent=viewbox.scene)

#vert_region2 = scene.LinearRegion([549.2, 700], [0.0, 1.0, 0.0, 0.5],
#                                  vertical=True,
#                                  parent=viewbox.scene)

vert_region2 = scene.LinearRegion([549.2, 700], [0.0, 1.0, 0.0, 0.3],
                                  vertical=True,
                                  parent=viewbox.scene)                                  

# add horizontal lines
pos = np.array([0.3, 0.0, -0.1], dtype=np.float32)
#hor_region1 = scene.LinearRegion(pos, [1.0, 0.0, 0.0, 0.5],
#                                 vertical=False,
#                                 parent=viewbox.scene)

#hor_region2 = scene.LinearRegion([-5.1, -2.0], [0.0, 0.0, 1.0, 0.5],
#                                 vertical=False,
#                                 parent=viewbox.scene)


# auto-scale to see the whole line.
viewbox.camera.set_range()


if __name__ == '__main__' and sys.flags.interactive == 0:
    app.run()

I d like to be able to select one of the limit lines and drag it around the graph and have the limit of the LinearRegion updated on real time. Is it possible?

So far i connected the mouse events to the callback functions but the problem i have is that when i start a drag event with the mouse the display moves to adjust the camera, but in this case i d like the display to stay still and just get the left or right limit of the linear region geting updated. Thanks a lot in advance!

[djhoese]

Am I correct that you would like the camera to still work if the linear region is not selected?

You mentioned that so far you've been able to connect the mouse events to the callback functions, but I don't see any callback functions in your example code? Could you give a minimal example that shows what you have so far including any callbacks and mouse event handling you have?

You may need to do some fancy "picking" to determine if you're clicking on a Visual (the linear region) versus clicking on the viewbox (controlled by the camera). But I can't tell you exactly where code would have to change without the code you have so far.

[lfreites]

What i d like is that when i have a mouse event close to the limits of the linear region, instead of moving the camera around, the mouse d just move the limits of the linear region. Here i have the code with the callback functions. So when i detect the event in the "picking regions" i d like to avoid camera moving and just drag the vertical lines. hope i am being more clear this time :S

import sys
import numpy as np
from vispy import app, scene

# vertex positions of data to draw
N = 200
pos = np.zeros((N, 2), dtype=np.float32)
x_lim = [50., 750.]
y_lim = [-2., 2.]
pos[:, 0] = np.linspace(x_lim[0], x_lim[1], N)
pos[:, 1] = np.random.normal(size=N)

# color array
color = np.ones((N, 4), dtype=np.float32)
color[:, 0] = np.linspace(0, 1, N)
color[:, 1] = color[::-1, 0]

canvas = scene.SceneCanvas(keys='interactive', show=True)
grid = canvas.central_widget.add_grid(spacing=0)

viewbox = grid.add_view(row=0, col=1, camera='panzoom')

# add some axes
x_axis = scene.AxisWidget(orientation='bottom')
x_axis.stretch = (1, 0.1)
grid.add_widget(x_axis, row=1, col=1)
x_axis.link_view(viewbox)
y_axis = scene.AxisWidget(orientation='left')
y_axis.stretch = (0.1, 1)
grid.add_widget(y_axis, row=0, col=0)
y_axis.link_view(viewbox)

vert_region2 = scene.LinearRegion([549.2, 700], [0.0, 1.0, 0.0, 0.3],
                                  vertical=True,
                                  parent=viewbox.scene) 

def on_mouse_press(event):
	modifiers = [key.name for key in event.modifiers]
	print('Mouse:    - pos: %r, button: %s, modifiers: %s, delta: %r' % (event.pos, event.button, modifiers, event.delta))
	left_x, right_x = vert_region2.pos
	tr = canvas.scene.node_transform(viewbox.scene)
	x, y, _, _ = tr.map(event.pos)
	print(f" transformed left X: {x}, transformed right X: {y}")
	if (abs(left_x - x) < 5 and abs(left_x - x) < abs(right_x - x)):
		print("Inside left margin picking region")
	elif (abs(right_x - x) < 5 and abs(right_x - x) < abs(left_x - x)):
		print("Inside right margin picking region")

def on_mouse_release(event):
	if event.is_dragging:
		print("Dragging release")


canvas.events.mouse_press.connect(on_mouse_press)
canvas.events.mouse_release.connect(on_mouse_release)


# add a line plot inside the viewbox
line = scene.Line(pos, color, parent=viewbox.scene)

# add vertical lines
color = np.array([[1.0, 0.0, 0.0, 1.0],
                  [0.0, 1.0, 0.0, 1.0],
                  [0.0, 0.0, 1.0, 1.0],
                  [0.0, 1.0, 0.0, 1.0],
                  [1.0, 0.0, 0.0, 1.0],
                  [0.0, 1.0, 0.0, 1.0]])


# auto-scale to see the whole line.
viewbox.camera.set_range()


if __name__ == '__main__' and sys.flags.interactive == 0:
    app.run()

[djhoese]

I'm playing around with the code. I'll get back to you on some ideas. In the mean time, be careful of your use of tabs in your python code. Always use spaces (preferably 4). You have tabs in your mouse press handler.

[djhoese]

Let me know if any part of this is not what you were hoping for, but here are some things I got to work. Even as the maintainer of vispy, I have to relearn a lot of this stuff every time it comes up (an obvious sign I need to put it in the documentation more). Also note there may be smarter ways of doing this, but this introduces some important concepts. The top portion of your script is unchanged as well as the very bottom with the Line creation.

vert_region2 = scene.LinearRegion([549.2, 700], [0.0, 1.0, 0.0, 0.3],
                                  vertical=True,
                                  parent=viewbox.scene)
# let the SceneCanvas "picking" logic know that this Visual can be "clicked on"
vert_region2.interactive = True

def on_mouse_press(event):
    mouse_event = event.mouse_event
    modifiers = [key.name for key in mouse_event.modifiers]
    print('Mouse:    - pos: %r, button: %s, modifiers: %s, delta: %r' % (event.pos, mouse_event.button, modifiers, mouse_event.delta))
    vis = event.visual
    print("Visual at mouse: ", vis)
    #vis = canvas.visual_at(event.pos)  # not used but could be useful

    if vis is vert_region2:
        # the SceneCanvas mouse handler will see this and not pass the event on
        # to the camera for handling
        event.handled = True


def on_mouse_move(event):
    if event.visual is vert_region2:
        event.handled = True
        print("Moving region", event.visual)
        # TODO: Do something to the visual based on mouse movement


def on_mouse_release(event):
    if event.mouse_event.is_dragging:
        print("Dragging release")


vert_region2.events.mouse_press.connect(on_mouse_press)
vert_region2.events.mouse_move.connect(on_mouse_move)
vert_region2.events.mouse_release.connect(on_mouse_release)

The key takeaways:

1. Set .interactive = True on any visual you want to be "clickable". For your use case you could possibly stick with your position ranges if you really wanted to. Note this does not handle cases where you want to know which vertex or edge of the Visual is clicked. For Markers and Meshes there are newer/better/smarter ways to do this. Note you could still use this and then do your range checks with the .bounds (I think that's the property) of the picked visual rather than using the linear regions .pos.
2. Set .handled = True on any event handler's event that you don't want the camera or parent Visual to receive.
3. Add your mouse handlers to the "picked" Visual rather than the canvas as a whole. This let's the mouse handlers only get called when the Visual of interest is clicked.
4. In the case shown in my code (with visual event handlers versus canvas handlers) the SceneCanvas is providing you a special SceneMouseEvent class (

   vispy/vispy/scene/events.py

   Line 10 in a1a639f

   class SceneMouseEvent(Event):

   ) which has the picked visual at .visual and the original mouse event at .mouse_event.

Hopefully this gives you enough information to be dangerous. Let me know if you have more questions.

[lfreites]

I have been playing around a bit considering your comments and i can actually update the linear region boundaries. But for some reason the data is not set on real time, i just see the result in the display after the mouse release. It also takes a long time to set the new boundaries. Is there a way to optimize this behaviour?

import sys
import numpy as np
from vispy import app, scene

# vertex positions of data to draw
N = 200
pos = np.zeros((N, 2), dtype=np.float32)
x_lim = [50., 750.]
y_lim = [-2., 2.]
pos[:, 0] = np.linspace(x_lim[0], x_lim[1], N)
pos[:, 1] = np.random.normal(size=N)

# color array
color = np.ones((N, 4), dtype=np.float32)
color[:, 0] = np.linspace(0, 1, N)
color[:, 1] = color[::-1, 0]

canvas = scene.SceneCanvas(keys='interactive', show=True)
grid = canvas.central_widget.add_grid(spacing=0)

viewbox = grid.add_view(row=0, col=1, camera='panzoom')

# add some axes
x_axis = scene.AxisWidget(orientation='bottom')
x_axis.stretch = (1, 0.1)
grid.add_widget(x_axis, row=1, col=1)
x_axis.link_view(viewbox)
y_axis = scene.AxisWidget(orientation='left')
y_axis.stretch = (0.1, 1)
grid.add_widget(y_axis, row=0, col=0)
y_axis.link_view(viewbox)


vert_region2 = scene.LinearRegion([549.2, 700], [0.0, 1.0, 0.0, 0.3],
                                  vertical=True,
                                  parent=viewbox.scene)
                                  
selected_region_left = False
selected_region_right = False

# let the SceneCanvas "picking" logic know that this Visual can be "clicked on"
vert_region2.interactive = True

def on_mouse_press(event):
    global selected_region_left
    global selected_region_right
    mouse_event = event.mouse_event
    modifiers = [key.name for key in mouse_event.modifiers]
    print('Mouse:    - pos: %r, button: %s, modifiers: %s, delta: %r' % (event.pos, mouse_event.button, modifiers, mouse_event.delta))
    vis = event.visual
    print("Visual at mouse: ", vis)
    #vis = canvas.visual_at(event.pos)  # not used but could be useful

    if vis is vert_region2:
        # the SceneCanvas mouse handler will see this and not pass the event on
        # to the camera for handling
        event.handled = True
        
    left_x, right_x = vert_region2.bounds(0)
    tr = canvas.scene.node_transform(viewbox.scene)
    x, y, _, _ = tr.map(mouse_event.pos)
    print(f" transformed left X: {x}, transformed right X: {y}")
    if (abs(left_x - x) < 5 and abs(left_x - x) < abs(right_x - x)):
        print("\nInside left margin picking region\n")
        selected_region_left = True
    elif (abs(right_x - x) < 5 and abs(right_x - x) < abs(left_x - x)):
        print("\nInside right margin picking region\n")
        selected_region_right = True

def on_mouse_move(event):
    mouse_event = event.mouse_event
    if event.visual is vert_region2:
        event.handled = True
        print("Moving region", event.visual)
        # TODO: Do something to the visual based on mouse movement
        left_x, right_x = vert_region2.bounds(0)
        tr = canvas.scene.node_transform(viewbox.scene)
        x, y, _, _ = tr.map(mouse_event.pos)
        print(f"Selected left linear region: {selected_region_left}")
        if selected_region_left:
            print("Setting data lext x for linear region!!!")
            vert_region2.set_data([x, right_x])
        if selected_region_right:
            print("Setting data right x for linear region!!!")
            vert_region2.set_data([left_x, x])
        
        


def on_mouse_release(event):
    print("Mouse release!!!!!!")
    if event.mouse_event.is_dragging:
        print("Dragging release")
    selected_region_left = False
    selected_region_right = False


vert_region2.events.mouse_press.connect(on_mouse_press)
vert_region2.events.mouse_move.connect(on_mouse_move)
vert_region2.events.mouse_release.connect(on_mouse_release)

# add a line plot inside the viewbox
line = scene.Line(pos, color, parent=viewbox.scene)

# add vertical lines
color = np.array([[1.0, 0.0, 0.0, 1.0],
                  [0.0, 1.0, 0.0, 1.0],
                  [0.0, 0.0, 1.0, 1.0],
                  [0.0, 1.0, 0.0, 1.0],
                  [1.0, 0.0, 0.0, 1.0],
                  [0.0, 1.0, 0.0, 1.0]])


# auto-scale to see the whole line.
viewbox.camera.set_range()


if __name__ == '__main__' and sys.flags.interactive == 0:
    app.run()

Thanks for all the effort!!

[djhoese]

Two things:

1. Missing global selected_region_left (and right) in your on_mouse_release.
2. Add vert_region2.update() after the 2 calls to vert_region2.set_data. I think this is just needed to force the canvas to redraw the visual, otherwise it waits for the next draw event.

This fixes the basic behavior, but it only seems to work on the first click and drag of a specific side. If you then click and drag the other side it works, but the original side is "snapped" back to its original position. This makes me think either the .bounds of the visual is broken and not returning the expected result or that something else is wrong with the logic you have, but nothing is jumping out at me right now.

[lfreites]

I made some debugging and i see that after the set_data and update the .bounds give the same initial values, so for some reason its not taking the values set with set_data.

[djhoese]

This is...a huge bug. I don't understand. The BaseVisual class implements the bounds method like this:

https://github.com/vispy/vispy/blob/2f3da299121894a4ddaa8379c0645dd943ef14d4/vispy/visuals/visual.py#L241C13-L255

The summary of this is that bounds seems to be cached in self._vshare.bounds. The only public-ish way to clear this to call _bounds_changed on the visual. It looks like there is an event that is supposed to be tied to this method, but it isn't even used anywhere and is never actually connected to this method from what I can tell:

https://github.com/search?q=repo%3Avispy%2Fvispy%20_bounds_change&type=code

@brisvag has napari seen this before? @almarklein @larsoner any historical context you can supply to this?

@lfreites adding a vert_region2._bounds_changed() after your set_data (before .update()) seems to fix your issue for me.

Edit: Unless I'm missing something, I can't imagine the number of updating-Visual cases that this breaks.

[larsoner]

No idea :(

[almarklein]

Interesting! I don't have any historical context, it looks like @campagnola wrote the bounds() method. I think that the idea was that the _bounds_changed() is called from the set_data() methods, but that this never happened.

[lfreites]

Actually as @djhoese commented adding vert_region2._bounds_changed() after set_data() fixed my problem. I ll just be carefull whenever i am updating data to consider if it ll be necesary to manually update the bounds too. Thanks to all for your help!

[brisvag]

@brisvag has napari seen this before? @almarklein @larsoner any historical context you can supply to this?

I actually encountered this bounds caching before, and had to work around it. At the time I thought it was an edge case and didn't think about checking the rest of vispy to see if others were misusing it :/

Right now there's just one place in napari where we explicitly use set_bounds(), and we handle the updating ourselves.

[djhoese]

From my skimming of code yesterday it seems there are a couple Visuals (maybe it was Markers and/or Meshes) where they manually reset the bounds in their own cache, but this doesn't change the .bounds() methods cache in the vshare object. For example, here is the LineVisual resetting the internal bounds cache when position is changed:

vispy/vispy/visuals/line/line.py

Lines 172 to 175 in 2f3da29

	if pos is not None:
	self._bounds = None
	self._pos = pos
	self._changed['pos'] = True

This makes ._compute_bounds recompute the bounds, but that doesn't change the vshare from what I can tell.