Touch movement of canvas in 20mb 360 degree Images

dc19

Hi all,

Hope you're well.

I'm running a Target Detection in 360-degree Images project. I'm comparing speed differences between VR and Flat Screen. For the Flat screen condition, I want to use a touch-based input to search/move through parts of an image.

In general images are 6000x3000 and 20mb in size. I anticipate there to be 50 images in each condition. I am concerned that this is an unusually large file size for OpenSesame. A Microsft Surface or Android Tablet will be used- the Surface will likely be more powerful.

When testing moving through the 360 around there is significant lag (5-10 seconds) and the experiment crashes with a memory error. I need to make it respond instantly or with no lag (what you would normally expect from moving around a photo on an iPad)

To test this touch interaction idea I used a slight adaptation (99%) of Edwin's python script from a 2014 thread (Thanks Edwin!)

from openexp.mouse import mouse
from openexp.canvas import canvas
my_mouse = mouse(exp)
my_canvas = canvas(exp)
path = exp.get_file(u'med.jpg')
my_canvas.image(path)
trialstart = my_canvas.show()
button, position, movstart = my_mouse.get_click(timeout=500)

while True:
    pos, time = my_mouse.get_pos()
    my_canvas.clear()
    my_canvas.image(path, x=pos[0], y=pos[1])
    timestamp = my_canvas.show()

movtime = timestamp - movstart
resptime = movstart - trialstart

exp.set("response", response)
exp.set("movetime", movtime)
exp.set("reponse_time", resptime)

Anyone have any ideas? or if this is possible? Ideally, the canvas shouldn't to render each time I move the view.

Cheers,

Daniel

sebastiaan

Hi Daniel,

There's most likely a more efficient way to do this; however, I'm not sure I understand fully what you want to do. My understanding is that:

• You have an extremely large image
• that you want to present centered around the cursor coordinates
• on a display that is far smaller than the image (but what's the resolution), so that most of the image is actually not visible.

Is that correct?

Cheers,
Sebastiaan

dc19

Yes that is all correct! The resolution of the display will be < 1/3 of the total picture. The exact scale I need is dependent image to image.

dc19

Hi Sebastiaan, thought I'd bump this Instead of using touch to move the image I'm going to use a joystick/aircraft yoke.

Any ideas on how to reduce the lag in the response?

sebastiaan

I made a simple example of how you can implement this directly in pygame. This is much faster (I actually made it on an Ubuntu tablet, and even there it's snappy), but it does require that you use the legacy backend.

Right now, the viewport is moved with the arrow keys of the keyboard; but you can easily change this into moving it around with a joystick/gamepad. You can do this with the OpenSesame joystick plugin, or directly in pygame. Read the code comments to understand the logic; it's not that complicated, really.

• http://osdoc.cogsci.nl/3.1/manual/response/joystick/
• http://www.pygame.org/docs/ref/joystick.html

Hope this gets you started!

import pygame

# Create a keyboard to control the viewport
kb = keyboard()
# Load an image
surf = pygame.image.load(pool['image.jpg']).convert()
# The width and height of the image
w = surf.get_width()
h = surf.get_height()
# The center point of the image
xc = w/2
yc = h/2
# The size of the window that you want to show (i.e. the viewport)
wx, wy = 100, 100
# The steps for shifting the viewport
stepsize_x, stepsize_y = 25, 25

# Where the viewport is shown on the display
viewport_topleft = exp.surface.get_width()/2-wx/2
viewport_topheight= exp.surface.get_height()/2-wy/2

while True:
    # Slice a subsurface out of the original image, and show it
    subsurf = surf.subsurface((xc-wx/2,yc-wy/2,wx,wy))
    exp.surface.blit(subsurf, (viewport_topleft, viewport_topheight))
    pygame.display.flip()
    # Check the keyboard. Use space to break, and the arrow keys to move
    # the viewport.
    key, time = kb.get_key()
    if key == 'space':
        break
    if key == 'left':
        xc -= stepsize_x
    elif key == 'right':
        xc += stepsize_x
    elif key == 'up':
        yc -= stepsize_y
    elif key == 'down':
        yc += stepsize_y
    # Make sure the viewport doesn't go outside of the image
    xc = max(wx/2, xc)
    xc = min(w-wx/2, xc)
    yc = max(wy/2, yc)
    yc = min(h-wy/2, yc)