A tablet, a small mirror, a few objects on the table. With Reflex, SENSEi research shows that learning can be taken beyond the screen in a simple, fun and affordable way, starting from the things children pick up with their hands.

Presented at the ACM CHI conference, Reflex is the first step in a line of research, phygital, that today lives on at room scale in SENSEi.

What is the research question?

Many educational apps ask the child to touch a screen. For children and young people with neurodevelopmental disorders, however, handling real objects is often an important part of learning.

The title of the article sums up the challenge: learning beyond the screen in a simple, fun and affordable way. Without expensive equipment, using devices that many schools and families already own.

The study in brief

Reflex is a game for smartphones and tablets. Its distinctive idea is a small mirror over the camera: this way the device “sees” the physical objects placed on the table and turns them into play. The physical and digital worlds meet.

The intended audience is children and adolescents, particularly those with neurodevelopmental disorders. The game’s features are co-designed, and the work includes a first pilot study.

What emerges?

For Mirko Gelsomini, who authored the work and now leads SENSEi, the new interaction paradigm, the co-designed features and the pilot study reveal a still unexplored potential for learning. There are no measures of effectiveness yet: it is the start of a journey.

And the journey continues. An article at the CHItaly conference presents Reflex as a training application for people with neurodevelopmental disorders, defining the phygital paradigm, that is, physical and digital together.

A later study in Multimedia Tools and Applications then evaluates Reflex with young adults with intellectual disability in a social care centre. We describe it in the case study on physical and digital objects for adults.

What does Reflex teach those with limited means?

Reflex is a good example of essential design. Some lessons also apply outside research:

• start from objects: an object to hold makes the task concrete;
• use what is there: a device you already have can become a different tool with a small adjustment;
• co-design: features grow out of dialogue with those who will actually use the game.

Those who want to experiment with limited means will find more ideas in the guide DIY multisensory room, which also explains what is best not to improvise.

Why does it matter today?

Reflex shows that innovation does not depend on the amount of technology, but on how close it is to people. It is the same spirit as SENSEi: no building work, activities that adapt to those who experience them, adults who stay in charge.

In the magic room, the phygital idea grows to room scale. Walls and floor come to life and interact with tactile objects and physical materials, while the whole body takes part in the experience. Discover how the SENSEi magic room works and the activities for education.

With Reflex, the SENSEi research group opens the phygital strand that, study after study, reaches all the way to services for adults. The original article is in the Research section: Reflex: Learning Beyond the Screen in a Simple, Fun, and Affordable Way. The follow-up work, presented at CHItaly, is here: the page on the phygital approach.

Frequently asked questions

What is Reflex?

It is a game for smartphones and tablets that, thanks to a mirror over the camera, recognises the physical objects placed on the table and uses them in the game. It is designed for children and adolescents, particularly those with neurodevelopmental disorders.

What does phygital mean?

It is the fusion of physical and digital: in Reflex the child moves real objects on the table and the game on the screen reacts.

Does Reflex prove that it improves learning?

The work includes a first pilot study and points to unexplored potential. It does not present measures of effectiveness.

How much does an environment like this cost?

It depends on the configuration: space, experiences, training and services. Contact us for an assessment.

Do you work in a school, a centre or a museum and want an environment like this? Tell us about your project: we start from the needs of the people who will use it.

Sources

• Gelsomini M. (2018). Reflex: Learning Beyond the Screen in a Simple, Fun, and Affordable Way. CHI EA 2018, ACM
• Spitale M., Gelsomini M., Beccaluva E., Viola L., Garzotto F. (2019). Meeting the needs of people with neuro-developmental disorder through a phygital approach. CHItaly 2019, ACM
• Gelsomini M., Spitale M., Garzotto F. (2021). Phygital interfaces for people with intellectual disability: an exploratory study at a social care center. Multimedia Tools and Applications, Springer