Artificial intelligence for autism is mainly about observing better: automatically measuring behaviours, movements, voice and emotions that today require many hours of expert observation. The AI4Autism project, funded by the Swiss National Science Foundation, is the broadest example among the research linked to SENSEi.
Here we describe it together with three related works, Emotify, WIYE and a study on fine-grained gesture recognition. With a clear premise: a SENSEi room does not make diagnoses. Diagnosis remains with clinicians.
The research question: observing better, earlier
The AI4Autism project starts from a figure from the US Centers for Disease Control and Prevention: 1 child in 36 receives a diagnosis of autism spectrum disorder. The team points out that, if the condition is identified early, interventions can markedly improve the trajectory of cognitive development in children with significant delays.
This leads to two needs: more scalable screening and diagnostic tools, and a finer description of profiles (the so-called phenotype) to better estimate symptoms.
The study in brief
AI4Autism is a four-year project. It studies how digital sensing, with cameras and sensors, can provide automatic measures of the broader autism phenotype. The aim is to better distinguish autism subtypes, from a precision medicine perspective. The team presents datasets and annotations that it intends to make public, and methods for analysing children’s behaviour.
Alongside AI4Autism, three studies look at emotions and movement:
- Emotify is a spoken game that helps autistic children recognise and express four states: happiness, sadness, anger and neutrality. First they learn, then they test what they have learned, with a multilingual recogniser that listens to the tone of voice;
- WIYE (What Is Your Emotion?) collects the voices, faces and stories of children aged 4 to 12 with an interactive storytelling app, one episode per emotion: sadness, anger, fear, surprise and joy;
- a study on fine-grained gesture recognition involves 17 participants in 4 tasks with an object fitted with a sensor, concluding that recognising fine-grained activities well requires a new approach.
What emerges?
The common message is that observation can become richer, more comparable and less demanding. AI4Autism makes data and methods available; Emotify and WIYE show that emotions can also be practised and studied through play; the study on fine-grained gestures honestly points out where current tools still fall short.
Why it matters today
This research matters for SENSEi because it teaches how to observe behaviour in a respectful, automatic and interpretable way. In a SENSEi room this translates into discreet observation: the room recognises how children move and take part, with nothing to wear, and each session leaves progress reports that help document the pathway. The adult decides whether to let the experience adapt on its own or to guide it themselves.
The work on emotions connects with the emotional expression and regulation activities of SENSEi CARE. The research strand continues with SUPSI, the University of Geneva and the Pôle Autisme: we describe it in the case The Pôle Autisme in Geneva.
In the SENSEi research section, the strand opens with the entry The AI4Autism Project. Those who work in the classroom will find practical guidance in the guide Sensory room and autism at school.
Frequently asked questions
What is AI4Autism?
A four-year research project funded by the Swiss National Science Foundation, which uses cameras and sensors to automatically measure aspects of the autism phenotype.
Can artificial intelligence diagnose autism?
Research studies methods to support diagnosis, but diagnosis remains a clinical act. SENSEi is not a diagnostic tool.
Does SENSEi record children?
SENSEi recognises behaviour to make the activities work and produces progress reports. How documentation is handled is agreed with the facility.
How much does an environment like this cost?
It depends on the configuration: space, experiences, training and services. Contact us for an assessment.
Do you work in a school, a centre or a museum and want an environment like this? Tell us about your project: we start from the needs of the people who will use it.
Sources
- Tafasca S., Gupta A., Kojovic N., Gelsomini M. et al. (2023). The AI4Autism Project. ICMI 2023 Companion. DOI 10.1145/3610661.3616239
- Rouhi A., Spitale M., Catania F., Cosentino G., Gelsomini M., Garzotto F. (2019). Emotify. IUI 2019 Companion. DOI 10.1145/3308557.3308688
- Spitale M., Catania F., Cosentino G., Gelsomini M., Garzotto F. (2019). WIYE. IUI 2019 Companion. DOI 10.1145/3308557.3308684
- Pandurangan S., Papandrea M., Gelsomini M. (2022). Fine-Grained Human Activity Recognition: A new paradigm. DOI 10.1145/3558884.3558893
