Machine learning for enhancing dementia screening in ageing deaf signers of British Sign Language

Anastasia Angelopoulou; Reda Al bata; Xing Liang; Bencie Woll; Epaminondas Kapetanios

Machine learning for enhancing dementia screening in ageing deaf signers of British Sign Language

Anastasia Angelopoulou, Reda Al bata, Xing Liang, Bencie Woll, Epaminondas Kapetanios

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Real-time hand movement trajectory tracking based on machine learning approaches may assist the early identification of dementia in ageing deaf individuals who are users of British Sign Language (BSL), since there are few clinicians with appropriate communication skills, and a shortage of sign language interpreters. In this paper, we introduce an automatic dementia screening system for ageing Deaf signers of BSL, using a Convolutional Neural Network (CNN) to analyse the sign space envelope and facial expression of BSL signers recorded in normal 2D videos from the BSL corpus. Our approach involves the introduction of a sub-network (the multi-modal feature extractor) which includes an accurate real-time hand trajectory tracking model and a real-time landmark facial motion analysis model. The experiments show the effectiveness of our deep learning based approach in terms of sign space tracking, facial motion tracking and early stage dementia performance assessment tasks.

Original language	English
Title of host publication	Published in: Efthimiou, Eleni , Fotinea, Stavroula-Evita , Hanke, Thomas , Hochgesang, Julie A. , Kristoffersen, Jette and Mesch , Johanna, (eds.) (2020) Proceedings of the LREC 2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. Paris, France : European Language Resources Association (ELRA). pp. 135-138. ISBN 9791095546542. Organising Body: Language Resources and Evaluation Conference Organising Body: Language Resources and Evaluation Conference
Publication status	Published - 11 May 2020

Bibliographical note

Note: Published in: Efthimiou, Eleni , Fotinea, Stavroula-Evita , Hanke, Thomas , Hochgesang, Julie A. , Kristoffersen, Jette and Mesch
, Johanna, (eds.) (2020) Proceedings of the LREC 2020
9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. Paris, France : European Language Resources Association (ELRA).
pp. 135-138. ISBN 9791095546542.

Organising Body: Language Resources and Evaluation Conference

Organising Body: Language Resources and Evaluation Conference

Keywords

Real-time hand tracking
Facial analysis
British Sign Language
dementia
Convolutional neural network
Computer science and informatics

Access to Document

https://lrec2020.lrec-conf.org/media/proceedings/Workshops/Books/SIGN2020book.pdf

1 Paper

Machine learning for enhancing dementia screening in ageing deaf signers of British Sign Language
Angelopoulou, A., Al bata, R., Liang, X., Woll, B. & Kapetanios, E., 11 May 2020.
Research output: Contribution to conference › Paper › peer-review

Cite this

Angelopoulou, A., Al bata, R., Liang, X., Woll, B., & Kapetanios, E. (2020). Machine learning for enhancing dementia screening in ageing deaf signers of British Sign Language. In Published in: Efthimiou, Eleni , Fotinea, Stavroula-Evita , Hanke, Thomas , Hochgesang, Julie A. , Kristoffersen, Jette and Mesch , Johanna, (eds.) (2020) Proceedings of the LREC 2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. Paris, France : European Language Resources Association (ELRA). pp. 135-138. ISBN 9791095546542. Organising Body: Language Resources and Evaluation Conference Organising Body: Language Resources and Evaluation Conference https://lrec2020.lrec-conf.org/media/proceedings/Workshops/Books/SIGN2020book.pdf

Angelopoulou, Anastasia ; Al bata, Reda ; Liang, Xing et al. / Machine learning for enhancing dementia screening in ageing deaf signers of British Sign Language. Published in: Efthimiou, Eleni , Fotinea, Stavroula-Evita , Hanke, Thomas , Hochgesang, Julie A. , Kristoffersen, Jette and Mesch , Johanna, (eds.) (2020) Proceedings of the LREC 2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. Paris, France : European Language Resources Association (ELRA). pp. 135-138. ISBN 9791095546542. Organising Body: Language Resources and Evaluation Conference Organising Body: Language Resources and Evaluation Conference. 2020.

@inproceedings{f76ed777cde34e84bc0bc06f113f1ccf,

title = "Machine learning for enhancing dementia screening in ageing deaf signers of British Sign Language",

abstract = "Real-time hand movement trajectory tracking based on machine learning approaches may assist the early identification of dementia in ageing deaf individuals who are users of British Sign Language (BSL), since there are few clinicians with appropriate communication skills, and a shortage of sign language interpreters. In this paper, we introduce an automatic dementia screening system for ageing Deaf signers of BSL, using a Convolutional Neural Network (CNN) to analyse the sign space envelope and facial expression of BSL signers recorded in normal 2D videos from the BSL corpus. Our approach involves the introduction of a sub-network (the multi-modal feature extractor) which includes an accurate real-time hand trajectory tracking model and a real-time landmark facial motion analysis model. The experiments show the effectiveness of our deep learning based approach in terms of sign space tracking, facial motion tracking and early stage dementia performance assessment tasks.",

keywords = "Real-time hand tracking, Facial analysis, British Sign Language, dementia, Convolutional neural network, Computer science and informatics",

author = "Anastasia Angelopoulou and \{Al bata\}, Reda and Xing Liang and Bencie Woll and Epaminondas Kapetanios",

note = "Note: Published in: Efthimiou, Eleni , Fotinea, Stavroula-Evita , Hanke, Thomas , Hochgesang, Julie A. , Kristoffersen, Jette and Mesch , Johanna, (eds.) (2020) Proceedings of the LREC 2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. Paris, France : European Language Resources Association (ELRA). pp. 135-138. ISBN 9791095546542. Organising Body: Language Resources and Evaluation Conference Organising Body: Language Resources and Evaluation Conference",

year = "2020",

month = may,

day = "11",

language = "English",

booktitle = "Published in: Efthimiou, Eleni , Fotinea, Stavroula-Evita , Hanke, Thomas , Hochgesang, Julie A. , Kristoffersen, Jette and Mesch , Johanna, (eds.) (2020) Proceedings of the LREC 2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. Paris, France : European Language Resources Association (ELRA). pp. 135-138. ISBN 9791095546542. Organising Body: Language Resources and Evaluation Conference Organising Body: Language Resources and Evaluation Conference",

}

Angelopoulou, A, Al bata, R, Liang, X, Woll, B & Kapetanios, E 2020, Machine learning for enhancing dementia screening in ageing deaf signers of British Sign Language. in Published in: Efthimiou, Eleni , Fotinea, Stavroula-Evita , Hanke, Thomas , Hochgesang, Julie A. , Kristoffersen, Jette and Mesch , Johanna, (eds.) (2020) Proceedings of the LREC 2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. Paris, France : European Language Resources Association (ELRA). pp. 135-138. ISBN 9791095546542. Organising Body: Language Resources and Evaluation Conference Organising Body: Language Resources and Evaluation Conference. <https://lrec2020.lrec-conf.org/media/proceedings/Workshops/Books/SIGN2020book.pdf>

Machine learning for enhancing dementia screening in ageing deaf signers of British Sign Language. / Angelopoulou, Anastasia; Al bata, Reda; Liang, Xing et al.
Published in: Efthimiou, Eleni , Fotinea, Stavroula-Evita , Hanke, Thomas , Hochgesang, Julie A. , Kristoffersen, Jette and Mesch , Johanna, (eds.) (2020) Proceedings of the LREC 2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. Paris, France : European Language Resources Association (ELRA). pp. 135-138. ISBN 9791095546542. Organising Body: Language Resources and Evaluation Conference Organising Body: Language Resources and Evaluation Conference. 2020.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Machine learning for enhancing dementia screening in ageing deaf signers of British Sign Language

AU - Angelopoulou, Anastasia

AU - Al bata, Reda

AU - Liang, Xing

AU - Woll, Bencie

AU - Kapetanios, Epaminondas

N1 - Note: Published in: Efthimiou, Eleni , Fotinea, Stavroula-Evita , Hanke, Thomas , Hochgesang, Julie A. , Kristoffersen, Jette and Mesch , Johanna, (eds.) (2020) Proceedings of the LREC 2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. Paris, France : European Language Resources Association (ELRA). pp. 135-138. ISBN 9791095546542. Organising Body: Language Resources and Evaluation Conference Organising Body: Language Resources and Evaluation Conference

PY - 2020/5/11

Y1 - 2020/5/11

N2 - Real-time hand movement trajectory tracking based on machine learning approaches may assist the early identification of dementia in ageing deaf individuals who are users of British Sign Language (BSL), since there are few clinicians with appropriate communication skills, and a shortage of sign language interpreters. In this paper, we introduce an automatic dementia screening system for ageing Deaf signers of BSL, using a Convolutional Neural Network (CNN) to analyse the sign space envelope and facial expression of BSL signers recorded in normal 2D videos from the BSL corpus. Our approach involves the introduction of a sub-network (the multi-modal feature extractor) which includes an accurate real-time hand trajectory tracking model and a real-time landmark facial motion analysis model. The experiments show the effectiveness of our deep learning based approach in terms of sign space tracking, facial motion tracking and early stage dementia performance assessment tasks.

AB - Real-time hand movement trajectory tracking based on machine learning approaches may assist the early identification of dementia in ageing deaf individuals who are users of British Sign Language (BSL), since there are few clinicians with appropriate communication skills, and a shortage of sign language interpreters. In this paper, we introduce an automatic dementia screening system for ageing Deaf signers of BSL, using a Convolutional Neural Network (CNN) to analyse the sign space envelope and facial expression of BSL signers recorded in normal 2D videos from the BSL corpus. Our approach involves the introduction of a sub-network (the multi-modal feature extractor) which includes an accurate real-time hand trajectory tracking model and a real-time landmark facial motion analysis model. The experiments show the effectiveness of our deep learning based approach in terms of sign space tracking, facial motion tracking and early stage dementia performance assessment tasks.

KW - Real-time hand tracking

KW - Facial analysis

KW - British Sign Language

KW - dementia

KW - Convolutional neural network

KW - Computer science and informatics

M3 - Conference contribution

BT - Published in: Efthimiou, Eleni , Fotinea, Stavroula-Evita , Hanke, Thomas , Hochgesang, Julie A. , Kristoffersen, Jette and Mesch , Johanna, (eds.) (2020) Proceedings of the LREC 2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. Paris, France : European Language Resources Association (ELRA). pp. 135-138. ISBN 9791095546542. Organising Body: Language Resources and Evaluation Conference Organising Body: Language Resources and Evaluation Conference

ER -

Angelopoulou A, Al bata R, Liang X, Woll B, Kapetanios E. Machine learning for enhancing dementia screening in ageing deaf signers of British Sign Language. In Published in: Efthimiou, Eleni , Fotinea, Stavroula-Evita , Hanke, Thomas , Hochgesang, Julie A. , Kristoffersen, Jette and Mesch , Johanna, (eds.) (2020) Proceedings of the LREC 2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. Paris, France : European Language Resources Association (ELRA). pp. 135-138. ISBN 9791095546542. Organising Body: Language Resources and Evaluation Conference Organising Body: Language Resources and Evaluation Conference. 2020

Machine learning for enhancing dementia screening in ageing deaf signers of British Sign Language

Abstract

Bibliographical note

Keywords

Access to Document

Fingerprint

Research output

Machine learning for enhancing dementia screening in ageing deaf signers of British Sign Language

Cite this