Abstract: Natural language processing (NLP) is becoming an important means for
automatic recognition of human traits and states, such as intoxication,
presence of psychiatric disorders, presence of airway disorders and states of
stress. Such applications have the potential to be an important pillar for
online help lines, and may gradually be introduced into eHealth modules.
However, NLP is language specific and for languages such as Dutch, NLP models
are scarce. As a result, recent Dutch NLP models have a low capture of long
range semantic dependencies over sentences. To overcome this, here we present
belabBERT, a new Dutch language model extending the RoBERTa architecture.
belabBERT is trained on a large Dutch corpus (+32 GB) of web crawled texts. We
applied belabBERT to the classification of psychiatric illnesses. First, we
evaluated the strength of text-based classification using belabBERT, and
compared the results to the existing RobBERT model. Then, we compared the
performance of belabBERT to audio classification for psychiatric disorders.
Finally, a brief exploration was performed, extending the framework to a hybrid
text- and audio-based classification. Our results show that belabBERT
outperformed the current best text classification network for Dutch, RobBERT.
belabBERT also outperformed classification based on audio alone.