In recent years, sensors from smart consumer devices have shown great
diagnostic potential in movement disorders. In this context, data modalities
such as electronic questionnaires, hand movement and voice captures have
successfully captured biomarkers and allowed discrimination between Parkinson's
disease (PD) and healthy controls (HC) or differential diagnosis (DD). However,
to the best of our knowledge, a comprehensive evaluation of assessments with a
multi-modal smart device system has still been lacking. In a prospective study
exploring PD, we used smartwatches and smartphones to collect multi-modal data
from 504 participants, including PD patients, DD and HC. This study aims to
assess the effect of multi-modal vs. single-modal data on PD vs. HC and PD vs.
DD classification, as well as on PD group clustering for subgroup
identification. We were able to show that by combining various modalities,
classification accuracy improved and further PD clusters were discovered.
In this context, data modalities such as electronic questionnaires, hand movement and voice captures have successfully captured biomarkers and allowed discrimination between Parkinson’s disease (PD) and healthy controls (HC) or differential diagnosis (DD).
In a prospective study exploring PD, we used smartwatches and smartphones to collect multi-modal data from 504 participants, including PD patients, DD and HC.
This study aims to assess the effect of multi-modal vs. single-modal data on PD vs. HC and PD vs. DD classification, as well as on PD group clustering for subgroup identification.
We were able to show that by combining various modalities, classification accuracy improved and further PD clusters were discovered.
様々なモダリティを組み合わせることで,分類精度が向上し,さらにPDクラスタが発見された。
0.72
Keywords Mobile Applications · Machine Learning · Movement Disorders · Parkinson’s Disease
キーワード モバイルアプリケーション ・ 機械学習 ・ 運動障害 ・ パーキンソン病
0.63
1 Introduction Parkinson’s Disease (PD) is a neurodegenerative disorder with well-known symptoms such as slowed movement, rigidity, tremor and various non-motor symptoms (NMS).
The appearance of these symptoms and the disease progress, however, highly differ from patient to patient and clinical documentation does not capture fine-grained objective phenotypical characteristics.
Clinical documentation of motor symptoms, for instance, only describes three main PD subtypes:
例えば、運動症状の臨床文書には3つの主要なpdサブタイプしか記述されていない。
0.56
1) Tremor-dominant PD,
1)Tremor-dominant PD,
0.37
2) Akineto-rigid PD,
2)Akineto-rigid PD,
0.39
3) Mixed/Equivalence type. Although there is no neuroprotective or regenerative treatment to date, early diagnosis and treatment is important in reducing burden and potential treatment costs [1].
Thus, there is a need for early objective biomarkers.
したがって、初期の客観的バイオマーカーが必要である。
0.69
Various systems have already demonstrated promising diagnostic potential when analyzing data modalities like electronic questionnaires, hand movement and voice captures [2, 3, 4].
These studies were able to differentiate between PD and healthy subjects based on digital biomarkers, yielding an important step towards potential clinical adaptation.
However, to the best of our knowledge, there is still a lack of comprehensive evaluation of combinations of these biomarkers in an interactive smart-device-based assessment setting.
In a compact assessment, the SDS was used to record self-completed electronic questionnaires and smartwatch-based sensor measures from a series of movement tasks.
Given this system, we have recorded a total of 503 patient sessions in a prospective study from 2018 to 2021.
本システムでは,2018年から2021年にかけて,503件の患者セッションを記録している。
0.68
Based on this study data, we have already found high diagnostic potential utilizing Machine Learning (ML) methods with movement and questionnaire data [5].
In a later stage of the study, further selected modalities were added to the assessments, these are speech recordings and a smartphone-based finger tapping task.
In this work, we analyze the advances of the multi-modality of our system.
本研究では,本システムにおけるマルチモーダリティの進歩を分析する。
0.77
We therefore 1) train ML models to discriminate PD from healthy controls and other movement disorders, and
したがって 1)健康管理や他の運動障害からPDを識別するためのMLモデルを訓練し、
0.66
2) perform cluster analysis within the PD group.
2) PDグループ内でクラスタ分析を行う。
0.83
Our research question is whether the usage of the multi-modal data compared to
我々の研究課題は、マルチモーダルデータの利用が比較されるかどうかである。
0.65
英語（論文から抽出）
日本語訳
スコア
single-modal data increases information gain and thus
シングルモーダルデータにより情報ゲインが増加し
0.68
1) improves diagnostic accuracy when combined, and
1)組み合わせによる診断精度の向上、及び
0.85
2) lets us discover distinguishable PD subgroups that go beyond the aforementioned three clinically established main types.
2) 臨床で確立された3つの主要型を超える識別可能なpdサブグループを見いだせる。
0.74
2 Study Data The study has been registered (ClinicalTrials.gov ID: NCT03638479) and approved by the ethical board of the University of Münster and the physician’s chamber of Westphalia-Lippe (Reference number: 2018-328-f-S).
The second part consists of 30 yes/no items about NMS based on Chaudhuri et al [7].
第2部はChaudhuri氏らによるNMSに関する30のイエス/ノー項目で構成されている。
0.63
2. Smartwatch-recorded movement tasks: 11 different movement tasks of 10 to 20 seconds length were performed with one smartwatch attached to each of the participants wrist respectively.
3. Voice recording: We computed Jitter via autocorrelation on all vocal tasks, measuring the extent of variation
3.音声記録:全ての音声課題の自己相関によりジッタを計算し,変動の程度を計測した
0.76
of the voice range. 4. Finger tapping: We divided the 15 seconds long record in three equal size segments and calculated the
声域のことです 4.指タッピング:15秒のレコードを3つの等間隔で分割し,計算した。
0.60
average speed and total count of display touches in every segment.
各セグメントの平均速度と ディスプレイの総数です
0.62
In addition, we generated a subset for the cluster analysis to account for the small sample size of the PD group with all data modalities (see Table 1).
We used the scikit-learn implementation of the support-vector machine (SVM) [9] and CatBoost, a gradient boosting decision-tree-based model [10].
我々は,scikit-learn実装である support-vector machine (svm) [9] と catboost を用いた。 訳抜け防止モード: scikit - learn implementation of the support - vector machine (svm ) [9] を使用した。 そしてcatboostは、勾配強化決定 - ツリーベースモデル [10] である。
0.62
To evaluate the potential information gain of combining features from different data modalities, we performed an adapted version of classifier stacking.
In this way, we trained one classifier for each data modality respectively and thus can utilize the additional samples for smartwatch and questionnaire data in the training process.
A simple linear model was trained on top of the individual outputs to consider all data modalities in the classification process and compute the final label for the input samples.
Figure 1 summarizes the architecture and the utilized classifiers.
図1はアーキテクチャと利用済みの分類器をまとめたものです。
0.63
To account for sample size differences, we used balanced class weighting in the training process and report results based on balanced classification accuracy.
For evaluation, a 3 times randomly repeated 5-fold cross-validation was used.
評価には,ランダムに5倍のクロスバリデーションを3回繰り返した。
0.65
Two classification tasks were performed: PD vs. HC and PD vs. DD.
PD vs. HC と PD vs. DD の2つの分類課題が実施された。
0.74
Figure 1: Stacking classifier for the classification of PD samples.
図1:PDサンプルの分類のためのスタック化分類器。
0.81
For each data modality, a selected classifier is trained.
各データモダリティに対して、選択された分類器を訓練する。
0.61
The outputs of each subclassifier are forwarded to a logistic regression that performs the final classification.
各サブ分類器の出力は、最終分類を実行するロジスティック回帰に転送される。
0.75
3.1 Results Classification performance has been evaluated for the individual classifiers for each respective data modality and the combination of all features using the previously described stacking approach.
Table 2 summarizes the averaged classification scores from the cross-validation.
表2は、クロスバリデーションから平均的な分類スコアを要約する。
0.60
Table 2: Evaluation of questionnaire, movement, voice and finger tapping data on the sample subset with all data records (44 samples for PD vs. HC, 48 samples for PD vs. DD).
表2:すべてのデータレコードを含むサンプルサブセットのアンケート、運動、音声、指タップデータの評価(44サンプルのpd vs. hc、48サンプルのpd vs. dd)。 訳抜け防止モード: 表2:全データ記録(PD対HCの44サンプル)によるサンプルサブセットの質問紙, 動き, 声, 指のタッピングデータの評価 48例 (PD vs. DD ) であった。
0.84
Performance is measured with balanced accuracy (STD).
性能は平衡精度(STD)で測定される。
0.86
Best results are marked in bold.
最良の結果は太字で示される。
0.71
Task Quest. Mov.
タスククエスト。 モブ。
0.55
Voice Finger Tapping Quest.
音声フィンガータッピングクエスト。
0.48
+ Mov. + Voice + Finger Tapping
+モブ。 +Voice + Finger Tapping
0.35
PD vs. HC 0.843 (0.098) 0.825 (0.112) 0.702 (0.154) 0.6383 (0.157) 0.918 (0.074)
PD vs. HC 0.843 (0.098) 0.825 (0.112) 0.702 (0.154) 0.6383 (0.157) 0.918 (0.074)
4 Clustering We conducted hierarchical clustering within the PD group using the scikit-learn implementation of the agglomerative cluster algorithm [9].
To analyze information gain through multi-modality, we compared clustering results of a single data modality (movement features) with multiple data modalities (movement, voice, finger tapping and questionnaire features).
The optimal number of clusters was determined from dendrograms by identifying the longest distance between joined clusters.
結合したクラスター間の最長距離を同定し,デンドログラムからクラスタの最適数を決定した。
0.72
For each cluster, we summarized the cluster composition by distinguishing between the clinically established PD types: The tremor-dominant type (T-type), the akineto-rigid type (AR-type) and the equivalence type (ART-type).
(a) a single data modality (movement features) and
(a)単一のデータモダリティ(動作特徴)及び
0.74
(b) multiple data modalities (movement, finger tapping, voice and questionnaire features).
(b)複数のデータモダリティ(移動、指のタップ、音声、アンケート機能)
0.70
In the single-modal analysis, clusters were labeled with the letter S (e g cluster S1), in the multi-modal analysis with the letter M (e g cluster M1).
単一モーダル解析では、クラスタを文字S(egクラスタS1)でラベル付けし、M(egクラスタM1)でマルチモーダル解析を行った。 訳抜け防止モード: 単一のモーダル解析では、クラスタは文字S(e g cluster S1 )でラベル付けされた。 multi-modal analysis with the letter M ( e g cluster M1 )
0.86
Table 3 displays the corresponding cluster composition.
テーブル3は、対応するクラスタ構成を表示する。
0.65
Figure 2: Dendrograms of the hierarchical cluster analysis for
図2:階層的クラスター分析のデンドログラム
0.68
(a) single-modal data (movement features) and
(a)シングルモーダルデータ(動作特徴)及び
0.78
(b) multi-modal data (movement, finger tapping, voice and questionnaire features).
b)マルチモーダルデータ(動作,指のタップ,音声,アンケート機能)
0.67
The gray horizontal line intersects the largest vertical distance between joined clusters.
灰色の横線は結合したクラスター間の最大垂直距離を交差する。
0.74
(a) (b) Table 3: The cluster composition by PD types corresponding to the cluster analysis in Figure 2.
(a) (b) 表3: 図2のクラスタ分析に対応するPDタイプによるクラスタ構成。
0.53
All values are given in percent and rounded to the second decimal place.
すべての値がパーセントで与えられ、第2の十進数に丸まる。
0.69
Movement Movement, Finger Tapping, Voice, Questionnaire
When using a single modality, the questionnaire classifier achieved the highest balanced accuracy in both classification task with 84.3% for PD vs. HC and 66.7% for PD vs. DD.
These results support our hypothesis that the proposed data recordings add informational value to the system, allowing more accurate discrimination of PD from healthy controls and, in particular, from other movement disorders.
Further, we observed that PD vs. DD generally yield far less accurate classification results, indicating that more research is needed to precisely characterize and distinct PD from other similar disorders.
that smartwatch based features capture the clinically established PD types.
スマートウォッチベースの機能は、臨床的に確立されたPDタイプをキャプチャします。
0.47
The cluster analysis with multiple data modalities resulted in a finer subdivision of participants.
複数のデータモダリティによるクラスタ分析の結果,参加者の細分化が生じた。
0.79
Cluster M2 included the same samples as cluster S1.
クラスタM2はクラスタS1と同じサンプルを含んでいる。
0.69
The remaining samples formed three additional cluster.
残りのサンプルは3つのクラスターを形成した。
0.61
Cluster M4 consisted equally of the AR- and the ART-type, whereas cluster M1 and cluster M3 consisted of the AR-type, the ART-type and samples labeled Unknown.
Because each cluster grouped at least two different PD types, we hypothesize that clusters cannot be explained by the clinically established PD types alone.
A limitation of our analysis is the relatively small sample size for the clustering within PD patients.
本分析の限界は, PD患者のクラスタリングにおいて, 比較的小さなサンプルサイズである。
0.85
Therefore, to find a stable and representative set of digital biomarkers, further evaluation with more multi-modal measurements - preferably > 200 PD participants and as many controls - is required.
6 Conclusion We have conducted a study with a multi-modal recording system based on mobile smart devices to research a broad phenotypical spectrum of PD.
In this work, we evaluated the information gain that results from using data from different modalities, including questionnaires, movement recordings, voice captures and smart-phone based finger tapping.
More importantly, we have seen a similar improvement in the classification between PD and the DD group, which consists of other movement disorders.
さらに,他の運動障害を含むDD群とPD群の分類に類似した改善が見られた。
0.66
These results indicate that the different data modalities complement each other and in this way aid in characterizing PD more precisely when comparing it to other disorders.
The second observation is related to the cluster analysis.
第2の観測はクラスタ分析に関連している。
0.82
Our methods have shown that we were able to identify certain subgroups within the PD group when utilizing movement data.
提案手法は, 移動データを利用した場合, pd群内の特定の部分群を同定できることを示した。
0.69
These representations are in line with medical expectation as PD is medically categorized based on movement symptoms.
PDは運動症状に基づいて医学的に分類されるため、これらの表現は医学的な期待と一致している。
0.52
However, when adding additional data modalities to the clustering, we observed a finer subdivision between clusters.
しかし,クラスタリングに付加的なデータモダリティを加えると,クラスタ間の細かな分割が観察される。
0.79
This observation indicates that there are potentially more PD sub-phenotypes beyond the well-known movement-based classifications.
この観察は、よく知られた運動に基づく分類以上のPDサブフェノタイプが存在する可能性を示唆している。
0.58
Finding and specifying such yet unknown groups could strongly aid in more personalized PD treatment.
このような未知のグループを見つけて特定することは、よりパーソナライズされたPD治療に強く役立つ。
0.52
As our system is fully based on consumer-grade devices, it could easily be integrated to support early diagnosis and disease monitoring by giving relevant indications from combinatory digital biomarkers.
[4] Benoit Carignan, Jean-François Daneault, and Christian Duval.
4]ベノワ・カリニャン、ジャン=フランソワ・ダノー、クリスチャン・デュヴァル。
0.51
Measuring tremor with a smartphone.
スマートフォンで震えを測定する。
0.68
In Mobile Health Technologies, pages 359–374.
モバイルでは ヘルス・テクノロジーズ 359-374頁。
0.66
Springer, 2015. [5] Julian Varghese, Catharina Marie van Alen, Michael Fujarski, Georg Stefan Schlake, Julitta Sucker, Tobias Warnecke, and Christine Thomas.
2015年、春。 5]Julian Varghese, Catharina Marie van Alen, Michael Fujarski, Georg Stefan Schlake, Julitta Sucker, Tobias Warnecke, Christine Thomas。
0.47
Sensor validation and diagnostic potential of smartwatches in movement disorders.
運動障害におけるスマートウォッチのセンサ検証と診断可能性
0.74
Sensors, 21(9):3139, 2021.
センサー21(9):3139,2021。
0.70
[6] Roongroj Bhidayasiri and Daniel Tarsy.
6]Roongroj Bhidayasiri氏とDaniel Tarsy氏。
0.41
Parkinson’s disease: Hoehn and yahr scale.
パーキンソン病: hoehnとyahr scale。
0.53
In Movement Disorders: A Video Atlas, pages 4–5.
運動障害におけるA ビデオ・アトラス 4-5頁。
0.66
Springer, 2012.
スプリンガー、2012年。
0.49
[7] Kallol Ray Chaudhuri, Pablo Martinez-Martin, Anthony HV Schapira, Fabrizio Stocchi, Kapil Sethi, Per Odin, Richard G Brown, William Koller, Paolo Barone, Graeme MacPhee, et al International multicenter pilot study of the first comprehensive self-completed nonmotor symptoms questionnaire for parkinson’s disease: the nmsquest study.
Kallol Ray Chaudhuri, Pablo Martinez-Martin, Anthony HV Schapira, Fabrizio Stocchi, Kapil Sethi, Per Odin, Richard G Brown, William Koller, Paolo Barone, Graeme MacPhee, et al International multicenter pilot study of the first comprehensive self-completed nonmotor symptoms symptoms for Parkinson's disease: the nmsquest study。 訳抜け防止モード: [7 ]Kallol Ray Chaudhuri, Pablo Martinez - Martin Anthony HV Schapira, Fabrizio Stocchi, Kapil Sethi, Per Odin Richard G Brown, William Koller, Paolo Barone, Graeme MacPhee パーキンソン病に対する最初の総合的自己-完結した非運動者症状の多施設試験 : nmsquest 研究
0.73
Movement disorders: official journal of the Movement Disorder Society, 21(7):916–923, 2006.
運動障害:the movement disorder society, 21(7):916–923, 2006年。
0.71
5
5
0.42
英語（論文から抽出）
日本語訳
スコア
[8] Julian Varghese, Stephan Niewöhner, Iñaki Soto-Rey, Stephanie Schipmann-Mileti´c, Nils Warneke, Tobias Warnecke, and Martin Dugas.
Julian Varghese, Stephan Niewöhner, Iñaki Soto-Rey, Stephanie Schipmann-Mileti ́c, Nils Warneke, Tobias Warnecke, Martin Dugas 訳抜け防止モード: [8 ]Julian Varghese, Stephan Niewöhner, Iñaki Soto - Rey, Stephanie Schipmann - Mileti ́c, Nils Warneke, Tobias Warnecke マーティン・デュガスとも。
0.83
A smart device system to identify new phenotypical characteristics in movement disorders.
運動障害における新しい表現型特徴を識別するスマートデバイスシステム
0.80
Frontiers in neurology, 10:48, 2019.
神経学のフロンティア、2019年10:48。
0.53
[9] Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake VanderPlas, Arnaud Joly, Brian Holt, and Gaël Varoquaux.
9] Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake VanderPlas, Arnaud Joly, Brian Holt, Gaël Varoquaux 訳抜け防止モード: 9]lars buitinck, gilles louppe, mathieu blondel fabian pedregosa, andreas mueller, olivier grisel, vlad niculae ピーター・プレッテンホーファー アレキサンダー・グラフォート ジャケス・グロブラー ロバート・レイトン jake vanderplas氏、arnaud joly氏、brian holt氏、gaël varoquaux氏。
0.59
API design for machine learning software: experiences from the scikit-learn project.
機械学習ソフトウェアのためのAPI設計: scikit-learnプロジェクトの経験。
0.90
In ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pages 108–122, 2013.
ECML PKDD Workshop: Languages for Data Mining and Machine Learning, page 108–122, 2013
0.41
[10] Anna Veronika Dorogush, Vasily Ershov, and Andrey Gulin.
10]アンナ・ヴェロニカ・ドログシュ、ヴァシーリー・エルショフ、アンドレイ・グリン
0.53
Catboost: gradient boosting with categorical features