Artificial intelligence literature suggests that minority and fragile
communities in society can be negatively impacted by machine learning
algorithms due to inherent biases in the design process, which lead to socially
exclusive decisions and policies. Faced with similar challenges in dealing with
an increasingly diversified audience, the museum sector has seen changes in
theory and practice, particularly in the areas of representation and
meaning-making. While rarity and grandeur used to be at the centre stage of the
early museum practices, folk life and museums' relationships with the diverse
communities they serve become a widely integrated part of the contemporary
practices. These changes address issues of diversity and accessibility in order
to offer more socially inclusive services. Drawing on these changes and
reflecting back on the AI world, we argue that the museum experience provides
useful lessons for building AI with socially inclusive approaches, especially
in situations in which both a collection and access to it will need to be
curated or filtered, as frequently happens in search engines, recommender
systems and digital libraries. We highlight three principles: (1) Instead of
upholding the value of neutrality, practitioners are aware of the influences of
their own backgrounds and those of others on their work. By not claiming to be
neutral but practising cultural humility, the chances of addressing potential
biases can be increased. (2) There should be room for situational
interpretation beyond the stages of data collection and machine learning.
Before applying models and predictions, the contexts in which relevant parties
exist should be taken into account. (3) Community participation serves the
needs of communities and has the added benefit of bringing practitioners and
communities together.
Social Inclusion in Curated Contexts: Insights from Museum Practices
キュレートされた文脈における社会的包摂性:博物館の実践から
0.50
HAN-YIN HUANG, Delft University of Technology, The Netherlands CYNTHIA C. S. LIEM, Delft University of Technology, The Netherlands Artificial intelligence literature suggests that minority and fragile communities in society can be negatively impacted by machine learning algorithms due to inherent biases in the design process, which lead to socially exclusive decisions and policies.
HAN-YIN HUANG, Delft University of Technology, The Netherlands CYNTHIA C. S. LIEM, Delft University of Technology, The Netherlands 人工知能文学は、社会の少数派と脆弱なコミュニティは、デザインプロセスに固有の偏りがあり、社会的に排他的な決定や政策につながることを示唆している。
0.77
Faced with similar challenges in dealing with an increasingly diversified audience, the museum sector has seen changes in theory and practice, particularly in the areas of representation and meaning-making.
While rarity and grandeur used to be at the centre stage of the early museum practices, folk life and museums’ relationships with the diverse communities they serve become a widely integrated part of the contemporary practices.
Drawing on these changes and reflecting back on the AI world, we argue that the museum experience provides useful lessons for building AI with socially inclusive approaches, especially in situations in which both a collection and access to it will need to be curated or filtered, as frequently happens in search engines, recommender systems and digital libraries.
We highlight three principles: (1) Instead of upholding the value of neutrality, practitioners are aware of the influences of their own backgrounds and those of others on their work.
CCS Concepts: • General and reference → Design; • Computing methodologies → Philosophical/theore tical foundations of artificial intelligence; • Applied computing → Arts and humanities.
In many classification tasks, the intention is to automatically label digital objects in ways that scale beyond human capacity.
多くの分類タスクでは、人間の能力を超えたスケールでデジタルオブジェクトを自動的にラベル付けする。
0.83
The necessary standardisation in this procedure has both been lauded for its efficiency, yet criticised for its taxonomic crudeness, which tends to be particularly oblivious to cultural and contextual sensitivities and minority perspectives.
Further downstream, machine learning technology empowers how vast collections of automatically describable digital objects get organised and re-organised, and how, when a human seeks to interact with the collections, they will be filtered down to much smaller-sized ‘relevant’ sub-selections.
On the positive side, automated filtering technologies
ポジティブな側面として、自動フィルタリング技術は
0.73
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.
FAccT ’22, June 21–24, 2022, Seoul, Republic of Korea
FAccT'22, 6月21-24, 2022, ソウル, 大韓民国
0.85
Han-Yin Huang and Cynthia C. S. Liem
Han-Yin HuangとCynthia C. S. Liem
0.37
can traverse digital collections more broadly and efficiently than humans can do.
デジタルコレクションを人間より広範に効率よく 横切ることができます
0.69
Generally, through digitalisation and unified access mechanisms, collection items that would be hard to reach in physical archives can now universally and globally be retrieved.
Yet, on the negative side, filtering and access mechanisms in our digital information space may not actually capitalise on the diversity they could in principle offer.
Many services do not naturally stimulate their audiences to venture beyond the filtered sub-selections offered to them, and perspectives provoking the strongest engagement easily get prioritised, at the risk of favoring ‘majority-group’ or heavily polarising world views.
Often, we face large collections reflecting a pluriformity of perspectives and interpretations, while people interested in interacting with these collections will only get to see limited selections of items.
How can curation be performed in ways that respect, engage and include audiences beyond mainstream perspectives, without necessarily going down populist routes?
When not applied consciously, AI technology may however actively work against these challenges, and it still is an open question how to avoid this.
しかし、意識的に適用されない場合、AI技術はこれらの課題に対して活発に機能する可能性がある。
0.64
However, in the physical world, over the last decades, similar questions and challenges have actually been posed and addressed in the museum community.
For at least two decades, the museum sector has been confronted with debates on its social role, potential biases in its work and issues of social inclusion and cultural diversity [12, 26, 32].
According to the current museum definition of the International Council for Museums (ICOM), a museum is an institution that ‘acquires, conserves, researches, communicates and exhibits’ both the intangible and tangible heritage [22].
In this process, a museum decides what is accepted into its collection and what is deemed significant heritage for the past, present and future generations.
Museums are, therefore, institutions with the power to endorse views.
したがって、博物館は見解を支持する力を持つ機関である。
0.66
Traditionally, museums are seen as places that house and exhibit items of historic, scientific or aesthetic values, where the criteria primarily have been based on the dominant groups whose cultures museums were founded on.
Such a curating process can actually be interpreted as one of exclusion (i.e., deciding what does not get to be accepted into a collection, and what thus would not be deemed significant heritage).
Recognising this, various activities and actions have been initiated to still foster social inclusion, as will be discussed in this paper.
これを認識しながら, 社会的包摂性を高めるための様々な活動や行動が, 本稿で論じられている。
0.66
Beyond these various initiatives in different museums, a recent debate surrounding the museum definition proposal of ICOM illustrates the sector’s attempt to officially formalise inclusive practices as part of its raison d’être.
Before its general assembly in 2019, ICOM announced that it would put a new museum definition proposal to be voted on during the assembly.
2019年の総会の前に、icomは議会期間中に新しい博物館の定義案を提出することを発表した。
0.60
This proposed new definition referred to museums as ‘democratising, inclusive and polyphonic spaces’, whose aims are to ‘contribute to human dignity and social justice, global equality and planetary wellbeing’ [14].
While applauded by supporters, the announcement was also met with criticisms, among which the lack of transparency in the decision-making process and insufficient consultation with members and national committees came to the fore.
According to the analysis and report of the ICOM Define Committee’s second consultation with national committees, among the terms mentioned by national committees, many are related to social inclusion and diversity, for example, ‘inclusive’ is mentioned by 66 percent of the national committees, ‘open to society/public’ 52 percent, ‘community/society’ 51 percent, ‘accessibility’ 45 percent, ‘service to society’ 44 percent and ‘diversity’ 41 percent [6].
Social Inclusion in Curated Contexts: Insights from Museum Practices
キュレートされた文脈における社会的包摂性:博物館の実践から
0.50
FAccT ’22, June 21–24, 2022, Seoul, Republic of Korea
FAccT'22, 6月21-24, 2022, ソウル, 大韓民国
0.85
definitions put forward by ICOM in February 2022, museums being inclusive and accessible to a variety of audiences is mentioned in all five definition proposals1.
Both worlds have a similar process, that starts with collecting what is deemed relevant to their purposes: in the museum’s case, the tangible and intangible heritage, and in case of AI technology, the data.
The collections are then interpreted, after which selected items and views are presented to audiences.
コレクションは解釈され、その後選択されたアイテムとビューが聴衆に提示される。
0.71
Drawing on this parallel, we argue that the existing debates and reform movements with regard to the museum experience can provide useful insights for developing socially inclusive approaches in AI.
They identify principles that the machine learning community can learn from, in particular on practices to increase transparency with regard to data collection.
In this paper, we complement these insights by taking a broader process and organisational perspective, looking beyond the collection phase, and also including the interpretation, application and presentation phases, that ultimately reach human audiences.
2 SOCIAL INCLUSION IN MUSEUMS Although, because of its more positive note, social inclusion is a term applied more often in policy and in practice, it is also used interchangeably with social exclusion, social cohesion, solidarity, integration and social capital [31, 40].
First coined in France in the 1970s, social exclusion referred to the marginalised groups of the French welfare system and has since then become part of the European political discussions.
As time went by, the discussion in policy moved to the more positive ‘social inclusion’ [40].
時が経つにつれ、政策に関する議論はよりポジティブな「社会的包摂性」へと移行した [40]。
0.75
According to Silver, social inclusion has different connotations and applications in different settings on the national and neighbourhood levels, and is also located in place [40].
Social inclusion is therefore a context-specific term.
したがって、社会的包含は文脈固有の用語である。
0.52
To many cultural institutions, what social inclusion entails is extremely fluid, and it thus is difficult to give a definition that fits all practices [31, 40].
Richard Sandell is one of the early museologists who champions museums as ‘agents of social inclusion’ and suggests that museums nowadays are involved in or committed to different degrees of socially inclusive practices and ideals [34].
Museums as agents of social regeneration take on the social, economic and political dimensions and seek to promote better quality of lives for individuals, also encouraging personal development of the disadvantaged groups.
Social inclusion in museum practice responds at first to the discussions of museum accessibility for a wider range of visitors (cultural dimension) and later includes the discussion the capacity of museums empowering its visitors (social, economic and political dimensions).
Engaging social inclusion in the museum sector has become an important part of the discussion about democratising the museum practice [35].
博物館における社会的包摂性は,博物館の実践の民主化に関する議論の重要な部分となっている[35]。
0.71
In contrast to its traditional image being situated in a power structure that posits museum staff as experts and a top-down practice model, the democratised museum practice seeks to revisit accessibility to the museum not just in terms of physical access, but also in terms of social, cultural and economic access, changing its relationship with the communities that have long been underrepresented by the museum sector.
The final report is scheduled to be published in May 2022.
最終報告は2022年5月に公開される予定である。
0.70
3
3
0.42
英語（論文から抽出）
日本語訳
スコア
FAccT ’22, June 21–24, 2022, Seoul, Republic of Korea
FAccT'22, 6月21-24, 2022, ソウル, 大韓民国
0.85
Han-Yin Huang and Cynthia C. S. Liem
Han-Yin HuangとCynthia C. S. Liem
0.37
strands of influences that intertwine to make socially inclusive practice an important aspect in the contemporary museum practice [25].
現代美術館の実践において,社会的な包括的実践に介入する影響が重要である[25]。
0.84
As the targeted audience shifts, from the elite educated and knowledgeable group to a more general public without specialist knowledge, subject matters also shift to include folk life and everyday collecting: the ordinary heritage that the visitors can recognise themselves in.
The third strand is the influence of the environmental movements in the 1960s and discussions on these in the museum sector, in the form of new museology [46] for ecomuseology [10].
第3のストランドは、1960年代の環境運動の影響と、博物館におけるそれらの議論であり、新しい博物館学 [46] for ecomuseology [10] の形式である。
0.69
At the core of the ecomuseum philosophy is the museum’s thinking beyond the confinement of its four walls, building relationships with communities.
Development in museum learning theories also informs new perspectives in the meaning-making of museum visits.
博物館学習理論の発展は、博物館訪問の意味づけにおける新しい視点を示唆する。
0.69
Literature of how museum visitors make sense of the messages conveyed in exhibitions and programmes suggests that there is no one universal manner of museum visit experience.
Visitors go to museums for different purposes with various expectations [13, 19].
来館者は様々な目的のために美術館に行く[13, 19].
0.76
It is therefore impossible for museums to create a ‘one thing fits all’ programme.
したがって、博物館が‘すべてに合致する’プログラムを作ることは不可能だ。
0.82
The visitors who come through the gate also bring with them their own personal, cultural, social and political context.
門を通ってくる訪問者は、彼ら自身の個人的、文化的、社会的、政治的文脈も持ってくる。
0.56
Every visitor can experience a museum visit very differently.
どの訪問者も美術館の訪問を全く異なる形で体験することができる。
0.52
To illustrate how these notions were implemented for socially inclusive museum experiences, we discuss three key principles, which in our perspective can be translated to parallels in AI practice.
First of all, we describe how instead of setting neutrality as the ultimate goal, an attitudinal change informed by cultural humility may provide better guidance.
Secondly, allowing room for situational interpretation at different stages of the development process ensures that different contexts are considered and acknowledged.
This implies that potential biases should keep being readdressed throughout the whole chain of services; that is, not only at the data collection, but also during the design, interpretation and implementation stages.
Lastly, it is important to actively engage communities so that the practice connects with the community experience.
最後に,実践がコミュニティ体験と結びつくように,コミュニティに積極的に関与することが重要である。
0.78
3 THE MUSEUM INSIGHTS 3.1 Neutrality Revisited As discussed earlier, to become more socially inclusive, a sectoral change is needed to transform the traditional museum practice.
3 The MUSEUM INSIGHTS 3.1 Neutrality Revisited 先に述べたように、より社会的に包括的になるためには、伝統的な博物館の慣行を変えるために、部分的な変化が必要である。 訳抜け防止モード: 3 MUSEUM INSIGHTS 3.1 Neutrality Revisited 前述したとおり。 より社会的に包摂的になる 伝統的な博物館の慣行を変えるには 部分的な変化が必要です
0.65
This does not trivially happen.
これはささいなことではありません。
0.46
Among the aspects identified by Sandell that inhibit the sectoral change, he first points out the attitudes of museum workers that do not subscribe to social inclusion [35].
To facilitate attitude changes, Sandell suggests training programmes based on first-hand experience and guided by the groups and communities, that have traditionally not seen themselves represented in the museums.
Such attitude change can be summarised as ‘cultural humility’.
このような態度の変化は「文化的謙虚」と要約できる。
0.72
The concept of cultural humility has originally been promoted in health care and library and information science, to tackle oppressive descriptions in the medical and archival practices, respectively.
It refers to the ability to acknowledge the influences personal backgrounds could have on the practitioners and their practices, and finding appropriate approaches to deal with these influences [42].
Museums do not exist in a vacuum and neither do the visitors.
博物館は真空中に存在せず、訪問者も存在しない。
0.71
As Stijn Schoonderwoerd, the general director of National Museum of World Cultures (NMVW), points out, “Museums, although they are often seen as places solely dedicated to beautiful things, and therefore as neutral and non-political spaces, are players in the social and political arena too” [37].
nmvw(national museum of world cultures)の所長であるstijn schoonderwoerd氏が指摘するように、museumsは美しいものだけにのみを捧げる場所と見なされることが多いが、中立的で非政治的な空間として、社会と政治の場におけるプレイヤーでもある [37]。
0.71
A museum collects, interprets and displays tangible and intangible heritage in its care.
博物館は、その管理において有形無形遺産を収集、解釈、展示している。
0.59
Its decisionmaking process illustrates a perspective curated and presented by the museum.
その意思決定プロセスは、博物館がキュレーションし提示した視点を示している。
0.56
The key word here is a perspective, rather than the perspective.
ここでのキーワードは視点ではなく視点です。
0.55
In this process, a museum makes decisions about what the museum deems relevant to the
この過程において、博物館は博物館が関連すると考えるものを決定する。
0.74
4
4
0.42
英語（論文から抽出）
日本語訳
スコア
Social Inclusion in Curated Contexts: Insights from Museum Practices
キュレートされた文脈における社会的包摂性:博物館の実践から
0.50
FAccT ’22, June 21–24, 2022, Seoul, Republic of Korea
FAccT'22, 6月21-24, 2022, ソウル, 大韓民国
0.85
institution in its exhibitions and education and outreach programmes.
展覧会、教育、アウトリーチプログラムの機関。
0.56
Museums function, therefore, in curated contexts that are not without implications of certain values.
したがって、博物館は特定の価値を含まないキュレートされた文脈で機能する。
0.62
Instead of emphasising how ‘neutral’ museums should be, an attitude of ‘cultural humility’ signifies changes in the curatorial practices of museums.
To facilitate an inclusive curatorial practice, the selection processes to decide what goes in the collections are informed by a museum’s mission statement and collection policy.
The collection policy will be reviewed at a regular interval to make sure the museum does not collect too many repeated, similar items, and the policy keeps up with developments within the museum.
Changes can also be seen in the calls for decolonising museum narratives, that go from seeing different cultures as the exotic others, to placing different cultures in their own contexts.
History and the heritage of the ruling and dominant groups are not the only heritage considered ‘qualified’ for museum collections.
支配的・支配的グループの歴史と遺産は、博物館コレクションの「資格」と見なされる唯一の遺産ではない。
0.69
One of the most telling examples in the museums’ efforts of putting cultural humility to practice, is the re-examination of terminology used in the museum sector.
When an item is accepted into the collection, museum curators conduct research into its material culture such as its origin, material, artist, maker and previous collectors, and update relevant findings and literature related to the item.
Derogatory terms in the catalogue descriptions and exhibition texts that are outdated in contemporary society are being identified and updated through research projects in museum terminology.
In the 1970s, the work was titled “Little Negress” by one of the curators then.
1970年代、この作品はキュレーターの1人によって「リトル・ネグレス」と題された。
0.50
In 2015, this title set off a public debate; as a result, the Museum assembled an inter-departmental team tasked to investigate the language used by the museum.
Although the task force originally gave the painting a new title “Young Woman with a Fan”, further research into the documents and communications in the Maris family archives unveiled in 2020 that the girl’s first name is Isabella, and that she was 12 years old when she posed for Maris.
“Isabella” sparked a series of term changes in the museum, for instance, ‘slaves’ are now described as ‘enslaved’ and ‘koelies’ are now referred to as ‘contract workers’.
Eurocentric terms have also been identified; for example, ‘Indians’ are now referred to as ’original residents of America’ and ‘exotic’, an adjective used to describe someone different from the European norm, is now avoided by the museum.
This project produced a work-in-progress document, compiling a list of terms that were previously used but are now identified as derogatory, accompanied by various commenting essays [30].
It also discusses how terminology used in museums have impacts on its work; not just in collection, but also in education and interpretation, and its relationship to the community.
Wayne Modest, one of the authors in the Words Matter document, says in response to controversies in the Netherlands about changing words and texts, that “Paying attention to words means acknowledging that the words we use affect a person or a group feels excluded or included, whether they feel a sense of belonging to society. This is about representation, recognition and respect.” [29]
Words Matterの著者の1人であるWayne Modest氏は、オランダでの言葉とテキストの変更に関する論争に対して、「言葉に注意を払うことは、私たちが使っている言葉が社会に属していると感じようとも、人やグループに影響を及ぼす言葉が除外されたり、含まれていると感じることを意味する」と述べている。
0.71
3.2 Situational Interpretation As mentioned in Section 2, social inclusion is a fluid concept and its definition and practice differ according to the context in which it is applied.
The museum practice of social inclusion goes beyond ensuring that a collection has a diverse representation of its communities and audiences: beyond the collection and curatorial departments, other areas
Apart from changing the outdated languages used in the museum sector, as outlined in Section 3.1, changes can be seen in how curators produce interpretive texts of museum exhibitions with more visitor-oriented approaches.
Accessible texts that are easy for the public to read and relate to, instead of jargon-loaded academic texts, are considered more effective in getting the messages across.
Moreover, there is no guarantee that the texts and messages will be
さらに、テキストとメッセージが保証されない。
0.59
6
6
0.42
英語（論文から抽出）
日本語訳
スコア
Social Inclusion in Curated Contexts: Insights from Museum Practices
キュレートされた文脈における社会的包摂性:博物館の実践から
0.50
FAccT ’22, June 21–24, 2022, Seoul, Republic of Korea
FAccT'22, 6月21-24, 2022, ソウル, 大韓民国
0.85
perceived and understood to the same degree and effect by all visitors [19].
あらゆるビジターから 同じ程度と効果を 認識され 理解されました [19]
0.76
Understanding that meanings of museum displays can be shifting from one visitor to another, museums start exploring the inclusion of texts written by community members, and the engagement of visitors in reflective questions in exhibitions.
In New York, for example, an exhibition by the New York Historical Society, Scenes of New York City, displays art works alongside texts written by New Yorkers who are connected to the subject matters of the works on display4.
For an oil painting by Gifford Beal which features a top-hat-wearing horse carriage driver, visitors see both interpretive texts from the exhibition curator, and from a horse carriage driver in Central Park.
While the curator’s text introduces the painter and touches on the carriage tour history at Central Park, the carriage driver’s text is a personal reflection of spring time and being a carriage driver in warm weather in the park, memories triggered by the painting [23].
Similarly, when museum educators plan to utilise an item in collection as part of their education programme or campaign, the research results (such a descriptive texts in the catalogue or labels from past exhibitions) will form the basis of this endeavour.
They are, however, often adapted in order to become suitable to the learning needs and specific contexts of the target audience.
しかし、それらはしばしば、対象の聴衆の学習ニーズや特定の文脈に適合するように適応される。
0.69
In the process of developing these resources, the collaboration between the curatorial department and the education department makes sure that the produced texts or knowledge promoted is in line with the research result of the curatorial department, but also incorporates the understandings of the education department on how best to convey messages across to the target group to achieve the desired learning outcomes.
With the museum collection as backbones, situational interpretation enables different parts of the museum services to be bridges between the collection and the visitors, by translating into languages or formats that visitors feel comfortable with.
3.3 Community Participation In the effort to offer inclusive practices, the community plays an important role, as informed by the new museology and ecomuseum philosophy.
The ecomuseum philosophy expands on this and argues that museums should not be confined to its four walls.
ecomuseumの哲学は、博物館は4つの壁に限定されるべきではないと主張する。
0.77
Instead of a museum building, an ecomuseum operates within a territory, instead of a building, and has a close tie with the communities whose lives are attached to this territory [7].
This way of collecting is usually a project-based, community-oriented approach to collecting.
このような収集方法は通常、プロジェクトベースのコミュニティ指向の収集アプローチです。
0.71
The contexts of this collecting process are documented through recordings of interviews of relevant parties and descriptions, so that these unique decision-making processes are not lost.
The first year of the project mainly consisted of consultation sessions with focus groups to discuss different themes surrounding the participants’ life experiences in Birmingham.
In the second year, the project team went to different venues and events in the local neighbourhoods.
2年目には、プロジェクトチームは地元の様々な場所やイベントに行きました。
0.65
More contentious objects related to race or religion emerged as a result of the diversity of respondents in the second year [2].
人種や宗教にまつわるより論争的な対象は、第2年度の回答者の多様性の結果として現れた[2]。
0.66
Echt Rotterdams Erfgoed (Authentic Rotterdam Heritage, ERE hereon) is an ongoing project started by Museum Rotterdam in the Netherlands and now is part of the Stichting Wijkcollectie (District collection Foundation) programmes67.
Kamen was travelling back and forth between Bulgaria and The Netherlands with this minivan, transporting his fellow Bulgarians who had been travelling between the two countries.
After identifying an initial collection of 30 items, Museum Rotterdam called on Rotterdam residents for additional proposals, and was overwhelmed by the number of proposals received.
The Museum then decided to put out a call for Rotterdam residents to form a collection committee.
博物館はその後、ロッテルダム住民に収集委員会を設立するよう呼びかけることに決めた。
0.72
The committee has, since 2017, determined collection criteria and met regularly for collection meetings.
委員会は2017年以降、収集基準を決定し、定期的に収集会議に出席している。
0.57
To date, one hundred exemplars of Rotterdam’s living heritage have been included in this collection, such as the Mayor Aboutaleb, who has been Mayor of Rotterdam since 2009, the Pauluskerk, a church that offers overnight shelters and warm food for those in need, and the Summer Carnival Rotterdam, reflecting the multicultural character of the harbour city.
In both examples, the museums rely on the local communities’ insights into lives in their cities, and explicitly let the communities identify what heritage matters to them.
7https://wijkcollect ie.nl/ere/, retrieved April 30, 2022.
7https://wijkcollect ie.nl/ere/ 2022年4月30日閲覧。
0.51
8
8
0.42
英語（論文から抽出）
日本語訳
スコア
Social Inclusion in Curated Contexts: Insights from Museum Practices
キュレートされた文脈における社会的包摂性:博物館の実践から
0.50
FAccT ’22, June 21–24, 2022, Seoul, Republic of Korea
FAccT'22, 6月21-24, 2022, ソウル, 大韓民国
0.85
4 REFLECTING ON AI PRACTICE Having described multiple key principles and actions observed in the museum community to promote social inclusion, we now reflect on ways in which these also can be useful for AI applications.
4 ReFLECTING ON AI PRACTICE 博物館のコミュニティで観察された、社会的包摂を促進するための重要な原則と行動について、我々は現在、これらがAIアプリケーションにも役立つ方法を反映している。
0.63
Here, we especially consider AI applications that involve supervised machine learning procedures on data that relies on human interpretation, and where intended applications will involve curation steps.
4.1 Neutrality Revisited As surveyed by Birhane et al , in many published research works in machine learning, generalisation and efficiency can be recognised as core important values defining achievement [3].
At the same time, dataset construction historically has been less valued and incentivised in the community, as also emphasised by Sambasivan et al [33].
同時に、歴史的にデータセットの構築は、sambasivan et al [33]によって強調されたように、コミュニティでの価値とインセンティブが低下してきた。
0.62
Thus, available large-scale datasets have commonly been reused, becoming ‘naturalised’ and appearing value-neutral in the process [11].
The stance that datasets—and the technologies built on them—are not actually value-neutral, has only recently been gaining traction and debate in the machine learning community [36].
However, this notion has already been recognised as early as in 1996, in Forsythe’s paper outlining insights from three years of anthropological observations in an AI development project for migraine sufferers [15].
Designers might think they cannot build their own cultural backgrounds and values into the system they are developing, but these explicit or tacit intentions often carry hidden presumptions.
These tools are not only ‘located’ in specific design settings, but also in specific practice settings.
これらのツールは、特定のデザイン設定だけでなく、特定のプラクティス設定でも‘位置’される。
0.72
The people who deal with data will consciously or unconsciously be influenced by their own background and those of others.
データを扱う人々は、意識的にも無意識的にも、自分自身のバックグラウンドや他人の影響を受けます。
0.69
By claiming to be neutral or striving to be neutral, the chances of addressing the influences and biases would actually be lost.
中立性を主張したり中立性を目指すことによって、影響や偏見に対処する可能性は実際に失われる。
0.67
As argued by Scheuerman et al [36], such an awareness in documentation practices is key to changing the culture of dataset development among the practitioners.
Thus, bias is not necessarily problematic per se, and always will be present to some degree.
したがって、偏見は必ずしも問題ではなく、常にある程度存在する。
0.57
Both professional archives [24] and museums will explicitly not seek to collect unbiased samples of the broad universe, but will set an explicit collection policy, purposefully making choices according to this policy and the institute’s overall mission statement.
As already argued by Jo and Gebru [24], more transparency on the policies and criteria behind machine learning datasets can be achieved through more explicit policy documentation, and existing proposals on datasheets for datasets [16] and model cards for model reporting [28] can naturally complement this.
It will be beneficial to explicitly write these with a mindset of cultural humility, accepting up front that the currently presented world view is not the one and only, canonical world view, and perspectives may need adjustment over time.
It should be noted that many ‘canonical’, heavily-employed, large-scale machine learning datasets were explicitly not constituted with a mindset of cultural humility.
Multiple datasets in computer vision have been heavily criticised for including problematic and offensive imagery [4] and labels [9], and displaying an Amerocentric and Eurocentric representation bias [39].
The follow-up to these criticisms had various forms.
これらの批判の続行には様々な形態があった。
0.55
Where the work of Shankar et al on geodiversity issues both sparked more inclusive benchmarking efforts [38] and well-publicised commentaries drawing attention to culturally specific AI biases [49], Crawford and Paglen pointing out offensive class labels in ImageNet [9] led to those categories
FAccT ’22, June 21–24, 2022, Seoul, Republic of Korea
FAccT'22, 6月21-24, 2022, ソウル, 大韓民国
0.85
Han-Yin Huang and Cynthia C. S. Liem
Han-Yin HuangとCynthia C. S. Liem
0.37
being removed from the ImageNet public dataset.
ImageNetパブリックデータセットから削除される。
0.75
Even more radically, Birhane and Prabhu’s audit [4] of the Tiny Images dataset [43] led to this dataset to be fully and permanently taken offline.
さらに急進的に、birhane氏とprabhu氏の“4] of the tiny images dataset [43]によって、このデータセットは完全にオフラインにされることになった。
0.68
As discussed in Section 3.1, it is not uncommon in museums that language and representation of collections items used to be accepted at collection time, but today would be considered derogatory and offensive.
For example, the Words Matter document explicitly addresses these terms, but with clear contextualisation of how these terms today are problematic, and a clear underlying message that these terms should not be used anymore.
Contrasting this to the removal actions undertaken on problematic machine learning categories and datasets, while full permanent removal prevents future harmful automatic inferences, it also fully cuts off the connection and context to earlier, existing models that were trained on these datasets.
4.2 Situational Interpretation The previous discussion raised that over time, descriptive language may need revision, updating, and re-tailoring to the presently relevant audiences.
In the discussion in Section 3.2, we raised that for museums, these audiences can be highly diverse, and in need of different languages and perspective highlights.
第3章2節の議論では、これらの観客は多種多様であり、異なる言語や視点のハイライトを必要としている。
0.57
These notions lead to multiple reflections on AI practice.
これらの概念は、AIの実践に複数の反映をもたらす。
0.60
First of all, in museums, the same collection item may be described in different ways, or purposefully draw attention to different aspects of the item.
When seeing a picture of a carriage driver with a top hat in Central Park, some people will focus on characteristics of the carriage, some will reminiscence about being a carriage driver, and some may remember a similar carriage trip elsewhere; these memories may be visual and factual, or abstract and affective.
They are very differently-natured responses, that jointly constitute the experience to the item, while the physical representation of the item itself remains the same.
Would these be considered in a single, general learning or tagging task, the same problems as encountered with ImageNet would occur [9], where not all terms would have a visual observable link, and some associations may not be desirable to learn.
Furthermore, considering the Isabella painting depicted in Figure 1 and discussed in Section 3.1, without context, one may describe the painting as a depiction of a seated young girl with a fan in fine clothing.
In contrast, a professional may describe it as an oil on canvas painting.
対照的に、プロはそれをキャンバスの絵の油と表現することができる。
0.62
However, knowing the context of the discussion this painting sparked, it may now be described as the art object that triggered the Rijksmuseum’s terminology revisions, and used to be problematically titled.
Again, this further strengthens the perspective that single, universal, ‘neutral’ descriptions and representations of the item should not be strived for.
Encoding these as multiple tags of a single vocabulary, which often would happen in supervised machine learning contexts, would both be too simple, and taxonomically problematic.
The Semantic Web naturally gives opportunities for richer and parallel linking of differing viewpoints, as e g outlined by Van Erp and De Boer [44].
セマンティックWebは、Van Erp氏とDe Boer氏[44]によって概説されたegのように、異なる視点のよりリッチで並列なリンクの機会を自然に与えます。 訳抜け防止モード: Semantic Webは自然に、異なる視点のよりリッチで並列なリンクの機会を与えます。 Van Erp と De Boer による e g の概要 [44 ]
0.85
Next to this, for digital music content, Solid Pods were proposed by Weigl et al as a means to link various content types (e g textual annotations, scores, and recordings) at various levels of authority and intended visibility (e g professional commentary vs. private notes), facilitating provenance information, while explicitly keeping governance in the hands of the owner [47].
Social Inclusion in Curated Contexts: Insights from Museum Practices
キュレートされた文脈における社会的包摂性:博物館の実践から
0.50
FAccT ’22, June 21–24, 2022, Seoul, Republic of Korea
FAccT'22, 6月21-24, 2022, ソウル, 大韓民国
0.85
Beyond technological questions of encoding and representation, more conceptually, questions can be asked about the nature and scope of personalisation.
In search engines and recommender systems, personalisation typically happens in the back-end of a system as part of the filtering process, leading to personalised curated samplings of a collection.
An interesting notion in the museum practice is that the filtering process was performed by a human curator, and the filtered sample of a collection for a given exhibition will be fixed: everyone going to the same exhibition will in principle be offered the same collection items to see.
Any possible personalised differentiation will be in the presentation and interpretation; the parallel in the digital world would be in the front-end user experience.
With regard to this, the discussion in Section 3.2 very strongly emphasised purposeful communication practice, that would link elements of curation and education.
In digital information access, tailored communication strategies focusing on item presentation are still relatively rare.
デジタル情報アクセスでは、アイテムの提示に焦点を絞った通信戦略が比較的稀である。
0.70
Furthermore, in machine learning pipelines, deliberate connections across different stages in the process—from collection through curation to presentation—are hardly seen.
Typically, datasets would be established in a general-purpose mindset: model learning may be tailored to a given learning task, but the user-facing presentation of the outcomes of this task are considered to be outside of the scope of the machine learning procedure.
The equivalent to the audience-tailored museum exhibit experience, where end-user-specific front-end requirements would affect criteria on data annotation and curation, has to the best of our knowledge not been implemented as a process yet.
Especially in current works emphasising the importance of quality assurance on data, calls for a more holistic take are emerging and increasing, explicitly connecting practice downstream to dataset creation processes upstream.
This e g is the case for the proposal of Hutchinson et al to more explicitly include software development practices in dataset development [21], but also for Sambasivan et al ’s work on data cascades, showing that uninformed choices made during dataset development will cause major problems in downstream applications [33].
In addition, it increasingly is acknowledged and recognised that more holistic connections will need to be made between machine learning prediction in AI and user experience (UX) considerations, e g in the call to action put forward by Cramer and Kim [8].
さらに、AIにおける機械学習の予測とユーザエクスペリエンス(UX)の考慮との間には、より包括的なつながりがあることが認識され、認識されるようになった。 訳抜け防止モード: さらに、AIにおける機械学習予測とユーザエクスペリエンス(UX)の考慮との間には、より包括的な接続が必要であると認識され、認識されるようになった。 Cramer と Kim [8 ] が提案した行動の呼び出しで、eg を g にします。
0.65
4.3 Community participation In striving for social inclusion, museums have emphasised a pro-active focus on communities whose voice has not yet been heard and uplifted.
Translating this to AI practice, care should be taken in doing this.
これをAIの実践に翻訳するには、注意が必要だ。
0.67
Often, minoritised communities were already implicitly involved in system development, but as low-paid laborers, conducting data labeling tasks during the dataset creation phase [11].
This labeling procedure would normally not solicit their personal perspectives, but rather ask for standardised descriptions.
このラベル付け手順は、通常、彼らの個人的視点を要求せず、むしろ標準化された記述を要求する。
0.55
Indeed, when participatory machine learning was coined as a way to more inclusively involve communities, warnings were put out by Sloane et al that this should not lead to further exploitation and participation-washin g, where community work does not benefit the community, but rather serves a party with power over this community [41].
Looking at how the museum world implemented community participation, all of the examples discussed in Section 3.3 clearly indicate agency of the community itself.
When inviting community members to perform community creation, it also is clear the suggestions and insights will be respected, even if they may differ from what an ‘authoritative’ curator would do.
Looking at participatory methods in the digital world, participation through a citizen science perspective may be a better form than through a crowdsourcing perspective.
Where in the latter case, participant knowledge is solicited by a
後者の場合、参加者の知識は、ある人によって勧誘される
0.56
11
11
0.43
英語（論文から抽出）
日本語訳
スコア
FAccT ’22, June 21–24, 2022, Seoul, Republic of Korea
FAccT'22, 6月21-24, 2022, ソウル, 大韓民国
0.85
Han-Yin Huang and Cynthia C. S. Liem
Han-Yin HuangとCynthia C. S. Liem
0.37
requesting party, to benefit that party’s intentions, in the earlier case, the participant is an actual, fully acknowledged participant, who will more comprehensively join the full process leading to relevant insights.
As e g documented in Hsu et al [20], the community citizen science approach can effectively include communities, engaging on concerns that demonstrably are relevant to them.
Hsu et al [20]に記録されているegのように、コミュニティ市民科学のアプローチは、コミュニティを効果的に含めることができる。 訳抜け防止モード: Hsu et al [ 20 ] で記録されている e g として コミュニティ市民科学のアプローチは 効果的にコミュニティを組み込むことができます 明らかに彼らに関連する懸念に 関わります
0.75
Finally, the community participation practices in the museum world again illustrate the importance of communication and education.
Earlier, in Section 3.1, it was emphasised that cultural humility may not naturally be ingrained in an institute, and minoritised communities will need to assist in educating the institutes about their blind spots.
The other way around, coming back to Sambasivan et al [33], data cascades can be avoided when data literacy is improved, which presently still gets little attention and investment.
別の方法として、Sambasivan氏ら[33]に戻ってくると、データのリテラシーが改善されたときに、データカスケードが回避される。 訳抜け防止モード: その逆です sambasivan et al [33]に戻る。 データカスケードは、いつ避けられるか データリテラシーは改善され、現時点ではほとんど注意と投資を受けていない。
0.61
5 ACTIONABLE ORGANISATIONAL CHANGE: LIBRARIES IN DIGITAL TRANSITION Having built forth on the Lessons from Archives paper [24], and now discussing insights from the museum sector, we have been broadening perspectives on social inclusion and AI from a wider range of the so-called GLAM sector (galleries, libraries, archives and museums).
While differently-natured in their provided services, all these four institution types are responsible for the collection, preservation, presentation and dissemination of cultural heritage.
To further illustrate the relevance of GLAM sector experiences for current social inclusion challenges in AI, as a final illustration, we wish to draw attention to the library sector.
In showing how this currently is tackled in the public library community in The Netherlands, we again have an inspirational example of how social inclusion in collection and curation contexts is taken up as an organisational challenge, driven by explicitly set policies.
However, it still very explicitly is a social meeting space, where people from different backgrounds and perspectives may naturally encounter one another, where this would not be trivial anymore in other aspects of daily life.
Against the backdrop of these developments, a dedicated Dutch law on public library facilities was passed in 2015 [1], quoting independence, reliability, accessibility, pluriformity and authenticity as core public values to uphold, and listing five core library tasks:
(1) to facilitate knowledge and information; (2) to offer possibilities for development and education; (3) to promote reading and introduce literature; (4) to organise encounters and debates; (5) to acquaint the audience with arts and culture.
Looking at this, it is striking that ‘neutrality’ indeed is not mentioned, but ‘pluriformity’ is.
これを見ると、実際には‘中立性’は言及されていないが、‘多重性’は顕著である。
0.56
Even more strongly, strategic library innovation agendas especially emphasise the pluriformity aspect [18], noticing that the public library is one of the few remaining trusted places where confrontations with a different perspective than one’s own may be tolerated.
Interestingly, the present eight ethical principles for using AI [45], authored by the National Library of the Netherlands, still call for neutrality as a core principle, explaining this as “not steering users into directions that they may not choose
興味深いことに、オランダ国立図書館(National Library of the Netherlands)が著した、AIを使用するための8つの倫理的原則[45]は、現在でも中立を中核原則として求めており、これを「ユーザーが選択できない方向にユーザーを誘導しない」と説明している。 訳抜け防止モード: 興味深いことに、AIを使うための8つの倫理的原則[45]。 オランダ国立図書館によって作成され、中核原理として中立を求める。 ユーザーを選択できない方向に誘導するものではない。
0.71
12
12
0.42
英語（論文から抽出）
日本語訳
スコア
Social Inclusion in Curated Contexts: Insights from Museum Practices
キュレートされた文脈における社会的包摂性:博物館の実践から
0.50
FAccT ’22, June 21–24, 2022, Seoul, Republic of Korea
FAccT'22, 6月21-24, 2022, ソウル, 大韓民国
0.85
outside of AI context”.
AIコンテキストの外部”。
0.69
At the same time, the libraries voice explicit interest in digital recommender systems, and a major concern voiced in strategic documents is that the application of AI in filtering processes would narrow down a user’s perspective in undesirable ways [18].
In other words, the neutrality core principle needs revision, and responsible AI recommenders in the public library context maybe even should actively steer users beyond the perspectives they would choose without the recommender.
Beyond the possible role of AI technology in this challenge, similarly to museums, libraries do not only seek to stimulate more inclusive practice as part of a filtering or book-lending procedure.
Considering the five core tasks of a public library, organised social activities and cultural programming explicitly are included as part of these tasks, and actively employed to further engage relevant audiences.
Here, the parallels with the community-oriented activities of museums can easily be seen.
ここで、博物館のコミュニティ指向の活動との類似性は容易に見ることができる。
0.60
In line with their task to offer possibilities for development and education, Dutch public libraries have today been designated as the place where citizens can learn important skills necessary for daily life.
This includes courses on reading comprehension, but also increasingly focuses on digitalisation.
これには読解に関するコースも含まれているが、デジタル化にも注目が集まっている。
0.52
Beyond introductory computer courses, this e g includes assistance with making use of digital public services, such as making vaccine appointments in the COVID-19 pandemic.
Furthermore, the public libraries also will be conducting outreach projects to raise citizen awareness of AI.
さらに、公立図書館は、AIに対する市民の意識を高めるために、アウトリーチプロジェクトも実施する。
0.59
Increasing this awareness is vital for allowing more inclusive and informed public debates on AI technologies, their impact on society, and the true needs and wishes of users.
In the United States of America, for similar reasons, the ‘We are AI’ course was developed and implemented as a collaboration between the Center for Responsible AI at New York University’s Tandon School of Engineering, the Peer 2 Peer University, and the Queens Public Library, in the form of a learning circle at the public library8.
同様の理由から、米国では、ニューヨーク大学のタンドン工科大学(Tandon School of Engineering)の責任AIセンター(Center for Responsible AI)、ピア2ピア大学(Peer 2 Peer University)、クイーンズ公共図書館(Queens Public Library)の協力のもと、公立図書館8の学習サークルという形で‘We are AI’コースが開発された。
0.79
6 CONCLUSION In this article, we reviewed how social inclusion practices have been integrated in the practice of museums, particularly focusing on the perception of ‘neutrality’, the need for situational interpretation, and the role of community participation.
We wish to emphasise that the museum sector did not yet ‘solve’ matters of inequality in representation and ownership.
博物館部門は、表現と所有権の不平等の問題をまだ解決していないことを強調したい。
0.62
With the ICOM museum definition discussion still ongoing, social inclusion has not formally been institutionalised yet.
ICOMの博物館定義に関する議論が続いているが、社会的包摂性はまだ公式には確立されていない。
0.49
Furthermore, some societal public museum statements have been criticised to be lip service.
さらに、いくつかの公共博物館の声明はリップサービスであると批判されている。
0.57
For example, during the global Black Lives Matter movement in 2020—in the middle of the COVID-19 pandemic and anger surrounding the death of George Floyd at the hand of the Minneapolis police—major museums such as the British Museum issued statements to express solidarity with the movement.
These gestures were, however, met with criticisms that museums need to do better to turn words into deeds by re-examining how Black communities have been portrayed.
In the case of the British Museum, for example, the Museum came under fire again for controversial colonial-period objects within its collection, such as the Benin Bronzes [5, 17].
Criticisms on museums following the Black Lives Matter movement show that the minority and underrepresented communities are not yet sufficiently satisfied, and that the museum sector is still a long way from resolving issues of social inclusion and cultural diversity in its practices.
Still, at the same time, many relevant discussions are being held, many policies have noticeably been adjusted, and many tangible actions can be noticed.
FAccT ’22, June 21–24, 2022, Seoul, Republic of Korea
FAccT'22, 6月21-24, 2022, ソウル, 大韓民国
0.85
Han-Yin Huang and Cynthia C. S. Liem
Han-Yin HuangとCynthia C. S. Liem
0.37
should be noted that conscious attitude changes are addressed at different stages of the curation and presentation process, with a strong focus on tailored presentation and its surrounding communicative intent.
Furthermore, the examples from the museum world also explicitly indicate how social inclusion requires strong organisational commitment, and how different departments within an organisation should actively be engaged in striving to improve it, while sustaining a mindset of cultural humility.
In the AI community, the rising awareness of socioculturally specific perspectives, interpretation dynamics, bias and power has so far largely been concentrated at the side of dataset creation and documentation [11, 16, 21, 27, 33].
However, in our current practice, commonly used datasets may get repurposed for different downstream tasks and applications, where the parties working on these tasks and applications may not always have been the dataset builders.
Furthermore, considering a final application, user-facing experience considerations will likely be handled by a very different team than the team working on data-driven components of the application.
Thus, more holistic quality assurance mechanisms will be needed.
したがって、より包括的な品質保証機構が必要である。
0.65
Even more strongly, any communicative efforts focusing on application adoption and acceptance may explicitly happen beyond technical application development cycles.
In our discussions of museum practice, we have highlighted that there are good reasons to actively make these different departments connect, also in the light of alignment with strategic mission statements.
We illustrated how the public libraries community in The Netherlands is presently going through a similar transformation, involving a more profound organisational mission reorientation, that manifests in a broad spectrum of associated activities.
Having positioned our current discussion next to Jo and Gebru’s Lessons from Archives [24], we have further strengthened the argument that the AI community can learn from organisational best practices of GLAM institutes.
These parties build upon long histories of experience with questions of curation, while at the same time being recognised as trusted, and in many cases public, spaces.
ACKNOWLEDGMENTS This work is part of the ‘Recommender requirements for pluriform perspectives in local public libraries’ project (NWA.1228.192.235) of the National Science Agenda (NWA) research programme, which is financed by the Dutch Research Council (NWO).
ACKNOWLEDGMENTS この研究は、オランダ研究協議会(NWO)が資金を拠出するNWA(National Science Agenda)研究プログラム(NWA.1228.192.235)のプロジェクト(NWA.1228.192.235)の一部である。 訳抜け防止モード: ACKNOWLEDGMENTS この研究は、National Science Agenda (NWA ) Research Programの'Recommender requirements for pluriform perspectives in local Public Library' Project (NWA.1228.192.235 )の一部である。 オランダ研究協議会(NWO)が出資している。
0.87
The National Library of The Netherlands (Koninklijke Bibliotheek, KB) is a partner in this project.
This partner role is that of a relevant domain stakeholder, and is neither financially compensated by the project grant, nor financially contributing to the project grant.
In addition, Dr Liem’s research time is supported through a personal Veni grant (016.Veni.192.131) of the NWO Talent Programme.
さらに、リーム博士の研究時間は nwo タレントプログラムの個人的 veni grant (016.veni.192.131) によって支えられている。
0.65
The NWO and the KB had no role in the design and conduct of the research presented in this paper; access and collection of data; analysis and interpretation of data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.
2004. Conflict and Complement: An Exploration of the Discourses Informing the Concept of the Socially Inclusive Museum in
2004. 論争と補足:社会包括的博物館の概念を語る言説の探索
0.51
Contemporary Britain. International Journal of Heritage Studies 10, 1 (2004), 49–73.
現代のイギリス。 International Journal of Heritage Studies 10, 1 (2004), 49–73。
0.60
[26] François Matarasso.
フランソワ・マタラッソ(François Matarasso)。
0.60
1997. Use or Ornament: the social impacts of participation in the arts.
1997. 使用または装飾:芸術に参加することによる社会的影響。
0.61
Comedia. [27] Milagros Miceli, Julian Posada, and Tianling Yang.
メディア。 【27】ミラフロス・ミッセリ、ジュリアン・ポサダ、天リング・ヤン
0.39
2022. Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?
2022. 機械学習データの研究:なぜパワーを意味するバイアスについて語るのか?
0.58
Proc. ACM Hum.
Proc ACMヒューム所属。
0.48
-Comput. Interact.
-計算。 相互作用する。
0.38
6, GROUP, Article 34 (jan 2022), 14 pages.
6グループ、34条(2022年2月)、14ページ。
0.62
https://doi.org/10.1 145/3492853
https://doi.org/10.1 145/3492853
0.17
[28] Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru.
28]マーガレット・ミッチェル、シモーネ・ウー、アンドリュー・ザルディヴァル、パーカー・バーンズ、ルーシー・ヴァッサーマン、ベン・ハッチンソン、エレナ・スピッツァー、イニオウルワ・デボラ・ラジ、ティムニット・ゲブル 訳抜け防止モード: マーガレット・ミッチェル、シモーヌ・ウー、アンドリュー・ザルディヴァル。 Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer Inioluwa Deborah RajiとTimnit Gebru。
0.62
2019. Model Cards for Model Reporting.
2019. モデルレポート用のモデルカード。
0.61
In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* ’19).
公正、説明責任、透明性に関する会議の議事録(fat* ’19)において。
0.63
ACM. [29] Wayne Modest.
acm。 ウェイン・モデスト(Wayne Modest)。
0.58
2020. Words Matter.
2020. 言葉が重要。
0.38
In Words Matter, Wayne Modest and Robin Lelijveld (Eds.).
NL, 6–10. [38] D. Sculley, Eric Breck, Igor Ivanov, James Atwood, Miha Skalic, Pallavi Baljekar, Pavel Ostyakov, Roman Solovyev, Weimin Wang, and Yoni Halpern.
NL, 6-10。 D. Sculley, Eric Breck, Igor Ivanov, James Atwood, Miha Skalic, Pallavi Baljekar, Pavel Ostyakov, Roman Solovyev, Weimin Wang, Yoni Halpern。 訳抜け防止モード: NL, 6-10。 [38 ] D. Sculley, Eric Breck, Igor Ivanov, James Atwood, Miha Skalic, Pallavi Baljekar, Pavel Ostyakov ロマン・ソロヴィエフ、ヴァイミン・ワン、ヨニ・ハルパーン。
0.58
2019. The Inclusive Images Competition.
2019. インクルーシブイメージコンペティション。
0.42
[39] Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, and D. Sculley.