Sociotechnical Audits: Broadening the Algorithm Auditing Lens to
Investigate Targeted Advertising
- URL: http://arxiv.org/abs/2308.15768v1
- Date: Wed, 30 Aug 2023 05:26:47 GMT
- Title: Sociotechnical Audits: Broadening the Algorithm Auditing Lens to
Investigate Targeted Advertising
- Authors: Michelle S. Lam, Ayush Pandit, Colin H. Kalicki, Rachit Gupta, Poonam
Sahoo, Dana\"e Metaxa
- Abstract summary: We propose the concept of sociotechnical auditing, focusing on the interplay between algorithms and users as each impacts the other.
We develop Intervenr, a platform for conducting longitudinal sociotechnical audits with consenting, compensated participants.
We deploy Intervenr in a two-week sociotechnical audit of online advertising to investigate the central premise that personalized ad targeting is more effective on users.
- Score: 1.6422305456306596
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Algorithm audits are powerful tools for studying black-box systems. While
very effective in examining technical components, the method stops short of a
sociotechnical frame, which would also consider users as an integral and
dynamic part of the system. Addressing this gap, we propose the concept of
sociotechnical auditing: auditing methods that evaluate algorithmic systems at
the sociotechnical level, focusing on the interplay between algorithms and
users as each impacts the other. Just as algorithm audits probe an algorithm
with varied inputs and observe outputs, a sociotechnical audit (STA)
additionally probes users, exposing them to different algorithmic behavior and
measuring resulting attitudes and behaviors. To instantiate this method, we
develop Intervenr, a platform for conducting browser-based, longitudinal
sociotechnical audits with consenting, compensated participants. Intervenr
investigates the algorithmic content users encounter online and coordinates
systematic client-side interventions to understand how users change in
response. As a case study, we deploy Intervenr in a two-week sociotechnical
audit of online advertising (N=244) to investigate the central premise that
personalized ad targeting is more effective on users. In the first week, we
collect all browser ads delivered to users, and in the second, we deploy an
ablation-style intervention that disrupts normal targeting by randomly pairing
participants and swapping all their ads. We collect user-oriented metrics
(self-reported ad interest and feeling of representation) and
advertiser-oriented metrics (ad views, clicks, and recognition) throughout,
along with a total of over 500,000 ads. Our STA finds that targeted ads indeed
perform better with users, but also that users begin to acclimate to different
ads in only a week, casting doubt on the primacy of personalized ad targeting
given the impact of repeated exposure.
Related papers
- Measuring Strategization in Recommendation: Users Adapt Their Behavior to Shape Future Content [66.71102704873185]
We test for user strategization by conducting a lab experiment and survey.
We find strong evidence of strategization across outcome metrics, including participants' dwell time and use of "likes"
Our findings suggest that platforms cannot ignore the effect of their algorithms on user behavior.
arXiv Detail & Related papers (2024-05-09T07:36:08Z) - Can Probabilistic Feedback Drive User Impacts in Online Platforms? [26.052963782865294]
A common explanation for negative user impacts of content recommender systems is misalignment between the platform's objective and user welfare.
In this work, we show that misalignment in the platform's objective is not the only potential cause of unintended impacts on users.
The source of these user impacts is that different pieces of content may generate observable user reactions (feedback information) at different rates.
arXiv Detail & Related papers (2024-01-10T18:12:31Z) - Online Decision Mediation [72.80902932543474]
Consider learning a decision support assistant to serve as an intermediary between (oracle) expert behavior and (imperfect) human behavior.
In clinical diagnosis, fully-autonomous machine behavior is often beyond ethical affordances.
arXiv Detail & Related papers (2023-10-28T05:59:43Z) - Sequence-aware item recommendations for multiply repeated user-item
interactions [0.0]
We design a recommender system that induces the temporal dimension in the task of item recommendation.
It considers sequences of item interactions for each user in order to make recommendations.
This method is empirically shown to give highly accurate predictions of user-items interactions for all users in a retail environment.
arXiv Detail & Related papers (2023-04-02T17:06:07Z) - Modeling Content Creator Incentives on Algorithm-Curated Platforms [76.53541575455978]
We study how algorithmic choices affect the existence and character of (Nash) equilibria in exposure games.
We propose tools for numerically finding equilibria in exposure games, and illustrate results of an audit on the MovieLens and LastFM datasets.
arXiv Detail & Related papers (2022-06-27T08:16:59Z) - Incentivizing Combinatorial Bandit Exploration [87.08827496301839]
Consider a bandit algorithm that recommends actions to self-interested users in a recommendation system.
Users are free to choose other actions and need to be incentivized to follow the algorithm's recommendations.
While the users prefer to exploit, the algorithm can incentivize them to explore by leveraging the information collected from the previous users.
arXiv Detail & Related papers (2022-06-01T13:46:25Z) - On the Convergence of Clustered Federated Learning [57.934295064030636]
In a federated learning system, the clients, e.g. mobile devices and organization participants, usually have different personal preferences or behavior patterns.
This paper proposes a novel weighted client-based clustered FL algorithm to leverage the client's group and each client in a unified optimization framework.
arXiv Detail & Related papers (2022-02-13T02:39:19Z) - Everyday algorithm auditing: Understanding the power of everyday users
in surfacing harmful algorithmic behaviors [8.360589318502816]
We propose and explore the concept of everyday algorithm auditing, a process in which users detect, understand, and interrogate problematic machine behaviors.
We argue that everyday users are powerful in surfacing problematic machine behaviors that may elude detection via more centrally-organized forms of auditing.
arXiv Detail & Related papers (2021-05-06T21:50:47Z) - Examining the Impact of Algorithm Awareness on Wikidata's Recommender
System Recoin [12.167153941840958]
We conduct online experiments with 105 participants using MTurk for the recommender system Recoin, a gadget for Wikidata.
Our findings include a positive correlation between comprehension of and trust in an algorithmic system in our interactive redesign.
Our results are not conclusive yet, and suggest that the measures of comprehension, fairness, accuracy and trust are not yet exhaustive for the empirical study of algorithm awareness.
arXiv Detail & Related papers (2020-09-18T20:06:53Z) - A black-box adversarial attack for poisoning clustering [78.19784577498031]
We propose a black-box adversarial attack for crafting adversarial samples to test the robustness of clustering algorithms.
We show that our attacks are transferable even against supervised algorithms such as SVMs, random forests, and neural networks.
arXiv Detail & Related papers (2020-09-09T18:19:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.