UA-Radar: Exploring the Impact of User Agents on the Web

• URL: http://arxiv.org/abs/2311.10420v1
• Date: Fri, 17 Nov 2023 09:53:32 GMT
• Title: UA-Radar: Exploring the Impact of User Agents on the Web
• Authors: Jean Luc Intumwayase, Imane Fouad, Pierre Laperdrix, Romain Rouvoy,
• Abstract summary: In the early days of the web, giving the same web page to different browsers could provide very different results. User Agent (UA) string was introduced for content negotiation. Over the past three decades, the UA string remained exposed by browsers.
• Score: 3.8373578956681547
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the early days of the web, giving the same web page to different browsers could provide very different results. As the rendering engine behind each browser would differ, some elements of a page could break or be positioned in the wrong location. At that time, the User Agent (UA) string was introduced for content negotiation. By knowing the browser used to connect to the server, a developer could provide a web page that was tailored for that specific browser to remove any usability problems. Over the past three decades, the UA string remained exposed by browsers, but its current usefulness is being debated. Browsers now adopt the exact same standards and use the same languages to display the same content to users, bringing the question if the content of the UA string is still relevant today, or if it is a relic of the past. Moreover, the diversity of means to browse the web has become so large that the UA string is one of the top contributors to tracking users in the field of browser fingerprinting, bringing a sense of urgency to deprecate it. In this paper, our goal is to understand the impact of the UA on the web and if this legacy string is still actively used to adapt the content served to users. We introduce UA-Radar, a web page similarity measurement tool that compares in-depth two web pages from the code to their actual rendering, and highlights the similarities it finds. We crawled 270, 048 web pages from 11, 252 domains using 3 different browsers and 2 different UA strings to observe that 100% of the web pages were similar before any JavaScript was executed, demonstrating the absence of differential serving. Our experiments also show that only a very small number of websites are affected by the lack of UA information, which can be fixed in most cases by updating code to become browser-agnostic. Our study brings some proof that it may be time to turn the page on the UA string and retire it from current web browsers.

Related papers

• Developers Insight On Manifest v3 Privacy and Security Webextensions [0.0]
  Currently, Chrome transitions to a modified set of APIs called Manifest v3.<n>This paper studies the challenges and opportunities of Manifest v3 with an in-depth structured qualitative research.
  arXiv  Detail & Related papers  (2025-07-18T14:00:16Z)
• Browser Fingerprinting Using WebAssembly [1.4732811715354452]
  This paper introduces an advanced fingerprinting method using WebAssembly (Wasm)<n>We present a new approach that leverages WebAssembly's computational capabilities to identify returning devices across different browsing sessions.<n>We validate this approach on a variety of platforms, including Intel, AMD, and ARM CPUs, operating systems such as Windows, Android, and iOS, and in environments like VMWare, KVM, and iOS.
  arXiv  Detail & Related papers  (2025-05-31T21:39:17Z)
• Snorkeling in dark waters: A longitudinal surface exploration of unique Tor Hidden Services (Extended Version) [2.498836880652668]
  The Onion Router (Tor) is a controversial network whose utility is constantly under scrutiny. In this work, we present a large-scale analysis of the Tor Network. We leverage our crawler, dubbed Mimir, which automatically collects and visits content linked within the pages to collect a dataset of pages from more than 25k sites.
  arXiv  Detail & Related papers  (2025-04-23T15:59:16Z)
• Web Privacy based on Contextual Integrity: Measuring the Collapse of Online Contexts [0.0]
  We operationalize the theory of Privacy as Contextual Integrity and measure persistent user identification within and between Web contexts. We crawl the top-700 popular websites across the contexts of health, finance, news & media, LGBTQ, eCommerce, adult, and education websites, for 27 days. This is a first modest step in measuring Web privacy as Contextual Integrity, opening new avenues for contextual Web privacy research.
  arXiv  Detail & Related papers  (2024-12-19T23:30:29Z)
• MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designs [50.274447094978996]
  Multi-Page Resource-Aware Webpage (MRWeb) generation task transforms UI designs into multi-page, functional web UIs with internal/external navigation, image loading, and backend routing. Our study applies existing methods to the MRWeb problem using a newly curated dataset of 500 websites (300 synthetic, 200 real-world). Specifically, we identify the best metric to evaluate the similarity of the web UI, assess the impact of the resource list on MRWeb generation, analyze MLLM limitations, and evaluate the effectiveness of the MRWeb tool in real-world.
  arXiv  Detail & Related papers  (2024-12-19T15:02:33Z)
• Fingerprinting Browsers in Encrypted Communications [0.12209039082584558]
  The study observed that different browsers use a different number of messages to communicate with the server. It was found that there was a 30%-35% dissimilarity in the behavior of different browsers.
  arXiv  Detail & Related papers  (2024-10-28T15:06:31Z)
• Infogent: An Agent-Based Framework for Web Information Aggregation [59.67710556177564]
  We introduce Infogent, a novel framework for web information aggregation. Experiments on different information access settings demonstrate Infogent beats an existing SOTA multi-agent search framework by 7%.
  arXiv  Detail & Related papers  (2024-10-24T18:01:28Z)
• Beyond Browsing: API-Based Web Agents [58.39129004543844]
  API-based agents outperform web browsing agents in experiments on WebArena. Hybrid Agents out-perform both others nearly uniformly across tasks. Results strongly suggest that when APIs are available, they present an attractive alternative to relying on web browsing alone.
  arXiv  Detail & Related papers  (2024-10-21T19:46:06Z)
• How Unique is Whose Web Browser? The role of demographics in browser fingerprinting among US users [50.699390248359265]
  Browser fingerprinting can be used to identify and track users across the Web, even without cookies. This technique and resulting privacy risks have been studied for over a decade. We provide a first-of-its-kind dataset to enable further research.
  arXiv  Detail & Related papers  (2024-10-09T14:51:58Z)
• Towards Fine-Grained Webpage Fingerprinting at Scale [18.201489295361892]
  Website Fingerprinting (WF) attacks can effectively identify the websites visited by Tor clients via analyzing encrypted traffic patterns. Existing attacks focus on identifying different websites, but their accuracy dramatically decreases when applied to identify fine-grained webpages. We propose Oscar, a WPF attack based on multi-label metric learning that identifies different webpages from obfuscated traffic by transforming the feature space.
  arXiv  Detail & Related papers  (2024-09-06T15:21:00Z)
• DarthShader: Fuzzing WebGPU Shader Translators & Compilers [19.345967816562364]
  A recent trend towards running more demanding web applications has led to the adoption of the WebGPU standard. This opens up a new attack surface: Untrusted web content is passed through to the GPU stack, which traditionally has been optimized for performance instead of security. DarthShader is the first language fuzzer that combines mutators based on an intermediate representation with those using a more traditional abstract syntax tree.
  arXiv  Detail & Related papers  (2024-09-03T12:06:19Z)
• Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs [112.89665642941814]
  Multimodal large language models (MLLMs) have shown impressive success across modalities such as image, video, and audio. Current MLLMs are surprisingly poor at understanding webpage screenshots and generating their corresponding HTML code. We propose a benchmark consisting of a new large-scale webpage-to-code dataset for instruction tuning.
  arXiv  Detail & Related papers  (2024-06-28T17:59:46Z)
• AutoScraper: A Progressive Understanding Web Agent for Web Scraper Generation [54.17246674188208]
  Web scraping is a powerful technique that extracts data from websites, enabling automated data collection, enhancing data analysis capabilities, and minimizing manual data entry efforts. Existing methods, wrappers-based methods suffer from limited adaptability and scalability when faced with a new website. We introduce the paradigm of generating web scrapers with large language models (LLMs) and propose AutoScraper, a two-stage framework that can handle diverse and changing web environments more efficiently.
  arXiv  Detail & Related papers  (2024-04-19T09:59:44Z)
• HDNA: A graph-based change detection in HTML pages(Deface Attack Detection) [0.0]
  HDNA (HTML DNA) is introduced for analyzing and comparing Document Object Model (DOM) trees. Method assigns an identifier to each HTML page based on its structure.
  arXiv  Detail & Related papers  (2023-10-05T20:49:54Z)
• A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understanding [66.6468787004067]
  We introduce the Wikipedia Webpage suite (WikiWeb2M) containing 2M pages with all of the associated image, text, and structure data. We design a novel attention mechanism Prefix Global, which selects the most relevant image and text content as global tokens to attend to the rest of the webpage for context.
  arXiv  Detail & Related papers  (2023-05-05T16:38:05Z)
• Uncovering Fingerprinting Networks. An Analysis of In-Browser Tracking using a Behavior-based Approach [0.0]
  This thesis explores the current state of browser fingerprinting on the internet. We implement FPNET to identify fingerprinting scripts on large sets of websites by observing their behavior. We track down companies like Google, Yandex, Maxmind, Sift, or FingerprintJS.
  arXiv  Detail & Related papers  (2022-08-15T18:06:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.