Creating an AI Observer: Generative Semantic Workspaces

• URL: http://arxiv.org/abs/2406.04555v1
• Date: Fri, 7 Jun 2024 00:09:13 GMT
• Title: Creating an AI Observer: Generative Semantic Workspaces
• Authors: Pavan Holur, Shreyas Rajesh, David Chong, Vwani Roychowdhury,
• Abstract summary: We introduce the $textbf[G]$enerative $textbf[S]$emantic $textbf[W]$orkspace (GSW)) GSW creates a generative-style Semantic framework, as opposed to a traditionally predefined set of lexicon labels.
• Score: 4.031100721019478
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An experienced human Observer reading a document -- such as a crime report -- creates a succinct plot-like $\textit{``Working Memory''}$ comprising different actors, their prototypical roles and states at any point, their evolution over time based on their interactions, and even a map of missing Semantic parts anticipating them in the future. $\textit{An equivalent AI Observer currently does not exist}$. We introduce the $\textbf{[G]}$enerative $\textbf{[S]}$emantic $\textbf{[W]}$orkspace (GSW) -- comprising an $\textit{``Operator''}$ and a $\textit{``Reconciler''}$ -- that leverages advancements in LLMs to create a generative-style Semantic framework, as opposed to a traditionally predefined set of lexicon labels. Given a text segment $C_n$ that describes an ongoing situation, the $\textit{Operator}$ instantiates actor-centric Semantic maps (termed ``Workspace instance'' $\mathcal{W}_n$). The $\textit{Reconciler}$ resolves differences between $\mathcal{W}_n$ and a ``Working memory'' $\mathcal{M}_n^*$ to generate the updated $\mathcal{M}_{n+1}^*$. GSW outperforms well-known baselines on several tasks ($\sim 94\%$ vs. FST, GLEN, BertSRL - multi-sentence Semantics extraction, $\sim 15\%$ vs. NLI-BERT, $\sim 35\%$ vs. QA). By mirroring the real Observer, GSW provides the first step towards Spatial Computing assistants capable of understanding individual intentions and predicting future behavior.

Related papers

• Federated UCBVI: Communication-Efficient Federated Regret Minimization with Heterogeneous Agents [13.391318494060975]
  We present the Federated Upper Confidence Bound Value Iteration algorithm ($textttFed-UCBVI$) We prove that the regret of $textttFed-UCBVI$ scales as $tildemathcalO(sqrtH3 |mathcalS| |mathcalA| T / M)$. We show that, unlike existing federated reinforcement learning approaches, $textttFed-UCBVI$'s communication complexity only marginally increases with the number of
  arXiv  Detail & Related papers  (2024-10-30T11:05:50Z)
• Guarantees for Nonlinear Representation Learning: Non-identical Covariates, Dependent Data, Fewer Samples [24.45016514352055]
  We study the sample-complexity of learning $T+1$ functions $f_star(t) circ g_star$ from a function class $mathcal F times mathcal G$. We show that as the number of tasks $T$ increases, both the sample requirement and risk bound converge to that of $r$-dimensional regression.
  arXiv  Detail & Related papers  (2024-10-15T03:20:19Z)
• Transformer In-Context Learning for Categorical Data [51.23121284812406]
  We extend research on understanding Transformers through the lens of in-context learning with functional data by considering categorical outcomes, nonlinear underlying models, and nonlinear attention. We present what is believed to be the first real-world demonstration of this few-shot-learning methodology, using the ImageNet dataset.
  arXiv  Detail & Related papers  (2024-05-27T15:03:21Z)
• Clustering Propagation for Universal Medical Image Segmentation [63.431147442243855]
  Prominent solutions for medical image segmentation are typically tailored for automatic or interactive setups. Inspired by clustering-based segmentation techniques, S2VNet makes full use of the slice-wise structure of data. S2VNet distinguishes itself by swift inference speeds and reduced memory consumption compared to prevailing 3D solutions.
  arXiv  Detail & Related papers  (2024-03-25T11:32:05Z)
• $\textit{Swap and Predict}$ -- Predicting the Semantic Changes in Words across Corpora by Context Swapping [36.10628959436778]
  We consider the problem of predicting whether a given target word, $w$, changes its meaning between two different text corpora. We propose an unsupervised method that randomly swaps contexts between $mathcalC$ and $mathcalC$. Our method achieves significant performance improvements compared to strong baselines for the English semantic change prediction task.
  arXiv  Detail & Related papers  (2023-10-16T13:39:44Z)
• Simplifying and Understanding State Space Models with Diagonal Linear RNNs [56.33053691749856]
  This work disposes of the discretization step, and proposes a model based on vanilla Diagonal Linear RNNs. We empirically show that, despite being conceptually much simpler, $mathrmDLR$ is as performant as previously-proposed SSMs. We also characterize the expressivity of SSMs and attention-based models via a suite of $13$ synthetic sequence-to-sequence tasks.
  arXiv  Detail & Related papers  (2022-12-01T18:53:06Z)
• Near-Optimal Regret Bounds for Multi-batch Reinforcement Learning [54.806166861456035]
  We study the episodic reinforcement learning (RL) problem modeled by finite-horizon Markov Decision Processes (MDPs) with constraint on the number of batches. We design a computational efficient algorithm to achieve near-optimal regret of $tildeO(sqrtSAH3Kln (1/delta))$tildeO(cdot) hides logarithmic terms of $(S,A,H,K)$ in $K$ episodes. Our technical contribution are two-fold: 1) a near-optimal design scheme to explore
  arXiv  Detail & Related papers  (2022-10-15T09:22:22Z)
• On the Power of Multitask Representation Learning in Linear MDP [61.58929164172968]
  This paper presents analyses for the statistical benefit of multitask representation learning in linear Markov Decision Process (MDP) We first discover a emphLeast-Activated-Feature-Abundance (LAFA) criterion, denoted as $kappa$, with which we prove that a straightforward least-square algorithm learns a policy which is $tildeO(H2sqrtfrackappa mathcalC(Phi)2 kappa dNT+frackappa dn)
  arXiv  Detail & Related papers  (2021-06-15T11:21:06Z)
• Categorical Representation Learning: Morphism is All You Need [0.0]
  We provide a construction for categorical representation learning and introduce the foundations of "$textitcategorifier$" Every object in a dataset $mathcalS$ can be represented as a vector in $mathbbRn$ by an $textitencoding map$ $E: mathcalObj(mathcalS)tomathbbRn$. As a proof of concept, we provide an example of a text translator equipped with our technology, showing that our categorical learning model outperforms the
  arXiv  Detail & Related papers  (2021-03-26T23:47:15Z)
• On Computing Stable Extensions of Abstract Argumentation Frameworks [1.1251593386108185]
  An textitabstract argumentation framework (sc af) is a directed graph $(A,R)$ where $A$ is a set of textitabstract arguments and $Rsubseteq A times A$ is the textitattack relation. We present a thorough, formal validation of a known backtracking algorithm for listing all stable extensions in a given sc af.
  arXiv  Detail & Related papers  (2020-11-03T05:38:52Z)
• Synthesizing Tasks for Block-based Programming [72.45475843387183]
  We propose a novel methodology to automatically generate a set $(rm Tout, rm Cout)$ of new tasks along with solution codes. Our algorithm operates by first mutating code $rm Cin$ to obtain a set of codes $rm Cout$.
  arXiv  Detail & Related papers  (2020-06-17T15:04:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.