Query-Specific Knowledge Graphs for Complex Finance Topics
- URL: http://arxiv.org/abs/2211.04142v1
- Date: Tue, 8 Nov 2022 10:21:13 GMT
- Title: Query-Specific Knowledge Graphs for Complex Finance Topics
- Authors: Iain Mackie and Jeffrey Dalton
- Abstract summary: We focus on the CODEC dataset, where domain experts create challenging questions.
We show that state-of-the-art ranking systems have headroom for improvement.
We demonstrate that entity and document relevance are positively correlated.
- Score: 6.599344783327053
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Across the financial domain, researchers answer complex questions by
extensively "searching" for relevant information to generate long-form reports.
This workshop paper discusses automating the construction of query-specific
document and entity knowledge graphs (KGs) for complex research topics. We
focus on the CODEC dataset, where domain experts (1) create challenging
questions, (2) construct long natural language narratives, and (3) iteratively
search and assess the relevance of documents and entities. For the construction
of query-specific KGs, we show that state-of-the-art ranking systems have
headroom for improvement, with specific failings due to a lack of context or
explicit knowledge representation. We demonstrate that entity and document
relevance are positively correlated, and that entity-based query feedback
improves document ranking effectiveness. Furthermore, we construct
query-specific KGs using retrieval and evaluate using CODEC's "ground-truth
graphs", showing the precision and recall trade-offs. Lastly, we point to
future work, including adaptive KG retrieval algorithms and GNN-based weighting
methods, while highlighting key challenges such as high-quality data,
information extraction recall, and the size and sparsity of complex topic
graphs.
Related papers
- Context Graph [8.02985792541121]
We present a context graph reasoning textbfCGR$3$ paradigm that leverages large language models (LLMs) to retrieve candidate entities and related contexts.
Our experimental results demonstrate that CGR$3$ significantly improves performance on KG completion (KGC) and KG question answering (KGQA) tasks.
arXiv Detail & Related papers (2024-06-17T02:59:19Z) - Automated Construction of Theme-specific Knowledge Graphs [21.68957334046064]
We propose a theme-specific knowledge graph (i.e., ThemeKG) constructed from a theme-specific corpus.
We design an unsupervised framework for ThemeKG construction (named TKGCon)
By constructing the theme-specific KG step by step, our model outperforms GPT-4 and could consistently identify accurate entities and relations.
arXiv Detail & Related papers (2024-04-29T23:14:14Z) - STaRK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Bases [93.96463520716759]
We develop STARK, a large-scale Semi-structure retrieval benchmark on Textual and K nowledge Bases.
Our benchmark covers three domains/datasets: product search, academic paper search, and queries in precision medicine.
We design a novel pipeline to synthesize realistic user queries that integrate diverse relational information and complex textual properties.
arXiv Detail & Related papers (2024-04-19T22:54:54Z) - Improving Retrieval in Theme-specific Applications using a Corpus
Topical Taxonomy [52.426623750562335]
We introduce ToTER (Topical taxonomy Enhanced Retrieval) framework.
ToTER identifies the central topics of queries and documents with the guidance of the taxonomy, and exploits their topical relatedness to supplement missing contexts.
As a plug-and-play framework, ToTER can be flexibly employed to enhance various PLM-based retrievers.
arXiv Detail & Related papers (2024-03-07T02:34:54Z) - FedCQA: Answering Complex Queries on Multi-Source Knowledge Graphs via
Federated Learning [55.02512821257247]
Complex logical query answering is a challenging task in knowledge graphs (KGs)
Recent approaches are proposed to represent KG entities into embedding vectors and find answers to logical queries from the KGs.
It remains unknown how to answer queries on multi-source KGs.
arXiv Detail & Related papers (2024-02-22T14:57:44Z) - DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain
Question Answering over Knowledge Base and Text [73.68051228972024]
Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge.
Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
arXiv Detail & Related papers (2023-10-31T04:37:57Z) - KG-ECO: Knowledge Graph Enhanced Entity Correction for Query Rewriting [15.243664083941287]
In this work, we propose KG-ECO: Knowledge Graph enhanced Entity COrrection for query rewriting.
To boost the model performance, we incorporate Knowledge Graph (KG) to provide entity structural information.
Experimental results show that our approach yields a clear performance gain over two baselines.
arXiv Detail & Related papers (2023-02-21T05:42:06Z) - UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question
Answering Over Knowledge Graph [89.98762327725112]
Multi-hop Question Answering over Knowledge Graph(KGQA) aims to find the answer entities that are multiple hops away from the topic entities mentioned in a natural language question.
We propose UniKGQA, a novel approach for multi-hop KGQA task, by unifying retrieval and reasoning in both model architecture and parameter learning.
arXiv Detail & Related papers (2022-12-02T04:08:09Z) - Exposing Query Identification for Search Transparency [69.06545074617685]
We explore the feasibility of approximate exposing query identification (EQI) as a retrieval task by reversing the role of queries and documents in two classes of search systems.
We derive an evaluation metric to measure the quality of a ranking of exposing queries, as well as conducting an empirical analysis focusing on various practical aspects of approximate EQI.
arXiv Detail & Related papers (2021-10-14T20:19:27Z) - AutoKG: Constructing Virtual Knowledge Graphs from Unstructured
Documents for Question Answering [19.72815568759182]
We propose a novel framework to automatically construct a knowledge graph from unstructured documents.
We first extract knowledges from unstructured documents and encode them with contextual information.
Entities with similar context semantics are linked through internal alignment to form a graph structure.
This allows us to extract desired information from multiple documents by traversing the generated KG without a manual process.
arXiv Detail & Related papers (2020-08-20T14:30:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.