Annotationsaurus: A Searchable Directory of Annotation Tools

• URL: http://arxiv.org/abs/2010.06251v2
• Date: Wed, 14 Oct 2020 06:41:06 GMT
• Title: Annotationsaurus: A Searchable Directory of Annotation Tools
• Authors: Mariana Neves and Jurica Seva
• Abstract summary: We create a comprehensive directory of annotation tools that currently includes 93 tools. We implement simple scripts and a Web application that filters the tools based on chosen criteria. We present two use cases using the directory and propose ideas for its maintenance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Manual annotation of textual documents is a necessary task when constructing benchmark corpora for training and evaluating machine learning algorithms. We created a comprehensive directory of annotation tools that currently includes 93 tools. We analyzed the tools over a set of 31 features and implemented simple scripts and a Web application that filters the tools based on chosen criteria. We present two use cases using the directory and propose ideas for its maintenance. The directory, source codes for scripts, and link to the Web application are available at: https://github.com/mariananeves/annotation-tools

Related papers

• A Web App for Teaching Finite State Automata [0.0]
  finsm.io is a tool for creating, simulating and exporting deterministic and non-deterministic finite state automata (DFA/NFA) We describe the conceptual background on which the tool is based, followed by a description of features and preliminary evaluation of the tool based on use spanning multiple years and hundreds of student users. Preliminary evaluation found that instructors and students overwhelmingly recommend the tool to others and agree that it has improved their learning and teaching.
  arXiv  Detail & Related papers  (2024-10-15T23:28:26Z)
• AutoFL: A Tool for Automatic Multi-granular Labelling of Software Repositories [6.0158981171030685]
  AutoFL is a tool for automatically labelling software repositories from source code. It allows multi-granular annotations including: textitfile, textitpackage, and textitproject -level.
  arXiv  Detail & Related papers  (2024-08-05T15:34:26Z)
• Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrieval [47.81307125613145]
  Re-Invoke is an unsupervised tool retrieval method designed to scale effectively to large toolsets without training. We employ a novel multi-view similarity ranking strategy based on intents to pinpoint the most relevant tools for each query. Our evaluation demonstrates that Re-Invoke significantly outperforms state-of-the-art alternatives in both single-tool and multi-tool scenarios.
  arXiv  Detail & Related papers  (2024-08-03T22:49:27Z)
• Chain of Tools: Large Language Model is an Automatic Multi-tool Learner [54.992464510992605]
  Automatic Tool Chain (ATC) is a framework that enables the large language models (LLMs) to act as a multi-tool user. To scale up the scope of the tools, we next propose a black-box probing method. For a comprehensive evaluation, we build a challenging benchmark named ToolFlow.
  arXiv  Detail & Related papers  (2024-05-26T11:40:58Z)
• EEVEE: An Easy Annotation Tool for Natural Language Processing [32.111061774093]
  We propose EEVEE, an annotation tool focused on simplicity, efficiency, and ease of use. It can run directly in the browser (no setup required) and uses tab-separated files (as opposed to character offsets or task-specific formats) for annotation.
  arXiv  Detail & Related papers  (2024-02-05T10:24:40Z)
• EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction [56.02100384015907]
  EasyTool is a framework transforming diverse and lengthy tool documentation into a unified and concise tool instruction. It can significantly reduce token consumption and improve the performance of tool utilization in real-world scenarios.
  arXiv  Detail & Related papers  (2024-01-11T15:45:11Z)
• MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use [82.24774504584066]
  Large language models (LLMs) have garnered significant attention due to their impressive natural language processing (NLP) capabilities. We introduce MetaTool, a benchmark designed to evaluate whether LLMs have tool usage awareness and can correctly choose tools. We conduct experiments involving eight popular LLMs and find that the majority of them still struggle to effectively select tools.
  arXiv  Detail & Related papers  (2023-10-04T19:39:26Z)
• Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models [90.96816639172464]
  Large language models (LLMs) are taught to use new tools by providing a few demonstrations of the tool's usage. We advocate the use of tool documentation, descriptions for the individual tool usage, over demonstrations.
  arXiv  Detail & Related papers  (2023-08-01T17:21:38Z)
• ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs [104.37772295581088]
  Open-source large language models (LLMs), e.g., LLaMA, remain significantly limited in tool-use capabilities. We introduce ToolLLM, a general tool-usetuning encompassing data construction, model training, and evaluation. We first present ToolBench, an instruction-tuning framework for tool use, which is constructed automatically using ChatGPT.
  arXiv  Detail & Related papers  (2023-07-31T15:56:53Z)
• SciAnnotate: A Tool for Integrating Weak Labeling Sources for Sequence Labeling [55.71459234749639]
  SciAnnotate is a web-based tool for text annotation called SciAnnotate, which stands for scientific annotation tool. Our tool provides users with multiple user-friendly interfaces for creating weak labels. In this study, we take multi-source weak label denoising as an example, we utilized a Bertifying Conditional Hidden Markov Model to denoise the weak label generated by our tool.
  arXiv  Detail & Related papers  (2022-08-07T19:18:13Z)
• SenTag: a Web-based Tool for Semantic Annotation of Textual Documents [4.910379177401659]
  SenTag is a web-based tool focused on semantic annotation of textual documents. The main goal of the application is two-fold: facilitating the tagging process and reducing or avoiding for errors in the output documents. It is also possible to assess the level of agreement of annotators working on a corpus of text.
  arXiv  Detail & Related papers  (2021-09-16T08:39:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.