Searching by Code: a New SearchBySnippet Dataset and SnippeR Retrieval Model for Searching by Code Snippets

• URL: http://arxiv.org/abs/2305.11625v2
• Date: Mon, 27 May 2024 05:44:48 GMT
• Title: Searching by Code: a New SearchBySnippet Dataset and SnippeR Retrieval Model for Searching by Code Snippets
• Authors: Ivan Sedykh, Dmitry Abulkhanov, Nikita Sorokin, Sergey Nikolenko, Valentin Malykh,
• Abstract summary: We argue that using a code snippet as a query while looking for bugfixing instructions and code samples is a natural use case not covered by prior art. We present a new SearchBySnippet dataset implementing the search-by-code use case based on StackOverflow data.
• Score: 7.948526577271158
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Code search is an important and well-studied task, but it usually means searching for code by a text query. We argue that using a code snippet (and possibly an error traceback) as a query while looking for bugfixing instructions and code samples is a natural use case not covered by prior art. Moreover, existing datasets use code comments rather than full-text descriptions as text, making them unsuitable for this use case. We present a new SearchBySnippet dataset implementing the search-by-code use case based on StackOverflow data; we show that on SearchBySnippet, existing architectures fall short of a simple BM25 baseline even after fine-tuning. We present a new single encoder model SnippeR that outperforms several strong baselines on SearchBySnippet with a result of 0.451 Recall@10; we propose the SearchBySnippet dataset and SnippeR as a new important benchmark for code search evaluation.

Related papers

• CoIR: A Comprehensive Benchmark for Code Information Retrieval Models [56.691926887209895]
  We present textbfname (textbfInformation textbfRetrieval Benchmark), a robust and comprehensive benchmark specifically designed to assess code retrieval capabilities. name comprises textbften meticulously curated code datasets, spanning textbfeight distinctive retrieval tasks across textbfseven diverse domains. We evaluate nine widely used retrieval models using name, uncovering significant difficulties in performing code retrieval tasks even with state-of-the-art systems.
  arXiv  Detail & Related papers  (2024-07-03T07:58:20Z)
• CoSQA+: Enhancing Code Search Dataset with Matching Code [27.10957318333608]
  CoSQA+ pairs high-quality queries with multiple suitable codes. CoSQA+ has demonstrated superior quality over CoSQA. We propose a new metric to assess one-to-N code search performance.
  arXiv  Detail & Related papers  (2024-06-17T14:34:14Z)
• STaRK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Bases [93.96463520716759]
  We develop STARK, a large-scale Semi-structure retrieval benchmark on Textual and K nowledge Bases. Our benchmark covers three domains/datasets: product search, academic paper search, and queries in precision medicine. We design a novel pipeline to synthesize realistic user queries that integrate diverse relational information and complex textual properties.
  arXiv  Detail & Related papers  (2024-04-19T22:54:54Z)
• Generation-Augmented Query Expansion For Code Retrieval [51.20943646688115]
  We propose a generation-augmented query expansion framework. Inspired by the human retrieval process - sketching an answer before searching. We achieve new state-of-the-art results on the CodeSearchNet benchmark.
  arXiv  Detail & Related papers  (2022-12-20T23:49:37Z)
• NS3: Neuro-Symbolic Semantic Code Search [33.583344165521645]
  We use a Neural Module Network architecture to implement this idea. We compare our model - NS3 (Neuro-Symbolic Semantic Search) - to a number of baselines, including state-of-the-art semantic code retrieval methods. We demonstrate that our approach results in more precise code retrieval, and we study the effectiveness of our modular design when handling compositional queries.
  arXiv  Detail & Related papers  (2022-05-21T20:55:57Z)
• Enhancing Semantic Code Search with Multimodal Contrastive Learning and Soft Data Augmentation [50.14232079160476]
  We propose a new approach with multimodal contrastive learning and soft data augmentation for code search. We conduct extensive experiments to evaluate the effectiveness of our approach on a large-scale dataset with six programming languages.
  arXiv  Detail & Related papers  (2022-04-07T08:49:27Z)
• Accelerating Code Search with Deep Hashing and Code Classification [64.3543949306799]
  Code search is to search reusable code snippets from source code corpus based on natural languages queries. We propose a novel method CoSHC to accelerate code search with deep hashing and code classification.
  arXiv  Detail & Related papers  (2022-03-29T07:05:30Z)
• Learning Deep Semantic Model for Code Search using CodeSearchNet Corpus [17.6095840480926]
  We propose a novel deep semantic model which makes use of the utilities of multi-modal sources. We apply the proposed model to tackle the CodeSearchNet challenge about semantic code search. Our model is trained on CodeSearchNet corpus and evaluated on the held-out data, the final model achieves 0.384 NDCG and won the first place in this benchmark.
  arXiv  Detail & Related papers  (2022-01-27T04:15:59Z)
• Search4Code: Code Search Intent Classification Using Weak Supervision [5.441318460204245]
  We propose a weak supervision based approach for detecting code search intent in search queries for C# and Java programming languages. We evaluate the approach against several baselines on a real-world dataset comprised of over 1 million queries mined from Bing web search engine. We are also releasing Search4Code, the first large-scale real-world dataset of code search queries mined from Bing web search engine.
  arXiv  Detail & Related papers  (2020-11-24T08:06:53Z)
• COSEA: Convolutional Code Search with Layer-wise Attention [90.35777733464354]
  We propose a new deep learning architecture, COSEA, which leverages convolutional neural networks with layer-wise attention to capture the code's intrinsic structural logic. COSEA can achieve significant improvements over state-of-the-art methods on code search tasks.
  arXiv  Detail & Related papers  (2020-10-19T13:53:38Z)
• Neural Code Search Revisited: Enhancing Code Snippet Retrieval through Natural Language Intent [1.1168121941015012]
  We study how code retrieval systems can be improved by leveraging descriptions to better capture the intents of code snippets. Building on recent progress in transfer learning and natural language processing, we create a domain-specific retrieval model for code annotated with a natural language description.
  arXiv  Detail & Related papers  (2020-08-27T15:39:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.