m2caiSeg: Semantic Segmentation of Laparoscopic Images using
Convolutional Neural Networks
- URL: http://arxiv.org/abs/2008.10134v2
- Date: Thu, 10 Dec 2020 21:34:59 GMT
- Title: m2caiSeg: Semantic Segmentation of Laparoscopic Images using
Convolutional Neural Networks
- Authors: Salman Maqbool, Aqsa Riaz, Hasan Sajid, Osman Hasan
- Abstract summary: We propose a deep learning based semantic segmentation algorithm to identify and label the tissues and organs in the endoscopic video feed of the human torso region.
We present an annotated dataset, m2caiSeg, created from endoscopic video feeds of real-world surgical procedures.
- Score: 4.926395463398194
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Autonomous surgical procedures, in particular minimal invasive surgeries, are
the next frontier for Artificial Intelligence research. However, the existing
challenges include precise identification of the human anatomy and the surgical
settings, and modeling the environment for training of an autonomous agent. To
address the identification of human anatomy and the surgical settings, we
propose a deep learning based semantic segmentation algorithm to identify and
label the tissues and organs in the endoscopic video feed of the human torso
region. We present an annotated dataset, m2caiSeg, created from endoscopic
video feeds of real-world surgical procedures. Overall, the data consists of
307 images, each of which is annotated for the organs and different surgical
instruments present in the scene. We propose and train a deep convolutional
neural network for the semantic segmentation task. To cater for the low
quantity of annotated data, we use unsupervised pre-training and data
augmentation. The trained model is evaluated on an independent test set of the
proposed dataset. We obtained a F1 score of 0.33 while using all the labeled
categories for the semantic segmentation task. Secondly, we labeled all
instruments into an 'Instruments' superclass to evaluate the model's
performance on discerning the various organs and obtained a F1 score of 0.57.
We propose a new dataset and a deep learning method for pixel level
identification of various organs and instruments in a endoscopic surgical
scene. Surgical scene understanding is one of the first steps towards
automating surgical procedures.
Related papers
- Synthesizing Multi-Class Surgical Datasets with Anatomy-Aware Diffusion Models [1.9085155846692308]
In computer-assisted surgery, automatically recognizing anatomical organs is crucial for understanding the surgical scene.
While machine learning models can identify such structures, their deployment is hindered by the need for labeled, diverse surgical datasets.
We introduce a multi-stage approach using diffusion models to generate multi-class surgical datasets with annotations.
arXiv Detail & Related papers (2024-10-10T09:29:23Z) - CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformers [66.15847237150909]
We introduce a self-supervised deep learning architecture to segment catheters in longitudinal ultrasound images.
The network architecture builds upon AiAReSeg, a segmentation transformer built with the Attention in Attention mechanism.
We validated our model on a test dataset, consisting of unseen synthetic data and images collected from silicon aorta phantoms.
arXiv Detail & Related papers (2024-03-21T15:13:36Z) - SAR-RARP50: Segmentation of surgical instrumentation and Action
Recognition on Robot-Assisted Radical Prostatectomy Challenge [72.97934765570069]
We release the first multimodal, publicly available, in-vivo, dataset for surgical action recognition and semantic instrumentation segmentation, containing 50 suturing video segments of Robotic Assisted Radical Prostatectomy (RARP)
The aim of the challenge is to enable researchers to leverage the scale of the provided dataset and develop robust and highly accurate single-task action recognition and tool segmentation approaches in the surgical domain.
A total of 12 teams participated in the challenge, contributing 7 action recognition methods, 9 instrument segmentation techniques, and 4 multitask approaches that integrated both action recognition and instrument segmentation.
arXiv Detail & Related papers (2023-12-31T13:32:18Z) - Visual-Kinematics Graph Learning for Procedure-agnostic Instrument Tip
Segmentation in Robotic Surgeries [29.201385352740555]
We propose a novel visual-kinematics graph learning framework to accurately segment the instrument tip given various surgical procedures.
Specifically, a graph learning framework is proposed to encode relational features of instrument parts from both image and kinematics.
A cross-modal contrastive loss is designed to incorporate robust geometric prior from kinematics to image for tip segmentation.
arXiv Detail & Related papers (2023-09-02T14:52:58Z) - Hierarchical Semi-Supervised Learning Framework for Surgical Gesture
Segmentation and Recognition Based on Multi-Modality Data [2.8770761243361593]
We develop a hierarchical semi-supervised learning framework for surgical gesture segmentation using multi-modality data.
A Transformer-based network with a pre-trained ResNet-18' backbone is used to extract visual features from the surgical operation videos.
The proposed approach has been evaluated using data from the publicly available JIGS database, including Suturing, Needle Passing, and Knot Tying tasks.
arXiv Detail & Related papers (2023-07-31T21:17:59Z) - Rethinking Surgical Instrument Segmentation: A Background Image Can Be
All You Need [18.830738606514736]
Data scarcity and imbalance have heavily affected the model accuracy and limited the design and deployment of deep learning-based surgical applications.
We propose a one-to-many data generation solution that gets rid of the complicated and expensive process of data collection and annotation from robotic surgery.
Our empirical analysis suggests that without the high cost of data collection and annotation, we can achieve decent surgical instrument segmentation performance.
arXiv Detail & Related papers (2022-06-23T16:22:56Z) - A unified 3D framework for Organs at Risk Localization and Segmentation
for Radiation Therapy Planning [56.52933974838905]
Current medical workflow requires manual delineation of organs-at-risk (OAR)
In this work, we aim to introduce a unified 3D pipeline for OAR localization-segmentation.
Our proposed framework fully enables the exploitation of 3D context information inherent in medical imaging.
arXiv Detail & Related papers (2022-03-01T17:08:41Z) - FUN-SIS: a Fully UNsupervised approach for Surgical Instrument
Segmentation [16.881624842773604]
We present FUN-SIS, a Fully-supervised approach for binary Surgical Instrument.
We train a per-frame segmentation model on completely unlabelled endoscopic videos, by relying on implicit motion information and instrument shape-priors.
The obtained fully-unsupervised results for surgical instrument segmentation are almost on par with the ones of fully-supervised state-of-the-art approaches.
arXiv Detail & Related papers (2022-02-16T15:32:02Z) - Multimodal Semantic Scene Graphs for Holistic Modeling of Surgical
Procedures [70.69948035469467]
We take advantage of the latest computer vision methodologies for generating 3D graphs from camera views.
We then introduce the Multimodal Semantic Graph Scene (MSSG) which aims at providing unified symbolic and semantic representation of surgical procedures.
arXiv Detail & Related papers (2021-06-09T14:35:44Z) - Generalized Organ Segmentation by Imitating One-shot Reasoning using
Anatomical Correlation [55.1248480381153]
We propose OrganNet which learns a generalized organ concept from a set of annotated organ classes and then transfer this concept to unseen classes.
We show that OrganNet can effectively resist the wide variations in organ morphology and produce state-of-the-art results in one-shot segmentation task.
arXiv Detail & Related papers (2021-03-30T13:41:12Z) - Robust Medical Instrument Segmentation Challenge 2019 [56.148440125599905]
Intraoperative tracking of laparoscopic instruments is often a prerequisite for computer and robotic-assisted interventions.
Our challenge was based on a surgical data set comprising 10,040 annotated images acquired from a total of 30 surgical procedures.
The results confirm the initial hypothesis, namely that algorithm performance degrades with an increasing domain gap.
arXiv Detail & Related papers (2020-03-23T14:35:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.