CCC++: Optimized Color Classified Colorization with Segment Anything Model (SAM) Empowered Object Selective Color Harmonization

• URL: http://arxiv.org/abs/2403.11494v2
• Date: Mon, 24 Jun 2024 05:29:10 GMT
• Title: CCC++: Optimized Color Classified Colorization with Segment Anything Model (SAM) Empowered Object Selective Color Harmonization
• Authors: Mrityunjoy Gain, Avi Deb Raha, Rameswar Debnath,
• Abstract summary: We formulate the colorization problem into a multinomial classification problem and then apply a weighted function to classes. To optimize the classes, we experiment with different bin sizes for color class transformation. We propose two new color image evaluation metrics, the Color Class Activation Ratio (CCAR) and the True Activation Ratio (TAR)
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this paper, we formulate the colorization problem into a multinomial classification problem and then apply a weighted function to classes. We propose a set of formulas to transform color values into color classes and vice versa. To optimize the classes, we experiment with different bin sizes for color class transformation. Observing class appearance, standard deviation, and model parameters on various extremely large-scale real-time images in practice we propose 532 color classes for our classification task. During training, we propose a class-weighted function based on true class appearance in each batch to ensure proper saturation of individual objects. We adjust the weights of the major classes, which are more frequently observed, by lowering them, while escalating the weights of the minor classes, which are less commonly observed. In our class re-weight formula, we propose a hyper-parameter for finding the optimal trade-off between the major and minor appeared classes. As we apply regularization to enhance the stability of the minor class, occasional minor noise may appear at the object's edges. We propose a novel object-selective color harmonization method empowered by the Segment Anything Model (SAM) to refine and enhance these edges. We propose two new color image evaluation metrics, the Color Class Activation Ratio (CCAR), and the True Activation Ratio (TAR), to quantify the richness of color components. We compare our proposed model with state-of-the-art models using six different dataset: Place, ADE, Celeba, COCO, Oxford 102 Flower, and ImageNet, in qualitative and quantitative approaches. The experimental results show that our proposed model outstrips other models in visualization, CNR and in our proposed CCAR and TAR measurement criteria while maintaining satisfactory performance in regression (MSE, PSNR), similarity (SSIM, LPIPS, UIUI), and generative criteria (FID).

Related papers

• Reinforcing Pre-trained Models Using Counterfactual Images [54.26310919385808]
  This paper proposes a novel framework to reinforce classification models using language-guided generated counterfactual images. We identify model weaknesses by testing the model using the counterfactual image dataset. We employ the counterfactual images as an augmented dataset to fine-tune and reinforce the classification model.
  arXiv  Detail & Related papers  (2024-06-19T08:07:14Z)
• CCC: Color Classified Colorization [0.0]
  We formulate the colorization problem into a multinomial classification problem and then apply a weighted function to classes. We propose a set of formulas to transform color values into color classes and vice versa. We compare our proposed model with state-of-the-art models using five different datasets.
  arXiv  Detail & Related papers  (2024-03-03T11:00:15Z)
• CoinSeg: Contrast Inter- and Intra- Class Representations for Incremental Segmentation [85.13209973293229]
  Class incremental semantic segmentation aims to strike a balance between the model's stability and plasticity. We propose Contrast inter- and intra-class representations for Incremental (CoinSeg)
  arXiv  Detail & Related papers  (2023-10-10T07:08:49Z)
• Incorporating Ensemble and Transfer Learning For An End-To-End Auto-Colorized Image Detection Model [0.0]
  This paper presents a novel approach that combines the advantages of transfer and ensemble learning approaches to help reduce training time and resource requirements. The proposed model shows promising results, with accuracy ranging from 94.55% to 99.13%.
  arXiv  Detail & Related papers  (2023-09-25T19:22:57Z)
• Unicom: Universal and Compact Representation Learning for Image Retrieval [65.96296089560421]
  We cluster the large-scale LAION400M into one million pseudo classes based on the joint textual and visual features extracted by the CLIP model. To alleviate such conflict, we randomly select partial inter-class prototypes to construct the margin-based softmax loss. Our method significantly outperforms state-of-the-art unsupervised and supervised image retrieval approaches on multiple benchmarks.
  arXiv  Detail & Related papers  (2023-04-12T14:25:52Z)
• Equalization and Brightness Mapping Modes of Color-to-Gray Projection Operators [0.0]
  The conversion of color RGB images to grayscale is covered by characterizing the mathematical operators used to project 3 color channels to a single one. Three classes of EQ modes and two classes of BM modes were found in linear operators, defining a 6-class taxonomy. It was found that most current metrics used to assess the quality of color-to-gray conversions better assess one of the two BM mode classes, but the ideal operator chosen by a human team belongs to the other class.
  arXiv  Detail & Related papers  (2022-08-21T19:23:06Z)
• Rank4Class: A Ranking Formulation for Multiclass Classification [26.47229268790206]
  Multiclass classification (MCC) is a fundamental machine learning problem. We show that it is easy to boost MCC performance with a novel formulation through the lens of ranking.
  arXiv  Detail & Related papers  (2021-12-17T19:22:37Z)
• Color Counting for Fashion, Art, and Design [0.0]
  First step in color modelling is to estimate the number of colors in the item / object. We propose a novel color counting method based on cumulative color histogram. This work is the first of its kind that addresses the problem of color-counting machine.
  arXiv  Detail & Related papers  (2021-10-13T12:42:15Z)
• On Model Calibration for Long-Tailed Object Detection and Instance Segmentation [56.82077636126353]
  We propose NorCal, Normalized for long-tailed object detection and instance segmentation. We show that separately handling the background class and normalizing the scores over classes for each proposal are keys to achieving superior performance.
  arXiv  Detail & Related papers  (2021-07-05T17:57:20Z)
• Multiscale Deep Equilibrium Models [162.15362280927476]
  We propose a new class of implicit networks, the multiscale deep equilibrium model (MDEQ) An MDEQ directly solves for and backpropagates through the equilibrium points of multiple feature resolutions simultaneously. We illustrate the effectiveness of this approach on two large-scale vision tasks: ImageNet classification and semantic segmentation on high-resolution images from the Cityscapes dataset.
  arXiv  Detail & Related papers  (2020-06-15T18:07:44Z)
• Improving Few-shot Learning by Spatially-aware Matching and CrossTransformer [116.46533207849619]
  We study the impact of scale and location mismatch in the few-shot learning scenario. We propose a novel Spatially-aware Matching scheme to effectively perform matching across multiple scales and locations.
  arXiv  Detail & Related papers  (2020-01-06T14:10:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.