PC-Droid: Faster diffusion and improved quality for particle cloud generation

• URL: http://arxiv.org/abs/2307.06836v3
• Date: Fri, 18 Aug 2023 16:33:39 GMT
• Title: PC-Droid: Faster diffusion and improved quality for particle cloud generation
• Authors: Matthew Leigh, Debajyoti Sengupta, John Andrew Raine, Guillaume Qu\'etant, Tobias Golling
• Abstract summary: Building on the success of PC-JeDi we introduce PC-Droid, a substantially improved diffusion model for the generation of jet particle clouds. By leveraging a new diffusion formulation, studying more recent integration solvers, and training on all jet types simultaneously, we are able to achieve state-of-the-art performance for all types of jets.
• Score: 0.9374652839580183
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Building on the success of PC-JeDi we introduce PC-Droid, a substantially improved diffusion model for the generation of jet particle clouds. By leveraging a new diffusion formulation, studying more recent integration solvers, and training on all jet types simultaneously, we are able to achieve state-of-the-art performance for all types of jets across all evaluation metrics. We study the trade-off between generation speed and quality by comparing two attention based architectures, as well as the potential of consistency distillation to reduce the number of diffusion steps. Both the faster architecture and consistency models demonstrate performance surpassing many competing models, with generation time up to two orders of magnitude faster than PC-JeDi and three orders of magnitude faster than Delphes.

Related papers

• FlowTurbo: Towards Real-time Flow-Based Image Generation with Velocity Refiner [70.90505084288057]
  Flow-based models tend to produce a straighter sampling trajectory during the sampling process. We introduce several techniques including a pseudo corrector and sample-aware compilation to further reduce inference time. FlowTurbo reaches an FID of 2.12 on ImageNet with 100 (ms / img) and FID of 3.93 with 38 (ms / img)
  arXiv  Detail & Related papers  (2024-09-26T17:59:51Z)
• Efficient and Scalable Point Cloud Generation with Sparse Point-Voxel Diffusion Models [6.795447206159906]
  We propose a novel point cloud U-Net diffusion architecture for 3D generative modeling. Our network employs a dual-branch architecture, combining the high-resolution representations of points with the computational efficiency of sparse voxels. Our model excels in all tasks, establishing it as a state-of-the-art diffusion U-Net for point cloud generative modeling.
  arXiv  Detail & Related papers  (2024-08-12T13:41:47Z)
• Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation [59.184980778643464]
  Fine-tuning Diffusion Models remains an underexplored frontier in generative artificial intelligence (GenAI) In this paper, we introduce an innovative technique called self-play fine-tuning for diffusion models (SPIN-Diffusion) Our approach offers an alternative to conventional supervised fine-tuning and RL strategies, significantly improving both model performance and alignment.
  arXiv  Detail & Related papers  (2024-02-15T18:59:18Z)
• Fast Training of Diffusion Transformer with Extreme Masking for 3D Point Clouds Generation [64.99362684909914]
  We propose FastDiT-3D, a novel masked diffusion transformer tailored for efficient 3D point cloud generation. We also propose a novel voxel-aware masking strategy to adaptively aggregate background/foreground information from voxelized point clouds. Our method achieves state-of-the-art performance with an extreme masking ratio of nearly 99%.
  arXiv  Detail & Related papers  (2023-12-12T12:50:33Z)
• One-Step Diffusion Distillation via Deep Equilibrium Models [64.11782639697883]
  We introduce a simple yet effective means of distilling diffusion models directly from initial noise to the resulting image. Our method enables fully offline training with just noise/image pairs from the diffusion model. We demonstrate that the DEQ architecture is crucial to this capability, as GET matches a $5times$ larger ViT in terms of FID scores.
  arXiv  Detail & Related papers  (2023-12-12T07:28:40Z)
• Co-training and Co-distillation for Quality Improvement and Compression of Language Models [88.94539115180919]
  Knowledge Distillation (KD) compresses expensive pre-trained language models (PLMs) by transferring their knowledge to smaller models. Most smaller models fail to surpass the performance of the original larger model, resulting in sacrificing performance to improve inference speed. We propose Co-Training and Co-Distillation (CTCD), a novel framework that improves performance and inference speed together by co-training two models.
  arXiv  Detail & Related papers  (2023-11-06T03:29:00Z)
• EPiC-ly Fast Particle Cloud Generation with Flow-Matching and Diffusion [0.7255608805275865]
  We present two novel methods that generate LHC jets as point clouds efficiently and accurately. epcjedi and ep both achieve state-of-the-art performance on the top-quark JetNet datasets.
  arXiv  Detail & Related papers  (2023-09-29T18:00:03Z)
• ProtoDiffusion: Classifier-Free Diffusion Guidance with Prototype Learning [0.0]
  In this work, we incorporate prototype learning into diffusion models to achieve high generation quality faster than the original diffusion model. We observe that our method, called ProtoDiffusion, achieves better performance in the early stages of training compared to the baseline method.
  arXiv  Detail & Related papers  (2023-07-04T21:18:39Z)
• PC-JeDi: Diffusion for Particle Cloud Generation in High Energy Physics [0.8246494848934447]
  We present a new method to efficiently generate jets in High Energy Physics called PC-JeDi. This method uses score-based diffusion models in conjunction with transformers which are well suited to the task of generating jets as particle clouds. PC-JeDi achieves competitive performance with current state-of-the-art methods across several metrics that evaluate the quality of the generated jets.
  arXiv  Detail & Related papers  (2023-03-09T16:23:49Z)
• Block Pruning For Faster Transformers [89.70392810063247]
  We introduce a block pruning approach targeting both small and fast models. We find that this approach learns to prune out full components of the underlying model, such as attention heads.
  arXiv  Detail & Related papers  (2021-09-10T12:46:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.