PortfolioMentor: Multimodal Generative AI Companion for Learning and Crafting Interactive Digital Art Portfolios

• URL: http://arxiv.org/abs/2311.14091v1
• Date: Thu, 23 Nov 2023 16:36:40 GMT
• Title: PortfolioMentor: Multimodal Generative AI Companion for Learning and Crafting Interactive Digital Art Portfolios
• Authors: Tao Long, Weirui Peng
• Abstract summary: Digital art portfolios serve as impactful mediums for artists to convey their visions, weaving together visuals, audio, interactions, and narratives. Without technical backgrounds, design students often find it challenging to translate creative ideas into tangible codes and designs. This tool guides and collaborates with students through proactive suggestions and responsible Q&As for learning, inspiration, and support. In detail, the system starts with the understanding of the task and artist's visions, follows the co-creation of visual illustrations, audio or music suggestions and files, click-scroll effects for interactions, and creative vision conceptualization.
• Score: 1.8130068086063336
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Digital art portfolios serve as impactful mediums for artists to convey their visions, weaving together visuals, audio, interactions, and narratives. However, without technical backgrounds, design students often find it challenging to translate creative ideas into tangible codes and designs, given the lack of tailored resources for the non-technical, academic support in art schools, and a comprehensive guiding tool throughout the mentally demanding process. Recognizing the role of companionship in code learning and leveraging generative AI models' capabilities in supporting creative tasks, we present PortfolioMentor, a coding companion chatbot for IDEs. This tool guides and collaborates with students through proactive suggestions and responsible Q&As for learning, inspiration, and support. In detail, the system starts with the understanding of the task and artist's visions, follows the co-creation of visual illustrations, audio or music suggestions and files, click-scroll effects for interactions, and creative vision conceptualization, and finally synthesizes these facets into a polished interactive digital portfolio.

Related papers

• Multimodal Large Language Models and Tunings: Vision, Language, Sensors, Audio, and Beyond [51.141270065306514]
  This tutorial aims to equip researchers, practitioners, and newcomers with the knowledge and skills to leverage multimodal AI. We will cover the latest multimodal datasets and pretrained models, including those beyond vision and language. Hands-on laboratories will offer practical experience with state-of-the-art multimodal models.
  arXiv  Detail & Related papers  (2024-10-08T01:41:56Z)
• Fostering Creative Visualisation Skills Through Data-Art Exhibitions [0.9463895540925061]
  We present our implementation of a data-art exhibition within a computing curriculum, for third-year degree-level students. Students create art-based visualisations from selected datasets and present their work in a public exhibition.
  arXiv  Detail & Related papers  (2024-08-29T12:16:13Z)
• Diffusion-Based Visual Art Creation: A Survey and New Perspectives [51.522935314070416]
  This survey explores the emerging realm of diffusion-based visual art creation, examining its development from both artistic and technical perspectives. Our findings reveal how artistic requirements are transformed into technical challenges and highlight the design and application of diffusion-based methods within visual art creation. We aim to shed light on the mechanisms through which AI systems emulate and possibly, enhance human capacities in artistic perception and creativity.
  arXiv  Detail & Related papers  (2024-08-22T04:49:50Z)
• MetaDesigner: Advancing Artistic Typography through AI-Driven, User-Centric, and Multilingual WordArt Synthesis [65.78359025027457]
  MetaDesigner revolutionizes artistic typography by leveraging the strengths of Large Language Models (LLMs) to drive a design paradigm centered around user engagement. A comprehensive feedback mechanism harnesses insights from multimodal models and user evaluations to refine and enhance the design process iteratively. Empirical validations highlight MetaDesigner's capability to effectively serve diverse WordArt applications, consistently producing aesthetically appealing and context-sensitive results.
  arXiv  Detail & Related papers  (2024-06-28T11:58:26Z)
• What's Next? Exploring Utilization, Challenges, and Future Directions of AI-Generated Image Tools in Graphic Design [2.0616038498705858]
  This study conducted semi-structured interviews with seven designers of varying experience levels to understand their current usage, challenges, and future needs for AI-generated image tools in graphic design. As our findings suggest, AI tools serve as creative partners in design, enhancing human creativity, offering strategic insights, and fostering team collaboration and communication. The findings provide guiding recommendations for the future development of AI-generated image tools, aimed at helping engineers optimize these tools to better meet the needs of graphic designers.
  arXiv  Detail & Related papers  (2024-06-19T10:51:56Z)
• Equivalence: An analysis of artists' roles with Image Generative AI from Conceptual Art perspective through an interactive installation design practice [16.063735487844628]
  This study explores how artists interact with advanced text-to-image Generative AI models. To exemplify this framework, a case study titled "Equivalence" converts users' speech input into continuously evolving paintings. This work aims to broaden our understanding of artists' roles and foster a deeper appreciation for the creative aspects inherent in artwork created with Image Generative AI.
  arXiv  Detail & Related papers  (2024-04-29T02:45:23Z)
• MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks [59.09343552273045]
  We propose a decoder-only model for multimodal tasks, which is surprisingly effective in jointly learning of these disparate vision-language tasks. We demonstrate that joint learning of these diverse objectives is simple, effective, and maximizes the weight-sharing of the model across these tasks. Our model achieves the state of the art on image-text and text-image retrieval, video question answering and open-vocabulary detection tasks, outperforming much larger and more extensively trained foundational models.
  arXiv  Detail & Related papers  (2023-03-29T16:42:30Z)
• Pathway to Future Symbiotic Creativity [76.20798455931603]
  We propose a classification of the creative system with a hierarchy of 5 classes, showing the pathway of creativity evolving from a mimic-human artist to a Machine artist in its own right. In art creation, it is necessary for machines to understand humans' mental states, including desires, appreciation, and emotions, humans also need to understand machines' creative capabilities and limitations. We propose a novel framework for building future Machine artists, which comes with the philosophy that a human-compatible AI system should be based on the "human-in-the-loop" principle.
  arXiv  Detail & Related papers  (2022-08-18T15:12:02Z)
• Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides [57.86931911522967]
  We test the capabilities of machine learning models in multimodal understanding of educational content. Our dataset contains aligned slides and spoken language, for 180+ hours of video and 9000+ slides, with 10 lecturers from various subjects. We introduce PolyViLT, a multimodal transformer trained with a multi-instance learning loss that is more effective than current approaches.
  arXiv  Detail & Related papers  (2022-08-17T05:30:18Z)
• Sketch-based Creativity Support Tools using Deep Learning [23.366634691081593]
  Recent developments in deep-learning models drastically improved machines' ability in understanding and generating visual content. An exciting area of development explores deep-learning approaches used to model human sketches, opening opportunities for creative applications. This chapter describes three fundamental steps in developing deep-learning-driven creativity support tools that consumes and generates sketches.
  arXiv  Detail & Related papers  (2021-11-19T00:57:43Z)
• Generating Music and Generative Art from Brain activity [0.0]
  This research work introduces a computational system for creating generative art using a Brain-Computer Interface (BCI) The generated artwork uses brain signals and concepts of geometry, color and spatial location to give complexity to the autonomous construction.
  arXiv  Detail & Related papers  (2021-08-09T19:33:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.