Leveraging Multimodal Generative AI to Foster a Creative Mindset and Expand Perception

?We have been getting increasing traction for our Generative AI for Creative professionals? at the University Of Oxford - especially post SORA. I am going to make some more announcements soon

But I have another interesting idea

Why not make creativity accessible for everyone through GenAI? Ie everyone can be an artist or a creator in their own way

I came to this idea in an unusual way

I like reading about art and literature. The logical (so-called left brain thinking) comes more naturally to me (maths/ AI etc) -but I have used my wider reading to explore about the creative side, especially to overcome my own limitations due to high functioning autism.?

I have been reading a book called Drawing on the right side of the brain by Betty Edwards which has sold over two million copies!??

The book teaches drawing by shifting the mental mode of the artist from a logical, left-brain perspective to a visual, right-brain perspective. In this sense, its true subject is perception.?

I can correlate it to two other books I have re Leonardo Da Vinci in the cover image (about whom I have read extensively). If you see Leonardo’s notebooks - you see the same thing ..? ie an entirely different way to look at the ordinary. Also disccussed extensively in Walter Isaacsson's biography of Leonardo

There is an endless creative way in which we, as individuals, can rethink the world creatively. My favourite example is Howard Rheinglod’s painted shoes :)??

So, if creativity can be thought of in terms of a different form of perception i.e. learning to see in a different way? - then how can we (both professional artists - but also everyone in general) adopt a creative mindset and improve perception using new gen AI multimodal tools

Multimodal AI tools like Dall-e integrate visual, textual, and other sensory modalities—to significantly improve perception and enhance creative processes in a number of ways (I used chatGPT for this list)

• AI-Generated Visual Breakdowns: Tools like DALL·E and Stable Diffusion can deconstruct complex images into simpler visual elements like shapes, colors, and textures, helping artists better analyze composition and relationships.
• Enhanced Detail Recognition: AI-powered zoom and resolution tools (e.g., Gigapixel AI) can amplify minute details in reference images, enhancing an artist’s ability to perceive subtle textures, patterns, and transitions.
• Style Transfer Models: AI models can convert images into the style of famous artists (e.g., Van Gogh, Monet). By observing these transformations, artists can study how different styles reinterpret light, form, and color.

• Textual Explanations of Visuals: Multimodal AI tools like OpenAI’s GPT-4 with vision can describe and analyze images, pointing out features like balance, contrast, and focus that an artist might overlook.
• Emotion and Mood Mapping: AI can analyze and generate visual cues to convey specific emotions or moods, providing artists with new frameworks for expressing themes and feelings.

• Dynamic Prompting for Exploration: Artists can input textual descriptions of concepts (e.g., "a surrealist dreamscape with flowing liquid textures") into multimodal tools to generate visual outputs, revealing interpretations they might not have envisioned.
• Combining Modalities: Tools like RunwayML or Adobe Firefly allow artists to experiment with combining text, images, and sounds, offering a holistic approach to visual storytelling.

• 3D Model Integration: AI tools like NVIDIA Omniverse can create 3D models from 2D sketches, allowing artists to explore spatial relationships and perspective in a virtual environment.
• Depth Perception and Light Simulation: AI can simulate light and shadow effects on objects or scenes, giving artists new ways to perceive and understand depth, form, and lighting dynamics.

• Interactive Feedback Loops: Tools like Procreate with AI plugins or Photoshop Sensei can provide real-time suggestions to improve balance, alignment, and proportions, training the artist’s eye for detail.
• Iterative Learning with AI: Artists can iteratively refine their work using AI-generated critiques and enhancements, which sharpen their ability to identify strengths and weaknesses in their creations.

• Cross-Modal Connections: AI tools that combine text, sound, and visuals can inspire artists to think beyond the visual domain, integrating abstract ideas or auditory cues into their work.
• Moodboards and Idea Generation: Platforms like Canva or Adobe Express now incorporate AI to create multimodal mood boards, helping artists visualize and expand their creative ideas.

• AI as a Co-Creator: Multimodal tools enable artists to collaborate with AI to explore alternative compositions, styles, and interpretations, improving their ability to perceive and adapt to different approaches.
• Dynamic Style Adaptation: Artists can feed their work into AI systems to explore how their style adapts across different genres or media, gaining a deeper understanding of their own creative signatures.

• Analyzing Masterpieces: AI can break down classic works of art, providing insights into composition, use of light, and symbolism, which can be applied to the artist’s own work.
• Exploring Cultural Aesthetics: Multimodal AI tools can generate visuals inspired by diverse cultural aesthetics, enhancing the artist’s perception of global artistic traditions and trends.

• Improving Perception for Neurodiverse Artists: AI tools can be customized to enhance visual perception for artists with neurodiverse conditions, offering personalized feedback or simplifications to aid their creative process.
• Augmented Vision for the Visually Impaired: Multimodal AI can translate visual elements into other modalities (e.g., sound or touch), enabling artists with visual impairments to perceive and engage with their creations in innovative ways.

• Visual Journals Powered by AI: Artists can use AI to create visual logs of their progress, reflecting on their evolving perceptual skills over time.
• AI-Generated Challenges: Tools can propose exercises to push artists outside their comfort zones, such as drawing scenes in unconventional perspectives or reinterpreting abstract concepts.

?I plan to explore some of these ideas to create an artefact of my own. I find these ideas inspiring i.e. use of Multimodal Generative AI to foster a creative mindset through expanded perception. I am planning to include a session on creativity in our Generative AI for Creative professionals? at the University Of Oxford and make creativity a key part of this course