Pushing Boundaries in Immersive Art: The Melt AI Success Story of Kamila Staszczyszyn & Kuba Matyka
In an era where pixels meet imagination, Kamila Staszczyszyn and Kuba Matyka have not only mastered the art of crafting immersive experiences but have also harnessed the power of AI to redefine creativity itself. As co-founders of MELT AI, their journey delves into the crossroads of AI and art, revealing how technology breathes new life into their immersive endeavors.
Journey into AI: Kamila and Kuba, could you share how your personal paths led you to co-found MELT.AI, and when did you first realize the potential of AI in art?
Kuba: melt.ai is kind of a little sibling to our main endeavor MELT IMMERSIVE studio that creates immersive experiences in the real world. We worked with many technologies over the years, as our first ever motto was ‘When Art meets Technology, Magic and Wonder will follow ✨’ and we’ve had many awe-strucked moments of recognizing infinite possibilities, like trying out the first Oculus prototype. So we instantly knew - generative AI is going to change the world of creativity.
Kamila: We were always drawn to technologies that could amplify the emotional impact of our work. It was an early summer of 22’ and we’ve been hooked on trying out Dall-e and other early models but we got really addicted to Midjourney. It was sleepless nights of trying to make it do what we wanted and seeing what it could imagine with the most abstract prompts and pop-culture mash-ups. There were a lot of Mickey Mouse, Moebius, Star Wars, Katsuhiro Otomo and H.R. Giger there ;) Our aim was to see how far we could push the boundaries, and we've been doing it ever since. We quickly figured out we wanted to do the same in motion, and we needed a lot more control over the effect.
Kuba: When we first started to integrate AI into our creative pipeline, the results were nothing short of magical. It was like gaining a new superpower; the ability to create visuals and narratives that were previously unimaginable. We knew then that AI would be central to our future endeavors.
The Creative Choreography: Can you walk us through the intricate dance between human ingenuity and AI algorithms that defines the development process of your latest AI-powered work?
Kamila: We always start with the concept - a visual story we want to tell. From the get-go, we sketch out the framework of what the video should convey. Then we create the video input - whether it’s a filmed video, a 3D animation or a video with special effects. Only then we start to experiment with generative AI.
Kuba: It’s a symbiotic creation. It's like a dialogue between our creative sensibilities and the almost limitless potential of AI to generate new forms and interpretations.
Kamila: Precisely. And it's this partnership between human creativity and AI capability that allows us to push the boundaries of what video art can be.
Kuba: It’s also a lot of back and forth. When we don’t get good results, we tweak the input videos until we’re satisfied. The input video is about 70% of the final output of most of our works - AI just adds a lot of layers of craziness to it 😀
Kamila: Worth mentioning that we use a lot of tools that use non-generative AI in the background for mundane tasks, like masking, motion capture, and such.
Technology at Your Fingertips: What specific software and tools have you found most effective in bringing your AI art projects to life?
Kuba: Stable Diffusion on the AI part. It’s open source, and the community is thriving, developing tools and techniques together and training hundreds of custom models. It’s an amazing thing to be a part of. Just check civitai.com to see for yourselves - there is a specific model for virtually everything.
Kamila: It reminds me of the times where there was no ControlNet yet. We’ve done some of our best work, but it was so uncontrollable ;D
Kuba: We also use our classic toolset a lot, Like Cinema 4D and Unreal Engine for 3D animation and After Effects for VFX and compositing.
Unraveling Challenges: Every project presents its unique obstacles. Could you recount a particularly vexing challenge you encountered while working with AI and how you overcame it?
Kamila: Yeah, our work is basically problem solving since always 😛 One of our great challenges when doing music videos for pop-stars was lip-sync. Unless the face was really huge in the frame, AI couldn’t get it right even when changing just 20-30% of the image. We ended up composting fragments of real faces back in the image manually.
Kuba: We also had to train custom models on their photos to have them somehow consistent.
Kamila: But the biggest challenge of our work always was to sell the idea before it’s ever been done. We think of ideas and concepts, but it’s a lot of work to visualize them. We help ourselves with references and moodboards, but it’s not the same as we imagined and clients have just to trust us based on our portfolio. It’s always the most intense part of the process before trying to finish up before the deadline xd.
The Pulse of Inspiration: As artists immersed in the realm of AI, what themes, concepts, or sources of inspiration particularly resonate with you when crafting your digital masterpieces?
Kamila: For me, the concept of 'realism meeting the surreal' is a rich vein of inspiration. AI has this unique ability to take everyday imagery and transform it into something completely otherworldly. We live in an era where the boundary between the real and virtual is increasingly blurry, and our work aims to explore this convergence in novel ways.
Kuba: My inspirations are much more down to earth, like scenes from the movies, games, anime, VR, and works of new-media artists, that made a lasting impact on me. I’m a huge movie geek and I love sci-fi. I watched the new Blade Runner 6 or 7 times in imax xd.
Kamila: When it comes to pop culture, especially movies, games, and anime, I'm enthralled by the world-building aspects. These mediums excel at creating universes that are not only visually stunning but also narratively complex. Whether it's the cyberpunk aesthetic of "Blade Runner" or the magical realism of Studio Ghibli films, they offer a rich tapestry of ideas that can be woven into our AI-generated videos.
Kuba: And let's not forget about our fascination with the human psyche. AI has the ability to generate imagery that taps into subconscious thoughts and emotions. It allows us to visualize abstract concepts like dreams and psychedelic trips - making the intangible tangible.
Kamila: Our work aims to be a melting pot of these diverse influences, channeled through the lens of AI. The end goal is always to create something that makes the viewer pause and get lost in it, to evoke a sense of wonder or astonishment.
AI's Sweet Spot: Within the realm of AI, do you have a preferred subject or area of exploration that excites you the most? What draws you to that aspect of AI?
Kuba: I think it’s the ability to conjure visually stunning sci-fi scenes. It's like stepping into a frame of a Ridley Scott or Stanley Kubrick movie—complex, multi-layered, and mind-bendingly beautiful, but presented in ways that even those creators couldn't have foreseen.
Kamila: Exactly, and when we add our own hypnotizing, trippy, and immersive vibe it’s a perfect combination. It's about pushing the boundaries of visual art into a realm that is as aesthetically mesmerizing and absorbing.
The Heart of Creation: When generating AI-driven art, what qualities or aesthetics are you most often seeking to infuse into your work?
Kamila: In every piece of art we create, there's a quest for an emotional resonance that captures the viewer. We aim for aesthetics that are not just visually striking, but also evoke a sense of wonder and contemplation. It's about adding layers of meaning beneath the beautiful façade.
Kuba: For me, it's about unpredictability and the element of surprise. I love it when our projects offer something unexpected, making each experience unique. This unpredictability, coupled with an engaging and immersive atmosphere, truly allows each piece to come to life in its own way.
Kamila: It's the combination of these aesthetics - emotional resonance, unpredictability, immersive engagement and a tone of detail - that forms the core of our creative philosophy at MELT IMMERSIVE. We want the viewer to not just see, but to feel and engage with our work on multiple levels.
Dreams and Aspirations: Beyond the present, what are your artistic ambitions when it comes to AI? How do you envision the fusion of AI and art evolving?
Kuba: I'm particularly enthralled by the capabilities of AI in real-time interaction. The potential for AI to react to user input or even environmental factors in real-time opens up avenues for truly immersive experiences.
Kamila. Imagine a dream-like, otherworldly virtual realm that you can not just see, but interact with. With real-time algorithms, these landscapes evolve as you engage with them, offering an ever-changing, dynamic art experience.
Kuba: It’s like stepping into an artificial dream.
Kamila: Haha, exactly! Artificial dream is the title of our first solo exhibition in our own immersive venue we’re opening in October in Warsaw called melt museum.
Kuba: It's not just about creating a ‘passive’ piece of art; it's about forming a living, interactive entity. The guest is not just a viewer but becomes an active participant in a dynamically changing environment. And this isn't confined to one digital screen; we're talking about full-blown immersive experiences that can transform physical spaces as well.
Kamila: This is what gets us out of bed in the morning, the potential to create not just groundbreaking art, but transformative experiences.
Idols of Influence: Who are your favorite artists, whether they hail from traditional or digital backgrounds, and how have they influenced your artistic journey?
Kamila: Sorayama's techno aesthetic and hyper-realistic forms have significantly influenced my desire to blend the organic with the mechanical in my art. Similarly, Six'n'five's genre-defining 3D aesthetic and digital landscapes encourage me to push the boundaries of what's possible.
Kuba: Spielberg's ability to evoke a sense of awe, wonder and discovery and Kubrick's revolutionary visual storytelling techniques have deeply inspired my creative methods. Jodorowsky and Moebius for connecting a surreal, psychedelic vibe with a massive sci-fi story and out-of-this-world visuals and designs.
Kamila: Science fiction and fantasy films in general. Even soundtracks from films like "Harry Potter" and "Blade Runner" have played a significant role in developing my imagination.
Kuba: Yayoi Kusama and James Turrell - the first immersive artists I became aware of.
Kamila: Hieronymus Bosch’s meticulous detailing in his paintings.The unique artistry of Tyler the Creator, whose every album presents a new persona, and Daniel Arsham's interpretations of pop culture, further shaped my approach to art and storytelling.
Kuba: In the realm of live experiences, Es Devlin and Willo Perron have both been especially influential. They have masterminded some of the most captivating concert setups for iconic musicians like Kanye West, Drake, and The Weeknd.
Guiding the Aspiring: Drawing from your first hand experience, what advice would you give to emerging artists who wish to delve into the captivating realm of AI-powered creativity?
Kuba: Experiment. Get lost in the tools. Try something new - it’s all trial and error. Right now we have all the tools in the world at the palm of our hand. If you have an overwhelming desire to create something - you can, and you will.
Kamila: Collaboration is key. AI is not just a tool; it's also a partner in the creative process. Interact with it, challenge it, and let it challenge you. But also, don't work in isolation. The AI art community is incredibly open and collaborative. Share your successes and failures alike. Some of our most innovative moments came from combining different perspectives and techniques.
NFTs and the Art Landscape: The advent of NFTs has stirred the art world. What's your take on the intersection of AI-generated art and the NFT movement?
Kuba: The advent of NFTs is an exciting development that adds another layer of complexity and opportunity to the art world. They make the ethereal tangible, and that's a revolutionary thing. On the other hand I think the speculative nature of NFTs has overshadowed the art itself.
Kamila: I find the convergence of AI-generated art and NFTs to be transformative for artists. NFTs not only create a unique opportunity for monetization but also open up novel methods of interactivity. Imagine a piece of AI art that evolves over time, or alters based on external conditions, all verified and tracked via NFTs. This offers an unprecedented layer of engagement between the artist, the art, and the audience.
Kuba: Both AI and NFTs have shaken up traditional notions of art, authorship, and ownership. When combined, they can offer a profoundly new way to experience and value art, but it's a landscape that should be navigated with both excitement and ethics in mind.
Peeking into the Future: With MELT.AI at the forefront of AI-infused art, what can enthusiasts and followers anticipate seeing in your next endeavors?
Kamila: Well, in November we’re opening MELT MUSEUM - a REAL immersive venue in the center of Warsaw, Poland.
Kuba: It’s a canvas we always dreamed of - where we can showcase our work outside of social media and your mobile phone and test our newest creations.
Kamila: Where we can see how people interact and respond to them.
Kuba: The concept transcends the traditional concept of a museum, to a sensory journey into the realm of new media art, technology, and immersive entertainment. We’ll have 11 immersive spaces, spanning over 600 square meters, filled with multisensory and interactive installations.
Kamila: And we’re not shying away from AI either, our first permanent exhibition, "Artificial Dreams" delves into the future relationship between humans and artificial intelligence, oscillating between techno-utopia and a cyberpunk vision of the world.
AI and the Canvas: Generative AI has become a force to reckon with in visual art. How have you harnessed its capabilities to infuse your creations with a touch of the digital sublime?
Kamila: For me, generative AI is like an additional layer of intuition, one that transcends human limitations. It's a powerful tool that amplifies the visual storytelling I'm aiming for, especially in the realm of sci-fi aesthetics and techno-futurism. Whether it's creating fantastical landscapes or generating detailed structures, AI offers me the ability to explore dimensions of imagination that were previously unreachable.
Kuba: AI has been a game-changer, but not an easy one to steer into any particular direction. That’s its biggest challenge, but also the greatest opportunity to create something original - combining your sensibilities and aesthetics with the ‘imagination’ of machines. We’re testing every tool that pops out and like I said before - it’s a lot of trial and error.
The Dawn of a New Market: Generative AI is opening doors to novel revenue streams for artists. Could you elaborate on how MELT.AI leverages this technology to forge a fresh path for creative entrepreneurs?
Kuba: It’s kind of a funny story. Apart from our classic business endeavors like doing commissioned work, like AI VFX and music videos, we briefly had an OnlyFans account. It was banned after just two weeks with no means of dispute, despite it having 100% original content made by us, with our own videos just slightly (like 20-30%) enhanced by AI.
Kamila: It broke us, because there were a lot of people asking for it and subscribing. It was a way to monetize the stuff we do other than commissions, and we were starting to build a community there. Each and every film we post on Instagram is a lot of work, and we kind of lost a spark after the ban.
Kuba: Plus, we’re super busy with getting MELT MUSEUM of the ground, so we had a little pause while posting new content.
Kamila: We’re coming back though, pinky promise. Mostly because we miss the little acts of creative expression unbounded by design rules and limitations of the real world.
Kuba: We’ll just have to find a way to make it unrestricted by Instagram's algorithm and rules, but still delivered to fans.
Navigating Uncharted Territories: Integrating AI into the art world isn't without its challenges. Could you share some insights into the hurdles you've encountered while employing generative AI in your projects?
Kamila: Using generative AI to create art is indeed a pioneering field, and the challenges are many. For one, there's the technical hurdle. Understanding and adapting AI algorithms and tools is not the same as picking up a brush. It demands a strong grip on both art and technology, which can be overwhelming at times. Moreover, people are still skeptical about whether AI-generated art can invoke the same emotional depth as traditional forms of art. It's a constant learning curve and a battle against preconceptions.
Kuba: I agree with Kamila; the technical aspects are one part, but the most intriguing challenge is ethical and philosophical. Who owns the art? Is it the algorithm, the person who trained it, or the one who fine-tuned the final output? It's a grey area, which makes it both exciting and precarious. Additionally, there's the question of originality. With an AI generating art, how do you ensure that the work is not just an amalgamation of past artistic styles but is genuinely novel? These are just a few of the issues we've had to navigate while integrating AI into our artistic practice.
Kamila: Exactly, what makes our approach unique is the collaboration between human and AI. We generally create about 70-80% of the piece manually, whether it's video, 3D animation or VFX. Then, we try to direct AI to add style, details and the finishing touches we want and prompt. It's almost like a jazz duet, where we lead with a tune, and then the AI chimes in, creating a more nuanced and complex composition. This collaborative effort allows us to explore artistic realms that we couldn't have reached otherwise.
Kuba: By starting with an almost complete piece, we ensure that the emotional and thematic core is distinctly human. The AI then builds upon that foundation, adding layers of complexity and often surprising us with its own form of creativity. It's an evolving process that has its challenges but also opens up incredibly exciting opportunities for artistic expression. This approach helps us overcome some of the ethical and originality concerns, as the end result is a co-creation that couldn't exist without either of us—human or AI.
Kamila: A lot of people still see it as a worthless work made ‘just by AI’, but we learned to not give a f**k.
Balancing Innovation and Tradition: As AI evolves, concerns arise about its impact on creative careers. How do you perceive the interplay between AI and human creativity in the context of your work?
Kuba: Unfortunately there’s no denying that a lot of people who do generic stuff in the creative fields will eventually lose their jobs. It’s a harsh reality and a challenge for every field in the coming future. But if you have your own voice, or develop your own style - there will always be an audience or a clientele for you. After all, it’s the emotional resonance that’s the most important in the works of art.
Kamila: I see AI as a powerful tool rather than a replacement for human creativity. In fact, I believe it can elevate the artist's vision, allowing us to dive into details or scale that were previously unimaginable. It's not about pitting machines against humans; it's about using advanced technology to enhance our capabilities. To be sure, the emergence of AI raises important ethical questions, especially in a field as intimate and human as art. But I think the key lies in how we integrate this technology into our practices while preserving the humanity and the essence of traditional art forms.
Kuba: The question of AI's impact on the arts often stirs a debate that I find slightly misplaced. AI, in the context of our work, acts more as a collaborator than a competitor. It gives us the ability to explore new dimensions of creative expression while retaining the core elements that make art relatable and emotionally resonant. We're living in a unique period where we can be the architects of this new relationship between human artists and AI. So rather than viewing it as a threat to creative careers, I see it as an expansion of what those careers can entail. Both can coexist, each enhancing the other in unprecedented ways.
The AI Imprint: AI's influence on creative industries is undeniable. How do you foresee AI shaping the future of immersive experiences and storytelling within the context of MELT.AI?
Kuba: In the realm of the future of AI and immersive experiences, we're not just talking about observing worlds - we're talking about stepping into them, experiencing them, and even molding their very fabric with our presence.
Kamila: Exactly, and that's where the real magic lies in the interactivity, in the dialogue between the art, the technology, and you. Think of the most vivid dream you've ever had and now imagine you can step back into it, interact with its elements, and invite others to join you in shaping its narrative.
Kuba: That’s the vision behind the MELT MUSEUM. Imagine rooms not just filled with art, but entire worlds, technology, and human imagination. We're talking about a blend of sights, sounds, and sensations that you can touch, alter, and share. It's interactive on a level that transcends entertainment.
Kamila: You’re not just consuming entertainment; you're getting lost in it.
Additionally, could you provide more information about the intriguing music video project "Stars"? We'd love to get an exclusive look behind the scenes, including any challenging aspects you encountered during this project.
Kuba: Ah, the 'STARS' music video! That was an adventure in itself—a space opera of sorts that we designed from scratch. Picture the quirks and eccentricities of 1970s science fiction, then throw in a dose of modern AI technology and a sprinkle of psychedelic animation. That's 'STARS' for you.
Kamila: It was like choreographing a dance between our artistic vision and the raw potential of AI. Anyone who has worked with AI knows it can be like taming a wild horse. You have some control, but there's always a massive amount of unpredictability. That was particularly challenging when we were coordinating across 4,200 frames while maintaining a cohesive narrative.
Kuba: We actually did a making-of video, it’s a bit long, so Instagram algorithm buried it in the depths of unseen, but it’s all there
