Systematiq Ai

CLAUDE + MAGNIFIC + RUNWAY WORKFLOW 3D-to-Video (full tutorial) ↓ From this post:?https://lnkd.in/eDyV92hU (More context below) PROCESS: 01. Build 3D Renders in Claude 3.7 02. Program camera movements 03. Screen record render 04. Upload video to Runway Gen-3 05. Extract 1st frame 06. Magnific Struct. Ref. 1st frame 07. Upload in Runway Restyle 08. Generate. CLAUDE PROMPT STRUCTURE: can you code a 3d version of [subject + env] in three.js? INITIAL CLAUDE PROMPT: can you code a 3d version of an epic castle atop a mountain plateau in a valley in three.js? MAGNIFIC STRUCT. REF PROMPT: editorial photo, epic castle on a plateau, intricate rocky textures and fine details, immaculate New Zealand landscape, white marble castle, high precision photography MAGNIFIC SETTINGS: + Model: Mystic 2.5 + Structure Reference + Structure Strength: 52% + Resolution: 2k + Creative Detailing: 75% + Engine: Magnific Sharpy CONTEXT: + This is far from perfect or without error + I have no idea what I'm doing w 3D. + I have a lot of respect for 3D artists + Their craft is extremely meticulous + This is a novice attempt at a complex task + I’m hoping there is some use for this + In concepting or pre-viz potentially + At the very least…knowing that it’s possible. QUICK THOUGHTS: + I’ve tried this with a few diff subjects + envs. + It’s replicable for most things? + Make sure to add “three.js” for claude + Or else it might not render the model? + Assuming this can be done in most 3D software + Without the need for Claude? + Runway needs an mp4 input file for Restyle + But it works off the 3D structure very well + I can imagine any 3D artist (who’s open to ai) + Could have a lot of fun playing with this Yes, you can do this w traditional I2V But 3D allows more controlled structure. I’m sure this translates to design pipelines… These tools allow us to “fail quicker” Iterate and ideate in a different way. I see benefit in that…others might not. This is also not immune to criticism.? It’s not immaculate quality or concepting. Just feels like something that could help. Try it out. #midjourney #runwayml #kling #claude #ai #innovation

So this is kinda ridiculous... Claude 3.7 + Magnific + Runway Restyle. (More context below) PROCESS: 01. Build 3D renders in Claude 3.7 02. Program camera movements 03. Screen record render 04. Upload video to Runway Gen-3 05. Extract 1st frame 06. Magnific Struct. Ref. 1st frame 07. Upload in Runway Restyle 08. Generate. INITIAL CLAUDE PROMPT: ok here's a crazy one...can you make a 3d model replica of an epic castle. so we can have full 360-degree view camera motion of the entire thing. Just kept chatting + editing from there. Asked for camera control functions. And screen recorded a video. MAGNIFIC STRUCT. REF PROMPT: high precision cinematic still, epic castle, white marble city, in a New Zealand open landscape, bright natural light, deep contrast and textures CONTEXT: + This is far from perfect or without error + I have no idea what I'm doing w 3D. + I have a lot of respect for 3D artists + Their craft is extremely meticulous + This is a novice attempt at a complex task + I’m hoping there is some use for this + In concepting or pre-viz potentially + At the very least…knowing that it’s possible. QUICK THOUGHTS: + I was unaware Claude could do this + It’s a relatively easy process to edit the model + Magnific does a great job with structure ref. + 3D models don’t have to be created in Claude + Assuming this could be done w any 3d software + But you’d need a video-file of the output? + Runway just needs an mp4 file for Restyle + I can imagine any 3D artist (who’s open to ai) + Could have a lot of fun playing with this Not sure what this means for design pipelines… I apologize in advance for butchering terms.? Like I said, I’m an infant in the 3D space. This is also not immune to criticism.? It’s not immaculate quality or concepting. Just feels like another tipping point. PS: Can do a longer tutorial for those who are interested. #midjourney #runwayml #kling #ai #innovation #claude?

RUNWAY RESTYLE → Video game nostalgia. Unexpected/fun little workflow. (There are probably better ways to do this) But something I was playing with. TOOLS: + VEO2: Video Gen + Premiere: Pixelation + FPS decrease + Magnific: Structure Reference Image + Runway: Restyle V2V PROCESS:? 01. Generate video in Veo2 02. Upload into Runway Gen-3 03. Extract 1st frame in RW 04. Magnific Struct. Ref. 1st frame 05. Upload Veo2 vid in Premiere 06. Increase pixelation + drop FPS 07. Upload vid + ref image into RW 08. Generate WHY: + RW tends to smooth the aesthetic + It’s working off edge detection + If the edges are smooth…it gens smooth? + I wanted a rougher/blocky edges + So adding the pixelation gave it a blocky texture + And dropping it to 18 FPS helped QUICK THOUGHTS: + This is a totally random use-case? + But the overall “ edge theory”…. + Might help someone with other uses + I am a novice with Premiere? + And there’s probably better workflows Somewhat of an interesting workaround. Curious if anyone else has found this in testing. Have fun. #midjourney #runwayml #Veo2 #ai #innovation #kling

RUNWAY RESTYLE → New Feature Alert. It’s just fun + kinda addicting. Using Veo2 + Midjourney + Runway. Opening the doors for motion capture. WHAT IT DOES: 01. You input a video 02. You add an image reference 03. RW "restyles" your video PROCESS: 01. Generate or curate a video. 02. Upload video to Runway Gen-3 03. Runway will extract the 1st frame. 04. Download the 1st frame. 05. Retexture 1st frame. 02. Upload retextured frame. 03. Generate. RETEXTURING: + You can use Midjourney Retexture + Or tools like Magnific Mystic 2.5 + With a “Structure Reference”? + Basically any tool utilizing controlnet MIDJOURNEY RETEXTURE PROCESS: 01. Go to “Edit” tab 02. Upload first frame 03. Select “Rextexture”? 04. Add a text prompt 05. Generate. MIDJOURNEY PROMPT: 8-bit video game style —s 25 —ar 5:3 QUICK THOUGHTS: + It’s super fast…typically 30s + 20 second max video upload + Outputs at 720p 24 FPS + Sometimes output can be choppy + Using topaz to increase FPS can help + Output can stray from ref image? + It’s very input image specific + It’s a super fun use for V2V + You can get creative with it + Runway’s previous V2v was good w. This + But having more control w ref image is great Going down the rabbit hole on this. More uses to come. Have fun. #midjourney #runwayml #Veo2 #kling #ai #innovation?

Kinda absurd Veo2 can do this… SB Mixtape Vol 1. Super complex…but not perfect. (Still impressive) TOOLS USED: + Veo2: Video gen + Topaz Theia: UpRes + FPS? + Topaz Starlight: Creative Upscale + MM Audio: Sound gen + ChatGPT: Prompt refinement + CapCut/Premiere: Editing VEO2 PROMPT EXAMPLE: The camera zooms in from below, looking up at the skater silhouetted against the glowing skyline. He pushes off, gaining speed, their hoodie rippling in the night air. A sudden ollie—they land perfectly onto the rail, the metal screeching under their trucks. The city rushes past below, cars lights streaking. The camera tracks tight beside them, locked into their movement. As they near the end of the rail, the shot snaps wide—revealing them landing smooth, rolling off into the glowing city as the neon lights reflect off wet pavement. 35mm film PROBLEM AREAS: + Board morphing? + Hand + arm morphing + Half-pipe physics is off? + Unrealistic board spin + tricks QUICK THOUGHTS: + SB is really complex for AI + It’s human + object + env. interaction + Esp. detaching the object from subject + So there’s a ton of variables at play + I haven’t been able to do this w. other models + And the small details are pretty impressive + The body balance + weight adjustment + The clothes/hair flowing w breeze appropriately + The camera action to follow the subject CONTEXT: + There’s a ton of issues w. this video. + I wouldn’t use this in place of real footage. + Pretty sure SB fans don’t want Ai videos. + We’re demo’ing the tool's capabilities.? + And watching the progression. Keep playing…try weird things. #midjourney #kling #veo2 #runwayml #ai #innovation?

Topaz Starlight → Creative Video Upscaler. Topaz recently released this Beta feature. The first diffusion model for video enhancement. For a nut that hasn't been cracked yet. It's a solid V1. PROCESS: 01. Open Topaz 02. Upload Video 03. Select “Try Starlight” 04. Select “Select Range”? 05. Select “Send to Cloud” No prompting or settings necessary INITIAL IMPRESSION: + Adds some contrast + depth + Works well with texture + Artifacts are still present + Easier to see in individual frames + It’s V1..still needs work…but promising TECHINCAL THOUGHTS: + Because of it’s size…it’s slower + Typically around 10-20 mins to generate + Exporting at 1080p + It’s also more expensive to run? + It’s about 90 credits/per minute + Everything is currently processed in the cloud + Local processing will be developed in the future OVERALL THOUGHTS: + This is one of the most requested features + It’s been a very difficult one to crack + If it wasn’t hard…you’d see it in more products + I would expect this to be a huge differentiation + For Topaz moving forward This feature is very needed. (More details from Topaz below) #midjourney #kling #minimax #ai #innovation #topaz

GOOGLE VEO2 → Just for fun. Arctic Rally. Testing more dynamic camera action. This is Veo2’s differentiator (imo). Especially with T2V. TOOLS USED: + VEO2: Video gen + ChatGPT: Prompt refinement + MMAudio: Audio gen + Topaz: Upscaling + FPS? + Capcut: Editing Prompt Structure: [shot type], [subject + action + details], [Environment description], [end result], [mood/key visuals] Prompt Ex: Fast cinematic action shot of a powerful rally car, drifting at high speed around a treacherous hairpin turn on a frozen Antarctic mountain pass. The vehicle kicks up a blizzard of snow and ice shards as its tires struggle for grip on the slick, unforgiving surface. Jagged ice cliffs and sharp black rock loom in the background, and the distant glow of the aurora borealis illuminates the dark, stormy sky. The driver’s precision is evident—every movement calculated in this life-or-death race against nature. The camera captures the scene from a low, dynamic angle, emphasizing the raw speed and peril of the moment, 35mm style. PROBLEM AREAS: + Car is different in every shot + Artifacts present with speed + Lighting consistencies? + Amongst many others QUICK THOUGHTS: + Consistency is tough w T2V + A super detailed car/object = variance + Ex. If I asked for a “black [insert car mode]” + The car would be more consistent + The camera action is super impressive + It’s less linear/smooth than other generators + And allows for some more dynamic shots + Very impressed with some of the subtle details + Ex headlight tracking/reflections + exhaust blasts/etc + This started with one random shot + Decided to build it out more (it’s addicting) It won’t capture the raw adrenaline of rally racing. But it’s definitely something fun to play with. Have fun. #midjourney #kling #veo2 #runwayml #ai #innovation

Veo2 → Redbull x Lord of the Rings. Not really...but kinda. Testing random stuff w. Veo2. Very surprised with a few things. VEO PROMPT STRUCTURE: [Shot Type], [Subject + Details], [Action/Motion Description], [Camera Motion + Description], [Enviornment + Description], [End Result], [Mood], [Key Visual Elements] PROMPT: Cinematic tracking shot of an adrenaline junkie riding a golden eagle. The rider barely has time to brace as the massive golden eagle shifts—its powerful wings flapping with epic force. The camera, locked behind them, struggles to keep up as gravity takes hold, plunging them toward the earth below. The vast Yukon wilderness pine canopy stretches out like an ocean of deep green. Then, the camera tilts down, mimicking the sickening pull of the dive, the horizon vanishing as the ground rockets closer. Wind screams past, pressure building, every heartbeat pounding in their ears. For a moment, it feels endless—just sky, speed, and the raw, terrifying pull of the earth below. 35mm film QUICK THOUGHTS: + I wasn’t expecting good output + Thought this rendered pretty well + The coherence wasn’t perfect + But I was asking for a lot + W. Veo small tokens make a difference + One word can alter the entire output + It’s a problem/strength w. coherent models + Veo reacts really well to “mood/emotion” + And a little more narrative in the prompt + Veo really wins on camera motion? + It has a more natural feel to it/less robotic + “Tracking shot” is a good holding category + For getting decent/less structured cam motion. My advice…because it’s Sunday… Try some weird stuff…and play. Have fun. #midjourney #kling #minimax #veo2 #ai #innovation ?

Quick Veo2 scene ↓ 80s Wall Street Energy. Testing character consistency. Still tough with t2v. VEO PROMPT STRUCTURE: [Shot Type/Camera Movement], [Subject + Action], [Depth of Field], [Lighting], [Color Palette], [Mood], [Key Visual Elements] EX PROMPT: A low-angle shot from the perspective of an old CRT computer monitor, looking outward at a chaotic 1980s stock exchange trading floor. A young Indian man in a pinstriped suit and a loose tie stands with A group of multi-racial frantic stockbrokers in sharp double-breasted suits and suspenders leaning in toward the screen, their faces twisted in shock and excitement. Green computer text reflects off their thick glasses, while some hold ticker tape slips clenched in sweaty fists. The room is packed with traders in a frenzy, papers flying, phones ringing, and stock numbers flashing on electronic boards overhead. The fluorescent lighting gives a harsh glow, casting deep shadows on their faces as they scream and point before exploding into celebration. The energy is electric—money is being made and lost in real time. 35mm style. QUICK THOUGHTS: + Char. Consistency is hard without i2v + Minor details are really hard to control + There’s a ton of continuity errors in here + Try focusing on a single subject + details + It tends to work better than multiple subs. + Also, isolating the character by themselves + Vs having them amongst a group of characters + Or objects / props…tends to be more consistent + Focus on the specific unique features like + What they’re wearing/hairstyle/mood/overall look + Small details like shoes/jewelry/etc will translate + It’s a super coherent model, but takes finessing. + Veo has a dynamic skill-set and will improve. + I also had a tough time with the sound here. + Respect to all the filmmakers/audio professionals. + That is not an easy skill set to pick up. Text-2-video is still the best way to learn (imo) Understanding the variables/tokens And how they affect the output. Will give you better grasp of the iterative process. PS: Veo2 is now available via Freepik. (Not sure it’s the full model, but it’s there) #midjourney #kling #veo2 #runwayml #ai #innovation?

Midjourney + Kling + Sora. Testing new workflows. Trying to push some visuals further. And mash a few things together. TOOLS: + Midjourney: Images + Kling Elements: Video gen + Sora: Video blend + upscale + MMAudio: Sound? + Capcut: Music + sound + Topaz: Upscaling Process: 01. Gen images in MJ 02. Kling Elements to combine 03. Sora Blend to mesh the videos 04. Sora Remix to upscale + clean up 04. Topaz upscale + increasing FPS MJ Prompt 1: cinematic still, showing the moment a conscious creator designs humanity --ar 16:9 --q 2 --p --s 650 MJ Prompt 2: cinematic still, Mayan shaman wearing intricate ceremonial attire, ancient ritual, light rays and shadows, smoke interacting with light --ar 16:9 --q 2 --p --s 650 Kling Elements Prompt: A cinematic tracking shot follows behind man conducting sorcery in a dark cave as he summons an ancient mayan spirt through a portal. The camera pushes in on the portal passing the mans hand from a low angle as he is conjuring a teal-green portal. Suddenly, an ancient mayan spirit emerges from the center of the portal-exploding onto the scene with a cinematic flair adorned in ritualistic attire and a massive headdress - showing his full body - as he enters the scene with presence and awe. Natural smoky textures billow and pulsate. Shot in cinematic 4k. QUICK THOUGHTS: + This is not perfect or cinema quality + But shows the tools can be more “complex” + Using them in concert to get more pop. + Kling Elements does a good job “mashing” + But you lose some quality and fidelity + The Kling gens weren’t exactly what I wanted + So I used Sora to blend (2) videos together + To get the best parts of both videos in one gen + Then Sora remix to increase to 1080p + Which also cleans up some artifacts? + Remix/Blend are Soras best qualities (imo)? + Adding sound is hard for an novice like me + I never know when to stop adding sound Overall, each tool has strengths/weaknesses. They can be problem solvers in their own way. Test them in weird ways…have fun. #midjourney #kling #minimax #runwayml #ai #innovation ?