AI summary by TwinMind
An AI artist explains his workflow
Vox
8m 18s
Character Introduction & Project Origins
- Stelfie: funny, clumsy time-traveling character, alter ego of artist (physically different)
- Project started to showcase Stable Diffusion potential combined with artist skills
- Goal: create scene of Stelfie boxing with Muhammad Ali
Initial Workflow & Challenges
- Always starts with hand-drawn sketch
- Stable Diffusion models extremely good but "cheeky" - easily drive away from original idea
- Tests random prompts to find good initial pose
- Couldn't find proper Stelfie pose on right side
- Moved to Photoshop to recreate pose manually
- ControlNet extension: same pose creation now takes 15 minutes vs 2 months ago
Technical Parameters & Samplers
- Always uses different samplers throughout process
- Sampler choice crucial for realism and details
- Skin replication: Euler sampler very synthetic/fake, DPM works great
- Steps parameter: tells Stable Diffusion how many times to work on prompt
- Can choose very low or very high numbers
Inpainting & Outpainting Techniques
- Inpaint: machine changes only selected parts of image
- Outpaint: machine imagines what's outside the box based on existing content
- Both techniques fundamental to workflow
Work Distribution & Process
- 50% Stable Diffusion, 40% Photoshop, 10% other
- Constant back-and-forth between Stable Diffusion and Photoshop
Character Model Training
- Created custom model trained specifically on Stelfie's face
- Process: created Stelfie in 3D, took snapshots from different angles
- Used images to train model with saved keyword
Noise Strength Control
- Last option in Stable Diffusion web UI
- Provides more/less control over image itself
- Critical parameter for fine-tuning
Face Creation Challenges
- Faces very difficult to achieve good results
- Especially challenging replicating popular persons (Muhammad Ali)
- Asked Stable Diffusion for Muhammad Ali-like face
- Manually warped facial features in Photoshop: nose, jaw, eyes
- Changed everything manually while maintaining correct pose
Character Design Specifications
- Wanted Stelfie slightly fluffy, not super fit, without six-pack
- Found satisfactory belly shape but needed more realism
- Muhammad Ali body accuracy: wasn't buff like modern athletes
- Big and healthy but not defined muscles of today's athletes
- Aimed for realistic Muhammad Ali representation
Project Timeline & Refinement
- 12-14 hours into project when scene properly set
- Continuous refinement process
- Nipple rendering issues noted humorously
Artist Philosophy & Background
- Must drive the machine, not other way around
- Views process as joint effort with AI
- Traditional artist for 2 decades (canvas painting)
- 5 years digital art experience
- Not threatened by AI - sees as opportunity
- Opens new creative branch different from existing digital art
Hand Creation Workaround
- 50% of hands in Stelfie artworks are artist's own hands
- Hand reproduction extremely challenging in AI
- Solution: photograph own hand in needed position, clean up, paste on top
