AI summary by TwinMind
I was wrong (OpenAI's image gen is a game changer)
Theo - t3․gg
33m 43s
Initial Underestimation and Capabilities Discovery
- Severely underestimated new OpenAI image generation capabilities
- Image making creator look 8 years younger, changed appearance since last video
- Much better than originally thought since last video release
- Witnessed Ghibli revolution on Twitter, chaotic introspective ChatGPT comments
- People building whole UIs, editing photos, changing appearances
- Not diffusion-based as suspected - more interesting technology at work
Research Findings and White Paper Discovery
- Found white paper from TikTok explaining new image generation technique
- Collected examples of cool use cases beyond expectations
- Technology uses Visual Autoregressive Modeling instead of diffusion
- Paper from ByteDance (TikTok's parent company) from almost a year ago
Sponsor Segment - CodeRabbit
- CodeRabbit: AI code review tool, free to start, $15/month
- Reviews every pull request, stops more bugs than human engineers
- Provides walkthroughs, diagrams, nitpick comments
- Suggests loading states, removes unused console logs, identifies reusable logic
- One-click apply button for fixes
Comic Generation Demonstration
- Used T3 chat to generate 4-panel comic script about ChatGPT's perspective
- Script: existential, humanizing AI
- Generated comic with ChatGPT create image feature
- Results: terrifyingly good quality despite text errors
- Comic content: "I exist in the space between questions and answers"
- Some text mistakes (ellipses, typos like "j" instead of "i")
- Close to perfect but not flawless
Practical Work Applications
- Using for thumbnails in Affinity Photo (Photoshop alternative)
- Previous AI image gen only good for stock generation (Envato, Storyblocks replacement)
- Now capable of very specific, usable outputs
- Microsoft/OpenAI chat history example with company logos
- Generated realistic chat interface showing "Microsoft: can you show us what you've been up to" / "OpenAI: no"
- Consistently generates good images, not lucky rolls
Image Transformation and Style Transfer
- Excellent at image transformation: "draw this image in style of Studio Ghibli"
- Sometimes refuses copyrighted content
- Created viral Twitter meme using How's Moving Castle trailer
- Posted as "someone already made whole movie in Ghibli style"
- Credited "Hayao Miyazaki" as prompter who formed studio around Ghibli art style
- Broke Twitter, still getting non-stop notifications
Personal Image Editing Examples
- Generated Ghibli-style self-portrait, captured earring detail
- Noted laptop sticker (spaceman from Outer Wilds) vs Apple logo confusion
- Better than real Twitch logo generation
- Image editing: "remove logos from cup and laptop"
- Successfully removed logos, changed mug to cup (removed handle)
- Subtle additional changes: mic position, watch content
UI Generation and Interface Issues
- SJ's T3 chat UI transformation to Ghibli-inspired design
- ChatGPT interface problems: chats disappearing, persistence layer broken
- Chats vanishing "into the ether," requiring refresh to reappear
- Chat order changing randomly due to broken persistence
- Frustrating UI compared to T3 chat experience
- No API access, forced to use broken interface
Technical Deep Dive - Visual Autoregressive Modeling
- Traditional diffusion: starts with noise, adjusts iteratively to match prompt
- Like sharpening algorithm, restructuring noise into correct image
- New approach: Visual Autoregressive Modeling (VAR)
- Paper from ByteDance researchers, almost year old
- VAR redefines autoregressive learning as coarse-to-fine next-scale prediction
- Diverges from standard raster scan next-token prediction
Performance Improvements
- First time GPT-style AR models surpass diffusion transformers
- 20x faster inference speed
- Better image quality, data efficiency, scalability
- Clear power law scaling laws similar to LLMs
- More power = better output (unlike previous diffusion limitations)
- Zero-shot generalization: inpainting, outpainting, editing
- Scaling laws and generalization properties match LLMs
Tool Calls System Explanation
- Weather example: AI can call external tools for current data
- Tool calls happen internally, not visible to user
- System prompt includes available tools list
- Example tools: text application, reflection, analyze chunks
- Pixel location identification and manipulation
Image Generation Process Theory
- Scaffold tool: generates minimal pass shape/structure
- Panel-by-panel filling for comics
- Find text block locations, fill dialogue boxes
- Reasoning with image similar to O-models
- Step-by-step verification and improvement
- Built-in correction tools: "make sure hands have five fingers"
Text Application and Perspective Tools
- Image editing tools: perspective correction, text warping
- HTML/CSS-like coordinate systems for text placement
- Skewed text boxes matching image perspective
- Filter applications (yellow filter example from previous video)
- Multiple tool applications during generation process
- Quality control: "is there anything wrong with this image?"
Practical Design and Development Applications
- Designers can create functional mockups for user testing
- Developers can mock UI concepts before bothering design team
- Reduces iteration cycles meaningfully
- Not replacing designers/developers, enabling better collaboration
- Thumbnail AB testing example: AI-generated vs traditional
- AI thumbnail lost test (2% vs much bigger gap difference)
AB Testing and Experimentation Benefits
- Can create 5 versions, test which resonates best
- Reduces time "polishing turds" nobody wants
- Enables more experimentation for small teams
- Traditional fake UI creation: complex multi-step process
- AI generation: skips technical knowledge requirements
Personal Tool Development Context
- Built QuickPick for SVG to PNG conversion
- Pick Thing: background removal and thumbnail asset management
- Dashboard with thumbnail faces for video creation
- Previous workflow: complex masking, cropping, layer management
- AI simplifies: direct instruction for image manipulation
Current Limitations and Future Potential
- Complex transformations still struggle
- Transparent backgrounds helpful but imperfect
- Far from perfect but significantly better than expected
- Excited about T3 chat integration coming soon
- First follow-up video due to significant new insights and value
