AI summary by TwinMind

I was wrong (OpenAI's image gen is a game changer)

Theo - t3․gg

33m 43s

Initial Underestimation and Capabilities Discovery

  • Severely underestimated new OpenAI image generation capabilities
  • Image making creator look 8 years younger, changed appearance since last video
  • Much better than originally thought since last video release
  • Witnessed Ghibli revolution on Twitter, chaotic introspective ChatGPT comments
  • People building whole UIs, editing photos, changing appearances
  • Not diffusion-based as suspected - more interesting technology at work

Research Findings and White Paper Discovery

  • Found white paper from TikTok explaining new image generation technique
  • Collected examples of cool use cases beyond expectations
  • Technology uses Visual Autoregressive Modeling instead of diffusion
  • Paper from ByteDance (TikTok's parent company) from almost a year ago

Sponsor Segment - CodeRabbit

  • CodeRabbit: AI code review tool, free to start, $15/month
  • Reviews every pull request, stops more bugs than human engineers
  • Provides walkthroughs, diagrams, nitpick comments
  • Suggests loading states, removes unused console logs, identifies reusable logic
  • One-click apply button for fixes

Comic Generation Demonstration

  • Used T3 chat to generate 4-panel comic script about ChatGPT's perspective
  • Script: existential, humanizing AI
  • Generated comic with ChatGPT create image feature
  • Results: terrifyingly good quality despite text errors
  • Comic content: "I exist in the space between questions and answers"
  • Some text mistakes (ellipses, typos like "j" instead of "i")
  • Close to perfect but not flawless

Practical Work Applications

  • Using for thumbnails in Affinity Photo (Photoshop alternative)
  • Previous AI image gen only good for stock generation (Envato, Storyblocks replacement)
  • Now capable of very specific, usable outputs
  • Microsoft/OpenAI chat history example with company logos
  • Generated realistic chat interface showing "Microsoft: can you show us what you've been up to" / "OpenAI: no"
  • Consistently generates good images, not lucky rolls

Image Transformation and Style Transfer

  • Excellent at image transformation: "draw this image in style of Studio Ghibli"
  • Sometimes refuses copyrighted content
  • Created viral Twitter meme using How's Moving Castle trailer
  • Posted as "someone already made whole movie in Ghibli style"
  • Credited "Hayao Miyazaki" as prompter who formed studio around Ghibli art style
  • Broke Twitter, still getting non-stop notifications

Personal Image Editing Examples

  • Generated Ghibli-style self-portrait, captured earring detail
  • Noted laptop sticker (spaceman from Outer Wilds) vs Apple logo confusion
  • Better than real Twitch logo generation
  • Image editing: "remove logos from cup and laptop"
  • Successfully removed logos, changed mug to cup (removed handle)
  • Subtle additional changes: mic position, watch content

UI Generation and Interface Issues

  • SJ's T3 chat UI transformation to Ghibli-inspired design
  • ChatGPT interface problems: chats disappearing, persistence layer broken
  • Chats vanishing "into the ether," requiring refresh to reappear
  • Chat order changing randomly due to broken persistence
  • Frustrating UI compared to T3 chat experience
  • No API access, forced to use broken interface

Technical Deep Dive - Visual Autoregressive Modeling

  • Traditional diffusion: starts with noise, adjusts iteratively to match prompt
  • Like sharpening algorithm, restructuring noise into correct image
  • New approach: Visual Autoregressive Modeling (VAR)
  • Paper from ByteDance researchers, almost year old
  • VAR redefines autoregressive learning as coarse-to-fine next-scale prediction
  • Diverges from standard raster scan next-token prediction

Performance Improvements

  • First time GPT-style AR models surpass diffusion transformers
  • 20x faster inference speed
  • Better image quality, data efficiency, scalability
  • Clear power law scaling laws similar to LLMs
  • More power = better output (unlike previous diffusion limitations)
  • Zero-shot generalization: inpainting, outpainting, editing
  • Scaling laws and generalization properties match LLMs

Tool Calls System Explanation

  • Weather example: AI can call external tools for current data
  • Tool calls happen internally, not visible to user
  • System prompt includes available tools list
  • Example tools: text application, reflection, analyze chunks
  • Pixel location identification and manipulation

Image Generation Process Theory

  • Scaffold tool: generates minimal pass shape/structure
  • Panel-by-panel filling for comics
  • Find text block locations, fill dialogue boxes
  • Reasoning with image similar to O-models
  • Step-by-step verification and improvement
  • Built-in correction tools: "make sure hands have five fingers"

Text Application and Perspective Tools

  • Image editing tools: perspective correction, text warping
  • HTML/CSS-like coordinate systems for text placement
  • Skewed text boxes matching image perspective
  • Filter applications (yellow filter example from previous video)
  • Multiple tool applications during generation process
  • Quality control: "is there anything wrong with this image?"

Practical Design and Development Applications

  • Designers can create functional mockups for user testing
  • Developers can mock UI concepts before bothering design team
  • Reduces iteration cycles meaningfully
  • Not replacing designers/developers, enabling better collaboration
  • Thumbnail AB testing example: AI-generated vs traditional
  • AI thumbnail lost test (2% vs much bigger gap difference)

AB Testing and Experimentation Benefits

  • Can create 5 versions, test which resonates best
  • Reduces time "polishing turds" nobody wants
  • Enables more experimentation for small teams
  • Traditional fake UI creation: complex multi-step process
  • AI generation: skips technical knowledge requirements

Personal Tool Development Context

  • Built QuickPick for SVG to PNG conversion
  • Pick Thing: background removal and thumbnail asset management
  • Dashboard with thumbnail faces for video creation
  • Previous workflow: complex masking, cropping, layer management
  • AI simplifies: direct instruction for image manipulation

Current Limitations and Future Potential

  • Complex transformations still struggle
  • Transparent backgrounds helpful but imperfect
  • Far from perfect but significantly better than expected
  • Excited about T3 chat integration coming soon
  • First follow-up video due to significant new insights and value

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome