Material Assistant (Director)

Material Director turns an instruction into an editable Open Painter material change. It runs Codex in a separate local process and applies validated proposals through the normal layer system: fill layers, masks, scene fields, filters and anchors. Codex can also generate surface textures and grayscale masks with its native image tool and assign them to layers. The result is always an ordinary stack you can keep editing, save, reopen and export with Codex disconnected.

Set up

1. Select a texture set with an assigned, UV-unwrapped mesh and open the Assistant sidebar tab.
2. Enable Blender's Preferences › System › Allow Online Access.
3. Click Sign in with ChatGPT. Wait for runtime setup and complete browser sign-in. Progress is shown in the panel; Disconnect cancels setup.
4. Enable Share Material Context with Codex before sending instructions. Signing in alone does not require sharing material context.

The add-on preferences offer OS Keyring to retain login or This Connection Only for memory-only credentials. An Advanced runtime override is optional. The managed runtime is pinned to 0.154.0; the current adapter accepts 0.154.x. Reconnect after changing runtime or account storage.

The panel reads Target → Instruction → Run → Changes. Refine is the default. What is sent lists actual visual inputs and summarizes the last request.

Prepare geometry evidence

Create, Refine and Plan require current, loaded and verified AO, Curvature, Normal, World Normal, Position and Thickness maps for every set in scope, with view and map sharing enabled. Missing or stale maps stop the request before sending; bake them explicitly from Bake Mesh Maps. Explain remains available without these prerequisites.

Direct a material

Instructions that work well:

• "Create dark green painted steel with irregular edge wear and a little cavity rust."
• "Reduce the existing rust, preserve its pattern, and make the bottom dirtier."
• "Add a roughness-only fingerprint layer. Keep colour and metalness unchanged."
• "The scratches look too uniform. Explain what to change."

For visual discussion, use Material Preview or Rendered shading, frame the material close enough to see it, and wait for the shader preview to finish before sending. The last six instruction/reply pairs for the current material stay in memory so "Fix those two problems" can follow an Explain; they are not saved in the .blend, and disconnecting, loading a file, Blender undo/redo or Clear AI History clears them.

Create and Refine commit the last valid proposal only when the turn completes, as one Blender undo step regardless of how many commands it contained. Ctrl+Z follows ordinary history; Undo Last AI Change uses the persisted receipt to preserve later artist edits where possible.

Generated textures

With Generate Surface Textures enabled, try: "Generate dark olive enamel and an irregular grayscale chipped-paint mask. Apply both, with separate exposed steel and cavity rust layers." Colour and emission textures import as sRGB; masks, scalar maps and detail inputs as Non-Color. Originals are packed into the .blend with their prompt recorded on the image, keep normal UV/triplanar projection, scale, levels, filters and opacity, and never overwrite existing pixels.

Limits: at most eight PNGs per instruction, each up to 4096 × 4096 and 16 MiB, shared across a scene instruction; 15 minutes per instruction (30 for a scene). Only verified artifacts from the active instruction can be assigned, Plan retains sources in a private cache until Apply (disposable previews can load temporary images), and Stop never applies pending images. Generated textures supply surface detail — geometry masks still determine cavities, edge wear and named regions, and a generated albedo is not a physical roughness, height or normal map. Seamlessness and calibrated, aligned PBR maps are not guaranteed by the image service; inspect the result before production use.

Surface generation and matching channels

Generated material work starts with a surface plan: components, substrate, handling history, detail scale and source roles. The Assistant captures orthographic observations of the evaluated surface with matching geometry-map evidence, generates appearance for those views and fuses it into the existing UV atlas. Color and scalar channels share geometric correspondence and fusion weights.

Coverage inspection reports good, weak, missing and conflicting regions. Corrective observations can target uncovered areas; unseen surfaces are reported rather than filled by stretching nearby pixels. Regional artwork can instead use an editable projected fill with its own frame, depth and mask.

Existing generated color can guide separate roughness, metallic and height maps without replacing its placement. Scalar maps use Non-Color and shared alpha coverage. Derived local-luminance detail is an approximation, not measured PBR; disable height when pigment or lighting creates false relief. Height feeds the normal result, so an independently generated normal need not repeat that relief.

For dimensioned details, declare physical size and its provenance first. Detail below available texel spacing is reported. Generated maps still require inspection for correspondence, scale and physical plausibility.

Candidate appearance and local corrections

The Assistant renders proposals in disposable Eevee scene copies, with multiple sides, closeups, assigned-face views and channel diagnostics. It reviews image evidence and can make up to two correction passes within a shared six-minute rendering budget and the instruction's image limit. The proposal must match the latest reviewed candidate and unchanged artist inputs before commit.

Targeted correspondence inspection relates a visible pixel to its mesh surface, UV island, seam pairing, source-image area and nearby geometry maps. Local source corrections preserve pixels outside the selected windows; global replacement changes a source's overall appearance. Tiled or reused sources can affect other occurrences, so all views are reviewed again.

A fused surface field is corrected through observations and mapping rather than a flat-canvas replacement. Coverage errors, fusion errors and source-appearance errors are distinguished. Candidate review is diagnostic evidence, not an independent aesthetic score or a guarantee of seamless output.

References: generated surfaces and surface-space generation.

Protected content and stale drafts

Paint layers and painted or stroke masks are protected automatically. Protected Content also lets you lock the selected layer, channel or mask; a protected folder includes its descendants, and shared anchor inputs are checked so editing a producer cannot indirectly alter a protected consumer. These protections constrain AI authoring; they do not freeze the final composed appearance against layers added elsewhere in the stack.

Each request captures the texture-set IDs, object bindings and a digest of the authoring and geometry inputs. Manual edits, changed inputs, missing targets, changed protection settings, a file load, Blender undo/redo, disconnecting or switching the active set reject a stale draft — run the next instruction against the current state. Receipts keep up to eight transactions within 8 MiB per texture set, survive saving and reopening, and are a bounded convenience history rather than a replacement for Blender undo or backups.

What it can and cannot change

• Fill layers and folders with stable IDs; enabled channels, values, colours, opacity, procedural sources, seeds, scales and projections.
• Generator, procedural, baked-map, scene-field, filter and published-input masks; channel and mask filters.
• Published anchors and validated connections; bundled catalog materials, effects and module recipes, inserted as BAKED (uses the catalog's requested maps) or LIVE (substitutes supported live fields).
• Generated PNGs assigned to channel content, scalar detail inputs or fill masks.

• It does not create UVs, change topology, segment parts, reassign material slots, alter resolution, export, bake, or run arbitrary Python, shell, RNA paths or free-form node wiring.
• Region-sensitive placement can use assigned faces, connected components, artist selections, named attributes, vertex groups or bounded geometric queries; ambiguous parts can still require an artist selection.
• Each proposal is bounded (64 commands, 24 new layers, 256 total layers, a conservative budget of 12 estimated shader texture inputs to leave room for the viewport on Metal). Validation or commit failure restores the prior state.
• Real-account inference was tested on Blender 5.2.1 LTS on macOS Apple Silicon. Candidate review does not guarantee artistic quality; Windows and Linux are untested.

Data and process boundaries

The Blender process gathers bounded summaries — object names, dimensions and counts, UV, attribute and group names, material structure and values, available anchors and maps, local recipe descriptions. The full mesh and .blend stay local. Views, map previews and the reference image are sent only when their checkboxes are enabled. Blender starts a bundled companion script with its own Python; a randomly authenticated loopback socket carries small JSON messages, and the companion owns Codex's stdio connection and network activity. Codex runs with execution, shell, web, apps, plugins and agent features disabled; the model receives constrained Painter authoring, inspection and candidate-review tools plus account-supported image generation and restricted runtime built-ins. Credentials and OAuth URLs are never stored in the project.

The full reference, live acceptance report and image-generation validation are in the add-on repository: docs/material-director.md. For where the rest of the painter fits, see Layers, channels and masks.