From prompt to mesh: what actually happens inside text-to-3D

You type weathered brass diving helmet, riveted seams, single object, wait about fifty seconds, and a mesh appears in the viewport. It has roughly 22,000 triangles, a 2048×2048 base color map, and a normal map that looks convincing at arm’s length. Most people treat that fifty seconds as a black box. It isn’t, and the parts inside it explain almost every quirk you will hit later: why the back of the helmet is blurrier than the front, why the rivets read as texture rather than geometry, why the wireframe looks like a fishing net instead of a modeled surface.

What follows is the pipeline as it actually runs in current-generation systems, including the ones behind MeshyFlix’s studio. Some vendors reorder these stages or fuse two of them into one network, but the shape of the process is now fairly standard across the field.

The old approach was optimization; the new one is prediction

The first wave of text-to-3D, around 2022, worked by optimization. DreamFusion and its descendants took a 2D image diffusion model, rendered a randomly initialized NeRF from a camera angle, asked the diffusion model “how would you denoise this render toward the prompt?”, and used that answer as a gradient to nudge the 3D representation. Repeat a few thousand times from many angles and a shape emerges. This is score distillation sampling, and it worked — at a cost of 40 to 90 minutes per asset on an A100, with heavy saturation and a strong tendency to grow a second face on the back of anything.

Nobody ships that anymore for interactive use. Current systems are feed-forward: a network trained directly on large corpora of 3D data that predicts a 3D representation in a single pass, the same way an image model predicts an image. Generation went from an hour to under a minute, and consistency improved dramatically because the model has actually seen millions of coherent 3D shapes rather than inferring them from 2D opinions.

Stage one: the shape model

The geometry stage never touches triangles. It produces a continuous field — usually a signed distance function or an occupancy field — encoded in a compact latent representation. Three encodings dominate:

• Triplanes. Three orthogonal 2D feature grids (XY, YZ, XZ), typically 128×128 with 32 channels each. Query a 3D point by projecting it onto all three planes, sampling, and running a tiny MLP. Cheap and memory-friendly, but the axis alignment leaks into results as a faint boxiness on organic shapes.
• Sparse voxel latents. A coarse occupancy pass at something like 64³ locates the surface, then a fine pass allocates detail only near it, often at an effective 512³ or 1024³. Far better at thin structures than triplanes.
• Vector sets. The shape is a set of a few thousand unordered latent tokens attended over by a transformer. Resolution-independent, and currently the best at fine detail, at the cost of a heavier decode.

The generative machinery on top is flow matching or rectified flow rather than classic DDPM diffusion, which is why sampling takes 25 to 50 steps instead of 250. Text conditioning arrives through cross-attention on CLIP or T5 embeddings of your prompt.

From field to mesh

The decoded field is sampled on a grid and converted to a surface by marching cubes or dual contouring. This is the step that determines what your wireframe looks like, and it constrains everything downstream.

Marching cubes at 512³ over a typical prop yields somewhere between 400,000 and 1.2 million triangles. Every triangle is roughly the same size, regardless of whether it sits on a flat panel or a curved lip. There is no edge flow, no loops around features, no density variation. It is an isotropic triangle soup that happens to be watertight and manifold. Dual contouring behaves better on sharp edges because it places vertices using gradient information rather than snapping to grid intersections, so a machined bevel comes out as an actual crease instead of a staircase.

That million-triangle surface is then decimated to whatever target you asked for — 30,000, 100,000, 300,000 — using quadric error metrics, which collapse the edges that change the shape least. Decimation preserves silhouette well and edge flow not at all. If you have ever wondered why generated meshes look uniform and slightly noisy in wireframe, this pair of steps is the reason.

Stage two: texture

Now there is geometry but no surface color. The texture stage typically runs like this:

1. UV unwrap. An automatic atlas tool (xatlas and its relatives) segments the mesh into charts by angle and curvature, flattens each, and packs them. Expect 60 to 75 percent packing efficiency and a lot of small islands. Perfectly serviceable for baked textures, useless for hand-painting or trim sheets.
2. Multi-view rendering. The mesh is rendered from 4 to 8 fixed cameras, usually with depth and normal buffers alongside.
3. Multi-view diffusion. An image model generates all views simultaneously with cross-view attention, conditioned on your prompt plus the depth and normal buffers. Generating them jointly rather than one at a time is what keeps the left and right sides agreeing about what color the straps are.
4. Back-projection. Each generated view is projected onto the mesh through its camera, writing into UV space, weighted by how directly each texel faced that camera. Grazing angles get low weight.
5. Inpainting. Areas no camera saw — armpits, undersides, the inside of a helmet — are filled by a UV-space inpainting pass. This is where quality drops off, and it is the honest answer to “why is the underside mushy?”
6. Delighting. The generated views contain baked highlights and shadows. A delighting network strips them so the base color is closer to true albedo. It is imperfect; you will still find soft ambient occlusion baked into crevices.
7. PBR decomposition. A final network predicts metallic, roughness, and tangent-space normal from the albedo and geometry. Metallic prediction is the weakest link. Brass and painted steel look similar in a photo, so models hedge and output mid-range metallic values that are physically meaningless.

Why preview and refine are separate

Splitting generation into an untextured preview and a textured refine is not an arbitrary product decision. The shape stage is cheap and the texture stage is expensive — roughly 4 to 6× the compute. Since most rejections happen on silhouette rather than color, showing you the bare mesh first lets you throw away bad candidates before paying for texturing. Generate four previews, refine the one that reads correctly in silhouette. That workflow costs about a third of refining all four.

What the prompt actually controls

The text encoder sees your prompt at the shape stage and again at the texture stage, but the two stages care about different words. Shape responds to nouns, structural adjectives, and part descriptions. Texture responds to materials, colors, and surface condition. Style words like “cinematic” or “8k” affect nothing useful, because the geometry model was never trained on renders.

• Do name one object and its parts: ceramic teapot with a bamboo handle and a hinged lid.
• Do state materials explicitly — brushed aluminium, oxidised copper, matte rubber — since these steer both the texture model and the metallic prediction.
• Do say in T-pose for anything you intend to rig; the shape model honors it and an A-pose sculpt is far harder to skin.
• Don’t describe a scene. Two objects in a prompt gives you one fused blob or an arbitrary winner.
• Don’t ask for readable text or logos. The multi-view stage cannot keep letterforms consistent between cameras and you get plausible-looking gibberish.
• Don’t use camera or render vocabulary — f/1.8, studio lighting, octane render. Lighting terms actively fight the delighting pass.

The failure modes worth recognising

Thin structures — wires, antennae, chain links, spectacle frames — sit below the extraction grid’s resolution and either vanish or fuse into the body. Thicken them in the prompt (thick wire handle) or expect to model them yourself.

Radial symmetry bias comes from training data: pots, wheels, and helmets come out clean while asymmetric mechanical assemblies drift. Deep concavities — bags, boxes with open lids, hollow interiors — get inpainted geometry that is often just wrong, because no training view showed the inside.

Where to go from here

Take one asset you already generated and inspect it in three passes. Open the wireframe and confirm the triangle distribution is uniform, which tells you it came out of marching cubes and needs retopology before any rigging. Open the metallic map alone and check it is close to 0 or 1 rather than a wash of 0.4 grey; if it is, flatten it manually. Then orbit to the side no camera saw during texturing and decide whether that blur is acceptable at your viewing distance.

If it isn’t, regenerate with the object rotated in the prompt rather than trying to fix it in Photoshop. Re-rolling the shape stage costs seconds; repairing an inpainted region by hand costs an afternoon.