Terrev

3DXML to OBJ - Converts LDD model captures to OBJ

Recommended Posts

Terrev

3DVIA Printscreen can capture LEGO Digital Designer models and save them as 3DXML files. It was discontinued years ago, but can still be downloaded from the web archives (direct installer link).

 

Not many programs can load 3DXML files, so I made a program to convert them to OBJ. It does some other helpful things too.

 

Download

 

The resulting models are far more optimized than exports from other LEGO building software, thanks to LDD's hidden stud/tube removal. For example, a small house model that's 50350 tris exported from LeoCAD is only 13548 tris with this method - and with some additional tricks, only 2586 trisMore info here.

 

I've only made this to work with LDD captures, compatibility with captures from other programs is not guaranteed. It will automatically name materials and textures with their official LEGO names/IDs.

 


 

Setup:

Usage:

  • Save your LDD model in 3DXML to OBJ's "Models" folder.
  • Launch 3DXML to OBJ and enter your LDD model name in the first text box, then click "Move camera". This saves a new LDD model with the camera set in the proper position for 3DXML capturing.
  • Open the new LDD model (it will have CAM_SET in the name) and capture it to a 3DXML file with 3DVIA Printscreen. Make sure it's saved in 3DXML to OBJ's "Models" folder, like the LDD models.
  • Enter the name of your 3DXML file into the second text box in 3DXML to OBJ. I recommend leaving "Weld duplicate vertices" enabled, though it may take a while on some large models. Choose a new color palette if you'd like, and click "Convert".

It sounds more complicated than it is. Here's a model being converted and imported into Unity:

 

 


 

And now, the advanced features, which are mostly for LEGO Universe-style color variation, but keep reading for some geometry removal tricks too.

 

For color variation, we need to randomly adjust the color of each brick... But, LDD often batches bricks of the same color together into one mesh. Thus, we can't do it after capturing a model. We have to have the randomized colors displayed in LDD itself. So, it's LDD modding time!

 

Setting up color variation:

  • Download this LIF Extractor.
  • Extract db.lif, found at C:\Users\YOURNAMEHERE\AppData\Roaming\LEGO Company\LEGO Digital Designer
  • Put the resulting db folder where the original db.lif file was, and rename db.lif to something else so LDD will use your extracted data instead of the original file.
  • Inside your db folder is Materials.xml. Put a copy of it in the same folder as the 3DXML to OBJ EXE.
  • Launch 3DXML to OBJ and click the "Advanced" button.
  • In the "Edit Materials.xml" box, choose what changes to make. You can add color variation and also apply changes to the base color palette. Click "Edit" and it will create a new XML file.
  • Now you'll want to update the program's own internal color definitions. Enter the name of the XML file you just created in the box below, and click "Update color definitions".
  • Replace LDD's Materials.xml with your new version.

Again, this all sounds more complicated than it is. There's a video further below showing how it's done.

 

Adding color variation to LDD models is simple enough; in advanced mode there's an option for it in the same box where you set the camera position in your LDD model. Just choose how strong you want the variation to be.

 

I strongly recommend converting all the materials to vertex colors for models with color variation. How you do this will depend on what software you prefer; in the video below I'm using Ultimate Unwrap 3D.

 

There's one more advanced feature: You can keep meshes of certain colors from receiving color variation and/or being exported. This is useful for "dummy bricks" only placed in the model to trip LDD's hidden geometry removal. In 3DXML to OBJ v1.7.0_Data\StreamingAssets, you will find "Color Export Exclusion.txt" and "Color Variation Exclusion.txt". Add the desired color IDs here, one per line.

 

Covering studs is obvious enough, but tubes are a bit more finicky. Original LDD model, vs export with the dummy bricks excluded:

 

hzpSjVm.png

 

JkOiUsp.png

 

E76r9Lh.png

  • A simple brick has the entire underside present. These quickly add up to loads of polygons.
  • Placing 1x1 round plates across the surface causes LDD to replace the underside with two triangles. Nice! There's a small handful of other parts with this effect, but 1x1 round plates are the best.
  • In fact, even just one 1x1 round plate can trigger this so long as the rest is still covered.
  • Covering the bottom entirely will cause LDD to remove it 100%.
  • Beware: Some bricks, like those rounded 2x2 pieces, will look like they have the same effect as 1x1 round plates, but actually don't... They still leave much of the tubes intact.

And finally, here's a video showing all of the advanced features, if the text wasn't clear enough:

 

 


 

More tips and tricks!

 

If you have developer mode enabled in LDD, you can press Shift W to toggle wireframe mode. You can also turn rendering of different parts of bricks on and off. LDD's bricks come in four sections, and can be toggled as such:

  • K: Toggles studs
  • Shift K: Toggles bottom and inside of tubes
  • Q: Toggles outside of brick
  • Shift Q: Toggles bottom and inside of brick

You can use this to, for example, capture a model without studs, and use that as a lower LOD.

 

Don't have developer mode enabled? Go to the same AppData folder as db.lif and developermode=1 to preferences.ini.

 

If you've modded LDD's decorations, or LDD has updated, you can update 3DXML to OBJ's internal texture definitions - just copy the Decorations folder from db to the same folder as 3DXML to OBJ's EXE, and click the button for it in advanced mode. (In case you're curious what this does: The program keeps a list of MD5 hashes of texture data, along side their file names/IDs. This allows the program to identify what textures are what in a 3DXML file, without actually containing any of the texture data itself.)

 

You can add your own custom color palettes, just go to 3DXML to OBJ v1.7.0_Data\StreamingAssets\Custom Palettes and use the existing files as examples of how they work.

 

If for some reason you need to, Shift R resets 3DXML to OBJ's saved preferences (resolution, most recent conversion options, etc).

 

Replacing colors when converting a 3DXML with color variation will work... But look very strange, as it'll only affect bricks that happen to be the original color values. The ones that have been slightly lightened or darkened won't be changed.

 

You can mod lower LODs of bricks from LU into LDD... But that'd be its own topic.

Share this post


Link to post
Share on other sites
tomfyhr

How does this converter differ from the converter that was posted on the forum previously, except for the advanced features? 

Will this allow the models to work in games without any major problems?

Share this post


Link to post
Share on other sites
Terrev

I'll write out more details later (if I'm lucky I can make the videos in the next few days), but in short: It works with all kinds of parts (the old method didn't work with things like flexible parts, decals, etc) and the models are more optimized by default. It won't make every model perfect for a game, but it does save a lot of work.

 

I may also end up working on the program itself to add some new things, before I record the videos... I have a few new features in mind.

 

This topic has more information too:

 

 

Share this post


Link to post
Share on other sites
tomfyhr

Is your new script, compatible using Mac? The old method was not compatible with Mac or Linux, only Windows. 

Do you have a link for 3DVIA printscreen for Mac?

 

Here is how I understood how to use Terrevs programs: (Due to the layout of your instructions in the post, some people may find it difficult to understand your instructions.) 

0) Export the LDD model in the format LXFML (if it is not already).

1) Place the LDD model (in LXFML format) in the convert ´ s folder called "models".

2) Launch the converter "LXFML to OBJ". Enter the name of model in the menu box and click on the buttom. A new LXFML-file will be saved in the folder "models".

2) Install 3DVIA Printscreen from this link (if you have not already): http://web.archive.org/web/20110717042507/http://www.3ds.com/fileadmin/PRODUCTS/3DVIA/3DXML/3dxml/3DVIA_PrintScreen_Setup_2.3.exe

3) Launch 3DVIA Printscreen and disable "group by textures". Do NOT change the camera.  

4) Launch LDD and import the new LXFML in LDD. Disable "high-quality render". (The model must be positioned at the origin (at exactly the centre), which is done automatically by LDD.)

5) In 3DVIA Printscreen, capture the the model, which exports the model as a 3DXML-file.

6) Place the 3DXML-file in the converts folder "models".

7) Launch the converter "3DXML to OBJ". Enter the name of the 3DXML-file in the menu box. Enable the option of duplicating verticles (unless the model is large). 

8) Optional: Select a custom palette to override LDD default color palette. 

9) A obj-file will be exported to another folder inside the converters folder "models". 

Disclaimer: You can use these models in your project, as long as you do not openly state that the models are from LDD.

 

Due to the increased number of steps in your methods, some people may find it easier to simply use the old method, even though it has more polygons. Unless someone is so far in developement to have they a finished fully-working game, which is unlikely unfortuntely, I believe that the old method will work just fine. It may not be practical in the long-run, but in short-term it will work. 

          An alternative instead of using LDD models, they can use models ripped from other LEGO games, since they will not disrupt the polygon count and are functional. All models that I have ripped are compatible with Unity, so that should not be a problem for them. If nothing else, those ripped models can be used as placeholders for the models the developers actually wants to use so they can get some progress done, if the models actually are the problem for them not being able to do progress.

 

Since the old method is no longer available as you pointed out in the post, I have uploaded the script, in case someone wants to use that one. You have to place the script inside the LDD folder. 

 

Link to old script: https://mega.nz/#!rpJgjZKC!eUkAoxObWBlK6d1UWqVJEfylN2GdUHTEmhgjoJIg9kY

 

Share this post


Link to post
Share on other sites
Terrev
9 hours ago, tomfyhr said:

Is your new script, compatible using Mac? The old method was not compatible with Mac or Linux, only Windows. 

Do you have a link for 3DVIA printscreen for Mac?

Unfortunately 3DVIA Printscreen was only made for Windows, as far as I know. So there still isn't a Mac compatible method.

Share this post


Link to post
Share on other sites
MineTimelapser

I gave this a spin earlier and it was pretty neat! Tried to stress-test it with a larger model, and while it took a bit (with fixing the vertices enabled) it came out good & had much better geometry of the bat. Fewer polygons, fewer objects, better performance, what's there not to like?

Share this post


Link to post
Share on other sites
Terrev
2 hours ago, MineTimelapser said:

I gave this a spin earlier and it was pretty neat! Tried to stress-test it with a larger model, and while it took a bit (with fixing the vertices enabled) it came out good & had much better geometry of the bat. Fewer polygons, fewer objects, better performance, what's there not to like?

Yeah, duplicate vertex removal is the slowest part of conversion. The more vertices in a single mesh, the longer that mesh takes to process, as each vertex is compared against the others.

 

The code for that was originally written by Bunny83, a very helpful fellow on Unity Answers:

http://answers.unity3d.com/questions/1382854/welding-vertices-at-runtime.html

 

I made some changes that sped it up dramatically, but I think it could still be improved further - if anybody wants to try their hand at it, feel free:

https://github.com/Terrev/3DXML-to-OBJ/blob/master/Assets/Scripts/MeshWelder.cs

 

And for posterity, here's the original unmodified code from Bunny83, in case the original Dropbox link in his Unity Answers comment ever goes down:

 

Spoiler

MeshWelder.cs


using System.Collections.Generic;
using System.Linq;
using UnityEngine;

namespace B83.MeshHelper
{
    public enum EVertexAttribute
    {
        Position = 0x0001,
        Normal = 0x0002,
        Tangent = 0x0004,
        Color = 0x0008,
        UV1 = 0x0010,
        UV2 = 0x0020,
        UV3 = 0x0040,
        UV4 = 0x0080,
        BoneWeight = 0x0100,
    }

    public class Vertex
    {
        public Vector3 pos;
        public Vector3 normal;
        public Vector4 tangent;
        public Color color;
        public Vector2 uv1;
        public Vector2 uv2;
        public Vector2 uv3;
        public Vector2 uv4;
        public BoneWeight bWeight;
        public Vertex(Vector3 aPos)
        {
            pos = aPos;
        }
    }

    public class MeshWelder
    {

        Vertex[] vertices;
        List<Vertex> newVerts;
        int[] map;

        EVertexAttribute m_Attributes;
        Mesh m_Mesh;
        public float MaxUVDelta = 0.0001f;
        public float MaxPositionDelta = 0.001f;
        public float MaxAngleDelta = 0.01f;
        public float MaxColorDelta = 1f / 255f;
        public float MaxBWeightDelta = 0.01f;

        public MeshWelder(Mesh aMesh)
        {
            m_Mesh = aMesh;
        }

        private bool HasAttr(EVertexAttribute aAttr)
        {
            return (m_Attributes & aAttr) != 0;
        }
        private bool CompareColor(Color c1, Color c2)
        {
            return
                (Mathf.Abs(c1.r - c2.r) <= MaxColorDelta) &&
                (Mathf.Abs(c1.g - c2.g) <= MaxColorDelta) &&
                (Mathf.Abs(c1.b - c2.b) <= MaxColorDelta) &&
                (Mathf.Abs(c1.a - c2.a) <= MaxColorDelta);
        }

        private bool CompareBoneWeight(BoneWeight v1, BoneWeight v2)
        {
            if (v1.boneIndex0 != v2.boneIndex0 || v1.boneIndex1 != v2.boneIndex1 ||
                v1.boneIndex2 != v2.boneIndex2 || v1.boneIndex3 != v2.boneIndex3) return false;
            if (Mathf.Abs(v1.weight0 - v2.weight0) > MaxBWeightDelta) return false;
            if (Mathf.Abs(v1.weight1 - v2.weight1) > MaxBWeightDelta) return false;
            if (Mathf.Abs(v1.weight2 - v2.weight2) > MaxBWeightDelta) return false;
            if (Mathf.Abs(v1.weight3 - v2.weight3) > MaxBWeightDelta) return false;
            return true;
        }

        private bool Compare(Vertex v1, Vertex v2)
        {
            if ((v1.pos - v2.pos).sqrMagnitude > MaxPositionDelta) return false;
            if (HasAttr(EVertexAttribute.Normal) && Vector3.Angle(v1.normal, v2.normal) > MaxAngleDelta) return false;
            if (HasAttr(EVertexAttribute.Tangent) && Vector3.Angle(v1.tangent, v2.tangent) > MaxAngleDelta || v1.tangent.w != v2.tangent.w) return false;
            if (HasAttr(EVertexAttribute.Color) && !CompareColor(v1.color, v2.color)) return false;
            if (HasAttr(EVertexAttribute.UV1) && (v1.uv1 - v2.uv1).sqrMagnitude > MaxUVDelta) return false;
            if (HasAttr(EVertexAttribute.UV2) && (v1.uv2 - v2.uv2).sqrMagnitude > MaxUVDelta) return false;
            if (HasAttr(EVertexAttribute.UV3) && (v1.uv3 - v2.uv3).sqrMagnitude > MaxUVDelta) return false;
            if (HasAttr(EVertexAttribute.UV4) && (v1.uv4 - v2.uv4).sqrMagnitude > MaxUVDelta) return false;
            if (HasAttr(EVertexAttribute.BoneWeight) && !CompareBoneWeight(v1.bWeight, v2.bWeight)) return false;
            return true;
        }

        private void CreateVertexList()
        {
            var Positions = m_Mesh.vertices;
            var Normals = m_Mesh.normals;
            var Tangents = m_Mesh.tangents;
            var Colors = m_Mesh.colors;
            var Uv1 = m_Mesh.uv;
            var Uv2 = m_Mesh.uv2;
            var Uv3 = m_Mesh.uv3;
            var Uv4 = m_Mesh.uv4;
            var BWeights = m_Mesh.boneWeights;
            m_Attributes = EVertexAttribute.Position;
            if (Normals != null && Normals.Length > 0) m_Attributes |= EVertexAttribute.Normal;
            if (Tangents != null && Tangents.Length > 0) m_Attributes |= EVertexAttribute.Tangent;
            if (Colors != null && Colors.Length > 0) m_Attributes |= EVertexAttribute.Color;
            if (Uv1 != null && Uv1.Length > 0) m_Attributes |= EVertexAttribute.UV1;
            if (Uv2 != null && Uv2.Length > 0) m_Attributes |= EVertexAttribute.UV2;
            if (Uv3 != null && Uv3.Length > 0) m_Attributes |= EVertexAttribute.UV3;
            if (Uv4 != null && Uv4.Length > 0) m_Attributes |= EVertexAttribute.UV4;
            if (BWeights != null && BWeights.Length > 0) m_Attributes |= EVertexAttribute.BoneWeight;

            vertices = new Vertex[Positions.Length];
            for (int i = 0; i < Positions.Length; i++)
            {
                var v = new Vertex(Positions[i]);
                if (HasAttr(EVertexAttribute.Normal)) v.normal = Normals[i];
                if (HasAttr(EVertexAttribute.Tangent)) v.tangent = Tangents[i];
                if (HasAttr(EVertexAttribute.Color)) v.color = Colors[i];
                if (HasAttr(EVertexAttribute.UV1)) v.uv1 = Uv1[i];
                if (HasAttr(EVertexAttribute.UV2)) v.uv2 = Uv2[i];
                if (HasAttr(EVertexAttribute.UV3)) v.uv3 = Uv3[i];
                if (HasAttr(EVertexAttribute.UV4)) v.uv4 = Uv4[i];
                if (HasAttr(EVertexAttribute.BoneWeight)) v.bWeight = BWeights[i];
                vertices[i] = v;
            }
        }
        private void RemoveDuplicates()
        {
            map = new int[vertices.Length];
            newVerts = new List<Vertex>();
            for (int i = 0; i < vertices.Length; i++)
            {
                var v = vertices[i];
                bool dup = false;
                for (int i2 = 0; i2 < newVerts.Count; i2++)
                {
                    if (Compare(v, newVerts[i2]))
                    {
                        map[i] = i2;
                        dup = true;
                        break;
                    }
                }
                if (!dup)
                {
                    map[i] = newVerts.Count;
                    newVerts.Add(v);
                }
            }
        }
        private void AssignNewVertexArrays()
        {
            m_Mesh.vertices = newVerts.Select(v => v.pos).ToArray();
            if (HasAttr(EVertexAttribute.Normal))
                m_Mesh.normals = newVerts.Select(v => v.normal).ToArray();
            if (HasAttr(EVertexAttribute.Tangent))
                m_Mesh.tangents = newVerts.Select(v => v.tangent).ToArray();
            if (HasAttr(EVertexAttribute.Color))
                m_Mesh.colors = newVerts.Select(v => v.color).ToArray();
            if (HasAttr(EVertexAttribute.UV1))
                m_Mesh.uv = newVerts.Select(v => v.uv1).ToArray();
            if (HasAttr(EVertexAttribute.UV2))
                m_Mesh.uv2 = newVerts.Select(v => v.uv2).ToArray();
            if (HasAttr(EVertexAttribute.UV3))
                m_Mesh.uv3 = newVerts.Select(v => v.uv3).ToArray();
            if (HasAttr(EVertexAttribute.UV4))
                m_Mesh.uv4 = newVerts.Select(v => v.uv4).ToArray();
            if (HasAttr(EVertexAttribute.BoneWeight))
                m_Mesh.boneWeights = newVerts.Select(v => v.bWeight).ToArray();
        }

        private void RemapTriangles()
        {
            for (int n = 0; n < m_Mesh.subMeshCount; n++)
            {
                var tris = m_Mesh.GetTriangles(n);
                for (int i = 0; i < tris.Length; i++)
                {
                    tris[i] = map[tris[i]];
                }
                m_Mesh.SetTriangles(tris, n);
            }
        }
        public void Weld()
        {
            CreateVertexList();
            RemoveDuplicates();
            RemapTriangles();
            AssignNewVertexArrays();
        }
    }
}

 

 

Features I'm looking into adding:

  • Collada/.dae exporting with vertex colors (this is especially crucial for models with color variation, or your material/draw call count will be absurd),
  • User selectable amounts of color variation, since LU's color variation (which it currently matches) was a bit intense, and some people may want to tone it down a bit

Share this post


Link to post
Share on other sites
tomfyhr

Just want to highlight that some people, such as @DarkianMaker, thinks @bhagavans old script for converitng lxf files to obj is shady and that it contains some kind of dangerous code, causing them to not trust it. This is a misconception and the script is perfectly fine use. Could you second that @Terrev ? :)

 

Is there any way to make the your script more automatic, so it requires less steps to be used? Even though it works just fine, I would in the best of words prefer if the script was a bit more automatic, which would make the process less time-consuming in comparision. 

 

Do you know whether  3DVIA Printscreen is compatible with the programe WINE?

Share this post


Link to post
Share on other sites
Terrev

It looks like the correct tool, maybe a slightly later version of it than I have. I'm not sure what the creator's stance is on people re-posting it publicly though, after he removed the original github repository.

 

I'm not aware of any ways to make my own program more automatic, since it relies on 3DVIA Printscreen and such.

 

I've never tried 3DVIA Printscreen with WINE, no idea if it'd work or not. If it does, and thus people can capture 3DXML files on Mac/Linux that way, I'll start releasing Mac/Linux versions of my program.

Share this post


Link to post
Share on other sites
tomfyhr

Well, unless we can get his personal opinion on this matter and he personally tells me to remove the link, then I will keep the link up. Besides, this is not paid stuff so it is in line with the forums rules. :) 

 

Technicaly speaking, even if he did tell me to remove the link, I am not actually obliged to do that @Terrev ? I mean, since the tool hasn´t been registered for copyright and a trademark legally, doesn´t that mean that it can be re-published by practically anyone with out the creator´s consent? I will listen to him out of respect on my own accord, but there isn´t anything that actually demands of me to do so. 

 

Can you make a bot, python code or a batch-converter? Do it by hand one-by-one will become time-consuming after a while. It it easier to use bhagavans script, in comparision to yours, since it requires only one step. So, if you are just making a render his script will work just fine. 

Share this post


Link to post
Share on other sites
Terrev
On 10/31/2017 at 11:54 AM, tomfyhr said:

Well, unless we can get his personal opinion on this matter and he personally tells me to remove the link, then I will keep the link up. Besides, this is not paid stuff so it is in line with the forums rules. :) 

 

Technicaly speaking, even if he did tell me to remove the link, I am not actually obliged to do that @Terrev ? I mean, since the tool hasn´t been registered for copyright and a trademark legally, doesn´t that mean that it can be re-published by practically anyone with out the creator´s consent? I will listen to him out of respect on my own accord, but there isn´t anything that actually demands of me to do so. 

I'm not an expert on legal stuff, and I'd also imagine it could depend somewhat on the license he originally published it under. I think basic respect is the most important factor in this case though.

 

On 10/31/2017 at 11:54 AM, tomfyhr said:

Can you make a bot, python code or a batch-converter? Do it by hand one-by-one will become time-consuming after a while. It it easier to use bhagavans script, in comparision to yours, since it requires only one step. So, if you are just making a render his script will work just fine. 

I don't know how I'd make using this tool more automatic. Most of my programming experience is just with C# in the context of Unity, so stuff like that is outside my scope of knowledge.

 

Also, Unity 2017.3 will include support for 32 bit mesh index buffers, which means my program will be able to display large meshes directly (it currently skips ones that are too large - they're still converted properly, just not shown to the user within the program). So once Unity 2017.3 launches I'll be upgrading this project to it.

Share this post


Link to post
Share on other sites
DarkianMaker

@tomfyhr my concern was that a complete stranger started talking to me and sending me links to a script which I of course don't even touch because only a dingus opens something from a source he doesn't know/trust. Now stop getting me involved in things I have no intention to be part of.

  • Like 3

Share this post


Link to post
Share on other sites
Terrev

Version 1.6.0 released. Added selectable color variation strength, and simple/advanced UI modes (so new users aren't blasted with lots of color variation-related UI boxes and options they may not care about). Get it here: https://github.com/Terrev/3DXML-to-OBJ/releases

 

Edit: Whoops, had a couple more ideas. v1.7.0 released, release notes in the link above.

Share this post


Link to post
Share on other sites
grappigegovert

Hey @Terrev, I've opened up a pull request to speed up the mesh welding quite a bit , care to take a look? :thumbsup:

Share this post


Link to post
Share on other sites
Terrev

Oooh, very nice, thanks! Merged.

 

Edit: I've been trying to figure out the slightly different vertex counts, no luck - so far it's only happened on one large model, going from 369665 verts with the old code to 369687 verts with the new code. Smaller models have been unaffected, though I haven't tried enough to really say for sure if size impacts it. And of course, with that big model, I couldn't find where the differences even were... Hmmm.

 

Edit 2: Imported both versions of the big model into Unity. Unity's own automatic mesh cleanup brought the model from the new code down to the vertex count of the model from the old code. Hrmmm. It's a shame Unity doesn't expose that functionality to the user - it achieves the exact same results as the old code, but runs much faster... Would save us the trouble of figuring this out.

Share this post


Link to post
Share on other sites
Terrev

Updated the first post with some guides and such. Finally.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now