Mass-exporting prefabs as DataNodes

[ArenMook]

In order to be able to serialize materials, the material needs to not be a built-in resource. If you place the material in the Resources folder, a reference to it will be kept. If you don't, the material will be serialized fully.

I use Unity 5.3.4.

DataNode won't serialize children if you are referencing a prefab located in the Resources somewhere, same as any other object. TNet keeps references, and only serializes objects it needs to.

5.4 is beta, so it may be possible that things are still broken in Unity. Quite likely really.

[Bradamante3D]

In order to be able to serialize materials, the material needs to not be a built-in resource. If you place the material in the Resources folder, a reference to it will be kept. If you don't, the material will be serialized fully.

I'm not sure I follow. My Assets/Resources folder is empty right now, except for the Sonic Ether Emissive shader. As it should, it gets referenced in an DataNode export:

1. shader = "Sonic Ether/Emissive/Textured"

But I am also usign Unity-builtin standard shaders. I can't include those in the Resources folder. The Standard shader however gets exported/imported correctly, i.e. it is being reproduced. Builtin shaders relating to particles however are not.

DataNode won't serialize children if you are referencing a prefab located in the Resources somewhere, same as any other object. TNet keeps references, and only serializes objects it needs to.

The children on my test objects are prefabs, yes (for example blinking lights that a lot of ship models share), but they are not in the Resources folder. Why does that matter anyway, since Unity breaks prefab connections once you use prefabs as children of other prefabs? I.e., no "nested prefabs"?
Now there sure are scenarios where only exporting the top-level prefab makes sense. But as soon as your assets become more complex, for example for modding, you need more than that?

Maybe future versions of DataNode should have a "export dependencies" option?

[ArenMook]

You need to explicitly break up prefab connections if you don't want prefab references to be saved. You can also try saving at run-time (prefab references are not available at run-time).

I'm not sure I follow the part about the particle shaders... why is it trying to reference particle shaders? What's using them? If it's particle systems, then my previous post still stands -- you need to put stuff in the Resources folder. Materials used by particle systems for example, or at the very least the shader used by them. By default Unity doesn't even include shaders that aren't referenced by prefabs/gameobjects used in the game at the time of the build. Unity lets you download the entire source for all the shaders. You can keep a local copy of the shader, placing it in the Resources folder.

[Bradamante3D]

Ok I experimented a bit with this and ... c'mon why do I have to download this massive shader package and put it in Resources? Wouldn't it be better if TNet just detected if it's a built-in shader and then write the path entry accordingly?

Also it seems that the models I load from DataNotes don't get reconstructed correctly. Materials that don't include a texture (Standard shader with only a color setting) are just a dark grey color. I guess it sticks with the first detected material color. Looking at the model in the Inspector however shows that the settings are there. So I guess that's more a Unity issue than TNet.
The Sonic Bloom materials do get reconstructed but they don't glow.

Also note that importing a text format DataNode never works for me, only binary format does.

See the attached screenshot for an example. Left is the DataNode import, on the right the native Prefab. Notice how the Inspector complains about the mesh. So maybe that's a problem with my assets? Thing is I won't recreate my models, that would be insane. And if Unity can cope with it, so can TNet

[Bradamante3D]

I emailed Unity QA about this via the Bug Reporter. They now came back to me:

You have to use AssemblyQualifiedName for GetType property.

Example:
instead of using

1. var type = System.Type.GetType(typeof(UnityEngine.Rendering.ReflectionProbeUsage).ToString());

you have to use this:

1. var type2 = System.Type.GetType(typeof(UnityEngine.Rendering.ReflectionProbeUsage).AssemblyQualifiedName);

Interesting... doing this:

1. Debug.Log(typeof(UnityEngine.Rendering.ReflectionProbeUsage));

gives:

1. UnityEngine.Rendering.ReflectionProbeUsage

Yet when I do this:

1. var type = System.Type.GetType(typeof(UnityEngine.Rendering.ReflectionProbeUsage).ToString());

The result is "null". This must be some Unity glitch.

[ArenMook]

That's great and all, but we are dealing with run-time types here. The example they gave you assumes that you already have the type. The type is exactly what we are trying to retrieve using its string name! They gave you a useless suggestion.

[ArenMook]

Actually, on second thought... maybe I was too hasty in my conclusion. Try this.

Find line 278 of TNUnityTools.cs:

1. t = System.Type.GetType("UnityEngine." + name + ", UnityEngine");

Replace it with:

1. var s = "UnityEngine." + name + ", UnityEngine, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null";
2. t = System.Type.GetType(s);

[Bradamante3D]

Hm, no that doesn't change anything. Importing my prefab as a text-based DataNode still errors out with:

Unknown prefix: 73 at position 1
UnityEngine.Debug:LogError(Object)
TNet.Tools:LogError(String, String, Boolean) (at Assets/TNet/Common/TNTools.cs:1001)
TNet.Serialization:ReadObject(BinaryReader, Object, Int32, Type, Boolean) (at Assets/TNet/Common/TNSerializer.cs:2739)
TNet.Serialization:ReadObject(BinaryReader) (at Assets/TNet/Common/TNSerializer.cs:2375)
TNet.Serialization:ReadObject(BinaryReader) (at Assets/TNet/Common/TNSerializer.cs:2366)
TNet.DataNode:Read(Byte[], SaveType) (at Assets/TNet/Common/DataNode.cs:585)
TNet.DataNode:Read(Byte[]) (at Assets/TNet/Common/DataNode.cs:540)
TNet.DataNode:Read(String, Boolean) (at Assets/TNet/Common/DataNode.cs:509)
DataNodeWriter:Start() (at Assets/Scripts/DataNodeWriter.cs:52)

NullReferenceException: Object reference not set to an instance of an object
TNet.ComponentSerialization.Instantiate (TNet.DataNode data) (at Assets/TNet/Client/TNUnitySerializers.cs:1024)
DataNodeWriter.Start () (at Assets/Scripts/DataNodeWriter.cs:59)

When I import the same prefab as a binary DataNode I get the result shown in the screenshot above. In text mode TNUnityTools @ line 278 doesn't seem to fire, only in binary mode it does. I put a few Debug.Logs there for the class and the result is:

UnityEngine.MeshFilter, UnityEngine, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null
UnityEngine.Debug:Log(Object)
TNet.UnityTools:GetTypeEx(String) (at Assets/TNet/Client/TNUnityTools.cs:283)
TNet.ComponentSerialization:DeserializeComponents(GameObject, DataNode) (at Assets/TNet/Client/TNUnitySerializers.cs:1002)
TNet.ComponentSerialization:Deserialize(GameObject, DataNode, Boolean) (at Assets/TNet/Client/TNUnitySerializers.cs:943)
TNet.ComponentSerialization:Instantiate(DataNode) (at Assets/TNet/Client/TNUnitySerializers.cs:1063)
DataNodeWriter:Start() (at Assets/Scripts/DataNodeWriter.cs:60)

UnityEngine.MeshRenderer, UnityEngine, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null
(...)

UnityEngine.SphereCollider, UnityEngine, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null
(...)

... which is what I would expect. However in binary mode the prefab is instantiated, but the materials are not applied, even when both the material and the shader is in Resources. Putting only the shader file does not work btw, it has to be the material.

Do you want me to send you the files so you can take a look?

[ArenMook]

You likely need to re-export it. Your first error message points to it reading it in binary, not text. What's DataNodeWriter? You can attach your text-exported file here, I can try importing it.

[Bradamante3D]

Nuh, I am constantly re-exporting. I just tried both text and binary again, to no avail. I am almost sure that my assets are just s**t, since I am absolutely not a content creator.

When you see "***Glow" materials, it's a Sonic Either/Emissive/Textures material with only color settings, no texture. Same goes for the "Materials/Shared/ColorName" materials, which are Standard shader materials with no texture, but an Albedo color.

I am attaching the text version.

[ArenMook]

Yeah, I don't know about that one. I just get that 73 error too. Can you email the original prefab you are trying to export as a unitypackage, to support [at] tasharen.com?

[Bradamante3D]

Alright, I tried the new TNet version that you sent me via email. In that mail you said I should rename the built-in shaders. Not sure why that makes a difference since the Sonic Either shaders don't get reproduced too. But ... I renamed the Stanard shader to "CCStandard", both the name of the file and the string info at the top of the shader file text. The good news is that both text and binary mode now give the same result:



The bad news is that as you can see this is still not the desired result. Unlike before, the UVed texture is gone ("octotoad1_auv"). That's a step back. The data is in the text file though. I guess, at 694 KB. Don't mind that in the screenshot "octotoad1_auv" does not yet have the "CCStandard" shader applied. I did that and it didn't help.

It seams to me that - as before - materials are not applied correctly. Instead, the last material is applied, right now it's one of the beiges, before it was dark grey.

Another note is that instantiating a text-based DataNode export does not give me console errors. Instantiating the same file as binary however gives me:

1. Unable to resolve type 'Rendering.ReflectionProbeUsage'
2. UnityEngine.Debug:LogError(Object)
3. TNet.Serialization:NameToType(String) (at Assets/Addons/TNet/Common/TNSerializer.cs:305)
4. TNet.Serialization:ReadType(BinaryReader, Int32&) (at Assets/Addons/TNet/Common/TNSerializer.cs:2372)
5. TNet.Serialization:ReadObject(BinaryReader, Object, Int32, Type, Boolean) (at Assets/Addons/TNet/Common/TNSerializer.cs:2404)
6. TNet.Serialization:ReadObject(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2390)
7. TNet.Serialization:ReadDataNode(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2282)
8. TNet.Serialization:ReadDataNode(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2287)
9. TNet.Serialization:ReadDataNode(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2287)
10. TNet.Serialization:ReadDataNode(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2287)
11. TNet.Serialization:ReadObject(BinaryReader, Object, Int32, Type, Boolean) (at Assets/Addons/TNet/Common/TNSerializer.cs:2432)
12. TNet.Serialization:ReadObject(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2390)
13. TNet.Serialization:ReadObject(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2381)
14. TNet.DataNode:Read(Byte[], SaveType) (at Assets/Addons/TNet/Common/DataNode.cs:585)
15. TNet.DataNode:Read(Byte[]) (at Assets/Addons/TNet/Common/DataNode.cs:540)
16. DataNodeWriter:Start() (at Assets/Scripts/DataNodeWriter.cs:55)
17.  
18. Argument cannot be null.
19. Parameter name: type
20. UnityEngine.Debug:LogError(Object)
21. TNet.Tools:LogError(String, String, Boolean) (at Assets/Addons/TNet/Common/TNTools.cs:1004)
22. TNet.TypeExtensions:Create(Type) (at Assets/Addons/TNet/Common/TypeExtensions.cs:70)
23. TNet.Serialization:ReadObject(BinaryReader, Object, Int32, Type, Boolean) (at Assets/Addons/TNet/Common/TNSerializer.cs:2713)
24. TNet.Serialization:ReadObject(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2390)
25. TNet.Serialization:ReadDataNode(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2282)
26. TNet.Serialization:ReadDataNode(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2287)
27. TNet.Serialization:ReadDataNode(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2287)
28. TNet.Serialization:ReadDataNode(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2287)
29. TNet.Serialization:ReadObject(BinaryReader, Object, Int32, Type, Boolean) (at Assets/Addons/TNet/Common/TNSerializer.cs:2432)
30. TNet.Serialization:ReadObject(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2390)
31. TNet.Serialization:ReadObject(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2381)
32. TNet.DataNode:Read(Byte[], SaveType) (at Assets/Addons/TNet/Common/DataNode.cs:585)
33. TNet.DataNode:Read(Byte[]) (at Assets/Addons/TNet/Common/DataNode.cs:540)
34. DataNodeWriter:Start() (at Assets/Scripts/DataNodeWriter.cs:55)
35.  
36. Unable to create an instance of
37. UnityEngine.Debug:LogError(Object)
38. TNet.Tools:LogError(String, String, Boolean) (at Assets/Addons/TNet/Common/TNTools.cs:1004)
39. TNet.Serialization:ReadObject(BinaryReader, Object, Int32, Type, Boolean) (at Assets/Addons/TNet/Common/TNSerializer.cs:2714)
40. TNet.Serialization:ReadObject(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2390)
41. TNet.Serialization:ReadDataNode(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2282)
42. TNet.Serialization:ReadDataNode(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2287)
43. TNet.Serialization:ReadDataNode(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2287)
44. TNet.Serialization:ReadDataNode(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2287)
45. TNet.Serialization:ReadObject(BinaryReader, Object, Int32, Type, Boolean) (at Assets/Addons/TNet/Common/TNSerializer.cs:2432)
46. TNet.Serialization:ReadObject(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2390)
47. TNet.Serialization:ReadObject(BinaryReader) (at Assets/Addons/TNet/Common/TNSerializer.cs:2381)
48. TNet.DataNode:Read(Byte[], SaveType) (at Assets/Addons/TNet/Common/DataNode.cs:585)
49. TNet.DataNode:Read(Byte[]) (at Assets/Addons/TNet/Common/DataNode.cs:540)
50. DataNodeWriter:Start() (at Assets/Scripts/DataNodeWriter.cs:55)

Which reminds of the one here, but that fix is in of course.

I am attaching the text-based exported file. Notice how only the "octotoad1_auv" material explicitely mentions the "CCStandard" shader where the materials from "Materials/Shared" don't. Could that be a problem?

[ArenMook]

To fix the ReflectionProbeUsage, change TNet.UnityTools.FindType to:

1.         static public System.Type FindType (string name)
2.         {
3.                 System.Type t = GetTypeEx(name); // <-- this right here
4.                 if (t != null) return t;
5.  
6.                 List<System.Type> types = TypeExtensions.GetTypes();
7.                 List<string> names = TypeExtensions.GetTypeNames();
8.  
9.                 for (int i = 0; i < names.size; ++i)
10.                         if (names[i] == name || types[i].Name == name)
11.                                 return types[i];
12.  
13.                 return null;
14.         }

As for materials... TNet will only save the path to the material if it's in the Resources folder. It will only save detailed info, like in your "octotoad1_auv" material, if the material is not in the resources folder.