Struggling with a serialization error?!?

For some reason, I can run my current Unity spike on the laptop, but not on the phone.  I finally managed to find some indication of what might be going wrong by attaching the debug process to the phone (having built a development version with script debugging) and restarting the app.

Conclusion : Assemblies break the traditional use of the “Editor folder” convention. This (invisibly) causes broken builds, and is a PITA when you have dependencies on third party plugins which use editor scripts.

I get errors like :

This seems to be a experience others have had (https://answers.unity.com/questions/1288797/a-script-behaviour-has-a-different-serialization-l.html) … but there could be any number of causes.  🙁   Time to start churning through the possibilities.

1. All filename/classnames match
2. No “undefined” Tags anywhere on any game objects (that I can see)
3. I have a [System.Serializable] attribute on both the DayAtoms and ESTrainingAtom classes… could this nesting be an issue?  If I remove the serialization from ESTrainingAtom, I lose the serialised list in the DayAtoms class (no surprise there). However, I can remove the serialization from the DayAtoms class.  It doesn’t change anything. 🙁
4. There are no #if UNITY_EDITOR preprocessing directives anywhere near anything I’m doing
5. ?!?

Time to create a new scene and check if it’s a systematic problem with the current project, or just something in the scene I’m working in. So… I’ve created a new scene with a simple ping ponging cube :

… added this to the build, and removed the original scene from the build (not just disabled it) so there should be no pollution. Low and behold, the cube moves in the editor but not on the phone. I think something annoying has gone wrong in this project. Time to come at it from the other angle – “additive” rather than “subtractive”. Next, I’ll create a new project and check that there isn’t some deeper OS weirdness happening… then gradually reconstruct the project until I find what broke, or until it just works because this time Unity hasn’t created some internal issue it can’t resolve itself.

ok. So a fresh project with a fresh scene works fine on the phone. Phew. It isn’t something even deeper down the stack (and it wouldn’t have been the first time!).

At this point, I think I’ll take a short-cut, by exporting the assets I need from the broken project, copying the bare minimum of scripts that I need (without the tests – as I have a suspicion that the custom assemblies necessary for the unit testing might have something to do with this situation) without referencing the EyeSkills Framework or other EyeSkills dependencies (which will mean creating an insitu “fake” of the framework’s AudioManager).  Then we’ll see how far that takes us.

19.03.2019 – UPDATE

So, I’ve found more time for this.  It turns out that the new project works flawlessly so far, in the IDE and on the phone.  I started by creating a project without dependencies on the framework, then introduced the framework and added support for VR devices. No problems.  My next step will be to reintroduce the unit tests I wrote, which will require me to explicitly create assemblies. I have a nasty feeling that this is where the problem may lie.

First of all… the PowerUI dependencies are constantly causing problems – reporting that assemblies are missing.  As usual, it just takes two or three restarts of Unity for it to gradually bludgeon its way through the dependencies tree. Don’t waste your time looking for reasons!

Next problem.  After reconfiguring Unity to run in .NET 4.x our serialised Atom data gets lost :-/  That data needs to be re-entered, as does the serialised data for the UXMolecule :-/  Why Unity? Why?  Sigh. Builds and runs in the IDE, but there are over 100 compilation errors related to the Framework Dependencies when trying to build for android.

It seems that this madness is due to a collision between the “old” way of specifying scripts as being relevant to the Editor by convention (directory name) and by ASM (assembly definition). The two approaches don’t match. I need to use ASM for the unit tests, which causes the editor scripts in the many plugins in the framework dependencies to break.

One thing I did differently between this blank project test and my original version, was to place the Framework and Dependencies as sibling folders within Assets.  Placing them as Siblings within a subfolder of Assets (e.g. EyeSkills) and creating an assembly on that subfolder, however, doesn’t work this time. *These compilation problems are only happening while trying to build for the phone*.

Next step. Try to reverse. Remove the test folders and remove all the assemblies. This actually causes the build to succeed!  The assemblies are somehow the root of the issue here, which is quite damning of Unity given that without explicitly creating assemblies you can’t unit test.

Let’s just iterate around one more time.  I use the IDE to create an empty Test folder (which includes an assembly). The build succeeds.

This time I’ll create an assembly directly under Assets (which contains my test folder, and everything else) and call this EyeSkillsAssets – perhaps this will be close enough to the automatic assembly creation in Unity that it somehow works and still allows my tests to get the dependencies they require on our classes. The build succeeds.  It runs correctly on the phone. It fails to run on the phone (i.e. the app builds and is installed, but fails to start executing scripts).

Let’s take this further and delete our tests folder again.  We’ll quit Unity and restart.  There is now only one difference between the project which runs on the phone and this one, and that is an explicitly created assembly file under the Assets folder, which has no dependencies, and is specified for “any platform”.

Again, it runs in the editor. It successfully compiles. It fails to run on the phone. No issues in adb logcat either.

Wait – for some reason it had auto selected TestAssemblies as true.  Removing that… and we are then back to 130 build time errors.

So.. now we finally go full circle – remove the Framework and Framework Dependencies. We check it builds and runs with an assembly under Assets. … Yes, it does.

This implies that something in the framework dependencies is breaking the build when used with an assembly.

One option is to just forget about unit testing.  Is this an option?  No. It is not.  How can a wider community work together without it. It can’t. So back to “dll hell” we go.  Next step…

I feel that many parts of the original framework are redundant (I’m looking at you, PowerUI).  It may be more sensible to separate our a true “micro core” which just provides what we need in terms of our EyeSkills Camera Rig and move from there, incrementally, with assemblies and unit tests, checking for breakage at each step.

The bare minimum I want to keep going from the framework are the Network/Config Managers, and the CameraRig. That’s pretty much it for now. So: deleting the UI scenes, the GVR and Reorderable Lists dependencies, and the Editor/buildLogic and (for reasons I do not understand) the EyeSkillsMenuOptions/Audio Creation scripts.

Actually, the Editor scripts might have been causing a problem precisely because of the incompatibility between using Editor folders and assemblies (there is still an assembly present in the project).  If that really is the cause, then we’ll just have to start adding editor entities with assemblies, one at a time as we need them.

Now we just need the JSON and WebSocket dependencies, and they work flawlessly (again, I am beginning to suspect, because they don’t contain any Editor subdirectories!).

For now, I shall use this opportunity to push on with actual development again, and perhaps spin out a new smaller framework with less dependencies, depending on what it turns out is needed.

So… lets add back in the unit tests, create assemblies for Framework, Dependencies, Tests, and UX, build and run! Fingers crossed….

Yes!

OK. Tests updated, and we’re off.