Category Archives: Unity

How to get your Unity LLAPI/WebSocket WebGL app to run under https with AutoSSL & stunnel

<continuing my “blog about whatever random issue I last dealt with in the hopes that some poor soul with the same issue will google it one day” series>

The problem

So you made your new Unity webGL game using the LLAPI and it works fine from a http:// address.  But when you try with https, even with a valid https cert being installed, you get this error:

“Uncaught SecurityError: Failed to construct ‘WebSocket’: An insecure WebSocket connection may not be initiated from a page loaded over HTTPS.”

This is your browser saying “Look, the website is https, but don’t let that fool you; it’s using a normal old web socket to send data under the hood which isn’t encrypted, so don’t trust this thing with your credit card numbers”.

Unity (at the time of this writing) has no internal support for what we really need to be using:  a Secure Web Socket.  So where http has https, ws has wss.  So how do we connect securely if our unity-based server binary can’t serve wss directly?

A little background info about CPanel & AutoSSL

Note: I’m using CentOS 7 on a dedicated server with WHM/CPanel

Setting up your website for proper SSL so it can have that wonderful green padlock used to be a painful and sometimes expensive ordeal.

But no longer!  Enter the magic of CPanel’s AutoSSL.  (I think it’s using Let’s Encrypt under the hood as a plugin?)  Behind the scenes, it will handle domain validation and setup everything for you.  While it does need to renew your cert everything three months, it’s free and automatic.  Add four new domains?  They will all get valid certs within a day or so, it’s great.

We can use this same cert to make your websockets secure as long as they are hosted at the same domain.

Setting up stunnel

This is an open source utility that is likely already included on  your linux server box, if it isn’t, go install it with yum or something.

It allows you to convert any socket into a secure socket.  For example, if you have a telnet port at 1000, you could setup stunnel to listen at 1001 securely and relay all information back to 1000.

The telnet connection has no idea what’s happening and sees no difference, but as long as the outside user can only access 1001, plain text information isn’t sent along the wire and one or both sides can be sure of the identity of who’s connecting.

Depending on the stunnel settings, it might be setup like https where the client doesn’t have to have any certain keys (what we want here), or it could be like a ssh where the client DOES need a whitelisted key.

A way to test a SSL port is to use OpenSSL from the command line on the host server via ssh.  For example (keep in mind 443 is the standard https port your website is probably using):

<at ssh prompt> openssl s_client -connect localhost:443

<info snipped>
subject=/OU=Domain Control Validated/OU=PositiveSSL/CN=host.toolfish.com
issuer=/C=US/ST=TX/L=Houston/O=cPanel, Inc./CN=cPanel, Inc. Certification Authority
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 4946 bytes and written 415 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
 Protocol : TLSv1.2
<info snipped>
Start Time: 1518495864
 Timeout : 300 (sec)
 Verify return code: 0 (ok)

Hitting enter after that will probably cause the website to an html error message because we didn’t send a valid request. That’s ok, it shows your website’s existing SSL stuff is working so we can move on.

So first edit your /etc/stunnel/stunnel.conf to something like this:

pid = /etc/stunnel/stunnel.pid

#we won't screw with changing this because we don't want to relocate/change permissions on our files right now
#setuid = nobody
#setgid = nobody

sslVersion = all
options = NO_SSLv2

#for testing purposes.. these should be removed later:
output = /etc/stunnel/log.txt
foreground = yes
debug = 7

[websitename1]
accept = 29000
connect = 80
cert = /var/cpanel/ssl/apache_tls/oversi.io/combined

[websitename2]
accept = 30000
connect = 20000
cert = /var/cpanel/ssl/apache_tls/oversi.io/combined

Next, still from the ssh prompt, run stunnel by typing stunnel.

Because we have foreground=yes set above it will run it in the shell, showing us all output directly, instead of in the background like it normally would. (Ctrl-C to cause stunnel to stop and quit)

Look for any issues or errors it reports.  The .conf file I listed aboveshows how to set it up for two or more tunnels at once, you likely only need one of those settings.

The “websitename1” part doesn’t matter or have to match anything.

The SSL cert is the most important setting.  You need to give it your private & public & CA info in  the same file.

Now, initially, you might try to setup your keys using the files in ~/ssl/keys and ~/ssl/certs but they seem to not have everything all in one nice file including the CA certs.  I figured out ‘bundled’ ones already exist in a cpanel directory so I linked straight to them there.  (replace oversi.io with your website name)

If stuff worked, you should be able to test your SSL’ed port with OpenSSL again.  In the example above under “websitename1” I told it to listen at 29000 and send to port 80, for no good reason.

So to test from a remote computer we can do:

(you did open those ports in your firewall so outside people can connect, right?)

C:\Users\Seth>openssl s_client -connect oversi.io:29000
Loading 'screen' into random state - done
CONNECTED(00000270)
depth=2 /C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/CN=oversi.io
 i:/C=US/ST=TX/L=Houston/O=cPanel, Inc./CN=cPanel, Inc. Certification Authority
 1 s:/C=US/ST=TX/L=Houston/O=cPanel, Inc./CN=cPanel, Inc. Certification Authority
 i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
 2 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
 i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
---
Server certificate
-----BEGIN CERTIFICATE-----
<snipped>
-----END CERTIFICATE-----
subject=/CN=oversi.io
issuer=/C=US/ST=TX/L=Houston/O=cPanel, Inc./CN=cPanel, Inc. Certification Authority
---
No client certificate CA names sent
---
SSL handshake has read 5129 bytes and written 453 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
 Protocol : TLSv1
<snipped>
 Key-Arg : None
 Start Time: 1518497616
 Timeout : 300 (sec)
 Verify return code: 20 (unable to get local issuer certificate)
read:errno=10093

Despite the errno=11093 and return code 20 errors, it’s working and properly sending our CA info (“cPanel, Inc. Certification Authority”).

Or, easier, let’s just use the browser instead for this one since we’re connecting to port 80 if it works in this case:

https://oversi.io:29000

It worked, see the green padlock?  Oh, ignore the error the website is sending, I assume that’s apache freaking out because the URL request is different from what it’s expecting (http vs https or the port difference?) so it can’t match up the virtual domain.

From here, you should probably remove the debug options in the .conf (including the foreground=yes) and set it up to run automatically.  I just placed “stunnel” in my /etc/rc.d/rc.local file. (this gets run at boot)

Actually connecting using the Unity LLAPI

Congratulations, everything is setup on the server and you’re sure your web socket port is listening and ready to go.

While your server binary doesn’t need to change anything, your webgl client does.

You now need to connect to WSS instead of WS.  Example:

try
 {
   _connectionID = NetworkTransport.Connect(_hostID, "wss://oversi.io", portNum, 0, out error);
 }
 catch (System.Exception ex)
 {
   Debug.Log("RTNetworkClient.Connect> " + ex.Message);
 }

That’s pretty much it.  If someone doesn’t care about https and decides to play over http, it still works fine. (internally the websocket code will still connect via wss)

If you want to see it in action, check out my webgl llapi multiplayer test project https://www.oversi.io

Unity snippet: Finding a GameObject by name, even inactive or disabled ones

I use GameObject.Find() in Unity for things like enabling or fading in/out a menu or to grab an object reference via code to store for later.   (I usually prefer doing things in code rather than drag and dropping references using the Unity Editor when I can)

A problem is GameObject.Find() won’t locate inactive gameobjects which causes me problems because I tend to have inactive object trees in a scene that are just turned on/off when they are being used, like a GUI menu for example.  It’s just kind of my programming style to do things that way.

I couldn’t find a clean full snippet for this online that used scene.GetRootGameObjects, so figured I’d post one.

Cut and paste this to MyUtils.cs or your own utils class:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.SceneManagement;

public class MyUtils 
{

    //hideously slow as it iterates all objects, so don't overuse!
    public static GameObject FindInChildrenIncludingInactive(GameObject go, string name)
    {

        for (int i = 0; i < go.transform.childCount; i++)
        {
            if (go.transform.GetChild(i).gameObject.name == name) return go.transform.GetChild(i).gameObject;
            GameObject found = FindInChildrenIncludingInactive(go.transform.GetChild(i).gameObject, name);
            if (found != null) return found;
        }

        return null;  //couldn't find crap
    }

    //hideously slow as it iterates all objects, so don't overuse!
    public static GameObject FindIncludingInactive(string name)
    {
        Scene scene = SceneManager.GetActiveScene();
        var game_objects = new List();
        scene.GetRootGameObjects(game_objects);

        foreach (GameObject obj in game_objects)
        {
            GameObject found = FindInChildrenIncludingInactive(obj, name);
            if (found) return found;
        }

        return null;
    }

}

And use it from anywhere like:

GameObject obj = MyUtil.FindIncludingInactive(“MyMenuName”);

How to do your Unity builds in the background

If you’ve looked at my recent Unity-related posts and downloaded the projects, you might have noticed I have .bat files like CreateAndUploadWebGLBuild.bat in there to cleanly create final versions easily.

Great.  But if you run this .bat file while you are working on the game with the Unity editor, you’ll get this error:

Aborting batchmode due to failure:
Fatal Error! It looks like another Unity instance is running with this project open.

Multiple Unity instances cannot open the same project.

Ugh.  WebGL builds are especially are incredible slow, so this is a big productivity waster if you’re doing a lot of WebGL testing.  (If you work at a big company and waiting for builds is your favorite time to make coffee and catch up on reddit, well, close this page right now and hope your boss never reads this!)

Cloud Build?

Maybe you could use Unity’s Cloud Build but there are some down sides:

  • Cost $9 a month?
  • Requires that you commit each change to a cvs such as svn, perforce, or git
  • I doubt it can do custom post build commands such as code signing, building a final installer, or rscying files to a linux server and restarting the process.  I guess you could do those things yourself when the build is done, but heck, why not just handle the build yourself from the start.
  • Requires that all your assets are also on cvs (?)

The DIY way

So let’s do it old school with … yep, you guessed it… even more .bat files!  The secret is very simple, Unity will allow you to build in the background if the project directory is different.

You just need to copy your entire project to a temporary folder, then run Unity.exe with parms to do a headless build like normal.

So when you are in a good place with your project and would like to start a background build, hit Ctrl-S to save, then run your “CopyToTempDirAndBuild.bat” file.

After a couple seconds the initial copy is done and it’s safe to continue working while the build is happening in the temp directory – any changes will not be in the temp directory, so your build in progress won’t be affected.  So you can keep working away, without ever closing your main unity editor window.

It’s not especially tricky to do, but here are some .bat files to look at as an example that could be tweaked.

To copy a directory tree to a temp dir: (it assumes the .bat is run from the directory that’s going to be copied)

CloneToTempDir.bat

:this sets some info about the project, for example, it causes %APP_NAME% to hold our main directory name
call app_info_setup.bat

rmdir ..\%APP_NAME%Temp%1 /S /Q
echo Cloning %APP_NAME% to temp dir %APP_NAME%Temp%1...
mkdir ..\%APP_NAME%Temp%1
xcopy . ..\%APP_NAME%Temp%1\ /E /F /Y /EXCLUDE:%cd%\CloneExclusionList.txt

Note:  You may wonder why I’m being repetitive and using “Temp” everywhere instead of including it in a variable.  It’s because you NEVER, NEVER use things like rmdir with only a variable if you can help it, because at some point, that variable is going to be set incorrectly.  It might be  .. or / or something and you’ll delete your whole hard drive.  Safety first.

Another note: %cd% is a DOS trick to get the current full directory

CloneExclusionList.txt contains dirs we don’t want to waste time copying:

Temp\
Library\
build\win
build\linux
build\web\Build

BuildAndUploadWebGLInClonedDir.bat

:this sets some info about the project, for example, it causes %APP_NAME% to hold our main directory name
call app_info_setup.bat
:Setting no pause causes our .bat files to not do pause commands when they are done
set NO_PAUSE=1
:First, let's customize the directory name we're going to close to, by adding a postfix to make it unique
SET CLONE_DIR_POSTFIX=WebGL
:Now let's actually make it, we'll pass in the postfix as a parm
call CloneToTempDir.bat %CLONE_DIR_POSTFIX%

:Move to build dir
cd ../%APP_NAME%Temp%CLONE_DIR_POSTFIX%
:Do the actual build
call BuildWebGL.bat
call UploadWebGLRsync.bat
:Move back out of it
cd ..
:Delete the temp dir we were just using
rmdir %APP_NAME%Temp%CLONE_DIR_POSTFIX% /S /Q
pause

This calls CloneToTempDir.bat with the parm “WebGL” which gets appended to the <AppName>Temp dir.

It then “calls” (this means run another .bat, and come back when it’s done) .bats to create the webgl build and also upload it to the website.

It then destroys the temp directory completely, a good idea because Unity will mark it as the last project and you don’t want to accidently work on that directory later.

Parallel Builds

If you’ve got 16 threads sitting around like I do, it might make sense to build MORE THAN ONE version at a time. (now you see why I use a custom temp dir name for each build)

Apparently, Unity doesn’t care how many simultaneous builds you’re doing on a single computer, as long as its license is valid. (I’m using a pro license)

The key to running parallel builds is to use the “start” command instead of “call”. This means “run this, but instead of waiting, just continue running the rest of this .bat file”.

MakeEverythingInCloneDirs.bat

start BuildAndUploadWebGLInClonedDir.bat
start BuildAndUploadLinux64InClonedDir.bat

So, including the Unity editor you have open, when this is run you’ll have THREE instances of Unity running on the same computer at once.  It all works fine!

Continuous integration as a background operation on the same computer you develop on

If you add a “goto :start” at the bottom of your .bat and a “:start” label at the top, you can “clone and build” non-stop all day.  I don’t see this as very useful as it’s going to break all the time as we’re not doing controlled commits with a cvs but I thought I’d throw that there.

How to get MySQL Connector/Net working with Unity 2017

So you thought my post on networking a few days ago was boring?  Get ready for database fun!

Normally I would just write about what worked, but in this case I’d rather trek through the foggy world of missteps I went through yesterday to make the various errors searchable, maybe my confusion and pain can be of help to some poor future googler.

Why do you want to directly talk to mySQL from a Unity app?

The first thing you normally see when searching online for help with integrating SQL and Unity is “you idiot, it’s too dangerous to ever connect directly to your server database!”

Well, for the CLIENT APP, that’s correct, it’s a horrible idea because anybody can steal your logon credentials and ruin your database. (I love you Team Meat, but … yeah)

Instead, you need a gateway of some kind (for high scores, this is usually a .php page on a website) that can do things like limit score additions from the same day from the same IP address and other types of ‘hardening’.

But for a game server that needs to access lots of data, a direct connection to a database from your Unity-based dedicated server is absolutely vital.

Step 1: Getting the MySQL Connector/Net files

I downloaded the files for “.NET & Mono” and drag and dropped the 4.5 branch files into my Unity project. (spoiler, this was dumb, I only needed one)

Note: Managed .dll files do NOT have to be dragged into any specific folder like C++ plugins do, anything inside of your Assets  folder is ok, even sub-folders.

Step 2: Activating Unity’s 4.5 .Net  support

From inside of Unity, I clicked on one of the newly added .dll files.  Near the bottom of the property window (it’s easy to miss) it gave this message:

“Plugin targets .NET 4.x and is marked as compatible with Editor, Editor can only use assemblies targeting .NET 3.5 or lower, please unselect Editor as compatible platform.”

To fix this, I changed Unity’s scripting runtime version to ‘Experimental .NET 4.6 Equivalent’.

Step 3: Noticing I added the wrong files

I added ‘using MySql.Data;‘ to a .cs as a test.  It compiled, but when I ran the game I got a “TypeLoadException” without any details.  I installed Unity 2017.3.01f (beta) because I heard it gave better description of these kinds of errors, and it did:

TypeLoadException: Could not load type of field 'MySql.Data.Entity.MySqlMigrationCodeGenerator+<>c:<>9__4_0' (1) due to: Could not load file or assembly 'EntityFramework, Version=5.0.0.0,'

EntityFramework?  Ohhh, so that zip had multiple versions of MySQL Connector and this one wants EntityFramework, whatever that is.  I removed them all (and the .xml files too) and re-added MySql.Data.dll.

Note: You probably don’t need to update to a beta version like I did; this isn’t recommended usually.  Obviously I just like to live dangerously.

Great, no errors and I can do a ‘using MySql.Data;‘ just fine.

Step 4: Adding support for System.Data

I added ‘using System.Data;‘ as we’ll need that too.  Uh oh, got an error there.  So I dragged and dropped  <unity install dir>\Editor\Data\MonoBleedingEdge\lib\mono\4.6-api\System.Data.dll into my project.  (I’m attempting to use the version that comes with Unity that matches the .NET version I’m using)

Well, it now compiles, but now I’m seeing ‘loading script assembly “assets/plugins/system.data.dll’ failed!’ error in the Unity log.  Is this the wrong version?

The error goes away if I use <Unity dir>\Editor\Data\MonoBleedingEdge\lib\mono\2.0-api\System.Data.dll instead. Fine, I’ll use that. (spoiler, this causes problems later)

I wrote some test code using MySQL Connector, it works fine in the Unity Editor!  I’m able to create/destroy/modify SQL databases. No warnings in Unity either!  But uh oh, now I’m seeing this in Visual Studio:

warning MSB3277: Found conflicts between different versions of the same dependent assembly that could not be resolved.".

I guess Visual Studio is set to use a newer version and having the old System.Data.dll referenced is bad?

Not only that, but when I build a Win64 stand-alone version, when running it, I got this error:

NullReferenceException: Object reference not set to an instance of an object
 at MySql.Data.MySqlClient.MySqlPoolManager.GetKey (MySql.Data.MySqlClient.MySqlConnectionStringBuilder settings) [0x00012] in <861c41359f7d4496a8fdf269ae744290>:0 
 at MySql.Data.MySqlClient.MySqlPoolManager.GetPool (MySql.Data.MySqlClient.MySqlConnectionStringBuilder settings) [0x00000] in <861c41359f7d4496a8fdf269ae744290>:0 
 at MySql.Data.MySqlClient.MySqlConnection.Open () [0x0016d] in <861c41359f7d4496a8fdf269ae744290>:0 
 at RTSqlManager.Init (System.String serverIP, System.String userName, System.String userPassword, System.String databaseName) [0x00083] in F:\Unity\SQLTest\Assets\RT\MySql\RTSqlManager.cs:51 
 at SQLTest.Start () [0x00030] in F:\Unity\SQLTest\Assets\_Script\SQLTest.cs:64

Uh… ok, I guess I used the wrong version of the .dll or something, I can see it’s being included but … when I look around, I can see other .DLL’s are coming from directory is actually from <Unity dir>\Editor\Data\MonoBleedingEdge\lib\mono\unityjit and not where I took them from.  Using the System.Data.dll from this “unityjit” dir fixed the Visual Studio warning AND fixed the standalone versions (Win and Linux at least, that’s all I tested).

It works!  So is it fast?

Hallelujah! Everything runs ok, both local in Windows (to test this, you need to setup a MySQL server on Windows) and on linux.  (I’m actually using MariaDB, a MySQL fork, that works fine too)

Speed seems comparable to using the C++ version.  As is my way, I wrote a Unity app with built-in Telnet server to test speeds, here are the results on a dedicated linux server:

Command > ?
/? for help
/create to create testtable
/delete to remove testtable
/add <count> to add a bunch of records
/show <index> to look at a specific record
/stats to show current status of testtable and record count
/seektest <count> to benchmark loading <count> random accounts and reading ID and RandomNum from them
/updatetest <count> to benchmark updating <count> random accounts with a new RandomNum and RandomIndexedNum
/ranktest <count> Rank, benchmark, and view top <count> records


Command > /updatetest 1000
Updating RandomNum and RandomIndexedNum for the first 1000 accounts, each in a different SQL statement
Updated 999 accounts in 5.860773 seconds. (0.005860773 p/account) (time taken: `w5.869883 seconds``)

Command > /seektest 1000
Reading ID and RandomNum for 1000 random accounts in 1000 different SQL SELECT statements...
Read from 1000 accounts in 0.1809998 seconds. (0.0001809998 p/account)
 (time taken: `w0.1875515 seconds``)

Command > /ranktest 1000
Ranking top 1000 random numbers...
#1 ID: 50117 RandomNum: 1999986
#2 ID: 31319 RandomNum: 1999854
#3 ID: 5009 RandomNum: 1999835
#4 ID: 7211 RandomNum: 1999784
#5 ID: 40406 RandomNum: 1999772
#6 ID: 9686 RandomNum: 1999698
#7 ID: 18099 RandomNum: 1999644
#8 ID: 41235 RandomNum: 1999593
#9 ID: 12447 RandomNum: 1999505
#10 ID: 37578 RandomNum: 1999396
(not displaying the rest...)Took 0.02680588 to rank and get 1000 records.
#1 ID: 35 RandomIndexedNum: 19954265
#2 ID: 552 RandomIndexedNum: 19942897
#3 ID: 161 RandomIndexedNum: 19898997
#4 ID: 874 RandomIndexedNum: 19872310
#5 ID: 484 RandomIndexedNum: 19859544
#6 ID: 453 RandomIndexedNum: 19777522
#7 ID: 649 RandomIndexedNum: 19776457
#8 ID: 851 RandomIndexedNum: 19769359
#9 ID: 488 RandomIndexedNum: 19741107
#10 ID: 727 RandomIndexedNum: 19711418
(not displaying the rest...)Took 0.0008430481 to rank and get 1000 indexed records.
 (time taken: `w0.02984619 seconds``)

To break it down (note: this database has 50,000 records):

Update a record: 5 ms

Seek and read from a random record: 0.18 ms

Sort 50k unindexed ints and return the top 1000:  27 ms

Sort 50k indexed ints and return the top 1000:  0.84 ms

Final thoughts

Well, I probably should have created the table with ENGINE=MEMORY as I suspect the update record test was hampered by IO writeback (this system doesn’t have an SSD), but overall I’m happy with these speeds.

I wonder how the speeds would compare with SQLite? Is the fact that I’m using a beta version of Unity change/break anything I did?  I don’t know, but it works fine now so whatever.