Tips – Robot Monkey Brain

Unity Input System: How to Pair Keyboard AND Mouse to the same InputUser

Unity’s new Input System is cool but the documentation needs a lot of work. Most tutorials on the new Input System are super basic and rely on the PlayerInput and PlayerInputManager components to do the magic of syncing users and control schemes which won’t work if you’re doing more custom gameplay or multiplayer setups.

So digging deeper you may have found the InputUser.PerformPairingWithDevice(...) method which will allow you to manually create a InputUser paired with a device like a gamepad. Which works great, however what is not obvious is how you pair 2 devices with a UserInput, like with the Keyboard AND the Mouse!

Well, after days of googling and banging my head against the wall I finally found the answer in a semi-unrelated forum thread:

var controls = new MyControls();
var user = InputUser.PerformPairingWithDevice(Keyboard.current);
InputUser.PerformPairingWithDevice(Mouse.current, user: user); // this is the key, non-intuitive part
user.AssociateActionsWithUser(controls);
user.ActivateControlScheme("KeyboardMouse");
controls.Enable();

So you still do the initial instantiation/pairing with one of the devices (the Keyboard in this case) and then follow it up with another call to PerformPairingWithDevice() this time passing the Mouse and the user that was just created.

This feels unfinished to me. It seems an odd process to pair two devices like this. Instead I wish you could pass in an array of devices when first instantiating the InputUser.

Hope others that struggle with this are able to find the answer faster now! Here’s the thread I found the solution: https://forum.unity.com/threads/solved-can-the-new-input-system-be-used-without-the-player-input-component.856108/#post-5669128

Convert Unity UI Screen Space position to World position

All credit goes to tosiabunio over on this forum post for this great solution.

Using Unity’s Canvas UI is excellent when you want your UI elements to scale and position correctly across different devices and aspect ratios. Learning to work with the Canvas UI properly is a bit of a nightmare but that’s a different story.

My issue was that while I do want UI elements to reposition across different aspect ratios I also want some GameObjects to have animations that seemingly interact with the UI elements. For instance, I wanted the Deck and Discard piles (UI Elements in Screen Space) to have Cards (Game Objects in World Space) come in/out of them.

GameObject Cards in World Space coming out of UI element Images in Screen Space. Works even as the UI elements move due to aspect ratio changes.

The problem is that you cannot simply have your World Space objects get the transform position of the Screen Space UI objects, the positions will not translate properly and you’ll end up with your World Space objects flying off the screen. This is where Tosiabunio’s solution comes into play when you want to ” Translate anchored overlay UI element to world position. “

var screenToWorldPosition = Camera.main.ScreenToWorldPoint(rectTransform.transform.position);

Basically, all you have to do is use the ScreenToWorldPoint method and pass it the desired UI object’s transform.position. It’s a simple one liner but this solution had evaded me for months where I was doing all kinds of crazy hacks trying to get these two positioning systems to play nice together.

Tosiabunio’s post also mentions a way to convert in the opposite direction, World Space into Screen Space:

Translate game object position from the world to overlay position in order to place UI element near that object.

To which he suggests:

RectTransformUtility.WorldToScreenPoint(Camera.main, obj.transform.position)

I haven’t tried this yet but it might be handy for others. Also, user karma0413 points out that WorldToScreenPoint can be slightly off in some situations and offers a solution which involves performing position calculations off the scale factor. Again, I haven’t tried it yet as the original one liner works perfectly for my purposes.

Let me know if this helps you and if you have any other Unity Canvas issues that you need help resolving. I’ve been working with it a lot lately and perhaps I can help or write more about common issues people face.

Unity Remote 5 not working: Fix

If you are attempting to use Unity Remote 5 on an Android device attached to a PC (and perhaps Mac/Linux) but upon clicking Play in Unity you game goes not show up in Unity Remote then these tips might help:

First, disconnect your Android device from your PC and shut down Unity to ensure you’re starting from a clean slate. This walkthrough also assumes you’ve completed the steps found in Unity’s documentation: https://docs.unity3d.com/Manual/UnityRemote5.html

Enable Developer Mode and USB debugging (or re-enable it)

On your Android Device, enable Developer Mode

Open Settings
Scroll down to and click About <Device> (Mine says About Tablet)
Scroll down to the Build Number
click Build Number many times (after a few times, a message will pop up telling you how many more times to click)

Now that Developer Mode is active

Go to Settings>Developer Options
Ensure Developer Options is toggled On (at the top of the list)
Optional: Enable Stay awake
Scroll down, under Debugging enable USB Debugging

Now you device should be ready.

Plug in your Android device to your PC.
Open Unity Remote 5 on your Android Device
Open you Unity project
Click Play

You should see you Unity project on your Android device. If not, try shutting down Unity and Unity Remote and restarting each.

Change Unity’s C# Template Code Style

The first thing I used to do every time I’d edit a new C# script in Unity is delete the comments and unused usings which are all included as part of the template. Going through that activity for every new script was bad enough, but on top of that the code style used in the template is pretty far off from what is considered standard for C#. Eventually, I decided it was time to stop my pre-scripting ritual and just make the default template into what I want. So to begin:

Navigate to the ScriptTemplates folder in your Unity install folder.

Route with a typical Windows install: C:\Program Files\Unity\Editor\Data\Resources\ScriptTemplates

Open 81-C# Script-NewBehaviourScript.cs.txt in your text editor of choice.

From here you can edit the file’s layout however you choose. Mine looks like this:

using UnityEngine;

public class #SCRIPTNAME# : MonoBehaviour 
{
    private void Start() 
    {
        #NOTRIM#
    }
    
    private void Update()
    {
        #NOTRIM#
    }
}

Note: You may be unable to save/overwrite the template file if you did not open your text editor As Administrator. Don’t worry, you can just Save As to your Desktop and drag/drop the new version from the Desktop to the ScriptTemplates directory.

Event Delegation in Unity

What is Event Delegation?

“…a helper object, known as a delegate, is given the responsibility to execute a task for the delegator“
– Delegation Pattern Wikipedia

Why use Event Delegation?

A good practice when working with Unity (or any framework) is to decouple your UI logic from your game logic. It’s fine to have the UI code reach down to the game logic layer and call methods directly on it but having the game logic layer directly call methods on the UI logic’s layer is, in general, a bad practice because doing so “marries” you to whatever UI you are currently using. So to get around this we can use C#’s Delegates and Events to fire off methods throughout our UI and even on the game logic layer.

How to use Event Delegation with Unity

All we have to do is:

create a static Event Manager class
- Add delegates and events to this Event Manager
Listeners will add functions to the events
Delegators will fire off the events

The Manager: Defining events

public class EventManager {
    public delegate void EnemyHovered(int laneIndex, int targetRange);
    public static event EnemyHovered EnemyTargetedRequest;
    public static void EnemyTargeted(int laneIndex, int targetRange) {
        if(EnemyTargetedRequest != null) EnemyTargetedRequest(laneIndex, targetRange);
    }
}

In this example we create a delegate with the type EnemyHovered which takes the arguments laneIndex and targetRange. Then we make an event EnemyTargetRequest which is the event that the listeners will subscribe functions to. Finally, there is a static method that can be called by delegators when they want to fire off this event, triggering all the functions that are subscribed to it.

The Listener: Subscribing to the event

In another class we can add the following:

void Awake() {
    EventManager.EnemyTargetedRequest += DoSomething;
}

public void DoSomething(int targetedIndex, int targetRange){
    // do something
}

In this class we define some function called DoSomething that takes the same arguments as our delegate. In the Awake() method we add the function to the static class’s EnemyTargetedRequest event.

The Delegator: Firing off events

In yet another class we can do the following:

public void OnPointerEnter(PointerEventData eventData) {      
    EventManager.EnemyTargeted(_index, _targetCount);
}

Here we’re just using Unity’s OnPointerEnter event as the UI event that will trigger our EnemyTargeted event. Because our Listener class subscribed it’s DoSomething function to the EnemyTargetRequest event when SignalManager.EnemyTargeted is called it will fire off the DoSomething function!

So with this, you can keep all your events off in the static Event Manager, subscribe to them when you need an object to react to an event and fire off the events whenever you need to which allows you to keep UI and game logic separated. You could even have GameObjects subscribe to events and unsubscribe from them as they are created and destroyed which is very useful when these objects are created and destroyed dynamically.

Good Enough Guide to Unity’s Unet Transport Layer (LLAPI)

I was excited to see that just as I was getting ready to start my first networked multiplayer Unity project, Unity was releasing their new Unet networking APIs. “What great timing!”, I thought. However, I was a total noob when it comes to working with sockets and communicating over a network and after playing around with Unet’s HLAPI (High-Level API) it quickly became clear to me that I’d have to dive deeper to get the functionality that I was looking for. So I began to read the documentation for the LLAPI (Low-Level API) which was not pleasant at all. After seeking advice and guidance from everyone I found shake down I managed to get everything working, connecting, and sending messages to and fro.

So here is my tutorial on working with Unity’s Unet LLAPI in C# for networking noobs. There are 4 main parts to any socket networking setup, opening a socket, connecting to another socket, sending messages and receiving messages:

Starting a “Host” (or Opening a Socket)

First make a new Unity project, 2D or 3D, it doesn’t matter for this tutorial. Then add an Empty GameObject to the scene and name it Transport. Then add a new C# script to the object and open the script.

We’re going to setup our socket right as the application begins so let’s begin in the Start() method. First thing we want to do is call initialize on the NetworkTransport class (don’t forget to add using UnityEngine.Networking to the class):

public void Start() {
  NetworkTransport.Init();
...

Next we just follow Unity’s documentation and add the ConnectionConfig. For most situations using the default config is fine, so let’s just go with that:

...
ConnectionConfig config = new ConnectionConfig();
...

Up next we need to add a channel and keep the channelId handy so we can use it later for sending/receiving. To do this we’ll add an int member variable to the class (not just the Start() method). We’ll also use the QosType of Reliable when setting up our channel. You don’t really need to know what this means right now (read: I don’t know enough about what this means and I still survive):

int myReliableChannelId; // above Start()
...
  myReliableChannelId = config.AddChannel(QosType.Reliable); // within Start()
...

This next part pretty much just sets the max number of connections allowed on your soon to exist socket:

...
  int maxConnections = 10;
  HostTopology topology = new HostTopology(config, maxConnections);
...

and finally we open the socket. One of the main stumbling blocks when trying to understand the LLAPI based on Unity’s documentation is that many thing are just named poorly. For starters, there is NetworkTransport.AddHost(). This doesn’t actually add a “Host” like the HLAPI does. This command actually just starts up a socket, I’m pretty sure they should change the name of the method to AddSocket, but who am I?? When we start the socket we have to tell it which port to listen to and in this case we use port 8888. Also the AddHost() method returns an int which is the “HostID” but screw that name we’re going to call it what it is, a SocketId. This SocketId also has to be accessible from other methods in this class so let’s put it up with the channelId:

int socketId; // above Start()
int socketPort = 8888; // Also a class member variable
... // back in Start()  
  socketId = NetworkTransport.AddHost(topology, socketPort);
  Debug.Log("Socket Open. SocketId is: " + socketId);
} // closing curly for Start()

And with that we have our code which sets up a socket! It’ll run right away when you click the Play button in Unity. You should see the debug message appear in the console, probably with the socketId of 0. It’s not that cool yet but we’re getting there.

Connecting to the socket

Once our socket is open we use it to connect to another socket, typically on another device. We can worry about getting another device in the mix later. For now, let’s just write our Connect() method.

int connectionId;
...

public void Connect() {
  byte error;
  connectionId = NetworkTransport.Connect(socketId, "localhost", socketPort, 0, out error);
  Debug.Log("Connected to server. ConnectionId: " + connectionId);
}

We need to keep the connectionId available to all methods because later we’ll use it to send messages. NetworkTransport.Connect() … connects us to another server (Hey! They did a proper job naming that one!). We pass in our socketId, the ip for the remote device (in this case we’re just connecting with ourselves, like a long weekend alone in the woods), the socketPort of the remote machine (same as ours in this case), 0 (I forget what this is but whatever), and if there is a problem we get an error out.

The Unity documentation mentions that we can check the error with:

if (error != kOk) // wat?

But I have no idea what kOk is and judging by all the red squiggles it certainly isn’t a byte. If you know what kOk is please tell someone (preferably me) we need to figure this out people! Basically, I just include the out error parameter because I have to.

I used my Android phone as the remote device but you can use another computer if you have one available. But for now you can do most of your testing by connecting to your own socket (I’m sure there is a rude joke in there).

Lastly, go into Unity and add a UI button and set the OnClick event to fire off our Connect() method. If you don’t want to do that I suppose you can just call Connect() at the end of Start() after our code that adds the socket. That would be good enough for now if you don’t want to make the button.

Sending a Message!

Finally (half of) the thing we actually want to do! Here we turn a string into a stream of bytes then send those bytes out our socket connection.

public void SendSocketMessage() {
  byte error;
  byte[] buffer = new byte[1024];
  Stream stream = new MemoryStream(buffer);
  BinaryFormatter formatter = new BinaryFormatter();
  formatter.Serialize(stream, "HelloServer");

  int bufferSize = 1024;

  NetworkTransport.Send(hostId, connectionId, myReliableChannelId, buffer, bufferSize, out error);
}

Quick run down: Declare another (unused) byte variable called error. Create byte array called buffer and set it’s length to 1024 (this is the max length of our message when it is in byte form). The next 3 lines do the conversion from string to byte array. I don’t know the details about it, but it works, so it’s good enough. We also send over the size of our buffer byte array as an int (which is bufferSize). Lastly we put all the stuff together, starting with out class variables hostId, connectionId, and myReliableChannelId then adding the rest.

Now that we’ve got our SendSocketMessage() method you can go back into Unity and add a UI button to fire off the method on the buttons OnClick event.

Listening for NetworkEvents (or Receiving a Message)

Last part! We can send a message to other connected devices but now we have to make our application do something when it gets these messages. Otherwise we’re just shouting into the void. We need to always be checking for new messages coming in so this portion is added to the Update() method.

void Update() {
  int recHostId;
  int recConnectionId;
  int recChannelId;
  byte[] recBuffer = new byte[1024];
  int bufferSize = 1024;
  int dataSize;
  byte error;
  NetworkEventType recNetworkEvent = NetworkTransport.Receive(out recHostId, out recConnectionId, out recChannelId, recBuffer, bufferSize, out dataSize, out error);
...

All we’re doing is declaring all the variables we need then setting up a NetworkTransport.Receive() method. Almost all those variables are assigned as a result of a message being received (all the parameters that begin with out). The recBuffer contains the byte array message that was received, while recNetworkEvent is an enum with 4 possible event types:

Nothing – When no messages are received.
ConnectionEvent – A socket connection is made.
DataEvent – A message is received.
DisconnectEvent – A device that was connected as told us it is closing the connection.

We’re going to use these event types to decide what to actually do with the messages we receive. So we’re going to continue to add on to our Update() method with a big ol’ switch statement:

...
switch (recNetworkEvent) {
  case NetworkEventType.Nothing:
    break;
  case NetworkEventType.ConnectEvent:
    Debug.Log("incoming connection event received");
    break;
  case NetworkEventType.DataEvent:
    Stream stream = new MemoryStream(recBuffer);
    BinaryFormatter formatter = new BinaryFormatter();
    string message = formatter.Deserialize(stream) as string;
    Debug.Log("incoming message event received: " + message);
    break;
  case NetworkEventType.DisconnectEvent:
    Debug.Log("remote client event disconnected");
    break;
}

So based on the type of network event we have a few different debug messages display. With the event type Nothing we do… nothing. This message comes in every Update() that a message of some other type is not received so we can ignore it. With ConnectionEvent we just want to be notified of the connection, but in a game you might do something like load a player prefab or display some text that a new player has connected.

DataEvent is the real meat and potatoes. This is where a message is received from a connected device and our application has to do something based on that message. Since our SendSocketMessage() method currently only sends a string we’ll only include code to handle string messages received (it’ll still try to convert any bytes into a string, so if you send something other than a string you’ll get an error or a garbage string (probably? Didn’t try it.)).

You can see that in the DataEvent case we’re doing almost the same thing as we did in the SendSocketMessage() method but instead of making a string into bytes, now we’re making bytes into a string. Once the message is back in a string form we display it as part of a debug message. Bam! Message Received!

Lastly there is the DisconnectEvent type. I don’t do anything with that event type here except log it to the console but you can image using this event as the time to do things like remove a player’s prefab, or display a disconnect message on screen.

Try it out

That should be all you need to get up and running. You have an application that, on startup, makes a socket available for connections. With the press of your Connect button the application uses it’s socket to connect to another device’s socket. Pressing your Send Message button will blast a string message from your socket to the remote devices socket and the remote device will receive the message, unpack it and display it in the console.

I did my testing initially just having my computer running Unity connect to it’s own socket, just as a way to rapidly test things. Because of this I saw the connect message in the console 2x, since one event was made for each side of the connection. After I got everything working like this, I changed the “localhost” to my computer’s IP address then built the project and sent it to my Android phone. I would then run the application in the Unity editor so I could see the console and I used the app on my phone to connect and send the message. It was really rewarding to hit the Send Message button on my phone and see the message appear in Unity’s console on my PC!

Expanding

This post is already super long, but I just want to give some final thoughts and extra tips.

How do I use this to make a game?

You can have another switch statement that executes different methods or series of methods based on the message received. So getting a message like “ChangeToNight” can trigger a method that changes the time in your game environment.

Does the message have to be a string?

No! It can be any object, all you have to do is serialize the object in a similar way to how we serialized the string. For instance I have experimented with sending JSON messages so that I can transport more complex objects. Using JSON .NET for Unity made this super easy.

Where can I read more about sockets?

While trying to figure all of this out I was directed to Beej’s Guide to Network Programming. It’s pretty advanced, written for C++ and I didn’t read much of it, but the What is a socket? section helped clear up some of my confusion about sockets.

I hope this helps some people who want to dive into Unity’s new networking API but don’t know a thing about network programming prior. If you are an experienced network programmer and have noticed things that I am wrong about, or things that could be expanded upon please let me know and I’ll update this guide.

Naming Things is Hard: Brainstorming Names

I’ve been working on my game for about 3 months now and I have yet to come up with a suitable name for it. I’ve been referring to it mainly as “My Game” since it’s really my first game project to make it this far. But I have given it a code name which I will use to refer to it when posting here, and that code name is Nano. This comes from the setting of the game which I’ll discuss more in a future post but suffice to say, nano technology plays an important roll in the setting.

How to Brainstorm Names

I came up with Robot Monkey Brain during a few short brainstorming sessions with a pal of mine. It’s real important to have someone to bounce ideas off of, and when brainstorming it’s best if there aren’t any negative judgments about ideas. Just get a couple of people to start spouting out names and phrases and follow the good ideas, building on them but feel free to jump tracks if one path is getting stale. We were hovering around the words Robot and Monkey since I’m a big fan of robotics and my online handle has included a misspelling of the word monkey since I was a freshman in high school. Once you find a good one make sure it isn’t taken:

Search the name, see if the name already has some pop culture weight around it.
See if the domain is taken. You want the .com extension unless you’re outside the US and don’t plan on making your game available internationally.
Check if there is already a Facebook page and twitter account using that name.
If all those things are clear, snatch up the domain, Facebook page and twitter name.

I kinda screwed up the twitter part in that RobotMonkeyBrain is 1 letter too long to be a twitter handle, so I had to go with @RoboMonkeyBrain. It’s not the end of the world but the OCD part of me would prefer all the names match and I do have the Robot Monkey Brain Facebook Page all set.

I’ll go through the same process when I’m ready to give an official name to Nano, but I still need to develop more of the core systems and setting details before I set that in stone.