Unity Input System: How to Pair Keyboard AND Mouse to the same InputUser

Unity’s new Input System is cool but the documentation needs a lot of work. Most tutorials on the new Input System are super basic and rely on the PlayerInput and PlayerInputManager components to do the magic of syncing users and control schemes which won’t work if you’re doing more custom gameplay or multiplayer setups.

So digging deeper you may have found the InputUser.PerformPairingWithDevice(...) method which will allow you to manually create a InputUser paired with a device like a gamepad. Which works great, however what is not obvious is how you pair 2 devices with a UserInput, like with the Keyboard AND the Mouse!

Well, after days of googling and banging my head against the wall I finally found the answer in a semi-unrelated forum thread:

var controls = new MyControls();
var user = InputUser.PerformPairingWithDevice(Keyboard.current);
InputUser.PerformPairingWithDevice(Mouse.current, user: user); // this is the key, non-intuitive part
user.AssociateActionsWithUser(controls);
user.ActivateControlScheme("KeyboardMouse");
controls.Enable();

So you still do the initial instantiation/pairing with one of the devices (the Keyboard in this case) and then follow it up with another call to PerformPairingWithDevice() this time passing the Mouse and the user that was just created.

This feels unfinished to me. It seems an odd process to pair two devices like this. Instead I wish you could pass in an array of devices when first instantiating the InputUser.

Hope others that struggle with this are able to find the answer faster now! Here’s the thread I found the solution: https://forum.unity.com/threads/solved-can-the-new-input-system-be-used-without-the-player-input-component.856108/#post-5669128

Convert Unity UI Screen Space position to World position

All credit goes to tosiabunio over on this forum post for this great solution.

Using Unity’s Canvas UI is excellent when you want your UI elements to scale and position correctly across different devices and aspect ratios. Learning to work with the Canvas UI properly is a bit of a nightmare but that’s a different story.

My issue was that while I do want UI elements to reposition across different aspect ratios I also want some GameObjects to have animations that seemingly interact with the UI elements. For instance, I wanted the Deck and Discard piles (UI Elements in Screen Space) to have Cards (Game Objects in World Space) come in/out of them.

GameObject Cards in World Space coming out of UI element Images in Screen Space. Works even as the UI elements move due to aspect ratio changes.

The problem is that you cannot simply have your World Space objects get the transform position of the Screen Space UI objects, the positions will not translate properly and you’ll end up with your World Space objects flying off the screen. This is where Tosiabunio’s solution comes into play when you want to ” Translate anchored overlay UI element to world position. “

var screenToWorldPosition = Camera.main.ScreenToWorldPoint(rectTransform.transform.position);

Basically, all you have to do is use the ScreenToWorldPoint method and pass it the desired UI object’s transform.position. It’s a simple one liner but this solution had evaded me for months where I was doing all kinds of crazy hacks trying to get these two positioning systems to play nice together.

Tosiabunio’s post also mentions a way to convert in the opposite direction, World Space into Screen Space:

Translate game object position from the world to overlay position in order to place UI element near that object.

To which he suggests:

RectTransformUtility.WorldToScreenPoint(Camera.main, obj.transform.position)

I haven’t tried this yet but it might be handy for others. Also, user karma0413 points out that WorldToScreenPoint can be slightly off in some situations and offers a solution which involves performing position calculations off the scale factor. Again, I haven’t tried it yet as the original one liner works perfectly for my purposes.

Let me know if this helps you and if you have any other Unity Canvas issues that you need help resolving. I’ve been working with it a lot lately and perhaps I can help or write more about common issues people face.

Unity Remote 5 not working: Fix

If you are attempting to use Unity Remote 5 on an Android device attached to a PC (and perhaps Mac/Linux) but upon clicking Play in Unity you game goes not show up in Unity Remote then these tips might help:

First, disconnect your Android device from your PC and shut down Unity to ensure you’re starting from a clean slate. This walkthrough also assumes you’ve completed the steps found in Unity’s documentation: https://docs.unity3d.com/Manual/UnityRemote5.html

Enable Developer Mode and USB debugging (or re-enable it)

On your Android Device, enable Developer Mode

  1. Open Settings
  2. Scroll down to and click About <Device> (Mine says About Tablet)
  3. Scroll down to the Build Number
  4. click Build Number many times (after a few times, a message will pop up telling you how many more times to click)

Now that Developer Mode is active

  1. Go to Settings>Developer Options
  2. Ensure Developer Options is toggled On (at the top of the list)
  3. Optional: Enable Stay awake 
  4. Scroll down, under Debugging enable USB Debugging

Now you device should be ready.

  1. Plug in your Android device to your PC.
  2. Open Unity Remote 5 on your Android Device
  3. Open you Unity project
  4. Click Play

You should see you Unity project on your Android device. If not, try shutting down Unity and Unity Remote and restarting each.

Change Unity’s C# Template Code Style

The first thing I used to do every time I’d edit a new C# script in Unity is delete the comments and unused usings which are all included as part of the template. Going through that activity for every new script was bad enough, but on top of that the code style used in the template is pretty far off from what is considered standard for C#. Eventually, I decided it was time to stop my pre-scripting ritual and just make the default template into what I want. So to begin:

Navigate to the ScriptTemplates folder in your Unity install folder.

Route with a typical Windows install: C:\Program Files\Unity\Editor\Data\Resources\ScriptTemplates

Open 81-C# Script-NewBehaviourScript.cs.txt in your text editor of choice.

From here you can edit the file’s layout however you choose. Mine looks like this:

using UnityEngine;

public class #SCRIPTNAME# : MonoBehaviour 
{
    private void Start() 
    {
        #NOTRIM#
    }
    
    private void Update()
    {
        #NOTRIM#
    }
}

Note: You may be unable to save/overwrite the template file if you did not open your text editor As Administrator. Don’t worry, you can just Save As to your Desktop and drag/drop the new version from the Desktop to the ScriptTemplates directory.

Event Delegation in Unity

What is Event Delegation?

“…a helper object, known as a delegate, is given the responsibility to execute a task for the delegator
– Delegation Pattern Wikipedia

Why use Event Delegation?

A good practice when working with Unity (or any framework) is to decouple your UI logic from your game logic. It’s fine to have the UI code reach down to the game logic layer and call methods directly on it but having the game logic layer directly call methods on the UI logic’s layer is, in general, a bad practice because doing so “marries” you to whatever UI you are currently using. So to get around this we can use C#’s Delegates and Events to fire off methods throughout our UI and even on the game logic layer.

How to use Event Delegation with Unity

All we have to do is:

  • create a static Event Manager class
    • Add delegates and events to this Event Manager
  • Listeners will add functions to the events
  • Delegators will fire off the events

The Manager: Defining events

public class EventManager {
    public delegate void EnemyHovered(int laneIndex, int targetRange);
    public static event EnemyHovered EnemyTargetedRequest;
    public static void EnemyTargeted(int laneIndex, int targetRange) {
        if(EnemyTargetedRequest != null) EnemyTargetedRequest(laneIndex, targetRange);
    }
}

In this example we create a delegate with the type EnemyHovered which takes the arguments laneIndex and targetRange. Then we make an event EnemyTargetRequest which is the event that the listeners will subscribe functions to. Finally, there is a static method that can be called by delegators when they want to fire off this event, triggering all the functions that are subscribed to it.

The Listener: Subscribing to the event

In another class we can add the following:

void Awake() {
    EventManager.EnemyTargetedRequest += DoSomething;
}

public void DoSomething(int targetedIndex, int targetRange){
    // do something
}

In this class we define some function called DoSomething that takes the same arguments as our delegate. In the Awake() method we add the function to the static class’s EnemyTargetedRequest event.

The Delegator: Firing off events

In yet another class we can do the following:

public void OnPointerEnter(PointerEventData eventData) {      
    EventManager.EnemyTargeted(_index, _targetCount);
}

Here we’re just using Unity’s OnPointerEnter event as the UI event that will trigger our EnemyTargeted event. Because our Listener class subscribed it’s DoSomething function to the EnemyTargetRequest event when SignalManager.EnemyTargeted is called it will fire off the DoSomething function!

So with this, you can keep all your events off in the static Event Manager, subscribe to them when you need an object to react to an event and fire off the events whenever you need to which allows you to keep UI and game logic separated. You could even have GameObjects subscribe to events and unsubscribe from them as they are created and destroyed which is very useful when these objects are created and destroyed dynamically.

 

Good Enough Guide to Unity’s Unet Transport Layer (LLAPI)

I was excited to see that just as I was getting ready to start my first networked multiplayer Unity project, Unity was releasing their new Unet networking APIs. “What great timing!”, I thought. However, I was a total noob when it comes to working with sockets and communicating over a network and after playing around with Unet’s HLAPI (High-Level API) it quickly became clear to me that I’d have to dive deeper to get the functionality that I was looking for. So I began to read the documentation for the LLAPI (Low-Level API) which was not pleasant at all. After seeking advice and guidance from everyone I found shake down I managed to get everything working, connecting, and sending messages to and fro.

So here is my tutorial on working with Unity’s Unet LLAPI in C# for networking noobs. There are 4 main parts to any socket networking setup, opening a socket, connecting to another socket, sending messages and receiving messages:

Starting a “Host” (or Opening a Socket)

First make a new Unity project, 2D or 3D, it doesn’t matter for this tutorial. Then add an Empty GameObject to the scene and name it Transport. Then add a new C# script to the object and open the script.

We’re going to setup our socket right as the application begins so let’s begin in the Start() method. First thing we want to do is call initialize on the NetworkTransport class (don’t forget to add using UnityEngine.Networking to the class):

public void Start() {
  NetworkTransport.Init();
...

Next we just follow Unity’s documentation and add the ConnectionConfig. For most situations using the default config is fine, so let’s just go with that:

...
ConnectionConfig config = new ConnectionConfig();
...

Up next we need to add a channel and keep the channelId handy so we can use it later for sending/receiving. To do this we’ll add an int member variable to the class (not just the Start() method). We’ll also use the QosType of Reliable when setting up our channel. You don’t really need to know what this means right now (read: I don’t know enough about what this means and I still survive):

int myReliableChannelId; // above Start()
...
  myReliableChannelId = config.AddChannel(QosType.Reliable); // within Start()
...

This next part pretty much just sets the max number of connections allowed on your soon to exist socket:

...
  int maxConnections = 10;
  HostTopology topology = new HostTopology(config, maxConnections);
...

and finally we open the socket. One of the main stumbling blocks when trying to understand the LLAPI based on Unity’s documentation is that many thing are just named poorly. For starters, there is NetworkTransport.AddHost(). This doesn’t actually add a “Host” like the HLAPI does. This command actually just starts up a socket, I’m pretty sure they should change the name of the method to AddSocket, but who am I?? When we start the socket we have to tell it which port to listen to and in this case we use port 8888. Also the AddHost() method returns an int which is the “HostID” but screw that name we’re going to call it what it is, a SocketId. This SocketId also has to be accessible from other methods in this class so let’s put it up with the channelId:

int socketId; // above Start()
int socketPort = 8888; // Also a class member variable
... // back in Start()  
  socketId = NetworkTransport.AddHost(topology, socketPort);
  Debug.Log("Socket Open. SocketId is: " + socketId);
} // closing curly for Start()

And with that we have our code which sets up a socket! It’ll run right away when you click the Play button in Unity. You should see the debug message appear in the console, probably with the socketId of 0. It’s not that cool yet but we’re getting there.

Connecting to the socket

Once our socket is open we use it to connect to another socket, typically on another device. We can worry about getting another device in the mix later. For now, let’s just write our Connect() method.

int connectionId;
...

public void Connect() {
  byte error;
  connectionId = NetworkTransport.Connect(socketId, "localhost", socketPort, 0, out error);
  Debug.Log("Connected to server. ConnectionId: " + connectionId);
}

We need to keep the connectionId available to all methods because later we’ll use it to send messages. NetworkTransport.Connect() … connects us to another server (Hey! They did a proper job naming that one!). We pass in our socketId, the ip for the remote device (in this case we’re just connecting with ourselves, like a long weekend alone in the woods), the socketPort of the remote machine (same as ours in this case), 0 (I forget what this is but whatever), and if there is a problem we get an error out.

The Unity documentation mentions that we can check the error with:

if (error != kOk) // wat?

But I have no idea what kOk is and judging by all the red squiggles it certainly isn’t a byte. If you know what kOk is please tell someone (preferably me) we need to figure this out people! Basically, I just include the out error parameter because I have to.

I used my Android phone as the remote device but you can use another computer if you have one available. But for now you can do most of your testing by connecting to your own socket (I’m sure there is a rude joke in there).

Lastly, go into Unity and add a UI button and set the OnClick event to fire off our Connect() method. If you don’t want to do that I suppose you can just call Connect() at the end of Start() after our code that adds the socket. That would be good enough for now if you don’t want to make the button.

Sending a Message!

Finally (half of) the thing we actually want to do! Here we turn a string into a stream of bytes then send those bytes out our socket connection.

public void SendSocketMessage() {
  byte error;
  byte[] buffer = new byte[1024];
  Stream stream = new MemoryStream(buffer);
  BinaryFormatter formatter = new BinaryFormatter();
  formatter.Serialize(stream, "HelloServer");

  int bufferSize = 1024;

  NetworkTransport.Send(hostId, connectionId, myReliableChannelId, buffer, bufferSize, out error);
}

Quick run down: Declare another (unused) byte variable called errorCreate byte array called buffer and set it’s length to 1024 (this is the max length of our message when it is in byte form). The next 3 lines do the conversion from string to byte array. I don’t know the details about it, but it works, so it’s good enough. We also send over the size of our buffer byte array as an int (which is bufferSize). Lastly we put all the stuff together, starting with out class variables hostId, connectionIdand myReliableChannelId then adding the rest.

Now that we’ve got our SendSocketMessage() method you can go back into Unity and add a UI button to fire off the method on the buttons OnClick event.

Listening for NetworkEvents (or Receiving a Message)

Last part! We can send a message to other connected devices but now we have to make our application do something when it gets these messages. Otherwise we’re just shouting into the void. We need to always be checking for new messages coming in so this portion is added to the Update() method.

void Update() {
  int recHostId;
  int recConnectionId;
  int recChannelId;
  byte[] recBuffer = new byte[1024];
  int bufferSize = 1024;
  int dataSize;
  byte error;
  NetworkEventType recNetworkEvent = NetworkTransport.Receive(out recHostId, out recConnectionId, out recChannelId, recBuffer, bufferSize, out dataSize, out error);
...

All we’re doing is declaring all the variables we need then setting up a NetworkTransport.Receive() method. Almost all those variables are assigned as a result of a message being received (all the parameters that begin with out). The recBuffer contains the byte array message that was received, while recNetworkEvent is an enum with 4 possible event types:

  • Nothing – When no messages are received.
  • ConnectionEvent – A socket connection is made.
  • DataEvent – A message is received.
  • DisconnectEvent – A device that was connected as told us it is closing the connection.

We’re going to use these event types to decide what to actually do with the messages we receive. So we’re going to continue to add on to our Update() method with a big ol’ switch statement:

...
switch (recNetworkEvent) {
  case NetworkEventType.Nothing:
    break;
  case NetworkEventType.ConnectEvent:
    Debug.Log("incoming connection event received");
    break;
  case NetworkEventType.DataEvent:
    Stream stream = new MemoryStream(recBuffer);
    BinaryFormatter formatter = new BinaryFormatter();
    string message = formatter.Deserialize(stream) as string;
    Debug.Log("incoming message event received: " + message);
    break;
  case NetworkEventType.DisconnectEvent:
    Debug.Log("remote client event disconnected");
    break;
}

So based on the type of network event we have a few different debug messages display.  With the event type Nothing we do… nothing. This message comes in every Update() that a message of some other type is not received so we can ignore it. With ConnectionEvent we just want to be notified of the connection, but in a game you might do something like load a player prefab or display some text that a new player has connected.

DataEvent is the real meat and potatoes. This is where a message is received from a connected device and our application has to do something based on that message. Since our SendSocketMessage() method currently only sends a string we’ll only include code to handle string messages received (it’ll still try to convert any bytes into a string, so if you send something other than a string you’ll get an error or a garbage string (probably? Didn’t try it.)).

You can see that in the DataEvent case we’re doing almost the same thing as we did in the SendSocketMessage() method but instead of making a string into bytes, now we’re making bytes into a string. Once the message is back in a string form we display it as part of a debug message. Bam! Message Received!

Lastly there is the DisconnectEvent type. I don’t do anything with that event type here except log it to the console but you can image using this event as the time to do things like remove a player’s prefab, or display a disconnect message on screen.

Try it out

That should be all you need to get up and running. You have an application that, on startup, makes a socket available for connections. With the press of your Connect button the application uses it’s socket to connect to another device’s socket. Pressing your Send Message button will blast a string message from your socket to the remote devices socket and the remote device will receive the message, unpack it and display it in the console.

I did my testing initially just having my computer running Unity connect to it’s own socket, just as a way to rapidly test things. Because of this I saw the connect message in the console 2x, since one event was made for each side of the connection. After I got everything working like this, I changed the “localhost” to my computer’s IP address then built the project and sent it to my Android phone. I would then run the application in the Unity editor so I could see the console and I used the app on my phone to connect and send the message. It was really rewarding to hit the Send Message button on my phone and see the message appear in Unity’s console on my PC!

Expanding

This post is already super long, but I just want to give some final thoughts and extra tips.

How do I use this to make a game?

You can have another switch statement that executes different methods or series of methods based on the message received. So getting a message like “ChangeToNight” can trigger a method that changes the time in your game environment.

Does the message have to be a string?

No! It can be any object, all you have to do is serialize the object in a similar way to how we serialized the string. For instance I have experimented with sending JSON messages so that I can transport more complex objects. Using JSON .NET for Unity made this super easy.

Where can I read more about sockets?

While trying to figure all of this out I was directed to Beej’s Guide to Network Programming. It’s pretty advanced, written for C++ and I didn’t read much of it, but the What is a socket? section helped clear up some of my confusion about sockets.

I hope this helps some people who want to dive into Unity’s new networking API but don’t know a thing about network programming prior. If you are an experienced network programmer and have noticed things that I am wrong about, or things that could be expanded upon please let me know and I’ll update this guide.

Twitch Streaming Game Development

I’ve recently started streaming some parts of Nano’s development on my Twitch channel at http://www.twitch.tv/munkeyxis and you can follow me there to get notified whenever I stream. There isn’t any scheduled time which I’ll be streaming, just whenever I have the time to work on the game for a bit.

I did a couple streams over the weekend and it went pretty well. Even with my petty number of viewers there was still discussion in the chat about coding, Unity and some aspects of the game itself. I’m going to try and provide some tutorial-ish commentary during my streams along with insight into some of the design choices.

The topics covered while streaming so far:

A map that allows for player movement when map tiles are clicked: http://www.twitch.tv/munkeyxis/b/662025812

Creating a Hearthstone-like Inventory/Deck UI in Unity, Part 1: http://www.twitch.tv/munkeyxis/b/662361477

Each of these topics will have a blog post associated with them, those are coming soon. Next time streaming will be Inventory/Deck UI Part 2!

Events

In Nano, Events are the primary way the player interacts with the game world. Events are also the most direct way the player learns about the environment they are taking part in, meaning Events play double duty by providing game play and story elements. While the story is something that will develop overtime, the functionality of Events is something that the game needs right away. Events are also the feature that I wrestled with the most when trying to come up with a solid development design.

It was clear what Events had to be, they were to display some story text to the player, a few buttons with text on them representing the player’s available choices, and upon selecting a choice a result is shown that also has story text as well as displaying any rewards the player is given. Think of the event system in games like FTL. This type of system is generally referred to as a dialog tree, decision tree, or most specifically a simple directed graph. There are many plugins for Unity that provide varying levels of decision tree functionality, but I wanted to take a stab at rolling my own first.

So far a very simple Event model is in place along with Choices and Results. When a player enters a new location an Event is chosen from the list of possible Events which is then displayed on screen. To manage the Event’s display is the Event UI Controller class which is attached to prefab in Unity. This Event prefab uses Unity’s new UI features to display text and buttons. This is my first attempt at working with this new UI system, before this all objects displaying text used Text Mesh components which I find to be clunky and lacking a lot of features that are needed for displaying even a modest amount of text.

The early Event UI screen.
The early Event UI screen.

There is still a lot to be done to get Events fully functional. The player still needs to be able to select one of the choices and be presented with the result. Events will also be the primary way that combat is triggered. Some choices should only be available to the player if certain conditions are met, like having a particular item equipped. There is also the daunting task of creating a large amount of Events all with multiple choices which in turn have a variety of rewards.

Nano with Unity: Fresh Start

I’ve started to remake Nano using Unity. Nano will no longer be a browser game, instead I plan to release it for PC, tablet and phones. It’s a bit odd for me to say “release it” as if I’ll ever finish it. It would be really cool if I did and that is the plan but you surely know how these kinds of personal projects go.

So far I’ve got some of the basic UI in place, enough for some testing and experimenting. So far this UI is limited to displaying the player’s health points, a few buttons with placeholder actions and an early take on the inventory system.

Next up will be improving the inventory system and adding the Deck feature. Cards in the player’s inventory can be added to their deck, which is used in Nano’s combat system. Though, before combat can take place Enemies will be need to be added.

So that’s the game plan. I’m excited to be working on this project again, here’s to the hope that it’ll go further and be better than the browser version. It wouldn’t take much to be better than that ol’ thing.

Lil Critters – Class structure and the Nervous System

I’ve started working on the Lil Critters “Evolution Sim” project again. First thing to do was some refactoring, so I made many more classes and split up the methods. The classes and structure of the project is kind of interesting, since I am loosely modeling it after how the nervous system in living creatures tend to work. I’m no expert in biology, neurology or anything like that so I’m taking a more broad and general approach to the nervous system.

A critter’s “nervous system” is made up of three main classes which are Stimulus, Brain, Response. As you can probably guess, Stimulus is in charge of receiving input from the world outside of the critter and relaying those inputs to the Brain. The Brain chooses a response based on the input and sends this choice to Response. Lastly, Response allows the critter to perform some action as dictated by the Brain. There are other classes each critter has as well such as Raycast (to draw sight line) and Metabolism (to manage energy consumption). There will also be plenty more added as the project grows. The way I’ve been thinking of these classes in more specific terms is:
BasicCritterClasses
Raycast: Photons that hit the eye.

Stimulus: The eye that forms information from the photons that hit it and the neurons that transport that information to the brain.

Brain: Takes care of brain stuff… makes choices to call for particular responses based on input.

Response: The muscle system, applying forces to the critter when instructed to by the brain.

Metabolism: The chemical reactions and burning of energy in order to power the muscles and also just to maintain life.

It’s almost soothing to organize the series of inputs, analysis and outputs like this. This project often makes me think about real life a bit differently. While we all get the general idea that out bodies and minds are set up in this way, we typically don’t give this idea any weight or importance. I find it interesting to be building these little critters with this structure as a guide, which developed over billions of years of evolution.