IMG_0228

Arduino, Unity and VR

January 17, 2015 devblog, Hardware, IgnisVR, Unity no comments

This will be the first in a series of blog posts about connecting electronics to interact with the real world from within Virtual Reality.
Virtual Reality is all about immersion. “Being there” is what distinguishes Virtual Reality from a flat screen conventional experience. Sometimes it might add value to an experience, to not only being able to have visual feedback of your new world.

You might also want to have tactile, audible and other sensory feedback. And for some experiences you may want to add other forms of input. Like riding and steering a virtual bike, dynamic heartbeat or brain driven environments or other custom build/printed devices to influence the new world you’re in.

These examples all require you to have some kind of electronics device to measure, trigger or control the physical real world and an interface with your computer to communicate with your Virtual Reality source-code.

Back in the days, because of its custom nature, it was only for the happy few electronic engineers to have enough knowledge to create such sensor reading and/or actuator triggering contraption. And even if they build one to completion, it would still cost a lot of money and time to develop the device. With little to no chance it would ever leave the room, to be used by the creative minds in the world to create something awesome with it.

Fortunately times have changed! Microcontrollers enabled programmable circuits and prototyping hardware kits evolved (PIC, Basic stamp). Making it easier and cheaper for everyone to start experimenting with electronics. In 2005 the Arduino was introduced to the world. It’s the cheapest, open source and easy to use micro processor prototyping kit around. It was definitely our first weapon of choice when thinking of hardware prototyping for virtual reality.

Arduino Uno rev. 3

It’s beyond the scope of this blog to introduce you with the basics of Arduino. I would recommend picking up the book Arduino for Dummies by John Nussey and browse to arduino.cc to get you going.

In this first blog I will try to explain how to get data in real-time from a Unity powered application to an LCD display using the Arduino. This setup will cover the concept of controlling external devices from inside your Unity developed VR world. For this I’m using Unity 4.6 and the Arduino Uno rev. 3.

For the hardware part, the base will be a slightly modified version of the “Hello world” example.

Arduino LCD hello world example

The only modifications I did were:
– hooking up the backlight to pin 13
– ignoring the pot resistor (Because I didn’t have any. But you can replace it with a fixed value resistor to level the contrast)
– changing the source-code to read and display the external data

The following sourcecode should be uploaded to the Arduino:

// Use the default LCD Arduino library
#include <LiquidCrystal.h>
// Setup the pins used
LiquidCrystal lcd(12, 11, 5, 4, 3, 2);
int backLight = 13;

void setup() {
  // Start reading the serial port at 9600 baud
  Serial.begin(9600);

  // Set pin 13 as output
  pinMode(backLight, OUTPUT);

  // turn backlight on. You can replace 'HIGH' with 'LOW' to turn it off.
  digitalWrite(backLight, HIGH); 
  // set up the LCD (16 characters in 2 rows) 
  lcd.begin(16, 2);
  // Print a message to the LCD.
  lcd.print("It works ! :)");
}

// Create a buffer to store our incoming data
String buffer = "";

void loop() { 
  // While our serial port buffer has any data in it, process it
  while (Serial.available() > 0) 
  {
    // Get and flush the first character in the queue     
    char c = Serial.read();

    // When a zero termination character arrives it should display the 
    // entire string that's inside the buffer. This avoids having to 
    // wait and guess when the serial port is done receiving.
    
    // If the incoming character is not our zero terminator: 
    // add the character to the buffer.
    if (c != '\0')
      buffer.concat(c);  
    else 
    {
      // Zero termination character found:
      // Clear all contents
      lcd.clear();
      // Display a label
      lcd.setCursor(0, 0);
      lcd.print("Coordinates:");
      // put the cursor on the LCD at the first character of the second line
      lcd.setCursor(0, 1);
      // display the entire buffer to the display
      lcd.print(buffer); 
      // clear the buffer
      buffer = "";  
    } 
  } 
}

When you are done uploading, the LCD should now display “It works ! :)” on its display.
If it’s not showing anything, check your cables and pin numbers.

Now for the Unity part.
Start a new Unity project. In the player settings make sure you use the full .NET 2.0 API (this includes the serial port assemblies) and not the default Unity .NET 2.0 subset.

Unity .NET 2.0 API

Create a new C# script and call it “ArduinoLCDTest”.

using UnityEngine;
using System.Collections;
using System.IO.Ports;
using System;

public class ArduinoLCDTest : MonoBehaviour 
{
	private SerialPort port;

	// initialization
	void Start()
	{ 
		// Our Arduino is on COM 5, at 9600 baud
		port = new SerialPort("COM5", 9600);
		// Check if port is not already in use
		if (!port.IsOpen)
		{
			try
			{ 
				// Open the port
				port.Open();
				// To avoid flooding our serial port, we use a coroutine with a delay.
				StartCoroutine(SendData());
			}
			catch(UnityException e)
			{
				// Basic exception handling
				Debug.LogError(e.Message);
			}
		}
		else 
			Debug.LogError("Port already open.");

	}

	IEnumerator SendData()
	{
		// Send data as long as the port is open.
		while (port.IsOpen)
		{
			// Write the coordinates of the game object to the serial port.
			// We round the values to fit the 16 character display and 
			// use a null terminated string to mark the ending.
			port.Write(String.Format("{0},{1},{2}\0",
			                         Mathf.RoundToInt(this.transform.position.x),
			                         Mathf.RoundToInt(this.transform.position.y),
			                         Mathf.RoundToInt(this.transform.position.z))
			           ); 
			// Wait for half a second before the next update is send.
			yield return new WaitForSeconds(0.5f);
		}
	} 

	void OnDestroy()
	{ 
		// Close the port when the program ends.
		if (port.IsOpen)
		{
			try
			{ 
				port.Close();
			}
			catch(UnityException e)
			{
				Debug.LogError(e.Message);
			}
		}
	}
}

Drag the ArduinoLCDTest class onto a game-object like the camera and run the scene. Now the LCD should display the label Coordinates with the rounded x,y,z coordinates of the camera. While running, drag the camera around in the editor to see the values on the LCD change.

Arduino Unity coordinates

You might experience some flickering when updating the screen. To reduce code complexity I didn’t implement this enhancement. It can be avoided very easily in a couple of ways by expanding the Arduino or Unity source-code:
– Only update the value when it’s changed.
– Update only the second line when a value changes.
– Avoid calling clear and fill the buffer with empty characters.

That’s it for today!

About the author

Virtual Reality enthusiast born in 1977 in a city called Breda (The Netherlands). Husband and father of two sons. Started programming in 1988. Experience with programming, music production, video production, 3d graphics, 2d design, web development, photography, electronics and hardware.

Join the discussion

Your email address will not be published. Required fields are marked *