Vuzix

• Introduction
• How to get started
• .NET Integration
• .NET integration: Vuzix on MAUI
• .NET MAUI Integration: First contact
• .NET MAUI Integration: Images & Image Size exploration
• .NET MAUI - Handling taps
• .NET MAUI - reporting events to the GUI
• .NET MAUI - reading glass status
• LVGL Image
• How it fits in the Vuzix stack
• .NET: Refactoring / Dependency Injection

Introduction

How to get started

The easiest way to get started has a few easy steps if you are familiar with Android Studio or building Android applications.

If you haven't gone through it before, it might not always be so clear. So this might help you to get through it in a couple of steps and help you get started.

 

Install the Vuzix SDK

1. Install the Vuzix Connect application
2. Pair the Z100 glasses

Setup your development environment

1. Install Android Studio
   1. Probably you'll need to put your Android phone into Developer mode
   2. And install the USB driver
2. Download the sample code
3. Start the sample code with Android Studio

 

.NET Integration

.NET integration: Vuzix on MAUI

What we'll try to achieve

We'll build an integration in MAUI that will hide all the complexity doing interfacing work on Android - and perhaps in a later stage on IOS as well.

Microsoft MAUI is a way to build code in C# / .NET and run it on several devices. And builds upon Microsoft Blazor - the coding experience is the same. The resulting application is either a web application or service vs a Mobile application.

This means, you can build "a desktop app that runs on Android or IPhone IOS" without any change in the code. This works with running HTML code with asyncronous calls to an interpreter that runs on either system. All this complexity is handled for you - so either you code in your usual .NET environment or write some JavaScript that will "drive the device in the background".

Lets get started!

Set up the project

1. Let's assume you've installed Visual Studio and checked the "mobile development" option (or have added this featureset
   
   
   
2. Create a new MAUI Blazor Hybrid  project
   

   

   
   
   
   
3. Click through the options and you'll see the folder structure.
   The folder "Platforms" contains code specific for the target system, in this guide, we'll only focus on Android for simplicity.
   
   

   
   

   
   
4. So lets right-click the project, to edit the project file
   

   
   

   
   
5. Remove the targets we don't use right now
   

   

   
   
6. We also need to set the minimum version to 30 for Android
   

   

   
   

Add the Nuget package

It is possible to create this project yourself - but I have taken the liberty to make a nuget package for it. (the generator requires some manual editting that is cumbersome, so a package will be easier)

1. Open nuget package manager
   
   
2. Find the nuget package and add it to the project
   

   

   
   

Application first run

If everything is setup properly, you should be able to select your target to deploy (or emulate) the application on.
If you've installed the USB drivers with Android studio, you should be able to put your Android phone in Debug-mode and run the application on your physical real phone.

You should now see your emulator running with your MAUI application that is set up to work with Vuzix Z100!

Continue to the next page, to start contacting your glasses.

.NET MAUI Integration: First contact

Introduction

On this page, it's assumed you have a MAUI project with the nuget package ready to go.

In the end,  you will be able to make a simple HTML website that receives text and it will be shown in the Z100 AR glasses.

If you don't want to spend time for going step by step, here are the source files:

Excercise1.zip

ExerciseFinal.zip

Background

Don't be overwhelmed: you don't need to study this, but it will help  you to conceptualize where which code will be run. And what you can access or in what context you are operating.

MAUI is a front that runs in a HTML engine with JQuery and Bootstrap which make it easy to make interactive web interfaces.
It then communicates to the actual back-end of the application via asynchronous calls - via their SignalR engine.

This helps to write "classic server code" but having the fluid easy GUI that web applications have.

In the back on the "server code" - there is a translation layer towards the target host technology. In this example, this will be Android.

We just have to somehow translate a mouse click, to go to the server code. And from the server code, towards the Android device.

For this reason, we're using the   CommunityToolkit.Mvvm.Messaging package, which allows to "listen" and "trigger" events that can be picked up via event handlers. We do not have to concern who triggered it, or where it was triggered. Via the magic of dependency injection, this is all handled for us.

Lets get to it!

Excercise1.zip

1. First, lets get the   CommunityToolkit.Mvvm.Messaging package into the solution.

2. Now we can write a simple code in the Home.razor page to take text.

This example connects the value of the variable @message into the textbox.
When the textbox is clicked, the function SendMessage() is triggered.
Because of the nature of Blazor - the page is refreshed. At the refresh, there is now a value. And your value is displayed.

@page "/"

@code{
    public string MyMessage;
    void SendMessage()
    {
        // Send the message to ANdroid ???
    }
}

<h1>Hello, world!</h1>
@if (MyMessage != null && MyMessage.Length>0 )
{
<p>You wrote: @MyMessage</p>
}

<input type="text" @bind="@MyMessage" />
<button class="btn btn-primary" @onclick="SendMessage">Send Message</button>

Run your emulator to try it out and see everything is working okay:

If you'd like you can set a breakpoint to inspect your variable as well, by clicking the left bar to set a red dot. And running your example. While hovering over your variable.

3. And now the real work (in 5 minutes) - Set up the GUI to send the message.

In this step we will send messages from the GUI towards Android. Know, that we can do the reverse, and send from Android towards the GUI if desired. But in our case, Android will forward the message to our glasses!

Add using directives, so the page knows where to find the nuget classes

@page "/"
@using CommunityToolkit.Mvvm.Messaging;
@using VuzixSDK.Class
@using VuzixSDK.Enum

And add the following 2 functions to your code.

These functions will use the Mvvm.Messaging WeakReferenceMessanger and send a class of UltraLiteMessage in one function, based on the string.

And an UltraLiteError in the second function. So we can display errors to the user.

    protected async static void UltraLiteMessage(String Message)
    {
        WeakReferenceMessenger.Default.Send(new UltraLiteMessage()
            {
                Data = Message
            });
    }

    protected async static void UltraLiteError(Exception Excpetion)
    {
        WeakReferenceMessenger.Default.Send(new UltraLiteError()
            {
                Source = "Counter",
                Exception = Excpetion
            });
    }

And update your SendMessage function to trigger the UltraLiteMessage function.

    void SendMessage()
    {
        // Send the message to ANdroid ???
        UltraLiteMessage(MyMessage);
    }

Your full home.razor page will look as such:

@page "/"
@using CommunityToolkit.Mvvm.Messaging;
@using VuzixSDK.Class
@using VuzixSDK.Enum


@code{
    public string MyMessage;
    void SendMessage()
    {
        // Send the message to ANdroid ???
        UltraLiteMessage(MyMessage);
    }

    protected async static void UltraLiteMessage(String Message)
    {
        WeakReferenceMessenger.Default.Send(new UltraLiteMessage()
            {
                Data = Message
            });
    }

    protected async static void UltraLiteError(Exception Excpetion)
    {
        WeakReferenceMessenger.Default.Send(new UltraLiteError()
            {
                Source = "Counter",
                Exception = Excpetion
            });
    }

}

<h1>Hello, world!</h1>
@if (MyMessage != null && MyMessage.Length>0 )
{
<p>You wrote: @MyMessage</p>
}

<input type="text" @bind="@MyMessage" />
<button class="btn btn-primary" @onclick="SendMessage">Send Message</button>

That's it for your personal code.

What is left, is to receive your text and show it on the glasses.

4. Receive the text in Android

Locate the file MainActivity.cs and add the using statement towards the Mvvm.Messaging library.

And create the following functions:

• OnCreate is an entrypoint in Android and will be run at the start of the application.
• We then register a listener for the messages that the front end is sending to us: UltraLiteError and UltraLiteMessage by the WeakReferenceMessenger declaration.
• showMessage will show a basic Android notification on the screen, it's always convenient to show errors if something goes wrong

In essense; when the application is created, when a message with object of class UltraLiteError is received, it will end up in the processUltraLiteError function.

When the application receives a message with object of class UltraLiteMessage, it will end up in the processUltraLiteMessage function.

        protected override void OnCreate(Bundle savedInstanceState)
        {
            base.OnCreate(savedInstanceState);
            try
            {
                WeakReferenceMessenger.Default.Register<UltraLiteError>(this, (sender, e) => { processUltraLiteError(e); });
                WeakReferenceMessenger.Default.Register<UltraLiteMessage>(this, (sender, e) => { processUltraLiteMessage(e.Data); });
            }
            catch (System.Exception ex)
            {
                showMessage(ex.Message);
            }
        }
        public void showMessage(string message)
        {
            MainThread.BeginInvokeOnMainThread(() =>
            {
                var toast = Toast.MakeText(this, message, ToastLength.Short);
                toast.Show();
            });
        }
        protected void processUltraLiteError(UltraLiteError error)
        {
        }
        protected void processUltraLiteMessage(String message)
        {
        }

At this point you can run or compile the code, but lets now connect the glasses.

Integrating the VUZIX SDK

ExerciseFinal.zip

First add using statements for the Vuzix SDK

using VuzixSDK.Class;
using Com.Vuzix.Ultralite;

Then, declare a variable _sdk, which will hold a reference to the Vuzix SDK

   IUltraliteSDK _sdk;

In the OnCreate function, asssign the variable

 _sdk = Com.Vuzix.Ultralite.IUltraliteSDK.Get(this);

And update our functions, to display the text on the glasses:

        protected void processUltraLiteError(UltraLiteError error)
        {
            if (_sdk.IsConnected)
            {
                if (!_sdk.IsControlledByMe)
                {
                    _sdk.RequestControl();
                }
                if (_sdk.IsControlledByMe)
                {
                    string _title = $"[Error]{(error.Source != null ? " " + error.Source : "")}";
                    string _error = (error.Exception != null ? $"Exception : {error.Exception.Message}" : "Error occured");
                    _sdk.SendNotification(_title, _error);
                }
            }
        }

        protected void processUltraLiteMessage(String message)
        {
            if (_sdk.IsConnected)
            {
                if (!_sdk.IsControlledByMe)
                {
                    _sdk.RequestControl();
                }
                if (_sdk.IsControlledByMe)
                {
                    _sdk.SetLayout(Layout.Canvas, 0, true);
                    int textId = _sdk.Canvas.CreateText(message, TextAlignment.Auto, UltraliteColor.White, Anchor.TopCenter, 0, 0, 640, -1, TextWrapMode.Wrap, true);
                    if (textId == -1)
                    {
                        showMessage("Text failed");
                    }
                    _sdk.Canvas.Commit();
                }
            }
        }

Your full MainActivity.cs will look like this:

using Android.App;
using Android.Content.PM;
using Android.OS;
using Android.Widget;
using CommunityToolkit.Mvvm.Messaging;
using VuzixSDK.Class;
using Com.Vuzix.Ultralite;
using Layout = Com.Vuzix.Ultralite.Layout;
using TextAlignment = Com.Vuzix.Ultralite.TextAlignment;

namespace MauiApp1
{
    [Activity(Theme = "@style/Maui.SplashTheme", MainLauncher = true, ConfigurationChanges = ConfigChanges.ScreenSize | ConfigChanges.Orientation | ConfigChanges.UiMode | ConfigChanges.ScreenLayout | ConfigChanges.SmallestScreenSize | ConfigChanges.Density)]
    public class MainActivity : MauiAppCompatActivity
    {
        IUltraliteSDK _sdk;
        protected override void OnCreate(Bundle savedInstanceState)
        {
            base.OnCreate(savedInstanceState);
            try
            {
                WeakReferenceMessenger.Default.Register<UltraLiteError>(this, (sender, e) => { processUltraLiteError(e); });
                WeakReferenceMessenger.Default.Register<UltraLiteMessage>(this, (sender, e) => { processUltraLiteMessage(e.Data); });
                _sdk = Com.Vuzix.Ultralite.IUltraliteSDK.Get(this);
            }
            catch (System.Exception ex)
            {
                showMessage(ex.Message);
            }
        }
        public void showMessage(string message)
        {
            MainThread.BeginInvokeOnMainThread(() =>
            {
                var toast = Toast.MakeText(this, message, ToastLength.Short);
                toast.Show();
            });
        }
        protected void processUltraLiteError(UltraLiteError error)
        {
            if (_sdk.IsConnected)
            {
                if (!_sdk.IsControlledByMe)
                {
                    _sdk.RequestControl();
                }
                if (_sdk.IsControlledByMe)
                {
                    string _title = $"[Error]{(error.Source != null ? " " + error.Source : "")}";
                    string _error = (error.Exception != null ? $"Exception : {error.Exception.Message}" : "Error occured");
                    _sdk.SendNotification(_title, _error);
                }
            }
        }

        protected void processUltraLiteMessage(String message)
        {
            if (_sdk.IsConnected)
            {
                if (!_sdk.IsControlledByMe)
                {
                    _sdk.RequestControl();
                }
                if (_sdk.IsControlledByMe)
                {
                    _sdk.SetLayout(Layout.Canvas, 0, true);
                    int textId = _sdk.Canvas.CreateText(message, TextAlignment.Auto, UltraliteColor.White, Anchor.TopCenter, 0, 0, 640, -1, TextWrapMode.Wrap, true);
                    if (textId == -1)
                    {
                        showMessage("Text failed");
                    }
                    _sdk.Canvas.Commit();
                }
            }
        }
    }
}

When you run the code on your Android phone, which has the Vuzix Connect app installed, the text from the input field will appear on the glasses when you click the button.

In the next chapter, we will update this code to send images from the GUI into the glasses.

.NET MAUI Integration: Images & Image Size exploration

Introduction

At the end of this page, you will generate an image and display it in the glasses.

We continue from the previous page. With this resulting project: ExerciseFinal.zip

And will end up with this project: ExerciseSendImages.zip - which will allow to specify the dimensions and generate an image with text that shows the dimensions. This image is then sent to the Z100 AR glasses and displayed.

The SDK allows us to send a "bitmap" and it will be displayed.
However, I haven't gotten such results with it as I wanted: at first I was a bit lost including and referencing files and resources in Android, from the MAUI application.

After I got that figured out, the images were either not displaying, with a strange aspect ratio either or image was scaled down and I couldn't really figure out if it was .NET or Android or the SDK that is doing the scaling. After reading up on "pixel density" and other complicated tutorials I have gone the more simple route.

To find the way to find the "most ideal image size" I generated the bitmap myself. I believe this guide is a good enough base to allow you to draw anything on the glasses and gain full control to its potential.


So, let's get to it!

Approach

We need to send a bitmap - which is an image - in binary form to the Android application. But we do not want to "depend on any framework" to do the scaling.

As we are in the GUI working with HTML5 - we have access to a canvas-element. Which is basically, a sort of image that can be built via JavaScript in the browser.

This way, we can easily try some things out in a local text-editor and browser to see if it is generated well. Before we get into a heavier compilation and deployment - which loses a lot of time (and motivation).

I will first step you through the preparation of the code, to be able to render a byte[] containing a bitmap. And then, we'll write a piece of code to generate an image and send it to the glasses.

Project adjustement: boiler plate

We first have to add some Android code, to listen to the new UltraLiteOperationRequest event, convert the byte to an image format the glasses can work with. And then send it to the glasses.

We start again in  MainActivity.cs

1. Add 3 functions to translate a List<byte[]> and a byte[] to LVGImage which Vuzix works with

private static Bitmap loadBitmap(byte[] bitmapbytes)
{
    BitmapFactory.Options options = new BitmapFactory.Options();

    // https://proandroiddev.com/image-decoding-bitmaps-android-c039790ee07e
    options.InSampleSize = 2;
    options.InPreferredConfig = Bitmap.Config.Argb8888;
    Bitmap bmp = BitmapFactory.DecodeByteArray(bitmapbytes, 0, bitmapbytes.Length, options);

    return bmp;// resize(bmp, 640, 480);
}
private static LVGLImage[] loadLVGLImage(List<byte[]> images)
{
    List<LVGLImage> _images = new List<LVGLImage>();
    foreach(var image in images)
    {
        _images.Add(LVGLImage.FromBitmap(loadBitmap(image), LVGLImage.CfIndexed1Bit));
    }
    return _images.ToArray();
}
private static LVGLImage loadLVGLImage(byte[] image)
{
    //ColorObject[] _colors = { LVGLImage.IColorMapper.White, LVGLImage.IColorMapper.Mid };
    //LVGLImage _img = new LVGLImage(LVGLImage.CfIndexed1Bit, 480, 640, _colors, image);
    LVGLImage _img2 =  LVGLImage.FromBitmap(loadBitmap(image), LVGLImage.CfIndexed1Bit);
    return _img2;
}

2. Add the function processUltraLiteOperation  that will process the incoming message

protected void processUltraLiteOperation(UltraLiteOperationRequest Request)
{
    if (_sdk.IsConnected)
    {
        if (!_sdk.IsControlledByMe)
        {
            _sdk.RequestControl();
        }
        if (_sdk.IsControlledByMe)
        {
            _sdk.SetLayout(Layout.Canvas, 0, true);
            if (Request.Operation == eUltraLiteOperation.ShowImage && Request.ImageBitMap != null)
            {
                LVGLImage image = loadLVGLImage(Request.ImageBitMap);                        
                int _imageID = _sdk.Canvas.CreateImage(image, Anchor.Center);
                if (_imageID == -1)
                {
                    showMessage("Image failed");
                }
                _sdk.Canvas.Commit();
            }
        }
    }
    else
    {
        showMessage("SDK is not connected");
    }         
}

3. Add, in the OnCreate function, the event listener for the message

        protected override void OnCreate(Bundle savedInstanceState)
        {
            base.OnCreate(savedInstanceState);
            try
            {
                WeakReferenceMessenger.Default.Register<UltraLiteError>(this, (sender, e) => { processUltraLiteError(e); });
                WeakReferenceMessenger.Default.Register<UltraLiteMessage>(this, (sender, e) => { processUltraLiteMessage(e.Data); });
                _sdk = Com.Vuzix.Ultralite.IUltraliteSDK.Get(this);
            }
            catch (System.Exception ex)
            {
                showMessage(ex.Message);
            }
        }

Your full Mainactivity.cs should look like this now:

using Android.App;
using Android.Content.PM;
using Android.OS;
using Android.Widget;
using CommunityToolkit.Mvvm.Messaging;
using VuzixSDK.Class;
using Com.Vuzix.Ultralite;
using Layout = Com.Vuzix.Ultralite.Layout;
using TextAlignment = Com.Vuzix.Ultralite.TextAlignment;
using VuzixSDK.Enum;
using Android.Graphics;

namespace MauiApp1
{
    [Activity(Theme = "@style/Maui.SplashTheme", MainLauncher = true, ConfigurationChanges = ConfigChanges.ScreenSize | ConfigChanges.Orientation | ConfigChanges.UiMode | ConfigChanges.ScreenLayout | ConfigChanges.SmallestScreenSize | ConfigChanges.Density)]
    public class MainActivity : MauiAppCompatActivity
    {
        IUltraliteSDK _sdk;
        protected override void OnCreate(Bundle savedInstanceState)
        {
            base.OnCreate(savedInstanceState);
            try
            {
                WeakReferenceMessenger.Default.Register<UltraLiteError>(this, (sender, e) => { processUltraLiteError(e); });
                WeakReferenceMessenger.Default.Register<UltraLiteMessage>(this, (sender, e) => { processUltraLiteMessage(e.Data); });
                WeakReferenceMessenger.Default.Register<UltraLiteOperationRequest>(this, (sender, e) => { processUltraLiteOperation(e); });

                _sdk = Com.Vuzix.Ultralite.IUltraliteSDK.Get(this);
            }
            catch (System.Exception ex)
            {
                showMessage(ex.Message);
            }
        }
        public void showMessage(string message)
        {
            MainThread.BeginInvokeOnMainThread(() =>
            {
                var toast = Toast.MakeText(this, message, ToastLength.Short);
                toast.Show();
            });
        }
        protected void processUltraLiteError(UltraLiteError error)
        {
            if (_sdk.IsConnected)
            {
                if (!_sdk.IsControlledByMe)
                {
                    _sdk.RequestControl();
                }
                if (_sdk.IsControlledByMe)
                {
                    string _title = $"[Error]{(error.Source != null ? " " + error.Source : "")}";
                    string _error = (error.Exception != null ? $"Exception : {error.Exception.Message}" : "Error occured");
                    _sdk.SendNotification(_title, _error);
                }   
            }
        }
        protected void processUltraLiteOperation(UltraLiteOperationRequest Request)
        {
            if (_sdk.IsConnected)
            {
                if (!_sdk.IsControlledByMe)
                {
                    _sdk.RequestControl();
                }
                if (_sdk.IsControlledByMe)
                {
                    _sdk.SetLayout(Layout.Canvas, 0, true);
                    if (Request.Operation == eUltraLiteOperation.ShowImage && Request.ImageBitMap != null)
                    {
                        LVGLImage image = loadLVGLImage(Request.ImageBitMap);
                        int _imageID = _sdk.Canvas.CreateImage(image, Anchor.Center);
                        if (_imageID == -1)
                        {
                            showMessage("Image failed");
                        }
                        _sdk.Canvas.Commit();
                    }
                }
            }
            else
            {
                showMessage("SDK is not connected");
            }
        }
        private static Bitmap loadBitmap(byte[] bitmapbytes)
        {
            BitmapFactory.Options options = new BitmapFactory.Options();

            // https://proandroiddev.com/image-decoding-bitmaps-android-c039790ee07e
            options.InSampleSize = 2;
            options.InPreferredConfig = Bitmap.Config.Argb8888;
            Bitmap bmp = BitmapFactory.DecodeByteArray(bitmapbytes, 0, bitmapbytes.Length, options);
            return bmp;// resize(bmp, 640, 480);
        }
        private static LVGLImage[] loadLVGLImage(List<byte[]> images)
        {
            List<LVGLImage> _images = new List<LVGLImage>();
            foreach (var image in images)
            {
                _images.Add(LVGLImage.FromBitmap(loadBitmap(image), LVGLImage.CfIndexed1Bit));
            }
            return _images.ToArray();
        }
        private static LVGLImage loadLVGLImage(byte[] image)
        {
            //ColorObject[] _colors = { LVGLImage.IColorMapper.White, LVGLImage.IColorMapper.Mid };
            //LVGLImage _img = new LVGLImage(LVGLImage.CfIndexed1Bit, 480, 640, _colors, image);
            LVGLImage _img2 = LVGLImage.FromBitmap(loadBitmap(image), LVGLImage.CfIndexed1Bit);
            return _img2;
        }

        protected void processUltraLiteMessage(String message)
        {
            if (_sdk.IsConnected)
            {
                if (!_sdk.IsControlledByMe)
                {
                    _sdk.RequestControl();
                }
                if (_sdk.IsControlledByMe)
                {
                    _sdk.SetLayout(Layout.Canvas, 0, true);
                    int textId = _sdk.Canvas.CreateText(message, TextAlignment.Auto, UltraliteColor.White, Anchor.TopCenter, 0, 0, 640, -1, TextWrapMode.Wrap, true);
                    if (textId == -1)
                    {
                        showMessage("Text failed");
                    }
                    _sdk.Canvas.Commit();
                }
            }
        }
    }
}

The MAUI code to send the image

We'll go back to the Home.razor page and  connect the GUI with the Android code

1. Add the function to send the UltraLiteOperationRequest containing a byte[] to the Android back end

        protected async static void UltraLiteOperation(eUltraLiteOperation operation,  byte[] BitMap = null)
        {
            WeakReferenceMessenger.Default.Send(new UltraLiteOperationRequest()
                {
                    Operation = operation,
                    ImageBitMap = BitMap

                });
        }

Add the folllowing function in a code block in the Home.razor page, this will expose the function to the script that we will write to trigger

    public static void SendGeneratedBMP(string response)
    {
        try
        {
            UltraLiteOperation(eUltraLiteOperation.ShowImage, Convert.FromBase64String(response));
        }
        catch (System.Exception ex)
        {
            UltraLiteError(ex);
        }
    }

You are all set for the backend! But, what will you send over the line ?

CANVAS to BITMAP

Let's take a step back and rephrase what we want to achieve: "I want to define a size for an image. And then send it to the Z100"

1. Write the Javascript code to handle the GUI

This is where it gets confusing for some: we're now in the context of the browser or the GUI - which can't just access the "machine code". We need to make use of the DotNet functions to pass data from the front to the back. While the drawing and clicking is happening on the front end.

If it sounds confusing, just take the code an play with it.

Open the index.html to add some javascript - this isn't the most "clean and proper way", but this will get it working for you while you write the gold standard code:

The first will get the element you specified and resize it to your specified width and height.

The other, will use standard JavaScript functions to parse the displayed graphic into a Base64 string (which is a text representation of a byte) - and will send it to the backend by this DotNet.invokeMethodAsync (you might need to replace the projectname MauiApp1 with your own for it to work)

<script type="text/javascript>
    function resizeCanvas(canvasID, width, height) {
        var _canvas = document.getElementById(canvasID);
        _canvas.width = width;
        _canvas.height = height;
        var ctx = _canvas.getContext("2d");

        ctx.font = (_canvas.width / 5) + "px Arial";

        ctx.fillStyle = "black";
        ctx.fillRect(0, 0, _canvas.width, _canvas.height);


        ctx.fillStyle = "white";
        ctx.fillText(_canvas.width + " x " + _canvas.height, 0, _canvas.height / 2);

    }

    function sendCanvas(canvasID) {

        var _canvas = document.getElementById(canvasID);
        var _data = _canvas.toDataURL("image/jpeg").split(';base64,')[1];
        DotNet.invokeMethodAsync("MauiApp1", "SendGeneratedBMP", _data);
    }
</script>

2. Adapt the Home.razor page to call your methods

<input type="text" @bind="@MyMessage" />
<button class="btn btn-primary" @onclick="SendMessage">Send Message</button>
<br />
<p>Here you can discoer the allowed width/height that is possible</p>
<p>It seems the most optimal is 560x560. 640x490 is also possible. But it seems scaled down, as 460x640 is the limit in the opposite direction</p>
<p>560x560 seems really the optimal.</p>
<input type="number" id="bmpWidth" maxlength="3" value="120" max="640" /> W x
<input type="number" maxlength="3" id="bmpHeight" value="120" max="640" /> H
<br />
<button class="btn btn-primary" onclick="resizeCanvas('myCanvas', document.getElementById('bmpWidth').value,document.getElementById('bmpHeight').value)">Resize Canvas</button>
    <br />
<button class="btn btn-primary" onclick="sendCanvas('myCanvas')">Send generated BMP</button>
<br />
<canvas id="myCanvas" width="560" height="560"</canvas>

Your full Home.razor would look like this

@page "/"
@using CommunityToolkit.Mvvm.Messaging;
@using VuzixSDK.Class
@using VuzixSDK.Enum


@code{
    public string MyMessage;
    void SendMessage()
    {
        // Send the message to ANdroid ???
        UltraLiteMessage(MyMessage);
    }

    protected async static void UltraLiteMessage(String Message)
    {
        WeakReferenceMessenger.Default.Send(new UltraLiteMessage()
            {
                Data = Message
            });
    }   

    protected async static void UltraLiteError(Exception Excpetion)
    {
        WeakReferenceMessenger.Default.Send(new UltraLiteError()
            {
                Source = "Counter",
                Exception = Excpetion
            });
    }
    [JSInvokable("SendGeneratedBMP")] // This is required in order to JS be able to execute it
    public static void SendGeneratedBMP(string response)
    {
        try
        {
            UltraLiteOperation(eUltraLiteOperation.ShowImage, Convert.FromBase64String(response));
        }
        catch (System.Exception ex)
        {
            UltraLiteError(ex);
        }
    }
    protected async static void UltraLiteOperation(eUltraLiteOperation operation, byte[] BitMap = null)
    {
        WeakReferenceMessenger.Default.Send(new UltraLiteOperationRequest()
            {
                Operation = operation,
                ImageBitMap = BitMap
            });
    }

}

<h1>Hello, world!</h1>
@if (MyMessage != null && MyMessage.Length>0 )
{
<p>You wrote: @MyMessage</p>
}

<input type="text" @bind="@MyMessage" />
<button class="btn btn-primary" @onclick="SendMessage">Send Message</button>
<br />
<p>Here you can discoer the allowed width/height that is possible</p>
<p>It seems the most optimal is 560x560. 640x490 is also possible. But it seems scaled down, as 460x640 is the limit in the opposite direction</p>
<p>560x560 seems really the optimal.</p>
<input type="number" id="bmpWidth" maxlength="3" value="120" max="640" /> W x
<input type="number" maxlength="3" id="bmpHeight" value="120" max="640" /> H
<br />
<button class="btn btn-primary" onclick="resizeCanvas('myCanvas', document.getElementById('bmpWidth').value,document.getElementById('bmpHeight').value)">Resize Canvas</button>
    <br />
<button class="btn btn-primary" onclick="sendCanvas('myCanvas')">Send generated BMP</button>
<br />
<canvas id="myCanvas" width="560" height="560"</canvas>

And whether you believe it or not - you have finished. You have now full control over your glasses.

When you enter a new size, and click the button. The image will be adjusted.

When you click "Send generated BMP" - it will show up in the glasses (if you deploy it on your Android phone, by connecting it and choosing it as debug target)

You can use it to try to see which image size looks best for your purpose.

You ca set a breakpoint in the code to see what comes back:

If you are in this stage trying to debug but can't find the source of an issue,  you can use chrome:

Enter the url chrome://inspect/devices and click the "inspect" link on the active debugging session (emulator or your phone) and then you can use the developer tools from the browser to read error or work on it.

Source code: ExerciseSendImages.zip

.NET MAUI - Handling taps

Now that we can send text and images, we can handle events.

We'll continue from the last project. But this one is quite straightforward now, so see the base example .NET MAUI Integration: First contact

Android will show a notification of how many times we tapped the glasses:

To "communicate it back to MAUI/.NET" - I will have to expand the SDK, and will write up the usage after this is done.

So lets get to it!

1. Locate you MainActivity.cs and let the compiler know we're implementing Com.Vuzix.Ultralite.IEventListener

   public class MainActivity : MauiAppCompatActivity, Com.Vuzix.Ultralite.IEventListener

2. In the Oncreate, let the SDK know we're handling the events
   
   

           IUltraliteSDK _sdk;
   		protected override void OnCreate(Bundle savedInstanceState)
           {
               base.OnCreate(savedInstanceState);
               try
               {
                   WeakReferenceMessenger.Default.Register<UltraLiteError>(this, (sender, e) => { processUltraLiteError(e); });
                   WeakReferenceMessenger.Default.Register<UltraLiteMessage>(this, (sender, e) => { processUltraLiteMessage(e.Data); });
                   WeakReferenceMessenger.Default.Register<UltraLiteOperationRequest>(this, (sender, e) => { processUltraLiteOperation(e); });
                   _sdk = Com.Vuzix.Ultralite.IUltraliteSDK.Get(this);
   				_sdk.AddEventListener(this); // <- this line we added
   			}
   			catch (System.Exception ex)
               {
                   showMessage(ex.Message);
               }
           }
   

3. We now create the functions of the known events we'll handle (these are the complete list)

   
   		public void OnTap(int tapCount)
   		{
   			processUltraLiteMessage("Tapped " + tapCount);
   			this.showMessage("Tapped " + tapCount);
   		}
   		private void onDisplayOff()
   		{
   			this.showMessage("onDisplayOff");
   		}
   		private void onDisplayOn()
   		{
   			this.showMessage("onDisplayOn");
   		}
   		private void onDisplayTimeout()
   		{
   			this.showMessage("onDisplayTimeout");
   		}
   		private void OnPowerButtonPress(bool turningOn)
   		{
   			this.showMessage("onPowerButtonPress, turnongOn:" + turningOn);
   		}
   		private void OnScrolled(bool isScreenEmpty)
   		{
   			this.showMessage("OnScrolled, isScreenEmpty:" + isScreenEmpty);
   		}

In your Visual Studio 2022 it will look like this:

Your full MainActivity might look among these lines:

using Android.App;
using Android.Content.PM;
using Android.OS;
using Android.Widget;
using CommunityToolkit.Mvvm.Messaging;
using VuzixSDK.Class;
using Com.Vuzix.Ultralite;
using Layout = Com.Vuzix.Ultralite.Layout;
using TextAlignment = Com.Vuzix.Ultralite.TextAlignment;
using VuzixSDK.Enum;
using Android.Graphics;
using System.Diagnostics.Tracing;
using static System.Net.Mime.MediaTypeNames;

namespace MauiApp1
{
    [Activity(Theme = "@style/Maui.SplashTheme", MainLauncher = true, ConfigurationChanges = ConfigChanges.ScreenSize | ConfigChanges.Orientation | ConfigChanges.UiMode | ConfigChanges.ScreenLayout | ConfigChanges.SmallestScreenSize | ConfigChanges.Density)]
    public class MainActivity : MauiAppCompatActivity, Com.Vuzix.Ultralite.IEventListener
	{
        IUltraliteSDK _sdk;
		protected override void OnCreate(Bundle savedInstanceState)
        {
            base.OnCreate(savedInstanceState);
            try
            {
                WeakReferenceMessenger.Default.Register<UltraLiteError>(this, (sender, e) => { processUltraLiteError(e); });
                WeakReferenceMessenger.Default.Register<UltraLiteMessage>(this, (sender, e) => { processUltraLiteMessage(e.Data); });
                WeakReferenceMessenger.Default.Register<UltraLiteOperationRequest>(this, (sender, e) => { processUltraLiteOperation(e); });
                _sdk = Com.Vuzix.Ultralite.IUltraliteSDK.Get(this);
				_sdk.AddEventListener(this);
			}
			catch (System.Exception ex)
            {
                showMessage(ex.Message);
            }
        }

		public void OnTap(int tapCount)
		{
			processUltraLiteMessage("Tapped " + tapCount);
			this.showMessage("Tapped " + tapCount);
		}
		private void onDisplayOff()
		{
			this.showMessage("onDisplayOff");
		}
		private void onDisplayOn()
		{
			this.showMessage("onDisplayOn");
		}
		private void onDisplayTimeout()
		{
			this.showMessage("onDisplayTimeout");
		}
		private void OnPowerButtonPress(bool turningOn)
		{
			this.showMessage("onPowerButtonPress, turnongOn:" + turningOn);
		}
		private void OnScrolled(bool isScreenEmpty)
		{
			this.showMessage("OnScrolled, isScreenEmpty:" + isScreenEmpty);
		}
		public void clearScreen()
		{
			_sdk.Canvas.RemoveText(_lastTextId);
			_sdk.Canvas.RemoveAnimation(_lastAnimationId);
			_sdk.Canvas.RemoveImage(_lastImageId);
		}
		public void showMessage(string message)
        {
            MainThread.BeginInvokeOnMainThread(() =>
            {
                var toast = Toast.MakeText(this, message, ToastLength.Short);
                toast.Show();
            });
        }
        protected void processUltraLiteError(UltraLiteError error)
        {
            if (_sdk.IsConnected)
            {
                if (!_sdk.IsControlledByMe)
                {
                    _sdk.RequestControl();
                }
                if (_sdk.IsControlledByMe)
                {
                    string _title = $"[Error]{(error.Source != null ? " " + error.Source : "")}";
                    string _error = (error.Exception != null ? $"Exception : {error.Exception.Message}" : "Error occured");
                    _sdk.SendNotification(_title, _error);
                }   
            }
        }
        int _lastTextId = -1;
        int _lastImageId = -1;
        int _lastAnimationId = -1;

		protected void processUltraLiteMessage(String message)
		{
			if (_sdk.IsConnected)
			{
				if (!_sdk.IsControlledByMe)
				{
					_sdk.RequestControl();
				}
				if (_sdk.IsControlledByMe)
				{
					_sdk.SetLayout(Layout.Canvas, 0, true);
					bool _messageSucceeded = false;
					if (_lastTextId >= 0)
					{
						_messageSucceeded = _sdk.Canvas.UpdateText(_lastTextId, message);
					}
					else
					{
						_lastTextId = _sdk.Canvas.CreateText(message, Anchor.Center);
						_messageSucceeded = (_lastTextId != -1);
					}
					if (!_messageSucceeded)
					{
						showMessage("Text failed");
					}
					_sdk.Canvas.Commit();
					SystemClock.Sleep(1000);
				}
			}
		}
		protected void processUltraLiteOperation(UltraLiteOperationRequest Request)
        {
            if (_sdk.IsConnected)
            {
                if (!_sdk.IsControlledByMe)
                {
                    _sdk.RequestControl();
                }
                if (_sdk.IsControlledByMe)
                {
                    _sdk.SetLayout(Layout.Canvas, 0, true);
                    if (Request.Operation == eUltraLiteOperation.ShowImage && Request.ImageBitMap != null)
                    {
                        LVGLImage image = loadLVGLImage(Request.ImageBitMap);
                        bool _imageSucceeded = false;
                        if (_lastImageId >= 0)
                        {
							_imageSucceeded = _sdk.Canvas.UpdateImage(_lastImageId, image);
                        }
                        else
                        {
                            _lastAnimationId = _sdk.Canvas.CreateImage(image, Anchor.Center);
							_imageSucceeded = (_lastImageId != -1);
						}
                        if(!_imageSucceeded) showMessage("Image failed");
                        _sdk.Canvas.Commit();
                    }
                    if (Request.Operation == eUltraLiteOperation.ShowAnimation && Request.AnimationBitMap != null)
                    {
                        LVGLImage[] image = loadLVGLImage(Request.AnimationBitMap);
                        int _animationDelay = 500;
                        if(_lastAnimationId >= 0)
                        {
                            _sdk.Canvas.RemoveAnimation(_lastAnimationId);
						}
						_lastAnimationId = _sdk.Canvas.CreateAnimation(image, Anchor.Center, _animationDelay);

						if (_lastAnimationId == -1)
                        {
                            showMessage("Animation failed");
                        }
                        _sdk.Canvas.Commit();
                    }
                }
                
            }
            else
            {
                showMessage("SDK is not connected");
            }
        }
        private static Bitmap loadBitmap(byte[] bitmapbytes)
        {
            BitmapFactory.Options options = new BitmapFactory.Options();

            // https://proandroiddev.com/image-decoding-bitmaps-android-c039790ee07e
            options.InSampleSize = 2;

            //options.InTargetDensity = 640 * 2;
            //options.InTargetDensity = 480 * 8;
            //options.InScaled = true;
            options.InPreferredConfig = Bitmap.Config.Argb8888;
            /* options.InMutable = true;
            
            options.InSampleSize = 8;
			options.OutWidth = 600;
			options.OutHeight = 400;
			options.InScaled = scaled;*/
            Bitmap bmp = BitmapFactory.DecodeByteArray(bitmapbytes, 0, bitmapbytes.Length, options);

            return bmp;// resize(bmp, 640, 480);
        }
        private static LVGLImage[] loadLVGLImage(List<byte[]> images)
        {
            List<LVGLImage> _images = new List<LVGLImage>();
            foreach (var image in images)
            {
                _images.Add(LVGLImage.FromBitmap(loadBitmap(image), LVGLImage.CfIndexed1Bit));
            }
            return _images.ToArray();
        }
        private static LVGLImage loadLVGLImage(byte[] image)
        {
            //ColorObject[] _colors = { LVGLImage.IColorMapper.White, LVGLImage.IColorMapper.Mid };
            //LVGLImage _img = new LVGLImage(LVGLImage.CfIndexed1Bit, 480, 640, _colors, image);
            LVGLImage _img2 = LVGLImage.FromBitmap(loadBitmap(image), LVGLImage.CfIndexed1Bit);
            return _img2;
        }
    }
}

When the code is run on your Android device - and you tap the side of the glasses, it will show a little message.
Vuzix's SDK mentions that 1, 2 and 3 taps are triggered.

.NET MAUI - reporting events to the GUI

In this page, we will send a notifiation of an event back to the .NET GUI.

The resulting project is as follows: ExcersisTapCallback.zip

This way, we're completely disconnected with the Android stack and can focus on development in MAUI or .NET and only consider a few functions to interact with the glasses.

Note on code design: in a real world application, you would register a handler in a service in .NET - as their "Dependency Injection" is very good. But this code will be sufficient to demonstrate the interface with the glasses.

So let's go!

Well start off again with the last project .NET MAUI - Handling taps 

1. Update the Nuget package, we require at least VuzixSDK 1.0.1
2. Change the event functions in the MainActivity to send the events via the WeakReferenceMessenger
   

   

   
   
   

public void OnTap(int tapCount)
{
    WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnTap>(new VuzixSDK.Class.Event.UltraLiteOnTap()
    {
        tapCount = tapCount
    });
}
private void onDisplayOff()
{
    WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnDisplayOff>(new VuzixSDK.Class.Event.UltraLiteOnDisplayOff());
}
private void onDisplayOn()
{
    WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnDisplayOn>(new VuzixSDK.Class.Event.UltraLiteOnDisplayOn());
}
private void onDisplayTimeout()
{
    WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnDisplayTimeout>(new VuzixSDK.Class.Event.UltraLiteOnDisplayTimeout());
}
private void OnPowerButtonPress(bool turningOn)
{
    WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnPowerButtonPress>(new VuzixSDK.Class.Event.UltraLiteOnPowerButtonPress()
    {
        turningOn = turningOn
    });
}
private void OnScrolled(bool isScreenEmpty)
{
    WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnScrolled>(new VuzixSDK.Class.Event.UltraLiteOnScrolled()
    {
        isScreenEmpty = isScreenEmpty
    });
}

Your MainActivy.cs will look as follows:

using Android.App;
using Android.Content.PM;
using Android.OS;
using Android.Widget;
using CommunityToolkit.Mvvm.Messaging;
using VuzixSDK.Class;
using Com.Vuzix.Ultralite;
using Layout = Com.Vuzix.Ultralite.Layout;
using TextAlignment = Com.Vuzix.Ultralite.TextAlignment;
using VuzixSDK.Enum;
using Android.Graphics;
using System.Diagnostics.Tracing;
using static System.Net.Mime.MediaTypeNames;

namespace MauiApp1
{
    [Activity(Theme = "@style/Maui.SplashTheme", MainLauncher = true, ConfigurationChanges = ConfigChanges.ScreenSize | ConfigChanges.Orientation | ConfigChanges.UiMode | ConfigChanges.ScreenLayout | ConfigChanges.SmallestScreenSize | ConfigChanges.Density)]
    public class MainActivity : MauiAppCompatActivity, Com.Vuzix.Ultralite.IEventListener
	{
        IUltraliteSDK _sdk;
		protected override void OnCreate(Bundle savedInstanceState)
        {
            base.OnCreate(savedInstanceState);
            try
            {
                WeakReferenceMessenger.Default.Register<UltraLiteError>(this, (sender, e) => { processUltraLiteError(e); });
                WeakReferenceMessenger.Default.Register<UltraLiteMessage>(this, (sender, e) => { processUltraLiteMessage(e.Data); });
                WeakReferenceMessenger.Default.Register<UltraLiteOperationRequest>(this, (sender, e) => { processUltraLiteOperation(e); });

                _sdk = Com.Vuzix.Ultralite.IUltraliteSDK.Get(this);
				_sdk.AddEventListener(this);
			}
			catch (System.Exception ex)
            {
                showMessage(ex.Message);
            }
        }

		public void OnTap(int tapCount)
		{
            WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnTap>(new VuzixSDK.Class.Event.UltraLiteOnTap()
            {
                tapCount = tapCount
            });
		}
		private void onDisplayOff()
		{
            WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnDisplayOff>(new VuzixSDK.Class.Event.UltraLiteOnDisplayOff());
        }
		private void onDisplayOn()
		{
            WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnDisplayOn>(new VuzixSDK.Class.Event.UltraLiteOnDisplayOn());
        }
        private void onDisplayTimeout()
		{
            WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnDisplayTimeout>(new VuzixSDK.Class.Event.UltraLiteOnDisplayTimeout());
        }
        private void OnPowerButtonPress(bool turningOn)
		{
            WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnPowerButtonPress>(new VuzixSDK.Class.Event.UltraLiteOnPowerButtonPress()
            {
                turningOn = turningOn
            });
		}
		private void OnScrolled(bool isScreenEmpty)
		{
            WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnScrolled>(new VuzixSDK.Class.Event.UltraLiteOnScrolled()
            {
                isScreenEmpty = isScreenEmpty
            });
        }
		public void clearScreen()
		{
			_sdk.Canvas.RemoveText(_lastTextId);
			_sdk.Canvas.RemoveAnimation(_lastAnimationId);
			_sdk.Canvas.RemoveImage(_lastImageId);
		}
		public void showMessage(string message)
        {
            MainThread.BeginInvokeOnMainThread(() =>
            {
                var toast = Toast.MakeText(this, message, ToastLength.Short);
                toast.Show();
            });
        }
        protected void processUltraLiteError(UltraLiteError error)
        {
            if (_sdk.IsConnected)
            {
                if (!_sdk.IsControlledByMe)
                {
                    _sdk.RequestControl();
                }
                if (_sdk.IsControlledByMe)
                {
                    string _title = $"[Error]{(error.Source != null ? " " + error.Source : "")}";
                    string _error = (error.Exception != null ? $"Exception : {error.Exception.Message}" : "Error occured");
                    _sdk.SendNotification(_title, _error);
                }   
            }
        }
        int _lastTextId = -1;
        int _lastImageId = -1;
        int _lastAnimationId = -1;

		protected void processUltraLiteMessage(String message)
		{
			if (_sdk.IsConnected)
			{
				if (!_sdk.IsControlledByMe)
				{
					_sdk.RequestControl();
				}
				if (_sdk.IsControlledByMe)
				{
					_sdk.SetLayout(Layout.Canvas, 0, true);
					bool _messageSucceeded = false;
					if (_lastTextId >= 0)
					{
						_messageSucceeded = _sdk.Canvas.UpdateText(_lastTextId, message);
					}
					else
					{
						_lastTextId = _sdk.Canvas.CreateText(message, Anchor.Center);
						_messageSucceeded = (_lastTextId != -1);
					}
					if (!_messageSucceeded)
					{
						showMessage("Text failed");
					}
					_sdk.Canvas.Commit();
					SystemClock.Sleep(1000);
				}
			}
		}
		protected void processUltraLiteOperation(UltraLiteOperationRequest Request)
        {
            if (_sdk.IsConnected)
            {
                if (!_sdk.IsControlledByMe)
                {
                    _sdk.RequestControl();
                }
                if (_sdk.IsControlledByMe)
                {
                    _sdk.SetLayout(Layout.Canvas, 0, true);
                    if (Request.Operation == eUltraLiteOperation.ShowImage && Request.ImageBitMap != null)
                    {
                        LVGLImage image = loadLVGLImage(Request.ImageBitMap);
                        bool _imageSucceeded = false;
                        if (_lastImageId >= 0)
                        {
							_imageSucceeded = _sdk.Canvas.UpdateImage(_lastImageId, image);
                        }
                        else
                        {
                            _lastAnimationId = _sdk.Canvas.CreateImage(image, Anchor.Center);
							_imageSucceeded = (_lastImageId != -1);
						}
                        if(!_imageSucceeded) showMessage("Image failed");
                        _sdk.Canvas.Commit();
                    }
                    if (Request.Operation == eUltraLiteOperation.ShowAnimation && Request.AnimationBitMap != null)
                    {
                        LVGLImage[] image = loadLVGLImage(Request.AnimationBitMap);
                        int _animationDelay = 500;
                        if(_lastAnimationId >= 0)
                        {
                            _sdk.Canvas.RemoveAnimation(_lastAnimationId);
						}
						_lastAnimationId = _sdk.Canvas.CreateAnimation(image, Anchor.Center, _animationDelay);

						if (_lastAnimationId == -1)
                        {
                            showMessage("Animation failed");
                        }
                        _sdk.Canvas.Commit();
                    }
                }
                
            }
            else
            {
                showMessage("SDK is not connected");
            }
        }
        private static Bitmap loadBitmap(byte[] bitmapbytes)
        {
            BitmapFactory.Options options = new BitmapFactory.Options();

            // https://proandroiddev.com/image-decoding-bitmaps-android-c039790ee07e
            options.InSampleSize = 2;

            //options.InTargetDensity = 640 * 2;
            //options.InTargetDensity = 480 * 8;
            //options.InScaled = true;
            options.InPreferredConfig = Bitmap.Config.Argb8888;
            /* options.InMutable = true;
            
            options.InSampleSize = 8;
			options.OutWidth = 600;
			options.OutHeight = 400;
			options.InScaled = scaled;*/
            Bitmap bmp = BitmapFactory.DecodeByteArray(bitmapbytes, 0, bitmapbytes.Length, options);

            return bmp;// resize(bmp, 640, 480);
        }
        private static LVGLImage[] loadLVGLImage(List<byte[]> images)
        {
            List<LVGLImage> _images = new List<LVGLImage>();
            foreach (var image in images)
            {
                _images.Add(LVGLImage.FromBitmap(loadBitmap(image), LVGLImage.CfIndexed1Bit));
            }
            return _images.ToArray();
        }
        private static LVGLImage loadLVGLImage(byte[] image)
        {
            //ColorObject[] _colors = { LVGLImage.IColorMapper.White, LVGLImage.IColorMapper.Mid };
            //LVGLImage _img = new LVGLImage(LVGLImage.CfIndexed1Bit, 480, 640, _colors, image);
            LVGLImage _img2 = LVGLImage.FromBitmap(loadBitmap(image), LVGLImage.CfIndexed1Bit);
            return _img2;
        }
    }
}

3. Go back to the MAUI section, Home.razor and add the following code. The OnInitialized() will be called by the framework when the page is ready. Then, we register a listener to a message of type UltraLiteOnTap and write some text in a string to show we received it

    string _eventText = "";
    protected override void OnInitialized()
    {
        WeakReferenceMessenger.Default.Register<VuzixSDK.Class.Event.UltraLiteOnTap>(this, (sender, e) => {
            _eventText = $"You have tapped {e.tapCount}.";
            StateHasChanged();
        });
    }

<h1>Hello, world!</h1>
@_eventText
@if (MyMessage != null && MyMessage.Length>0 )
{
<p>You wrote: @MyMessage</p>
}

We do the same for all the remaining events:

    protected override void OnInitialized()
    {
        WeakReferenceMessenger.Default.Register<VuzixSDK.Class.Event.UltraLiteOnTap>(this, (sender, e) =>
        {
            _eventText = $"You have tapped {e.tapCount}.";
            StateHasChanged();
        });

        WeakReferenceMessenger.Default.Register<VuzixSDK.Class.Event.UltraLiteOnDisplayOff>(this, (sender, e) =>
        {
            _eventText = $"Display off event.";
            StateHasChanged();
        });
        WeakReferenceMessenger.Default.Register<VuzixSDK.Class.Event.UltraLiteOnDisplayOn>(this, (sender, e) =>
        {
            _eventText = $"Display on event.";
            StateHasChanged();
        });
        WeakReferenceMessenger.Default.Register<VuzixSDK.Class.Event.UltraLiteOnDisplayTimeout>(this, (sender, e) =>
        {
            _eventText = $"Display timeout event.";
            StateHasChanged();
        });
        WeakReferenceMessenger.Default.Register<VuzixSDK.Class.Event.UltraLiteOnPowerButtonPress>(this, (sender, e) =>
        {
            _eventText = $"Power button press {(e.turningOn? "ON" : "OFF")}";
            StateHasChanged();
        });
        WeakReferenceMessenger.Default.Register<VuzixSDK.Class.Event.UltraLiteOnScrolled>(this, (sender, e) =>
        {
            _eventText = $"Scrolled {(e.isScreenEmpty?"Empty screen":"Non empty screen")}.";
            StateHasChanged();
        });

    }

At this point, you should be set to build your GUI as imagined. And display them on the glasses with ease.

.NET MAUI - reading glass status

Once you're playing around and get some life in the glasses, there are moments you're not sure if your glasses aren't listening or wonder about the battery or status. 

The resulting project is this one: ExerciseGlassesInfoCallBack.zip

 

 

In this project, we'll extend the previous project and add a button to read the status of the glasses.

You will need VuzixSDK 1.0.2 - it has the extra classes and the classes are organized a little more logical.
To migrate from 1.0.1 to 1.0.2 you only need to change one namespace.

We continue here from the tutorial  .NET MAUI - reporting events to the GUI

This is the code base to aadjust ExcersisTapCallback.zip

Let's get to it!

 

Set up the android space

We will add the class to listen for. And when we receive it, send the information of the SDK back in Android towards the GUI/.NET stack. (Blazor, MAUI, .. how you want to label it: "the thing in the front")

1. Open your (familiar) solution and update the VuzixSDK package

2. Include the new namspace in your MainActivity.cs

using VuzixSDK.Class.Request;

3. and add a listener for the new request in your MainActivity.cs

    WeakReferenceMessenger.Default.Register(this, (sender, e) => { processUltraLiteStatusRequest()});

4. Write the function that sends the information back 

        private void processUltraLiteStatusRequest()
        {
            if(_sdk != null)
            {
                WeakReferenceMessenger.Default.Send<UltraLiteStatusResponse>(new UltraLiteStatusResponse()
                {
                    isAvailable = _sdk.IsAvailable,
                    isCharging = _sdk.IsCharging,
                    isConnected = _sdk.IsConnected,
                    isControlled = _sdk.IsControlled,
                    isControlledByMe = _sdk.IsControlledByMe,
                    isLinked = _sdk.IsLinked,
                    BatteryLevel = _sdk.BatteryLevel,
                    Name = _sdk.Name
                });
            }
        }

Now your MainActivity.cs should look as follows:

using Android.App;
using Android.Content.PM;
using Android.OS;
using Android.Widget;
using CommunityToolkit.Mvvm.Messaging;
using VuzixSDK.Class;
using Com.Vuzix.Ultralite;
using Layout = Com.Vuzix.Ultralite.Layout;
using TextAlignment = Com.Vuzix.Ultralite.TextAlignment;
using VuzixSDK.Enum;
using VuzixSDK.Class.Request;
using Android.Graphics;
using System.Diagnostics.Tracing;
using static System.Net.Mime.MediaTypeNames;

namespace MauiApp1
{
    [Activity(Theme = "@style/Maui.SplashTheme", MainLauncher = true, ConfigurationChanges = ConfigChanges.ScreenSize | ConfigChanges.Orientation | ConfigChanges.UiMode | ConfigChanges.ScreenLayout | ConfigChanges.SmallestScreenSize | ConfigChanges.Density)]
    public class MainActivity : MauiAppCompatActivity, Com.Vuzix.Ultralite.IEventListener
	{
        IUltraliteSDK _sdk;
		protected override void OnCreate(Bundle savedInstanceState)
        {
            base.OnCreate(savedInstanceState);
            try
            {
                WeakReferenceMessenger.Default.Register<UltraLiteError>(this, (sender, e) => { processUltraLiteError(e); });
                WeakReferenceMessenger.Default.Register<UltraLiteMessage>(this, (sender, e) => { processUltraLiteMessage(e.Data); });
                WeakReferenceMessenger.Default.Register<UltraLiteOperationRequest>(this, (sender, e) => { processUltraLiteOperation(e); });
                WeakReferenceMessenger.Default.Register<UltraLiteStatusRequest>(this, (sender, e) => { processUltraLiteStatusRequest();  });

                _sdk = Com.Vuzix.Ultralite.IUltraliteSDK.Get(this);
				_sdk.AddEventListener(this);
			}
			catch (System.Exception ex)
            {
                showMessage(ex.Message);
            }
        }

		public void OnTap(int tapCount)
		{
            WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnTap>(new VuzixSDK.Class.Event.UltraLiteOnTap()
            {
                tapCount = tapCount
            });
		}
		private void OnDisplayOff()
		{
            WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnDisplayOff>(new VuzixSDK.Class.Event.UltraLiteOnDisplayOff());
        }
		private void OnDisplayOn()
		{
            WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnDisplayOn>(new VuzixSDK.Class.Event.UltraLiteOnDisplayOn());
        }
        private void OnDisplayTimeout()
		{
            WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnDisplayTimeout>(new VuzixSDK.Class.Event.UltraLiteOnDisplayTimeout());
        }
        private void OnPowerButtonPress(bool turningOn)
		{
            WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnPowerButtonPress>(new VuzixSDK.Class.Event.UltraLiteOnPowerButtonPress()
            {
                turningOn = turningOn
            });
		}
		private void OnScrolled(bool isScreenEmpty)
		{
            WeakReferenceMessenger.Default.Send<VuzixSDK.Class.Event.UltraLiteOnScrolled>(new VuzixSDK.Class.Event.UltraLiteOnScrolled()
            {
                isScreenEmpty = isScreenEmpty
            });
        }
        private void processUltraLiteStatusRequest()
        {
            if(_sdk != null)
            {
                WeakReferenceMessenger.Default.Send<UltraLiteStatusResponse>(new UltraLiteStatusResponse()
                {
                    isAvailable = _sdk.IsAvailable,
                    isCharging = _sdk.IsCharging,
                    isConnected = _sdk.IsConnected,
                    isControlled = _sdk.IsControlled,
                    isControlledByMe = _sdk.IsControlledByMe,
                    isLinked = _sdk.IsLinked,
                    BatteryLevel = _sdk.BatteryLevel,
                    Name = _sdk.Name
                });
            }
        }
		public void clearScreen()
		{
			_sdk.Canvas.RemoveText(_lastTextId);
			_sdk.Canvas.RemoveAnimation(_lastAnimationId);
			_sdk.Canvas.RemoveImage(_lastImageId);
		}
		public void showMessage(string message)
        {
            MainThread.BeginInvokeOnMainThread(() =>
            {
                var toast = Toast.MakeText(this, message, ToastLength.Short);
                toast.Show();
            });
        }
        protected void processUltraLiteError(UltraLiteError error)
        {
            if (_sdk.IsConnected)
            {
                if (!_sdk.IsControlledByMe)
                {
                    _sdk.RequestControl();
                }
                if (_sdk.IsControlledByMe)
                {
                    string _title = $"[Error]{(error.Source != null ? " " + error.Source : "")}";
                    string _error = (error.Exception != null ? $"Exception : {error.Exception.Message}" : "Error occured");
                    _sdk.SendNotification(_title, _error);
                }   
            }
        }
        int _lastTextId = -1;
        int _lastImageId = -1;
        int _lastAnimationId = -1;

		protected void processUltraLiteMessage(String message)
		{
			if (_sdk.IsConnected)
			{
				if (!_sdk.IsControlledByMe)
				{
					_sdk.RequestControl();
				}
				if (_sdk.IsControlledByMe)
				{
					_sdk.SetLayout(Layout.Canvas, 0, true);
					bool _messageSucceeded = false;
					if (_lastTextId >= 0)
					{
						_messageSucceeded = _sdk.Canvas.UpdateText(_lastTextId, message);
					}
					else
					{
						_lastTextId = _sdk.Canvas.CreateText(message, Anchor.Center);
						_messageSucceeded = (_lastTextId != -1);
					}
					if (!_messageSucceeded)
					{
						showMessage("Text failed");
					}
					_sdk.Canvas.Commit();
					SystemClock.Sleep(1000);
				}
			}
		}
		protected void processUltraLiteOperation(UltraLiteOperationRequest Request)
        {
            if (_sdk.IsConnected)
            {
                if (!_sdk.IsControlledByMe)
                {
                    _sdk.RequestControl();
                }
                if (_sdk.IsControlledByMe)
                {
                    _sdk.SetLayout(Layout.Canvas, 0, true);
                    if (Request.Operation == eUltraLiteOperation.ShowImage && Request.ImageBitMap != null)
                    {
                        LVGLImage image = loadLVGLImage(Request.ImageBitMap);
                        bool _imageSucceeded = false;
                        if (_lastImageId >= 0)
                        {
							_imageSucceeded = _sdk.Canvas.UpdateImage(_lastImageId, image);
                        }
                        else
                        {
                            _lastAnimationId = _sdk.Canvas.CreateImage(image, Anchor.Center);
							_imageSucceeded = (_lastImageId != -1);
						}
                        if(!_imageSucceeded) showMessage("Image failed");
                        _sdk.Canvas.Commit();
                    }
                    if (Request.Operation == eUltraLiteOperation.ShowAnimation && Request.AnimationBitMap != null)
                    {
                        LVGLImage[] image = loadLVGLImage(Request.AnimationBitMap);
                        int _animationDelay = 500;
                        if(_lastAnimationId >= 0)
                        {
                            _sdk.Canvas.RemoveAnimation(_lastAnimationId);
						}
						_lastAnimationId = _sdk.Canvas.CreateAnimation(image, Anchor.Center, _animationDelay);

						if (_lastAnimationId == -1)
                        {
                            showMessage("Animation failed");
                        }
                        _sdk.Canvas.Commit();
                    }
                }
                
            }
            else
            {
                showMessage("SDK is not connected");
            }
        }
        private static Bitmap loadBitmap(byte[] bitmapbytes)
        {
            BitmapFactory.Options options = new BitmapFactory.Options();

            // https://proandroiddev.com/image-decoding-bitmaps-android-c039790ee07e
            options.InSampleSize = 2;

            //options.InTargetDensity = 640 * 2;
            //options.InTargetDensity = 480 * 8;
            //options.InScaled = true;
            options.InPreferredConfig = Bitmap.Config.Argb8888;
            /* options.InMutable = true;
            
            options.InSampleSize = 8;
			options.OutWidth = 600;
			options.OutHeight = 400;
			options.InScaled = scaled;*/
            Bitmap bmp = BitmapFactory.DecodeByteArray(bitmapbytes, 0, bitmapbytes.Length, options);

            return bmp;// resize(bmp, 640, 480);
        }
        private static LVGLImage[] loadLVGLImage(List<byte[]> images)
        {
            List<LVGLImage> _images = new List<LVGLImage>();
            foreach (var image in images)
            {
                _images.Add(LVGLImage.FromBitmap(loadBitmap(image), LVGLImage.CfIndexed1Bit));
            }
            return _images.ToArray();
        }
        private static LVGLImage loadLVGLImage(byte[] image)
        {
            //ColorObject[] _colors = { LVGLImage.IColorMapper.White, LVGLImage.IColorMapper.Mid };
            //LVGLImage _img = new LVGLImage(LVGLImage.CfIndexed1Bit, 480, 640, _colors, image);
            LVGLImage _img2 = LVGLImage.FromBitmap(loadBitmap(image), LVGLImage.CfIndexed1Bit);
            return _img2;
        }
    }
}

 

 

 

Set up the pages and navigation in the UI

 

 

We now have to initiate the request for this information and then show it.
We'll make another page, add a button and write the information in text on the screen.

1. Right-click Pages and add Razor-component

2. Enter the name of the razor page and Add

3. You'll see this as te page

4. Add a link to the page in the navigation, through going in ComponentsLayout/NavMenu.razor.

Just copy paste an existing element and adjust the link and the text

        <div class="nav-item px-3">
            <NavLink class="nav-link" href="VuzixStatus">
                <span class="bi bi-list-nested-nav-menu" aria-hidden="true"></span> Status
            </NavLink>
        </div>

 

While here, lets also remove the Counter item - we haven't been using it.

5. Now you could test if everything is in the right place and builds. - in this case we see we have to  add the namespaces for the new nuget version.

 

-> So add the includes

@page "/"
@using CommunityToolkit.Mvvm.Messaging;
@using VuzixSDK.Class.Event
@using VuzixSDK.Class
@using VuzixSDK.Class.Request
@using VuzixSDK.Enum

 

-> Rebuild.

 

-> Start debug

 

You should now see the Status menu item if you press the hamburger menu.

And when pressed, see our page/component

 

Making the two ends meet together (trigger request and read response)

1. Include the namespaces in VuzixStatus.razor (or your name) and the Mvvm namespace

@page "/VuzixStatus"
@using VuzixSDK.Class.Event
@using VuzixSDK.Class
@using VuzixSDK.Class.Request
@using VuzixSDK.Enum
@using CommunityToolkit.Mvvm.Messaging;

2. Write a button to trigger the function to trigger the function: @onclick denotes it is ran on the server. So we can write this functin in a @code{ } segment

<h3>VuzixStatus</h3>
<button @onclick="GetZ100Info">Get Info</button><br />
@code {

    protected void GetZ100Info()
    {
        WeakReferenceMessenger.Default.Send(new UltraLiteStatusRequest());
    }
}

3. Declare a variable to hold the information and write the receiver in the OnInitialized function to set the variable in the @code{} segment

    UltraLiteStatusResponse Z100Status;
    protected override void OnInitialized()
    {
        WeakReferenceMessenger.Default.Register<UltraLiteStatusResponse>(this, (sender, e) =>
        {
            Z100Status = e;
            StateHasChanged();
        });
    }

4. Write the GUI code to display the variable, outside of the @code{} segment - the @ denotes server code. So we check the variable on the server and then render the GUI if not empty. (unset)

@if(Z100Status != null)
{
    <table>
        <tr>
            <td>Name</td>
            <td>@Z100Status.Name</td>
        </tr>
        <tr>
            <td>isAvailable</td>
            <td>@Z100Status.isAvailable</td>
        </tr>
        <tr>
            <td>isCharging</td>
            <td>@Z100Status.isCharging</td>
        </tr>
        <tr>
            <td>BatteryLevel</td>
            <td>@Z100Status.BatteryLevel %</td>
        </tr>
        <tr>
            <td>isConnected</td>
            <td>@Z100Status.isConnected</td>
        </tr>
        <tr>
            <td>isControlled</td>
            <td>@Z100Status.isControlled</td>
        </tr>        <tr>
            <td>isControlledByMe</td>
            <td>@Z100Status.isControlledByMe</td>
        </tr>
        <tr>
            <td>isLinked</td>
            <td>@Z100Status.isLinked</td>
        </tr>
        <tr>
            <td>isConnected</td>
            <td>@Z100Status.isConnected</td>
        </tr>
    </table>

}

Your full razor page would look something as this:

@page "/VuzixStatus"
@using CommunityToolkit.Mvvm.Messaging;
@using VuzixSDK.Class.Event
@using VuzixSDK.Class
@using VuzixSDK.Class.Request
@using VuzixSDK.Enum

<h3>VuzixStatus</h3>
<button @onclick="GetZ100Info">Get Info</button><br />
@code {
    protected UltraLiteStatusResponse Z100Status;
    protected override void OnInitialized()
    {
        WeakReferenceMessenger.Default.Register<UltraLiteStatusResponse>(this, (sender, e) =>
        {
            Z100Status = e;
            StateHasChanged();
        });
    }
    protected void GetZ100Info()
    {
        WeakReferenceMessenger.Default.Send(new UltraLiteStatusRequest());
    }
}


@if(Z100Status != null)
{
    <table>
        <tr>
            <td>Name</td>
            <td>@Z100Status.Name</td>
        </tr>
        <tr>
            <td>isAvailable</td>
            <td>@Z100Status.isAvailable</td>
        </tr>
        <tr>
            <td>isCharging</td>
            <td>@Z100Status.isCharging</td>
        </tr>
        <tr>
            <td>BatteryLevel</td>
            <td>@Z100Status.BatteryLevel %</td>
        </tr>
        <tr>
            <td>isConnected</td>
            <td>@Z100Status.isConnected</td>
        </tr>
        <tr>
            <td>isControlled</td>
            <td>@Z100Status.isControlled</td>
        </tr>        <tr>
            <td>isControlledByMe</td>
            <td>@Z100Status.isControlledByMe</td>
        </tr>
        <tr>
            <td>isLinked</td>
            <td>@Z100Status.isLinked</td>
        </tr>
        <tr>
            <td>isConnected</td>
            <td>@Z100Status.isConnected</td>
        </tr>
    </table>

}

 

Run the application,  And select the status in the navigation. Press the button.

 

 

Resulting project: ExerciseGlassesInfoCallBack.zip

LVGL Image

How it fits in the Vuzix stack

It appears that the Vuzix100 glasses use LVGLImage to render the GUI.

As far I have understood from the API of the Vuzix SDK documentation that the only interface we have from our code, is to load a Bitmap. Which is logical, as the LVGL format looks like embedded code. And we are programming towards a Bluetooth protocol and not the rendering of the GUI on the glasses directly.

Playing with the library

It's fairly easy to pull some code as example into Visual Studio -

git clone --recurse-submodules https://github.com/lvgl/lv_port_pc_visual_studio.git

And then run the solution

 

It is then very clear this is embedded code. We'll park it here for now, and might explore it further in some ESP32 project with an OLED screen. Like the ESP32 LilyGO TT-Display but right now we'll continue with our more direct approach of beaming a bitmap we've generated in Canvas.

 

 

 

 

.NET: Refactoring / Dependency Injection

This section will be a more around architectural and design considerations.

Someone who is affluent with .NET Core will probably have wondered "why didn't we start off like this", while we focused on the integration and left the considerations of other aspects of the application to the reader. 

Someone who isn't so used to .NET Core, Dependency Injection (DI) or other of such design concepts - might've gotten quite easily overwhelmed without understanding what problem in the design we'll solve now.

Context

So far, we've considered the application as some sort of webpage and could sneak around into Android via the MvvM messenger.
Which behaves pretty much as a message bus in engineering software.

Now, we could write a single "page" with all functionality and logic, and depending on clicks and events, we could model it towards the single task the user is trying to get done.

But this would violate the "seperation of function" and breaks all the rules of modern "OOP" (Object Oriented Programming).

Consider our example: if we want to trigger a function from the GUI, we need to prepare the server side code for this "page" to receive and send these messages back and forth.

If we create a "new page", we have to do it again ? And what about, when someone comes back to a previous page, does it need to re-initialize everything ? Or, are we still interested in messages when we're not watching ?

.NET Core is - actually - a program rendering a GUI. And interfacing with this GUI over a sort of asynchronous back and forth to read the state, or see if the state changed (and it has to redraw) - while the logical engine is in front of either the Iphone, or ANdroid, or web, ...

BUt then the problem becomes: we could create an instance in the main program to handle everything. but then, how to get it into "a page" ? And how can "a page" communicate with this instance in the main program ?

Services

This is what is called "services" in .NET Core: we'll create a class that handles everything around a certain function or context in the program. And, whichever component i nthe program that wants to use this class - we can use the @inject keyword, so, when the page is created, the framework will inject the service into the page which references the single instance in the main application. So, every "page" will consume the same service.

To get "notified of changes", we can add event handlers. 

IN this way, we only have one instance of this "service" we'll make. And, the service will know who to notify, based on event handlers that are added to it.

The code

Create a folder services and add a new class "VuzixService"

Now, collect all the functions that were written to interface with the glasses into this class.