Log in to follow projects

Following a project allows you to receive updates via email. It also lets the owner know that you like them.

×

Femto Paint

By Oliver Smith

A paint app for my iPad where I can micromanage the UX

35% complete
1 follower 2165 views
About Log (6) Discussion (0)
See more posts

Making a paint app in Unity


WIP of Femto, as of writing this.


I was recently working on some terrain tools for the Unity editor, and while using the stylus to draw terrain, I realized I wasn't that far away from writing a paint app. Since I spend a lot of time drawing on my iPad, I got really excited about being able to practice drawing and painting in my own app where I could have the UI and toolset exactly how I wanted it. So I pivoted to doing that instead.

There are a few interesting hurdles that I've encountered so far:

  1. Unity is designed for making games, not apps
  2. I don't have an Apple device to build iOS apps with
  3. Unity doesn't have good support for the Apple Pencil stylus
  4. My Windows installation crashed so I switched to Linux halfway

I think it would be fun to write a short blog post on each of those topics, so this post is about the first hurdle.

Unity for Application Development


Unity is meant for making games, which are usually fullscreen experiences where the screen is completely redrawn as fast as possible (ideally 60 times per second or more). For games, it's okay to use a ton of CPU and GPU resources - the user isn't expecting to multitask anyway. Crucially, that means a game will probably drain your battery much faster on a tablet than an app would, which is no good for a paint app that you might be working in for hours.

An application doesn't have this need to refresh the screen as fast as possible, so it would be designed to refresh only when needed, or even to refresh only the parts of the screen that have changed. Unity just doesn't work that way.

So why write an app in Unity at all? It's probably not a good idea, but I can think of a few reasons:

  1. To leverage existing knowledge of Unity
  2. To use Unity's graphics API
  3. Unity runs cross-platform almost everywhere

In my experience, paying too much attention to #1 above is usually a mistake, and it's better to use the right tools for the job than to limit yourself to your existing skill-set. But I think reasons #2 and #3 are interesting, and I'm just doing this for fun anyway.

If you're like me, your first instinct might be to just set Application.targetFramerate to something low, like 1 FPS, when the user is not actively doing something. But the issue there is that Unity only handles input events during Update(), so if you do that there will be a noticeable delay from when the user tries to do something, to when your app notices it and is able to bring the frame-rate back up.

Fortunately, Unity does give us another option. This article at secretlab.institute explains how to shut off Unity's core game loop when idle, which means your program will only use CPU and GPU resources when the user is actively doing something. That alone should improve battery life by a lot, assuming the user isn't constantly interacting with your app.

(Edit - As of 7/22 I'm no longer using the below technique, since I learned about Unity's UnityEngine.Rendering.OnDemandRendering feature - I'd recommend looking into that first, since it is a simpler way of doing more or less the same thing!)

The linked technique is a bit complicated though, since it requires writing native plugins for each supported platform in order to detect when the user has moved the mouse or touched the screen, in order to turn Unity's game loop back on again. I found that with Unity's "new" Input System at least, that's no longer required and you can do it without writing any native code. The trick is to leave some Unity SubSystems running even when idle - in particular, the ones that allow Unity to detect user input and tell your code about it.

Okay, but there's still a problem - now the SubSystem that regulates Unity's framerate is disabled. Since Unity no longer has to wait for VSync or for the GPU to render, the remaining SubSystems run as fast as possible and use 100% CPU! I was able to work around that by inserting a simple 1ms delay to give the CPU a break.

On Windows, this approach seems to work quite well. Resource usage goes to near 0 whenever the user is not doing anything, and responds instantly when the user touches the mouse or stylus. I therefore assume it is working on iPad, since anecdotally the battery usage of my app seems comparable to other paint apps. However, I don't know how to accurately test the power usage on iPad yet, so take this with a grain of salt. On Linux, I think there is still an issue and something about this approach isn't working perfectly, but I haven't had time to research it yet.

If you want to try this technique yourself, here's the code I'm using.