In case of desktop app can't one use frameworks like Qt or GTK or Cairo. I believe they are available for all platforms.
And now that Google has introduced a brand new cross platform Graphic Engine cum UI toolkit Skia one can also use that.
I program a lot in Delphi for creating desktop apps. Here if I include Skia in the project the performance of the app and responsiveness of the UI increased by almost 30%.
I personally would suggest you to give Skia a closer look.
One thing that I like about Go is that it is very very fast in execution.