This is the last entry in the Google Summer of Code series that
I have been writing weekly for the last three months. It is different from the
usual updates in that I won’t be discussing development progress: rather, this
will be the submission report for the project as a whole. I’ll be discussing the
why? behind the project, the plan that my mentor and
I came up with to execute the project, the work I have done over the summer
including a video of the result, the things that are
left to work on, what I’ve learned during
the project and finally, the links to the code that I have
written for the actual submission. Of course I finish with a
For the uninitiated, my mentor is Peter Hutterer and the project was done under the X.Org organization.
So why did Peter and I propose this project? Well, it was Google Summer of Code, I wanted to gain experience in the real world and Peter wanted free labor so he didn’t have to do it 😉
Before the Summer of Code, Piper wasn’t user-friendly, feature complete or even actively developed in the first place. When Peter and Benjamin (bentiss) started the libratbag project in September 2015, they had a clear idea for the library itself but not its integration with the desktop:
Eventually we want this to be integrated into the desktop environments, either in the respective control panels or in a standalone application. libratbag already provides SVGs for some devices we support but we’ll need some designer input for the actual application. Again, any help you want to provide here will be much appreciated.
Help never arrived, and so in February 2016 Peter started himself. He threw something together over the course of two months and released Piper version 0.2.1. It wasn’t yet feature complete and the UI was… well, let’s just call it not user friendly. Piper wasn’t touched since that release, even though libratbag was steadily improving as was the gaming on Linux story in general.
This is why Peter and I proposed this project: to rewrite Piper from the ground up so that it can fully expose all features offered by libratbag and present a unified graphical user interface to those that want to configure their gaming mice on their Linux desktop. (something something year of the Linux desktop.)
The goal thus was to rewrite Piper in order to make it more user-friendly and to enable support for features found in today’s gaming mice. This support was already present in libratbag, the library created specifically to configure gaming mice. In particular, libratbag allows you to:
- configure your device’s resolutions, through:
- adding and removing resolutions;
- switching between resolutions;
- setting a resolution’s DPI and report rate;
- setting the active and default resolutions.
- configure your device’s buttons, through:
- remapping a physical button to another logical button;
- assigning special actions such as switching resolutions to physical buttons;
- assigning macros to physical buttons.
- configure your device’s LEDs through different effect modes, each with their own settings such as color, brightness and effect rate.
- do all of the above as part of a profile, through adding and removing profiles.
In order to make Piper the best tool for the job, it had to be able to do all of this as well. With this, the requirements of my project were established. Initially I dedicated sprint one to requirement analysis, but I finished it even before the coding period began (what can I say, I was excited!). This tentative plan divided the work into mostly two week sprints, where each sprint would result in a deliverable. All deliberables were required for a successful completion of my project:
|Week||Start / End date||Milestone||Explanation|
|1-2||30-5 / 12-6||Identify the feature set to be supported in the GUI||This includes looking what features most gaming mice provide and considering if they can be supported with libratbag/ratbagd. Perhaps support can be added for missing features.|
|3-4||13-6 / 10-7||Start with the GUI||Draft a mockup and discuss with my mentor and GNOME’s design team if possible. Implement all features currently supported by Piper.|
|5-6||27-6 / 10-7||Enable the infrastructure to make a GUI possible||Ratbagd has to be evaluated and maybe extended in order to support the new GUI with more features. This can likely be started in parallel to the previous deliverable while I wait for feedback on the mockup.|
|7||11-7 / 17-7||Implement profiles and profile switching||Allow the user to save the current settings as a profile that can be restored later.|
|8||18-7 / 24-7||Enable resolution configuration||Expose the resolution of the mice as a configurable setting.|
|9-10||25-7 / 7-8||Implement LED configuration||Currently not supported; allow the user to configure the mice’s LEDs.|
|10-11||8-8 / 21-8||Start implementing button mappings||Allow the user to remap the mice’s buttons to user-set functions. Start off by moving interfaces around to support this feature and starting on the implementation.|
|11-12||22-8 / 28-8||Finish implementing button mappings||The button mappings will be implemented to completion in these last two weeks.|
The work I did
If you’ve been following along with my weekly progress updates, you’ll know that I didn’t exactly follow this schedule. As I already mentioned, the first sprint was done even before it was supposed to start, while others took longer than planned. I also moved some things around, as you will read below.
The biggest mistake I made in the schedule was to say that the rewrite would support all features supported by the old Piper at the end of week four. I mean, I had a whole sprint dedicated to bring support for, for example, resolutions, so what was I even thinking here?
In any case, this is why I called it a tentative schedule. Luckily Peter wasn’t too strict on my schedule and was open to switching things around as I went, which was a nice gesture that I took as a sign of trust in me getting things done.
Let’s run through each of the sprints, in the order of which I actually executed them.
Creating the mockups
Creating mockups of how the new user interface should look was the basis of my project, together with establishing the feature set. I wasn’t really looking forward to this part, partly because I’d never made real mockups before but also because I wasn’t familiar with Inkscape, the tool I’d be using.
All of this was misplaced, as I quickly got the hang of it and was enjoying the
whole process! My process consisted of looking up concepts in GNOME’s Human
Interface Guidelines and similar
mockups from the GNOME design team that are published on a separate GitHub
account. To save me some
time I mostly copied & pasted elements from their mockups into mine; taking the
standing on the shoulders of giants quite literally!
To discuss the mockups, Peter created a Redesign wiki
page where I’d publish
the mockups together with an explanation, on which Peter and I would
comment. In the process I commented out some comments to keep things clear for
myself, but if there’s interest I can publish them again; all the data is still
there. Data for the Wiki, including history for the mockups, can be found in the
After a few iterations, the mockups were ready according to Peter and I, at
which point I contacted GNOME’s design team on
#gnome-design. To my surprise,
it was Jakub Steiner who replied and gave his
feedback. I won’t discuss that feedback in detail here, because I discussed it
at length in part 4, but it was inspiring to talk with
someone of his status and I learned quite a bit from the conversations we had
Probably the largest change that I made due to Jakub’s feedback was to go from
legend approach (i.e. having an image with labels and having to search
for those labels in an adjacent list to configure that object) to a visual
mapping approach, where a
map of the device is displayed with lines
stretching from an object on the mouse to its corresponding control widget. To
visualize that, we went from
Creating the MouseMap
A central element of the new mockups was thus this
map: it was to be used
in all three configuration pages. As you can guess, there is no standard Gtk
widget to achieve this, so I had to write my own. This widget had to be able to
do the following:
- Position child widgets on arbitrary x- and y-coordinates, relative to their markings in the SVG;
- Draw the SVG in the background of the widget;
- Update said SVG dynamically to highlight an element on the device when its corresponding control widget is hovered by the cursor.
I go in-depth into the creation of this custom container widget in part 4 of my development series. The creation of this widget spanned a whole two and a half weeks, because of 1) my final university exams, 2) optimizing the highlighting of the SVG elements and 3) because, based on another discussion with Jakub, I also added support for left-aligned widgets for devices that have too many buttons to position them all on the right side. This required a significant rewrite of the layout code and required updating all the SVGs we had as well. You can read all about that in part 6.
The resolutions page
With the MouseMap finished, it was time to get started on the real work. The first step was to do the resolutions page, because this had the most support on the ratbag side. This meant that I had to focus only on the Piper side of things; something we didn’t think would be the case for the buttons and the LEDs.
This turned out to be a Good Thing™, because as I soon found out PyGObject does
not support Gtk composite
templates. Such a template
allows you to define composite widgets (widgets composed of other widgets, e.g.
a window containing a few buttons) through
.ui files, which are XML files
generated by Glade. This removes all boilerplate
code to instantiate widgets, set their properties and connect their signals.
Luckily, there is an unofficial
prototype that brings
this to PyGObject. After an experiment and a short discussion with Peter, I
all clear signal to try this in Piper. To this day, I haven’t
noticed a single issue yet and the UI code is significantly cleaner and easier
With this out of the way I started on the resolutions page. I discuss all of that (and the PyGI templates) in part 7.
The LED page
Next up was the LED configuration page. I deviated from the original mockup shown below:
This mockup uses multiple toggle buttons that are pressed to highlight the current mode. One not-obvious interaction was that you’d have to click an already pressed button to change the current mode, for example when you want to change the solid mode’s color. After a few iterations, I got it to what’s there now:
The largest issue that I ran into here was a race condition where we’d set an LED’s mode on DBus, then query that mode on the same bus directly after that without returning to the main loop to give our DBus proxy a chance to update. You can read about this and other issues with ratbagd in part 8.
The buttons page
When making the planning, Peter told me that the buttons would be the most difficult part. He wasn’t wrong, and I think overall I have spent most of my time on the buttons.
The buttons page has been implemented according to the mockups, but these intentionally left the configuration dialog empty. Too much was unclear to me when I was designing the mockups, so I thought it’d be best to experiment and iterate towards a final design once I got to this point.
At first I tried a dialog with a stack page for each kind of mapping. Ratbag exposes four kinds of mappings, one of which is to be removed:
- Button mapping, where a physical button to another, logical button. This way you can for example make the right mouse button emit a left click;
- Key mapping, where a physical button is assigned a series of keys to press and release on the keyboard;
- Special mapping, where a physical button is assigned a special, device-specific function. Libratbag supports several such actions that are supported by most devices;
- Macros, where a physical button is assigned a series of key events (presses with or without releases) and time intervals, to make a button emit a certain key combo with a certain interval in between.
The key mappings are to be removed because they are in essence a subset of macros. The idea is to keep supporting them on the devices, but expose them as a simple macro.
This first iteration looked like this:
However, the different options are quite similar to the user: the general user won’t think of the (technical) difference between a button mapping and a special mapping. Similarly, as explained, key mappings are just a subset of macros. Because of this, I have iterated away from separating these mappings and finalized on a single list with categories instead:
As you can see, we’ve also prevented users from shooting themselves in the foot by remapping the left mouse click. The only valid scenario we could think of for wanting to remap the primary mouse buttons was to switch them around for left-handed users; if you want to remap them for another purpose, please open an issue. If you are interested in the technical details behind this configuration dialog, you can read part 9 and part 13.
I moved the profiles to the end of the summer (as opposed to the beginning according to the schedule) because I wanted to have a solid architecture for the rest of the code that I could then thread the profiles through; if I’d done profiles first I would have been working in the dark, so to speak. I’m happy with this decision and think it worked out well.
Supporting profiles means that all control widgets
- update their state to reflect new values in the new profile. For example, a resolution scale displaying the first resolution’s DPI of 50 in one profile should, when the profile changes to another profile, display the first resolution’s DPI of 100 in that profile (if the first resolution in that profile has a resolution of 100 of course, but bear with me here), and
- apply their changes to the correct resolution object. Using the same example, the scale should now change the DPI of the first resolution in the other profile, and not the first resolution in the original profile.
Each control widget thus needs to be aware of profile changes so that it can
talk with the active profile to do items one and two above. The
different approaches to do this throughout the architecture are discussed in
part 11. Once Peter and I settled on this,
implementing the code wasn’t much work (which I took as a confirmation of both
the approach and shuffling plans around being the right choices).
The welcome- and error screens
Both these features didn’t appear in the original schedule, but were brought forward by Peter as feedback on my mockups. He rightly said that Piper should be able to deal with devices being unplugged or otherwise disconnecting, and having zero or more than one devices connected. For these scenarios, I designed the welcome- and error screens:
I moved these to the end of the summer as, to me, they were the least essential features to implement: having Piper deal with errors when it can’t even configure everything on your device is useless, and owning more than one device (let alone using them simultaneously) is a niche case.
There was, however, another reason to implement these screens. Contrary to initial plans, there are ideas to support a subset of gaming keyboards as well. This wasn’t part of my work, but naturally if I could make Piper open for extension that was the way to go. In essence, this would then just be another such screen.
That means there would be four screens, each providing a different
or perspective into the same application window. I therefore added the concept
perspective, which I define as a certain view into Piper. The
implementation details can be found in part
11, but in short the window has a stack of
perspectives, one of which it activates depending on what happens. Adding
keyboard support is now as simple as adding a keyboard perspective that contains
all the necessary control widgets.
Libratbag and ratbagd changes
Over the course of the summer there have also been substantial changes to
libratbag and ratbagd. The most visible change on the user-side is that
libratbag now supports SVG themes. This was done after my design discussions
with Jakub, where he suggested that Piper uses toned-down SVG’s instead of the
colored SVGs that libratbag shipped until then. However, since Piper is a GUI
for libratbag intended to integrate with the GNOME Desktop Environment, it is
entirely possible that a hypothetical KDE version would want different graphics.
For this reason libratbag thus supports SVG themes, where currently a
gnome theme are present.
A developer and packager-side change is the merging of ratbagd and libratbag into a single project. Anything using libratbag needs to run as root to have sufficient permissions to access the devices, at which point you’ll need some daemon to use it from any user-facing application, whether this be graphical or command-line. At this point, you’re going to need IPC between the daemon and its client(s) and then you might as well merge the two projects.
Specifically to Piper, part of my contributions were adding missing bits to ratbagd (an API for LEDs comes to mind) and fixing things that were broken in one way or another. Peter went ahead and added support for macros specifically because I required those bits in Piper, which otherwise might have been done further down the line instead. Finally, the DBus interface was refactored significantly to make most properties read-write instead of read-only with a setter method. This was done not only to clean up the API but also to solve issues in the bindings where the property would have to be updated in the DBus proxy immediately in order to avoid race conditions.
Here’s a short summary of what Piper can currently do:
As I mentioned in part 11, I finished all large features. In the weeks after that, I also fixed (the most important) issues to improve the user experience and Piper’s robustness. All in all, I achieved what I set out to do and all my code got merged. That doesn’t mean, however, that there’s nothing left to do in both Piper and libratbag.
In the mockups, I showed the ability to add and remove resolutions. The control widgets are there in Piper, but libratbag does not yet have this ability. This means we’ll have to implement this in the coming weeks and hook the widgets up to the corresponding DBus calls to get this to work. Likewise, we also cannot yet change the default resolution through Piper. There’s been a longstanding PR that was recently closed in order to start over. In short, there are issues with propagating this change through ratbagd (more details are in part 8). However, recently I made similar changes to ratbagd so I believe that this shouldn’t be too difficult anymore.
The report rate is currently part of the resolution, but not many devices support this; it should therefore be set per profile instead. In fact, Piper already makes sure to synchronize a change in one resolution’s report rate to all resolutions part of the same profile.
We are also in the process of making commits to the device
asynchronous, so that we
circumvent the timout that would be reached on some devices. Furthermore, the
initial window size is too big,
the device SVGs in the welcome screen need to be
aligned, the mousetrap picture
in the error screen should be
symbolic, the scale for the last
resolution’s DPI can be masked in the
list, the scales need stops and
reasonable ranges, the macro
capture dialog should allow the user to capture
timeouts and finally, the
Off page in the LED dialog needs an empty
placeholder. Libratbag is also in
the process of adding support for named
profiles, after which Piper
will do so too. Libratbag also
needs to report sane
defaults, and finally,
libratbag’s GNOME theme is still lacking
SVGs for five currently
supported devices. I also want to get around to write some tests for Piper,
possibly using dogtail.
Those are all changes for the short term. In the long term, we are rethinking
the user experience yet again. When discussing my mockups with Jakub, he had
the idea that we could introduce a
capture button phase, where you would
press the physical button on your device that you want to configure. This would
then be highlighted in the device SVG, and upon button release the respective
configuration dialog would open. Open questions for this design are:
- How to initiate the capture phase;
- How to present the current mappings and LED modes, and
- How to configure LEDs and resolutions in this new form of interaction.
At GUADEC, Bastien Nocera suggested a slightly different alternative, where we wouldn’t introduce a capture mode but rather make the different elements in the SVG clickable. This would solve questions 1 and 3 of the capture-based approach, but has the problem that it isn’t yet clear how to make the SVG elements clickable. I have some ideas, but I haven’t experimented with this yet.
If you want to get involved, I have labelled some of the open
issues with the labels
help wanted! If you have questions, you can contact us on
#libratbag on Freenode, or on GitHub.
What I’ve learned
I went into this project with some previous open source contributions and experience with previous (university) projects. Yet every project is different and every time I do one I learn something new and improve myself as a developer. It’s hard to compile a list of things I’ve learned, as I picked up most things subconciously. Here’s an attempt, but it is very likely incomplete.
The most important skill I have picked up is working with people in different
"it's the middle of the night in Australia, you'll have to wait
until whot wakes up") and communicating
over IRC; especially being explicit in what you’re saying.
Also new to me was interacting with different communities, where I have for
example had to consult GNOME’s design team to discuss my mockups and the Gtk+
team when I had a Gtk-related issue. It was almost surreal to be talking with
these giants in the open source world, and to be given advice as
them was an experience in and off itself. Thanks to Jakub I have picked up
some pitfalls in UI design, and thanks to the Gtk+ guys I have deepened my
experience with Gtk+’s internals.
This project also exposed me to working with DBus and the Meson build system for the first time. It was also my first Python project, as opposed to single file scripts.
Finally, I experienced debugging across multiple projects; sometimes I noticed a bug in Piper, tracked it to and fixed it in either the bindings, ratbagd or libratbag.
All of my code can be found in the master branch of Piper’s GitHub
repository. The commits between version
0.2.1 and 0.2.900 encompass my summer of code (until
d2a94f833e0aec2bd74dfe94b439afe701212b13), as announced on the
You can also
commits online on GitHub.
I have also had to work on ratbagd and libratbag. The ratbagd project merged
with the libratbag project, but history was preserved when the repositories were
merged. All of my contributions to both projects are thus found in the master
branch on libratbag’s GitHub
repository. My commits are found in
version 0.9.900 (until
6ddba76c93e51c15acf6e59b316532a4332835ac), as announced
You can also
the commits online on GitHub.
I would like to thank my mentor, Peter Hutterer, for offering me this opportunity. It goes without saying (I’m doing it anyway!) that this was an amazing experience that would not have been possible without Peter. I also want to thank Benjamin for stepping in when Peter wasn’t available, the X.Org organisation for having us under their umbrella and the GNOME design and Gtk+ teams for their openness and advice. Finally, thank you Google for hosting the summer of code!