April 23, 2014

Contributions Welcome

If you are interested in coding for GNOME, but haven’t figured out what to work on, this post is for you.

In my last post, I described an experiment that I’m running for the GNOME 3.14 development cycle. The goal is to make it easier of people to contribute to GNOME, by making it easy to find tasks to work on and getting rapid and effective feedback.

Since I wrote that post, I’ve been working with a number of GNOME application maintainers to get their bugs in a state where it is easy for people to contribute. The result is three apps that have a clear set of bugs that contributors can get to work on today.


Music Screenshot

The Music app has been around for a couple of cycles now. It is currently fairly basic, but manages playback fairly well and gives a really nice view of your music collection. This cycle some big new features are planned, like new and improved search and Last.fm integration.

Music has a great development team around it, and is written in Python.

As of today, there are 32 bugs that are available for contributors. Every one of these is in a state where you can get to work on them, and they have all been reviewed by Vadim (the Music maintainer) and myself – so you can be sure that they are things we want.

Some of the bugs are small and address UI niggles, like bug 723144: which aims to give the Artists view a consistent visual style to other sidebars in GNOME. Other bugs are for bigger features, like allowing you to view and play music stored in ownCloud instances. There’s plenty to choose from, and there should be a bug to suit your tastes.

Music is a really promising app, and there are opportunities to play a serious role as it matures.


Documents Screenshot

Documents is one of the original GNOME 3 applications. It has come a long way, and has a lot of cool functionality. I’m not sure that people have made the most of this app in the past, but I think that its utility will become much more obvious with a few changes we have planned, particularly when managing document collections.

Debarshi Ray is currently leading the Documents effort. He’s a great guy and an active maintainer. The application is written in JavaScript.

There are 43 available bugs for Documents. They include functional enhancements that will make the application much more useful. Adding the ability to sort documents in different ways is one of these. Another is making the list view more usable.

We also have some nice UI polish planned, such as using a popover for search options and having a smoother full screen mode.


Contacts Screenshot

Contacts is one of the older GNOME 3 apps. It is written in Vala, and is maintained by Erick Pérez Castellanos, who is awesome.

This is a nice app that can really shine with a bit of work. Right now there are 60 bugs that are available for contributors. Again, there’s lots of small UI issues: in the spirit of Every Detail Matters, these fixes would make a big difference to the overall user experience. For example:

  • Bug 696384 – porting the contact linking suggestion box to GtkActionBar.
  • Bug 699319 – making the app look better when you don’t have any contacts.
  • Bug 703201 – allowing users to select contacts using the right mouse button.

There are also some fun bugs that will hopefully make Contacts a bit more engaging, such as showing maps and status messages from contacts.

What Next

If you want to get involved in GNOME, these applications, and the lists of bugs I’ve pointed to, are the best place to start. The nice thing about these apps is that small fixes will go a really long way, and they all have active maintainers. It would be fantastic if people could help us to make them shine for 3.14.

If other application maintainers want to get involved in this initiative, just get in touch or follow the procedure I described in my last post.

format all comments
As part of our series of trying to solve in-house needs LibreOffice 4.3 will have a "format all comments" feature to change the character properties of all comments in a document.

April 21, 2014

Home entertainment implementations are pretty appalling
I picked up a Panasonic BDT-230 a couple of months ago. Then I discovered that even though it appeared fairly straightforward to make it DVD region free (I have a large pile of PAL region 2 DVDs), the US models refuse to play back PAL content. We live in an era of software-defined functionality. While Panasonic could have designed a separate hardware SKU with a hard block on PAL output, that would seem like unnecessary expense. So, playing with the firmware seemed like a reasonable start.

Panasonic provide a nice download site for firmware updates, so I grabbed the most recent and set to work. Binwalk found a squashfs filesystem, which was a good sign. Less good was the block at the end of the firmware with "RSA" written around it in large letters. The simple approach of hacking the firmware, building a new image and flashing it to the device didn't appear likely to work.

Which left dealing with the installed software. The BDT-230 is based on a Mediatek chipset, and like most (all?) Mediatek systems runs a large binary called "bdpprog" that spawns about eleventy billion threads and does pretty much everything. Runnings strings over that showed, well, rather a lot, but most promisingly included a reference to "/mnt/sda1/vudu/vudu.sh". Other references to /mnt/sda1 made it pretty clear that it was the mount point for USB mass storage. There were a couple of other constraints that had to be satisfied, but soon attempting to run Vudu was actually setting a blank root password and launching telnetd.

/acfg/config_file_global.txt was the next stop. This is a set of tokens and values with useful looking names like "IDX_GB_PTT_COUNTRYCODE". I tried changing the values, but unfortunately made a poor guess - on next reboot, the player had reset itself to DVD region 5, Blu Ray region C and was talking to me in Russian. More inconveniently, the Vudu icon had vanished and I couldn't launch a shell any more.

But where there's one obvious mechanism for running arbitrary code, there's probably another. /usr/local/bin/browser.sh contained the wonderful line:
export LD_PRELOAD=/mnt/sda1/bbb/libSegFault.so
, so then it was just a matter of building a library that hooked open() and launched inetd and dropping that into the right place, and then opening the browser.

This time I set the country code correctly, rebooted and now I can actually watch Monkey Dust again. Hurrah! But, at the same time, concerning. This software has been written without any concern for security, and it listens on the network by default. If it took me this little time to find two entirely independent ways to run arbitrary code on the device, it doesn't seem like a stretch to believe that there are probably other vulnerabilities that can be exploited with less need for physical access.

The depressing part of this is that there's no reason to believe that Panasonic are especially bad here - especially since a large number of vendors are shipping much the same Mediatek code, and so probably have similar (if not identical) issues. The future is made up of network-connected appliances that are using your electricity to mine somebody else's Dogecoin. Our nightmarish dystopia may be stranger than expected.

comment count unavailable comments

April 17, 2014

What is GOM¹
Under that name is a simple idea: making it easier to save, load, update and query objects in an object store.

I'm not the main developer for this piece of code, but contributed a large number of fixes to it, while porting a piece of code to it as a test of the API. Much of the credit for the design of this very useful library goes to Christian Hergert.

The problem

It's possible that you've already implemented a data store inside your application, hiding your complicated SQL queries in a separate file because they contain injection security issues. Or you've used the filesystem as the store and threw away the ability to search particular fields without loading everything in memory first.

Given that SQLite pretty much matches our use case - it offers good search performance, it's a popular thus well-documented project and its files can be manipulated through a number of first-party and third-party tools - wrapping its API to make it easier to use is probably the right solution.

The GOM solution

GOM is a GObject based wrapper around SQLite. It will hide SQL from you, but still allow you to call to it if you have a specific query you want to run. It will also make sure that SQLite queries don't block your main thread, which is pretty useful indeed for UI applications.

For each table, you would have a GObject, a subclass of GomResource, representing a row in that table. Each column is a property on the object. To add a new item to the table, you would simply do:

item = g_object_new (ITEM_TYPE_RESOURCE,
"column1", value1,
"column2", value2, NULL);
gom_resource_save_sync (item, NULL);

We have a number of features which try to make it as easy as possible for application developers to use gom, such as:
  • Automatic table creation for string, string arrays, and number types as well as GDateTime, and transformation support for complex types (say, colours or images).
  • Automatic database version migration, using annotations on the properties ("new in version")
  • Programmatic API for queries, including deferred fetches for results
Currently, the main net gain in terms of lines of code, when porting SQLite, is the verbosity of declaring properties with GObject. That will hopefully be fixed by the GProperty work planned for the next GLib release.

The future

I'm currently working on some missing features to support a port of the grilo bookmarks plugin (support for column REFERENCES).

I will also be making (small) changes to the API to allow changing the backend from SQLite to a another one, such as XML, or a binary format. Obviously the SQL "escape hatches" wouldn't be available with those backends.

Don't hesitate to file bugs if there are any problems with the API, or its documentation, especially with respect to porting from applications already using SQLite directly. Or if there are bugs (surely, no).

Note that JavaScript support isn't ready yet, due to limitations in gjs.

¹: « SQLite don't hurt me, don't hurt me, no more »

April 16, 2014

Printing comments in margins
Because a fellow RedHat employee requested it on Friday, LibreOffice Writer 4.3 will be able to print comments in the margin effectively as they appear on screen, which should take care of the old fdo#36815 feature request. There is now an additional "place comments in margin" option in the print dialog (and writer print options). On screen the comments are placed outside the real page area, so to actually get them onto the paper when printing, the contents of the page needs to be scaled down by approximately 75% of its original size to make space to fit the comments in.

Here's the additional comment place option in the print dialog

Here's some sample pdf output

April 14, 2014

JDLL 2014 report
The 2014 "Journées du Logiciel Libre" took place in Lyon like (almost) every year this past week-end. It's a francophone free software event over 2 days with talks, and plenty of exhibitors from local Free Software organisations. I made the 600 metres trip to the venue, and helped man the GNOME booth with Frédéric Peters and Alexandre Franke's moustache.

Our demo computer was running GNOME 3.12, using Fedora 20 plus the GNOME 3.12 COPR repository which was working pretty well, bar some teething problems.

We kept the great GNOME 3.12 video running in Videos, showcasing the video websites integration, and regularly demo'd new applications to passers-by.

The majority of people we talked to were pretty impressed by the path GNOME has taken since GNOME 3.0 was released: the common design patterns across applications, the iterative nature of the various UI elements, the hardware integration or even the online services integration.

The stand-out changes for users were the Maps application which, though a bit bare bones still, impressed users, and the redesigned Videos.

We also spent time with a couple of users dispelling myths about "lightness" of certain desktop environments or the "heaviness" of GNOME. We're constantly working on reducing resource usage in GNOME, be it sluggishness due to the way certain components work (with the applications binary cache), memory usage (cf. the recent gjs improvements), or battery usage (cf. my wake-up reduction posts). The use of gnome-shell using tablet-grade hardware for desktop machines shows that we can offer a good user experience on hardware that's not top-of-the-line.

Our booth was opposite the ones from our good friends from Ubuntu and Fedora, and we routinely pointed to either of those booths for people that were interested in running the latest GNOME 3.12, whether using the Fedora COPR repository or Ubuntu GNOME.

We found a couple of bugs during demos, and promptly filed them in Bugzilla, or fixed them directly. In the future, we might want to run a stable branch version of GNOME Continuous to get fixes for embarrassing bugs quickly (such as a crash when enabling Zoom in gnome-shell which made an accessibility enthusiast tut at us).

GNOME and Rhône

Until next year in sunny Lyon.

(and thanks Alexandre for the photos in this article!)
Real-world Secure Boot attacks
MITRE gave a presentation on UEFI Secure Boot at SyScan earlier this month. You should read the the presentation and paper, because it's really very good.

It describes a couple of attacks. The first is that some platforms store their Secure Boot policy in a run time UEFI variable. UEFI variables are split into two broad categories - boot time and run time. Boot time variables can only be accessed while in boot services - the moment the bootloader or kernel calls ExitBootServices(), they're inaccessible. Some vendors chose to leave the variable containing firmware settings available during run time, presumably because it makes it easier to implement tools for modifying firmware settings at the OS level. Unfortunately, some vendors left bits of Secure Boot policy in this space. The naive approach would be to simply disable Secure Boot entirely, but that means that the OS would be able to detect that the system wasn't in a secure state[1]. A more subtle approach is to modify the policy, such that the firmware chooses not to verify the signatures on files stored on fixed media. Drop in a new bootloader and victory is ensured.

But that's not a beautiful approach. It depends on the firmware vendor having made that mistake. What if you could just rewrite arbitrary variables, even if they're only supposed to be accessible in boot services? Variables are all stored in flash, connected to the chipset's SPI controller. Allowing arbitrary access to that from the OS would make it straightforward to modify the variables, even if they're boot time-only. So, thankfully, the SPI controller has some control mechanisms. The first is that any attempt to enable the write-access bit will cause a System Management Interrupt, at which point the CPU should trap into System Management Mode and (if the write attempt isn't authorised) flip it back. The second is to disable access from the OS entirely - all writes have to take place in System Management Mode.

The MITRE results show that around 0.03% of modern machines enable the second option. That's unfortunate, but the first option should still be sufficient[2]. Except the first option requires on the SMI actually firing. And, conveniently, Intel's chipsets have a bit that allows you to disable all SMI sources[3], and then have another bit to disable further writes to the first bit. Except 40% of the machines MITRE tested didn't bother setting that lock bit. So you can just disable SMI generation, remove the write-protect bit on the SPI controller and then write to arbitrary variables, including the SecureBoot enable one.

This is, uh, obviously a problem. The good news is that this has been communicated to firmware and system vendors and it should be fixed in the future. The bad news is that a significant proportion of existing systems can probably have their Secure Boot implementation circumvented. This is pretty unsurprisingly - I suggested that the first few generations would be broken back in 2012. Security tends to be an iterative process, and changing a branch of the industry that's historically not had to care into one that forms the root of platform trust is a difficult process. As the MITRE paper says, UEFI Secure Boot will be a genuine improvement in security. It's just going to take us a little while to get to the point where the more obvious flaws have been worked out.

[1] Unless the malware was intelligent enough to hook GetVariable, detect a request for SecureBoot and then give a fake answer, but who would do that?
[2] Impressively, basically everyone enables that.
[3] Great for dealing with bugs caused by YOUR ENTIRE COMPUTER BEING INTERRUPTED BY ARBITRARY VENDOR CODE, except unfortunately it also probably disables chunks of thermal management and stops various other things from working as well.

comment count unavailable comments

April 12, 2014

Westcoast summit, day three

This being last day of the hackfest, people started to disappear in the afternoon. Before that, we had a planning session for Wayland in GNOME 3.14, and came up with a number of concrete tasks and goals. We decided to use bugzilla to track the tasks, so I’ve updated the existing tracker bug.  One of our goals for this cycle is to make Wayland sessions day-to-day usable and keep them that way. Therefore, we want to get Wayland testing into gnome-continuous and aim for having Wayland sessions working well in Fedora rawhide by the end of this month.

April 11, 2014

Westcoast Summit, day two

San Francisco
The second day of the hackfest in San Francisco was punctuated by hour-long presentation of the Endless Mobile team about their mission and product. Part of the presentation was done by video stream from Rio.

Around this highlight, the usual small group discussions and hacking continued all day.

I got a demonstration of california from its author. CaliforniaIt is still young, but looks very nice already. ‘Just enough calendar’, was my impression. Since it uses eds underneath, all my stuff shows up right away.

I also had a good exchange with Daniel from elementary about their widget and theming needs.

One of the bigger discussions in the afternoon was about application  development, how to make it easier and remove obstacles. To kick it off, Christian demonstrated his IDE prototype, which looks really nice. The discussion circulated around the expected topics: better documentation, better tools, distribution and revenue, etc. I don’t think there were any grand conclusions, but as far as GTK+ is concerned, I think we should do a few things:

  • Add more topic- or recipe-based documentation. I’ve made a start on this with my Building Applications talk. Some of the examples that were mentioned include: A preference dialog from scratch, Side-bar best practices.
  • Get serious about considering glade part of the (wider) GTK+ project. The challenge for 3.14: Have all widgets supported in glade at the day of the release.
  • Adopt gtkparasite. We’ve let gtkparasite languish in random forks on github for too long. This is really useful technology, and should be fully integrated into glade and other ides.

Throughout the day, I fixed several GTK+ bugs that were pointed out by Daniel and Jim. I also continued to port applications to  DBus-activatable.

April 10, 2014

Westcoast Summit, day one

The first day of the Westcoast summit is now over. We enjoyed the excellent hospitality of the Endless Mobile crew.

Before coming here, I set up a GNOME goal for converting applications to DBus activation, and with the help of Cosimo, I converted a few applications. This will make applications fit nicely into the coming session setup with systemd –user and kdbus.

One of the longer discussions we had today was around this topic. Lennart explained how things might work: The systemd user instance creates the user bus when a user logs in, and every application that is started via DBus activation is actually spawned by the systemd user instance, and thus gets its own cgroup, and D-Bus policy, which will be used to limit the access that sandboxed applications have on the bus.

Kay demonstrated systemd –user and kdbus launching a full GNOME session on his laptop. Most things just work already. Only a few things will need fixes:

  •  PolicyKit currently uses the session of the caller in determining which authentication agent to talk to, etc.  Applications that are activated on the user bus are not technically part of a session (as in: a descendent of the gnome-session process), so this won’t work anymore.
  • gdm keeps a process around for the lifetime of the session that provides the pam stack used e.g. when unlocking the session, and without a clear session association, this may not work in the same way anymore.

We can work on these issues while we are waiting for kdbus to find its way in the kernel.

After discussing this, and many other topics, we ended the day in a fantastic Chinese restaurant, followed by beer.

April 09, 2014

Enabling Participation

With 3.12 out the door, it’s time to think about what we want to be doing for 3.14. I have a long list of design projects that I want to work on for the next release, but I also want to spend some time on how the GNOME project is working and how we can improve it.

One of my reoccurring interests is how we, as a project, can ensure that each module is in a healthy state. We want modules to have active developer teams around them, and we want it to be easy for people to get involved – not just because it is good for our software, but also because openness is an important part of our mission.

This interest in helping people to contribute isn’t just reserved for new, inexperienced contributors. There are experienced coders out there who are interested in GNOME but haven’t found a way in. Even members of the GNOME project itself don’t always know how to contribute to different apps and modules.

Making it easy for people to contribute takes work. Simply putting the code online is not enough: we need to provide potential contributors with the information they need, and we need to give them feedback and support as they work. We need to enable them to participate by creating the conditions in which it is easy to contribute.

There are a number of reasons why people sometimes find it difficult to participate, and we aren’t going to solve them all overnight. Thinking about this topic, though, one of the main reasons why people struggle to contribute is that it is difficult, if not impossible, to know which tasks to work on. In my opinion, the way that we manage bugs in GNOME is a major factor here [1].

In GNOME we often don’t do a good job of indicating which bugs we want to be fixed, and we don’t spell out what needs doing to fix them. This leaves potential participants with no way to contribute. A long list of unconfirmed bugs, often with no guidance on what needs to be done to resolve them, is a brick wall. It can be the end of the story for potential contributors.

This is the issue I want to address. Interestingly, though, improvements in this area can also help with other aspects of project management: if we are clear about which bugs we want fixing, it prompts forward planning, and it stimulates discussions about which issues should be prioritised over others. It also creates opportunities for conversations about the direction of modules, which can help to include contributors in taking on leadership roles.

To make it easier for people to participate in GNOME, our bugs need to be organised so that they give clear guidance about where contributions are needed. This requires that we have a different process for how bugs are processed and categorised. I’ve spent some time talking about this with various maintainers, as well as members of the Bug Squad, and I have come up with a set of procedures that could work.

This procedure won’t be for everyone, and I am not proposing that GNOME adopts it on a project-wide basis. What I am suggesting is that a small number of applications try it out for the 3.14 cycle as an experiment (I’m focusing on applications because I think they are the best place for new contributors to get involved). If it has a positive impact, then we can think about involving more applications in the following cycle. If it doesn’t, then that’s fine: we’ll have learnt something.

How it could work

The main goal for the bug management procedure I’ve come up with is to remove uncertainty from bug reports. This is something that we are bad at: thousands of ambiguous bugs sit in Bugzilla, which contributors have little chance of knowing what to do with. The procedure uses this schema for bug reports that are on the path to being fixed:

  • UNCONFIRMED: new bugs that haven’t been validated. These reports are uncertain – they might not correspond to real issues.
  • NEW: reports that have been validated, and therefore correspond to actual issues.
  • NEW with “needs_design” whiteboard: valid bugs that are waiting for a design to be produced in order to fix them.
  • NEW with “available” whiteboard: NEW bugs that are ready to be fixed. These bugs should have an identified solution which has been stated, and they shouldn’t be blocked by other bugs.

In this schema, “available” bugs are the reports that you point potential contributors to. They are items of outstanding work that can be tackled today. You can link to the list of these bugs from your wiki page, blog posts, or IRC topic. Having an available status is also helpful to maintainers: it helps them see which tasks are pending.

This bug schema also makes UNCONFIRMED a meaningful category. These are the bugs that triagers and maintainers need to process in order to give them a definite status. A large or growing number of UNCONFIRMED bugs in your product is a sign that you need to do a sweep through to clean them up.

In this approach to bug management, you need to regularly review bugs that don’t fall into the “available” category, in order to try and resolve them, either by identifying a solution (and therefore making them available) or by closing them as WONTFIX or NOTABUG or so on. It’s a fairly aggressive approach, in which you have to routinely say what is desirable and what isn’t, but in doing so you open the doors to new contributors who know what the project wants and how they can help.

What’s going to happen next

My plan for 3.14 is to trial this bug management approach with a small number of applications. I’m already working with Debarshi on the Documents bugs, which he has blogged about, and I’m also looking at Contacts with Erick. Once one or two more applications are involved, I’ll post an update on how to get involved.

We want to see if this approach helps to attract new contributors and to manage projects more effectively. We also want to see if the bug classification schema needs to be improved in any way. Towards the end of the cycle, I’ll be talking to people to see how they think it went: was it useful? Do any changes need to be made? Would you recommend this approach for other modules?

If you’re an application maintainer and are interested in this initiative, I’d love to hear from you. Also, I really, really want to hear what people think about the classification schema and the process around it.

[1] I realise that, in focusing on bug reports, I am restricting this to code contributions. In doing so, i don’t mean to suggest that patches are the only way to contribute to GNOME. This is merely a way of enabling participation in one – obviously important, but by no means exclusive – area.

April 07, 2014

LGM 2014

I have just returned home from this year’s Libre Graphics Meeting, which was held in Leipzig, Germany. As always, it was a great event, which is somewhat unique in bringing together art and design practitioners with programmers and engineers.

LGM is a good opportunity to meet with friends in other projects, especially graphics applications. I was really happy to be able to spend time with members of the GIMP and Inkscape projects, and hope that this will lead to closer ties and working relationships in the future.

GNOME and Libre Graphics have a lot in common. GNOME design uses free tools developed by the Libre Graphics community, and we practice open design in the way that many of those at LGM also do. I think that GNOME also helps to bring people into the Libre Graphics community, and it was nice to see a good contingent of people from GNOME at LGM this year. This is something that Jakub and I talked about in our presentation on the last conference day.

The Libre Graphics community is creative and passionate, and I always feel refreshed after LGM. Big thanks to the organisers for putting on another great conference.

April 03, 2014

Mozilla and leadership
A post I wrote back in 2012 got linked from a couple of the discussions relating to Brendan Eich being appointed Mozilla CEO. The tldr version is "If members of your community doesn't trust their leader socially, the leader's technical competence is irrelevant". That seems to have played out here.

In terms of background[1]: in 2008, Brendan donated money to the campaign for Proposition 8, a Californian constitutional amendment that expressly defined marriage as being between one man and one woman[2]. Both before and after that he had donated money to a variety of politicians who shared many political positions, including the definition of marriage as being between one man and one woman[3].

Mozilla is an interesting organisation. It consists of the for-profit Mozilla Corporation, which is wholly owned by the non-profit Mozilla Foundation. The Corporation's bylaws require it to work to further the Foundation's goals, and any profit is reinvested in Mozilla. Mozilla developers are employed by the Corporation rather than the Foundation, and as such the CEO is responsible for ensuring that those developers are able to achieve those goals.

The Mozilla Manifesto discusses individual liberty in the context of use of the internet, not in a wider social context. Brendan's appointment was very much in line with the explicit aims of both the Foundation and the Corporation - whatever his views on marriage equality, nobody has seriously argued about his commitment to improving internet freedom. So, from that perspective, he should have been a fine choice.

But that ignores the effect on the wider community. People don't attach themselves to communities merely because of explicitly stated goals - they do so because they feel that the community is aligned with their overall aims. The Mozilla community is one of the most diverse in free software, at least in part because Mozilla's stated goals and behaviour are fairly inspirational. People who identify themselves with other movements backing individual liberties are likely to identify with Mozilla. So, unsurprisingly, there's a large number of socially progressive individuals (LGBT or otherwise) in the Mozilla community, both inside and outside the Corporation.

A CEO who's donated money to strip rights[4] from a set of humans will not be trusted by many who believe that all humans should have those rights. It's not just limited to individuals directly affected by his actions - if someone's shown that they're willing to strip rights from another minority for political or religious reasons, what's to stop them attempting to do the same to you? Even if you personally feel safe, do you trust someone who's willing to do that to your friends? In a community that's made up of many who are either LGBT or identify themselves as allies, that loss of trust is inevitably going to cause community discomfort.

The first role of a leader should be to manage that. Instead, in the first few days of Brendan's leadership, we heard nothing of substance - at best, an apology for pain being caused rather than an apology for the act that caused the pain. And then there was an interview which demonstrated remarkable tone deafness. He made no attempt to alleviate the concerns of the community. There were repeated non-sequiturs about Indonesia. It sounded like he had no idea at all why the community that he was now leading was unhappy.

And, today, he resigned. It's easy to get into hypotheticals - could he have compromised his principles for the sake of Mozilla? Would an initial discussion of the distinction between the goals of members of the Mozilla community and the goals of Mozilla itself have made this more palatable? If the board had known this would happen, would they have made the same choice - and if they didn't know, why not?

But that's not the real point. The point is that the community didn't trust Brendan, and Brendan chose to leave rather than do further harm to the community. Trustworthy leadership is important. Communities should reflect on whether their leadership reflects not only their beliefs, but the beliefs of those that they would like to join the community. Fail to do so and you'll drive them away instead.

[1] For people who've been living under a rock
[2] Proposition 8 itself was a response to an ongoing court case that, at the point of Proposition 8 being proposed, appeared likely to support the overturning of Proposition 22, an earlier Californian ballot measure that legally (rather than constitutionally) defined marriage as being between one man and one woman. Proposition 22 was overturned, and for a few months before Proposition 8 passed, gay marriage was legal in California.
[3] http://www.theguardian.com/technology/2014/apr/02/controversial-mozilla-ceo-made-donations-right-wing-candidates-brendan-eich
[4] Brendan made a donation on October 25th, 2008. This postdates the overturning of Proposition 22, and as such gay marriage was legal in California at the time of this donation. Donating to Proposition 8 at that point was not about supporting the status quo, it was about changing the constitution to forbid something that courts had found was protected by the state constitution.

comment count unavailable comments
XDG Summit: Day #4
During the wee hours of the morning, David Faure posted a new mime applications specification which will allow to setup per-desktop default applications, for example, watching films in GNOME Videos in GNOME, but DragonPlayer in KDE. Up until now, this was implemented differently in at least KDE and GNOME, even to the point that GTK+ applications would use the GNOME default when running on a KDE desktop, and vice-versa.

This is made possible using XDG_CURRENT_DESKTOP as implemented in gdm by Lars. This environment variable will also allow implementing a more flexible OnlyShowIn and NotShowIn desktop entry fields (especially for desktops like Unity implemented on top of GNOME, or GNOME Classic implemented on top of GNOME) and desktop-specific GSettings/dconf configurations (again, very useful for GNOME Classic). The environment variable supports applying custom configuration in sequence (first GNOME Classic then GNOME in that example).

Today, Ryan and David discussed the desktop file cache, making it faster to access desktop file data without hitting scattered files. The partial implementation used a custom structure, but, after many kdbus discussions earlier in the week, Ryan came up with a format based on serialised GVariant, the same format as kdbus messages (but implementable without implementing a full GVariant parser).

We also spent quite a bit of time writing out requirements for a filesystem notification to support some of the unloved desktop use cases. Those use cases are currently not supported by either inotify and fanotify.

That will end our face-to-face meeting. Ryan and David led a Lunch'n'Learn in the SUSE offices to engineers excited about better application integration in the desktops irrespective of toolkits.

Many thanks to SUSE for the accommodation as well as hosting the meeting in sunny Nürnberg. Special thanks to Ludwig Nussel for the morning biscuits :)

April 02, 2014

Freedesktop Hackfest: Day #3
Wednesday, Mittwoch. Half of the hackfest has now passed, and we've started to move onto other discussion items that were on our to-do list.

We discussed icon theme related simplifications, especially for application developers and system integrators. As those changes would extend into bundle implementation, being pretty close to an exploded-tree bundle, we chose to postpone this discussion so that the full solution includes things like .service/.desktop merges, and Intents/Implements desktop keys.

David Herrman helped me out with testing some Bluetooth hardware (which might have involved me trying to make Mario Strikers Charged work in a Wii emulator on my laptop ;)

We also discussed a full-fledged shared inhibition API, and we agreed that the best thing to do would be to come up with an API to implement at the desktop level. The desktop could then proxy that information to other session- and/or system-level implementations.

David Faure spent quite a bit of time cleaning up after my bad copy/pasted build system for the idle inhibit spec (I copied a Makefile with "-novalidate" as an option, and the XML file was full of typos and errors). He also fixed the KDE implementation of the idle inhibit to match the spec.

Finally, I spent a little bit of time getting kdbus working on my machine, as this seemed to trigger the infamous "hidden cursor bug" without fail on every boot. Currently wondering why gnome-shell isn't sending any events at all before doing a VT switch and back.

Due to the Lufthansa strike, and the long journey times, tomorrow is going to be the last day of the hackfest for most us.
why I suck at finishing stuff , or how I learned to stop working and love DisplayPort MST
DisplayPort 1.2 Multi-stream Transport is a feature that allows daisy chaining of DP devices that support MST into all kinds of wonderful networks of devices. Its been on the TODO list for many developers for a while, but the hw never quite materialised near a developer.

At the start of the week my local Red Hat IT guy asked me if I knew anything about DP MST, it turns out the Lenovo T440s and T540s docks have started to use DP MST, so they have one DP port to the dock, and then dock has a DP->VGA, DP->DVI/DP, DP->HDMI/DP ports on it all using MST. So when they bought some of these laptops and plugged in two monitors to the dock, it fellback to using SST mode and only showed one image. This is not optimal, I'd call it a bug :)

Now I have a damaged in transit T440s (the display panel is in pieces) with a dock, and have spent a couple of days with DP 1.2 spec in one hand (monitor), and a lot of my hair in the other. DP MST has a network topology discovery process that is build on sideband msgs send over the auxch which is used in normal DP to read/write a bunch of registers on the plugged in device. You then can send auxch msgs over the sideband msgs over auxch to read/write registers on other devices in the hierarchy!

Today I achieved my first goal of correctly encoding the topology discovery message and getting a response from the dock:
[ 2909.990743] link address reply: 4
[ 2909.990745] port 0: input 1, pdt: 1, pn: 0
[ 2909.990746] port 1: input 0, pdt: 4, pn: 1
[ 2909.990747] port 2: input 0, pdt: 0, pn: 2
[ 2909.990748] port 3: input 0, pdt: 4, pn: 3

There are a lot more steps to take before I can produce anything, along with dealing with the fact that KMS doesn't handle dynamic connectors so well, should make for a fun tangent away from the job I should be doing which is finishing virgil.

I've ordered another DP MST hub that I can plug into AMD and nvidia gpus that should prove useful later, also for doing deeper topologies, and producing loops.

Also some 4k monitors using DP MST as they are really two panels, but I don't have one of them, so unless one appears I'm mostly going to concentrate on the Lenovo docks for now.

April 01, 2014

Freedesktop Summit: Day #2
Today, Ryan carried on with writing the updated specification for startup notification.

David Faure managed to get Freedesktop.org specs updated on the website (thanks to Vincent Untz for some chmod'ing), and removed a number of unneeded items in the desktop file specification, with help from Jérôme.

I fixed a number of small bugs in shared-mime-info, as well as preparing for an 8-hour train ride.

Lars experimented with technics to achieve a high score at 2048, as well as discussing various specifications, such as the possible addition of an  XDG_CURRENT_DESKTOP envvar. That last suggestion descended into a full-room eye-rolling session, usually when xdg-open code was shown.
XDG Hackfest: Day #1
I'm in Nürnberg this week for the Freedesktop Hackfest, aka the XDG Summit, aka the XDG Hackfest aka... :)

We started today with discussions about desktop actions, and how to implement them, such as whether showing specific "Edit" or "Share" sub-menus and how to implement them. We decided that that could be implemented through specific desktop keys which a file manager could use. This wasn't thought to be generally useful to require a specification for now.

The morning is stretching to discuss "splash screens". A desktop implementor running on low-end hardware is interested in having a placeholder window show up as soon as possible, in some cases even before the application has linked and the toolkit is available. This discussion is descending into slightly edge cases, such as text editors launching either new windows or new tabs depending on a number of variables.

Specific implementation options were discussed after a nice burrito lunch. We've decided that the existing X11 startup notification would be ported to D-Bus, using signals instead of X messages. Most desktop shells would support both versions for a while. Wayland clients that want startup notification would be required to use the D-Bus version of the specification. In parallel, we would start passing workspace information along with the DESKTOP_STARTUP_ID envvar/platform data.

Jérôme, David and I cleared up a few bugs in shared-mime-info towards the end of the day.

Many thanks to SUSE for the organisation, and accommodation sponsorship.

Update: Fixed a typo

March 28, 2014

Viewing the Xorg.log with journalctl

Those running Fedora Rawhide or GNOME 3.12 may have noticed that there is no Xorg.log file anymore. This is intentional, gdm now starts the X server so that it writes the log to the systemd journal. Update 29 Mar 2014: The X server itself has no capabilities for logging to the jornal yet, but no changes to the X server were needed anyway. gdm merely starts the server with a /dev/null logfile and redirects stdin/stderr to the journal.

Thus, to get the log file use journalctl, not vim, cat, less, notepad or whatever your $PAGER was before.

This leaves us with the following commands.

journalctl -e /usr/bin/Xorg
Which would conveniently show something like this:

Mar 25 10:48:41 yabbi Xorg[5438]: (II) UnloadModule: "wacom"
Mar 25 10:48:41 yabbi Xorg[5438]: (II) evdev: Lenovo Optical USB Mouse: Close
Mar 25 10:48:41 yabbi Xorg[5438]: (II) UnloadModule: "evdev"
Mar 25 10:48:41 yabbi Xorg[5438]: (II) evdev: Integrated Camera: Close
Mar 25 10:48:41 yabbi Xorg[5438]: (II) UnloadModule: "evdev"
Mar 25 10:48:41 yabbi Xorg[5438]: (II) evdev: Sleep Button: Close
Mar 25 10:48:41 yabbi Xorg[5438]: (II) UnloadModule: "evdev"
Mar 25 10:48:41 yabbi Xorg[5438]: (II) evdev: Video Bus: Close
Mar 25 10:48:41 yabbi Xorg[5438]: (II) UnloadModule: "evdev"
Mar 25 10:48:41 yabbi Xorg[5438]: (II) evdev: Power Button: Close
Mar 25 10:48:41 yabbi Xorg[5438]: (II) UnloadModule: "evdev"
Mar 25 10:48:41 yabbi Xorg[5438]: (EE) Server terminated successfully (0). Closing log file.
The -e toggle jumps to the end and only shows 1000 lines, but that's usually enough. journalctl has a bunch more options described in the journalctl man page. Note the PID in square brackets though. You can easily limit the output to just that PID, which makes it ideal to attach to the log to a bug report.

journalctl /usr/bin/Xorg _PID=5438
Previously the server kept only a single backup log file around, so if you restarted twice after a crash, the log was gone. With the journal it's now easy to extract the log file from that crash five restarts ago. It's almost like the future is already here.

700 conversion target reached

Pushed through the 700 widget conversion barrier this week. The FindDialogs wiki page has helped quite a bit. There remains quite a few blanks in there though, so help still needed to identify the route to launch those unknown dialogs and tabpages via the UI.

Of the newly identified ones I'm a little amused by RID_OFAPAGE_SMARTTAG_OPTIONS which (we think) can only be seen when a smart tags extension is installed. Is there a known working extension out there that enables this to be shown when installed ?

Trivia titbit: we have converted eight different passwords dialogs. And there remains one unconverted one whose route to being launched is currently unknown

Current conversion stats are:
706 .ui files currently exist
There are 67 unconverted dialogs
There are 68 unconverted tabpages
An estimated additional 135 .ui are required
We are 83% of the way through.

March 26, 2014

My GNOME 3.12 in numbers
1 new GNOME Videos, 1 updated Bluetooth panel, 2 new thumbnailers, 9 grilo sources, and 1 major UPower rework.

I'm obviously very attached to the GNOME Videos UI changes, the first major UI rework in its 12-year existence.

GNOME Videos watching itself

GNOME Software on Ubuntu (II)

So I did a bit more hacking on PackageKit, appstream-glib and gnome-software last night. We’ve now got screenshots from Debian (which are not very good) and long application descriptions from the package descriptions (which are also not very good). It works well enough now, although you now need PackageKit from master as well as appstream-glib and gnome-software.




This is my last day of hacking on the Ubuntu version, but I’m hopeful other people can take what I’ve done and continue to polish the application so it works as well as it does on Fedora. Tasks left to do include:

  • Get aptcc to honour the DOWNLOADED filter flag so we can show applications in the ‘Updates’ pane
  • Get aptcc to respect the APPLICATION filter to speed up getting the installed list by an order of magnitude
  • Get gnome-software (or appstream-glib) to use the system stock icons rather than the shitty ones shipped in the app-install-data package
  • Find out a way to load localized names and descriptions from the app-install-data gettext archive and add support to appstream-glib. You’ll likely need to call dgettext(), bindtextdomain() and bind_textdomain_codeset()
  • Find out a way how to populate the ‘quality’ stars in gnome-software, which might actually mean adding more data to the app-install desktop files. This is kind of data we need.
  • Find out why aptcc sometimes includes the package summary in the licence detail position
  • Improve the package details to human readable code to save bullet points and convert to a UTF-8 dot
  • Get the systemd offline-updates code working, which is completely untested
  • Find out why aptcc seems to use a SHA1 hash for the repo name (e.g. pkcon repo-list)
  • Find out why aptcc does not set the data part of the package-id to be prefixed with installed: for installed packages

If you can help with any of this, please grab me on #PackageKit on freenode.

March 25, 2014

GNOME Software on Ubuntu

After an afternoon of hacking on appstream-glib, I can show the fruits of my labours:


This needs gnome-software and appstream-glib from git master (or gnome-apps-3.14 in jhbuild) and you need to manually run PackageKit with the aptcc backend (--enable-aptcc).


It all kinda works with the data from /usr/share/app-install/*, but the icons are ugly as they are included in all kinds of sizes and formats, and also there’s no long descriptions except for the two (!) installed applications new enough to ship local AppData files.Also, rendering all those svgz files is muuuuch slower than a pre-processed png file like we ship with AppStream. The installed view also seems not to work. Only the C locale is present too, as I’ve not worked out how to get all the translations from an external gettext file in appstream-glib. I’d love to know how the Ubuntu software center gets long descriptions and screenshots also. But it kinda works. Thanks.

Looking Forward to 3.12


I usually do a review of what is coming in the run up to a release. However, there have been so many blog posts about 3.12 already that I don’t feel I need to go over individual features. If you haven’t read Planet GNOME in a while, now is a good time to check it out: there’s lots of great content on there right now.

It is worth looking at what the individual features in 3.12 add up to though. A release is more than the sum of its parts, and this is especially true of 3.12.

One important thing you will see in 3.12 is that, more and more, GNOME’s core applications are coming together. Videos will look and behave like a GNOME 3 app: it will let you browse your content, and it offers a modern, streamlined viewing experience. gedit has also had the GNOME 3 treatment. It has retained all its existing functionality, but in a more compact interface [1]. Many of the other apps have also matured of course, Software and Web in particular.

The other big news for 3.12 is that a number of significant gaps have been filled in. For a long time people have wanted to be able to manually organise their apps: now they can with the new apps folder feature. We’ve also added functionality to make installing sofware updates easier and more convenient, as well as the addition of wired networking controls to the system status area.

There are also major developments in the developer space, with the new notifications API, new GTK+ widgets, new capabilities for launching processes, and improved documentation. I think that 3.12 is probably our strongest for developers in a long time.

Finally, and for me perhaps most significantly, 3.12 looks set to be the best quality release so far. Signs of ongoing improvements are everywhere. There are performance gains for startup and (hopefully) memory usage, the theme and animations in the shell has been refined in quite a few subtle ways, high-resolution display support has been extended, and a great many bugs have been fixed. As each release comes and goes, GNOME 3 gets better and better, and 3.12 is no exception.

There’s plenty more that I could mention about this release, of course, and the release notes will provide full details, but what is important is the progress that GNOME is making. 3.12 feels like another significant upgrade, and is another release where it feels like things are coming together more and more.

[1] The other day I did a quick comparison, and found that the chrome in the new version is around 60 pixels shorter than before. That’s an impressive space saving, and makes the app much more focused on what you are editing.

March 24, 2014

CyPS/2 Cypress Trackpad and firmware-based button emulation

For a longer story of this issue, please read Adam Williamson's post. The below is the gist of it, mostly for archival purposes.

The Dell XPS13 (not the current Haswell generation, the ones before) ships with a touchpad that identifies as "CyPS/2 Cypress Trackpad". This touchpad is, by the looks of it, a ClickPad and identifies itself as such by announcing the INPUT_PROP_BUTTONPAD evdev property. In the X.Org synaptics driver we enable a couple of features for those touchpads, the most visible of which is the software-emulated button areas. If you have your finger on the bottom left and click, it's a left click, on the bottom right it's a right click. The size and location of these areas are configurable in the driver but also trigger a couple of other behaviours, such as extra filters to avoid erroneous pointer movements.

The Cypress touchpad is different: it does the button emulation in firmware. A normal clickpad will give you a finger position and a BTN_LEFT event on click. The Cypress touchpads will simply send a BTN_LEFT or BTN_RIGHT event depending where the finger is located, but no finger position. Only once you move beyond some threshold will the touchpad send a finger position. This caused a number of issues when using the touchpad.

Fixing this is relatively simple: we merely need to tell the Cypress that it isn't a clickpad and hope that doesn't cause it some existential crisis. The proper way to do this is this kernel patch here by Hans de Goede. Until that is in your kernel, you can override it with a xorg.conf snippet:

Section "InputClass"+Section "InputClass"
Identifier "Disable clickpad for CyPS/2 Cypress Trackpad"
MatchProduct "CyPS/2 Cypress Trackpad"
MatchDriver "synaptics"
Option "ClickPad" "off"
This snippet is safe to ship as a distribution-wide quirk.

Keeping gnome-shell approachable

One aspect that I always found very appealing about gnome-shell is that you could just go in /usr/share/gnome-shell/js, make a few changes, hit Alt-F2 r and try them out. This was a very low barrier to entry – no development environment needed, no days of jhbuilding dependencies. This is at least part of the explanation why shell extensions exist in large numbers. Sure, you still have to make yourself familiar with the internal and external APIs that are used in gnome-shell, and if you are unlucky, then Alt-F2 r will show you not your cool hack, but the fail whale.

I was a bit sad to see that we’ve lost a bit of this newcomer friendliness in 3.12, when all the JavaScript and css files were wrapped up in resources and included in the gnome-shell binary (to be exact, they are located in /usr/lib64/gnome-shell/libgnome-shell.so, not in /usr/bin/gnome-shell itself). Why was this done ? I guess having everything in one file and not spread across the file system makes gnome-shell start up a tiny bit faster (although I’m not sure if anybody has measured this).

But how do I now try gnome-shell changes quickly ? Let see…

For some background, gnome-shell is using the GResource mechanism for embedding the js files in the binary. Under the covers, this puts the files in a separate ELF section and makes their content available in a filesystem-like structure. The application itself can get at the resources e.g.  by constructing GFiles from resource:// URIs, like this:

file = g_file_new_for_uri ("resource:///org/gnome/software/gtk-style.css");

To access the embedded resources from the outside, you can use the gresource utility that is shipped with GLib. It can list the resources and also extract their content. Sadly, there is currently no easy way to replace existing resources with newer versions, since that requires recreating the ELF section and relinking the application.

gnome-shell has quite a few resources; the list looks like this:

gresource list /usr/lib64/gnome-shell/libgnome-shell.so

Here is how I used the gresource tool to get back to gnome-shell tweakability. Since the gresource commandline is not very versatile, I wrote this little script:

#! /bin/sh


cd $HOME/gnome-shell-js

mkdir -p ui/components ui/status misc perf extensionPrefs gdm

for r in `gresource list $gs`; do
  gresource extract $gs $r > ${r/#\/org\/gnome\/shell/.}

After running this script, all the js files that make up the gnome-shell UI can be found in $HOME/gnome-shell. Now I can point gnome-shell at these files with the (undocumented) GNOME_SHELL_JS variable:

GNOME_SHELL_JS=$HOME/gnome-shell-js gnome-shell

And – voila! – gnome-shell is as hackable as it always was.

GNOME Software 3.12.0 Released!

Today I released gnome-software 3.12.0 — with a number of new features and a huge number of bugfixes:


I think I’ve found something interesting to install — notice the auto-generated star rating which tells me how integrated the application is with my environment (i.e. is it available in my language) and if the application is being updated upstream. Those thumbnails look inviting:


We can continue browsing while the application installs — also notice the ‘tick’ — this will allow me to create and modify application folders in gnome-shell so I can put the game wherever I like:


The updates tab looks a little sad; there’s no update metadata on rawhide for my F20 GNOME 3.12 COPR, but this looks a lot more impressive on F20 or the yet-to-be-released F21. At the moment we’re using the AppData metadata in place of update descriptions there. Yet another reason to ship an AppData file.


We can now safely remove sources, which means removing the applications and addons that we installed from them. We don’t want applications sitting around on our computer not being updated and causing dependency problems in the future.


Development in master is now open, and we’ve already merged several large patches. The move to libappstream-glib is a nice speed boost, and other more user-visible features are planned. We also need some documentation; if you’re interested please let us know!

What free software means to me
I was awarded the Free Software Foundation Award for the Advancement of Free Software this weekend[1]. I'd been given some forewarning, and I spent a bunch of that time thinking about how free software had influenced my life. It turns out that it's a lot.

I spent most of the 90s growing up in an environment that was rather more interested in cattle than in computers, and had very little internet access during that time. My entire knowledge of the wider free software community came from a couple of CDs that contained a copy of the jargon file, the source code to the entire GNU project and an early copy of the m68k Linux kernel.

But that was enough. Before I'd even got to university, I knew what free software was. I'd had the opportunity to teach myself how an operating system actually worked. I'd seen the benefits of being able to modify software and share those modifications with others. I met other people with the same interests. I ended up with a job writing free software and collaborating with others on integrating it with upstream code. And, from there, I became more and more involved with a wider range of free software communities, finding an increasing number of opportunities to help make changes that benefited both me and others.

Without free software I'd have started years later. I'd have lost the opportunity to collaborate with people spread over the entire world. My first job would have looked very different, as would my entire career since then. Without free software, almost everything I've achieved in my adult life would have been impossible.

To me, free software means I've lived a significantly better life than would otherwise have been the case. But more than that, it means doing what I can to make sure that other people have the same opportunities. I am here because of the work of others. The most rewarding part of my continued involvement is the knowledge that I am part of a countless number of people working to make sure that others can tell the same story in future.

[1] I'd link to the actual press release, but it contains possibly the worst photograph of me in the entire history of the universe

comment count unavailable comments

March 21, 2014

Dialogs in GTK+ 3.12

Dialogs are getting a face-lift in GTK+ 3.12.

Most of the work on this was done by Jon McCann, I’ve only helped out here and there. The main visible change is the switch to client-side decorations and headerbars.

Here are some examples of GTK+’s built-in complex dialogs with their new look:

File Chooser

Color Chooser

The application chooser has had a bit more work done – we have a search button in the header bar, which makes a search bar appear when clicked.

Application Chooser

The most common dialogs in applications are preference dialogs. gedit shows how these can look with client-side decorations.


And then there are simple prompts.  GTK+ has the GtkMessageDialog class for these.  Their new look is maybe the boldest part of this refresh.


Of course, GTK+ is used in many places, and client-side decorations may look foreign in some of them. Together with these changes, we introduced a dialogs-use-header setting. Built-in dialogs will fall back to a more traditional appearance if it is not set:

Traditional File Chooser

Traditional Color Chooser

Note that some of the details in my screenshots, such as the blue color for suggested actions, depend on the theme. What you see here is how dialogs will appear with the Adwaita theme in GNOME 3.12.

The GNOME HIG contains a lot of helpful advice on how to make best use of dialogs in your application.

March 20, 2014

Stacking xorg.conf.d snippets

We've had xorg.conf.d snippets for quite a while now (released with X Server 1.8 in April 2010). Many people use them as a single configuration that's spread across multiple files, but they also can be merged and rely on each other. The order they are applied is the lexical sort order of the directory, so I recommend always prefixing them with a number. But even within a single snippet, you can rely on the stacking. Let me give you an example:

$ cat /usr/share/X11/xorg.conf.d/10-evdev.conf
Section "InputClass"
Identifier "evdev touchpad catchall"
MatchIsTouchpad "on"
MatchDevicePath "/dev/input/event*"
Driver "evdev"

$ cat /usr/share/X11/xorg.conf.d/50-synaptics.conf
Section "InputClass"
Identifier "touchpad catchall"
MatchIsTouchpad "on"
MatchDevicePath "/dev/input/event*"
Driver "synaptics"
The first one applies the evdev driver to anything that looks like a touchpad. The second one, sorted later, overwrites this setting with the synaptics driver. Now, the second file also has a couple of other options:

Section "InputClass"
Identifier "Default clickpad buttons"
MatchDriver "synaptics"
Option "SoftButtonAreas" "50% 0 82% 0 0 0 0 0"
This option says "if the device is assigned the synaptics driver, merge the SoftButtonAreas option". And we have another one, from ages ago (which isn't actually needed anymore):

Section "InputClass"
Identifier "Disable clickpad buttons on Apple touchpads"
MatchProduct "Apple Wireless Trackpad"
MatchDriver "synaptics"
Option "ClickPad" "on"
This adds on top of the other two, provided your device has the name and is assigned the synaptics driver.

The takeaway of this is that when you have your own xorg.conf.d snippet, there is almost never a need for you to write more than a 5-line snippet merging exactly that one or two options you want. Let the system take care of the rest.

March 19, 2014

Fedora rawhide should have GL 3.3 on radeonsi supported hardware
So to enable OpenGL 3.3 on radeonsi required some patches backported to llvm 3.4, I managed to get some time to do this, and rebuilt mesa against the new llvm, so if you have an AMD GPU that is supported by radeonsi you should now see GL 3.3.

For F20 this isn't an option as backporting llvm is a bit tricky there, though I'm considering doing a copr that has a private llvm build in it, it might screw up some apps but for most use cases it might be fine.
AppStream Logs, False Positives and You

Quite a few people have asked me how the AppStream distro metadata is actually generated for thier app. The actual extraction process isn’t trivial, and on Fedora we also do things like supply missing AppData files for some key apps, and replacing some upstream screenshots on others.

In order to make this more transparent, I’m going to be uploading the logs of each generation run. If you’ve got a few minutes I’d appreciate you finding your application there and checking for any warnings or errors. The directory names are actually Fedora package names, but usually it’s either 1:1 or fairly predictable.

If you’ve got a application that’s being blacklisted when it shouldn’t be, or a GUI application that’s in Fedora but not in that list then please send me email or grab me on IRC. The rules for inclusion are here. Thanks.

Lenovo T440 touchpad button configuration

Update March 19 2014: this post is outdated, please read X.Org synaptics support for the Lenovo T440, T540, X240, Helix, Yoga, X1 Carbon instead.

The T440 has a rather unusual touchpad with the buttons painted on top of the touchpad rather than the bottom. In addition, the separate set of buttons for the trackstick have gone the way of the dodo. Moving the software-emulated buttons up on the touchpad is obviously quite important for trackstick users but it throws up a bunch of problems. There are some limitations with the current synaptics X.Org driver: we can only have one region each designated for the right and the middle button. The rest of the touchpad is a left button click. In the case of the T440, the default Windows config has a right button up the top and another one at the bottom of the touchpad. An ASCII-art of that would look like this:

| |
| |
| |
| |
| |
| |
We simply can't do that at the moment, best we can is split the touchpad so that the whole right side is a right-click and a strip in the middle that is a middle click. So the best we can do is:

I'm working on a solution for the proper config, but for now you'll have to be content with this.

The easiest approach for local configuration is a new InputClass section in the form:

Section "InputClass"
Identifier "t440 top buttons"
MatchDriver "synaptics"
# right btn|middle btn
Option "SoftButtonAreas" "60% 0 0 0 40% 60% 0 0"
Drop that into /etc/X11/xorg.conf.d/99-t440-synaptics.conf and you're good to go.

The problem is finding a generic solution to this that we can ship in a distribution. That requires a two-step progress. The touchpads look the same as all others, the only differentiator we have is the DMI information on the box. We can't check that in the xorg.conf snippets yet (Daniel Martin is working on a MatchDMI tag, but it won't happen until server 1.16). For now, we need a udev rule to help the xserver.

ACTION!="add|change", GOTO="touchpad_quirks_end"
KERNEL!="event*", GOTO="touchpad_quirks_end"
ENV{ID_INPUT_TOUCHPAD}!="1", GOTO="touchpad_quirks_end"

ATTR{[dmi/id]product_version}=="*T440*", \

If our product matches T440, we tag the touchpad, and that tag is something we can match against. Our revised InputClass section now looks like this:

Section "InputClass"
Identifier "t440 top buttons"
MatchDriver "synaptics"
MatchTag "touchpad_softbutton_top"
Option "SoftButtonAreas" "60% 0 0 0 40% 60% 0 0"
I've pushed this configuration into Fedora now (rawhide, F20, F19), let's see what the feedback is. Having the whole right-side of the touchpad work as right button may cause a few issues so this is one change I may have to revert in the future.

X.Org synaptics support for the Lenovo T440, T540, X240, Helix, Yoga, X1 Carbon

This is a follow-up to my post from December Lenovo T440 touchpad button configuration. Except this time the support is real, or at least close to being finished. Since I am now seeing more and more hacks to get around all this I figured it's time for some info from the horse's mouth.

[update] I forgot to mention: synaptics 1.8 will have all these, the first snapshot is available here

Lenovo's newest series of laptops have a rather unusual touchpad. The trackstick does not have a set of physical buttons anymore. Instead, the top part of the touchpad serves as software-emulated buttons. In addition, the usual ClickPad-style software buttons are to be emulated on the bottom edge of the touchpad. An ASCII-art of that would look like this:

| |
| |
| |
| |
| |
| |
Getting this to work required a fair bit of effort, patches to synaptics, the X server and the kernel and a fair bit of trial-and-error. Kudos for getting all this sorted goes to Hans the Goede, Benjamin Tissoires, Chandler Paul and Matthew Garrett. And in the process of fixing this we also fixed a bunch of other issues that have been plaguing clickpads for a while.

The first piece in the puzzle was to add a second software button area to the synaptics driver. Option "SecondarySoftButtonAreas" now allows a configuration in the same manner as the existing one (i.e. right and middle button). Any click in that software button area won't move the cursor, so the buttons will behave just like physical buttons. Of course, we expect that button area to work out of the box, so we now ship configuration files that detect the touchpad and apply that automatically. This requires an xserver fix and a kernel/udev fix, more on that later.

The second piece in the puzzle was to work around the touchpad firmware. The touchpads speak two protocols, RMI4 over SMBus and PS/2. Windows uses RMI4, Linux still uses PS/2. Apparently the firmware never got tested for PS/2 so the touchpad gives us bogus data for its axis ranges. A kernel fix for this is in the pipe.

Finally, the touchpad needed to be actually usable. So a bunch of patches that tweak the clickpad behaviours were merged in. If a finger is set down inside a software button area, finger movement does no longer affect the cursor. This stops the ever-so-slight but annoying movements when you execute a physical click on the touchpad. Also, there is a short timeout after a click to avoid cursor movement when the user just presses and releases the button. The timeout is short enough that if you do a click-and-hold for drag-and-drop, the cursor will move as expected. If a touch started outside a software button area, we can now use the whole touchpad for movement. And finally, a few fixes to avoid erroneous click events - we'd sometimes get the software button wrong if the event sequence is off.

Another change changed the behaviour of the touchpad when it is disabled through the "Synaptics Off" property. If you use syndaemon to disable the touchpad while typing, the buttons now work even when the touchpad is disabled. If you don't like touchpads at all and prefer to use the trackstick only, use Option "TouchpadOff" "1". This will disable everything but physical clicks on the touchpad.

On that note I'd also like to mention another touchpad bug that was fixed in the recent weeks: plenty of users reported synaptics having a finger stuck after suspend/resume or sometimes even after logging in. This was an elusive bug and finally tracked down to a mishandling of SYN_DROPPED events in synaptics 1.7 and libevdev. I won't provide a fix for synaptics 1.7 but we've fixed libevdev - please use synaptics 1.8 RC1 or later and libevdev 1.1 RC1 or later.

The kernel fix is more complicated. The problem is that while the touchpads all have a unique PNPID, that ID is not easily accessible because it shows up on a device that is not a parent of the device we see in the driver. Hence MatchPnPID doesn't apply. Matthew Garrett had a preliminary patch for this but it turned out to break some other use-case so we're back at square one at the moment. For now, I'm using a couple of udev rules together with the MatchTag configuration:

ATTR{[dmi/id]product_version}=="*T540*", ENV{ID_INPUT.tags}="T540"
and with the matching xorg.conf snippet:

Section "InputClass"
Identifier "Lenovo T540 trackstick software button buttons"
MatchTag "T540"
Option "SecondarySoftButtonAreas" "3363 0 0 2280 2717 3362 0 2280"
Oh btw, do you note the magic numbers here? This really should be in percent of the touchpad but we don't have the kernel patch. So for now I just ship the absolute numbers in the Fedora packages and glare pointedly at everyone who thinks that's a permanent solution.

Fedora users: everything is being built in rawhide and I have a F20 Copr that I'll keep pushing to once I get the various other bits for F20 sorted.

March 18, 2014

Announcing Appstream-Glib

For a few years now Appstream and AppData adoption has been growing. We’ve got client applications like GNOME Software consuming the XML files, and we’ve got several implementations of metadata generators for a few distros now. We’ve also got validation tools we’re encouraging upstream applications to use.

The upshot of this was the same code was being duplicated across 3 different projects of mine, all with different namespaces and slightly different defined names. Untangling this mess took a good chunk of last week, and I’ve factored out 2759 lines of code from gnome-software, 4241 lines from createrepo_as, and the slightly less impressive 178 lines from appdata-tools.

The new library has a simple homepage, and so far a single release. I’d encourage people to check this out and provide early comments, as as soon as gnome-software branches for 3-12 I’m going to switch it to using this. I’m also planning on switching createrepo_as and and appdata-tools for the next releases too so things like jhbuild modulesets need to be updated and tested by somebody.

Appstream-Glib 0.1.0 provides just enough API to make sense for a first release, but I’m going to be continuing to abstract out useful functionality from the other projects to share even more code. I’ve spent a few long nights profiling the XML parsing code, and I’m pleased to say the load time of gnome-software is 160ms faster with this new library, and createrepo_as completes the metadata generation 4 minutes faster. Comments, suggestions and patches very welcome. There’s a Fedora package linked from the package review bug if you’d rather test that. Thanks.

New in GTK+ 3.12: popovers

In the third part of my recap of the GNOME 3.12 development cycle, I’ll talk about some of the changes in GTK+ that I have been involved in.

Popovers have already been discussed quite a bit. Most of the popover implementation has been done by Carlos Garnacho, generalizing his earlier work on touch selection popups that has been in GTK+ since 3.8.

One of the nice things about popovers is that they are just normal containers – you can put any widget into them, and keyboard navigation and input works like everywhere else. This is a marked contrast to menus, which are very specialized. Attempts to put entries, sliders or buttons into menus usually end badly.

I recently acquired a laptop with a touchscreen, so I can say with confidence that popovers are also much easier to use with touch than menus.

Here are some examples of popovers in gedit:


My own contribution to popovers has been to convert GtkVolumeButton to use a popover:


I also made it possible to populate popovers from a GMenuModel, giving you instant popover menus:


Popovers are still very new, so their adoption in GNOME 3.12 will be somewhat limited. But we are in the lucky position that we already have quite good design guidance for popovers, so this will probably change soon.

March 17, 2014

App folder configuration

Continuing my 3.12 recap, this post is about gnome-software. I’ve done much less work on it this cycle than the previous one. All the heavy lifting has been done by Richard. The one feature that I did add to gnome-software this cycle is app folder configuration.

GNOME SoftwareGNOME has been moving away from hierarchical menus for applications. It is problematic for many reasons. One problem is the need for a global, hierarchical classification (‘categories’) – the world is just not that simple, and applications don’t always fit into these predefined categories. Another problem is that menus don’t really scale beyond a single level of submenus or beyond more than 10-15 items per menu.   Not to mention that menus are hard to use on touch devices.

The transition from menus and categories to a scrollable grid for applications was pretty much complete in 3.10. But there is still some need for grouping of related applications, and this is where app folders come in. In 3.10, we provided predefined folders for ‘Utilities’ and ‘Sundry’.

In 3.12,  we are adding an easy way for users to create  their own folders.  We chose to add this feature in the application that always shows you a list of all installed applications anyway, gnome -software.

Installed appsThe alternative would be do implement this directly in the shell overview, but that would be pretty complicated, requiring either a selection mode or complex drag-and-drop, so we decided not to do this (at least for now).

Once you’ve selected the apps you want to group, you can select an existing folder in the ‘Add to Folder’ dialog:

Add to FolderOr you can click on the ‘+’ button to create a new folder:

New FolderOnce you have done this, the new dialog will show up in the GNOME shell overview:

OverviewAnd that’s all there is to this feature!

If you are not using gnome-software, the app folder configuration is also available via gsettings.  It is using relocatable schemas, so the required gsettings command-line looks a little different from the usual, and may be worth showing. First,

$ gsettings get org.gnome.desktop.app-folders folder-children
['Utilities', 'Sundry', 'Feet']

will show you the list of defined app folders. Then,

$ gsettings get org.gnome.desktop.app-folders.folder:/org/gnome/desktop/app-folders/folders/Feet/ apps
['dconf-editor.desktop', 'd-feet.desktop', 'devhelp.desktop']

will list the apps that are in the folder named ‘Feet’. The folder schema has a few more settings that you can explore or change with similar commands.

March 16, 2014

The new gnome-initial-setup

As the development cycle for 3.12 is winding down, I want to take the time to look back at some of the things I’ve worked on this cycle.

First, gnome-initial-setup has received a design overhaul that I’ve implemented together with Jasper. The pages now look a lot more uniform and polished. We use headerbars and we are consistently using list boxes for selections.

The first few pages are about language, region and input.




The network page is skipped if a we have a connection.


The timezone map is now properly sized.


Online accounts have been moved earlier.

Online Accounts

This lets us pick up avatar and name for the account page from a configured online account, which is something we’ve wanted to do all along:


The on-screen keyboard works during initial setup now:

On-Screen Keyboard

Setting a password has been separated from the account creation:


And thats all!


March 12, 2014

Dealing with Apple ACPI issues
I wrote about Thunderbolt on Apple hardware a while ago. Since then Andreas Noever has somehow managed to write a working Thunderbolt stack, which awesome! But there was still the problem I mentioned of the device not appearing unless you passed acpi_osi="Darwin" on the kernel command line, and a further problem that if you suspended and then resumed it vanished again.

The ACPI _OSI interface is a mechanism for the firmware to determine the OS that the system is running. It turns out that this works fine for operating systems that export fairly static interfaces (Windows, which adds a new _OSI per release) and poorly for operating systems that don't even guarantee any kind of interface stability in security releases (Linux, which claimed to be "Linux" regardless of version until we turned that off). OS X claims to be Darwin and nothing else. As I mentioned before, claiming to be Darwin in addition to Windows was enough to get the Thunderbolt hardware to stay alive after boot, but it wasn't enough to get it powered up again after suspend.

It turns out that there's two sections of ACPI code where this Mac checks _OSI. The first is something like:

if (_OSI("Darwin")) Store 0x2710 OSYS; else if(_OSI("Windows 2009") Store 0x7D9 OSYS; else…

ie, if the OS claims to be Darwin, all other strings are ignored. This is called from \_SB._INI(), which is the first ACPI method the kernel executes. The check for whether to power down the Thunderbolt controller occurs after this and then works correctly.

The second version is less helpful. It's more like:

if (_OSI("Darwin")) Store 0x2710 OSYS; if (_OSI("Windows 2009")) Store 0x7D9 OSYS; if…

ie, if the OS claims to be both Darwin and Windows 2009 (which Linux will if you pass acpi_osi="Darwin"), the OSYS variable gets set to the Windows 2009 value. This version gets called during PCI initialisation, and once it's run all the other Thunderbolt ACPI calls stop doing anything and the controller gets powered down after suspend/resume. That can be fixed easily enough by special casing Darwin. If the platform requests Darwin before anything else, we'll just stop claiming to be Windows.

Phew. Working Thunderbolt! (Well, almost - _OSC fails and so we disable PCIe hotplug, but that's easy to work around). But boo, no working battery. Apple do something very strange with their ACPI battery interface. If you're running anything that doesn't claim to be Darwin, Apple expose an ACPI Control Method battery. Control Method interfaces abstract the complexity away into either ACPI bytecode or system management traps - the OS simply calls an ACPI method, magic happens and it gets an answer back. If you claim to be Darwin, Apple remove that interface and instead expose the raw ACPI Smart Battery System interface. This provides an i2c bus over which the OS must then speak the Smart Battery System protocol, allowing it to directly communicate with the battery.

Linux has support for this, but it seems that this wasn't working so well and hadn't been for years. Loading the driver resulted in modprobe hanging until a timeout occurred, and most accesses to the battery would (a) take forever and (b) probably fail. It also had the nasty habit of breaking suspend and resume, which was unfortunate since getting Thunderbolt working over suspend and resume was the whole point of this exercise.

So. I modified the sbs driver to dump every command it sent over the i2c bus and every response it got. Pretty quickly I found that the failing operation was a write - specifically, a write used to select which battery should be connected to the bus. Interestingly, Apple implemented their Control Method interface by just using ACPI bytecode to speak the SBS protocol. Looking at the code in question showed that they never issued any writes, and the battery worked fine anyway. So why were we writing? SBS provides a command to tell you the current state of the battery subsystem, including which battery (out of a maximum of 4) is currently selected. Unsurprisingly, calling this showed that the battery we wanted to talk to was already selected. We then asked the SBS manager to select it anyway, and the manager promptly fell off the bus and stopped talking to us. In keeping with the maxim of "If hardware complains when we do something, and if we don't really need to do that, don't do that", this makes it work.

Working Thunderbolt and working battery. We're even getting close to getting switchable GPU support working reasonably, which is probably just going to involve rewriting the entirety of fbcon or something similarly amusing.

comment count unavailable comments

March 10, 2014

Using git - the next level

There's a million tutorials out there how to learn git. This isn't one of them. I'm going to assume that you learned git a while ago, you've been using it a bit and you're generally familiar with its principles. I'm going to show is a couple of things that improved my workflow. Chances are, it will improve yours too. This isn't a tutorial though. I'm just pointing you in the direction of things, you'll have to learn how to use them yourself.

Use tig

Seriously. Don't tell me you use gitk or git log is good enough for you. Use tig. tig is to git log what mutt is to mail(1). It has been the source of the biggest efficiency increase for me. Screenshots don't do it justice because the selling point is that it is interactive. But anyway, here are some official screenshots: tig blame shows you the file and the commits, you just need to select the line, hit enter and you see the actual commit. The main view by default shows you tags, branch names, remote branch names, etc. So not only do you immediately know which branch you're on, you will see local branches that have been merged, tags that have been applied, etc. It gives you an awareness that git log doesn't. Do yourself a favour, install it, use it for a day or two and I'm pretty sure you won't go back.

tig also supports custom configurations. Here is my $HOME/.tigrc:

bind generic X !git cherry-pick -x %(commit)
bind generic C !git cherry-pick %(commit)
bind generic R !git revert %(commit)
bind generic E !git format-patch -1 %(commit)
bind generic 0 !git checkout %(commit)
bind generic 9 !git checkout %(commit)~
bind generic A !git commit --amend -s
bind generic S !git show %(commit)
So with a couple of key strokes I can cherry-pick, export patches, revert, check out a single tree, etc. Especially cherry-picking is extremely efficient: check out the target branch, run "tig master", then simply select each commit, it "C" or "X" and done.

Use branches

Anytime it takes you more than 5 minutes to fix an issue, create a new branch. I'm getting torn between multiple things all the time. I may spend a day or two on one bug, then it's back to another, unrelated issue. With the review requirements on some projects I may have multiple patches waiting for feedback, but I can't push them yet. Hence - a branch for each feature/bugfix. master is reserved for patches that can be pushed immediately.

This approach becomes particularly useful for fixes that may need some extra refacturing. You start on a feature-based branch, but halfway through realise you need a few extra patches to refactor things. Those are easy to review so you send them out to gather reviews, then cherry-pick them to master and push. Back to your feature branch, rebase and you're done - you've managed two separate streams of fixes without interference. And most importantly, you got rid of a few patches that you'd otherwise have to carry in your feature branch.

Of course, it takes a while to get used this and it takes discipline. It took me a few times before I really managed to always work like this but the general rule for me is now: if I'm hacking on the master branch, something is off. Remember: there's no real limit to how many branches you can create - just make sure you clean them up when you're done to keep things easy for your brain.

Use the branch names to help you. You can rename branches (git branch -m), so I tend to name anything that's a bigger rewrite with "wip/somefeature" whereas normal bug fixes go on branches with normal names. And because I rebase local feature branches it doesn't matter what I name them anyway, the branches are deleted once I merge them. Branches where I do care about the branch history (i.e. those I pull them into master with a merge commit) I rename before pulling to get rid of the "wip" prefix.

Use branch descriptions

Hands up if you have a "devel" branch from 4 months ago. Hands up if you still remember what the purpose of that branch was. Right, I didn't think so. git branch --edit-description fires up an editor and lets you add a description for the branch. Sometimes a single sentence is enough to refresh your memory. Most importantly: when you task-switch to a different feature, edit the description to note where you left off, what the plan was, etc. This reduces the time to get back to work. git config branch.<branchname>.description shows you the description for the matching branch.

I even have a git hook to nag me when I check out a branch without a description. Note that branch descriptions are local only, they are not pushed to the remote.

Amend and rebase until the cows come home

The general rule: what is committed, doesn't get lost. At least not easily, it is still in the git reflog. So commit when you think you're done. Then review, test, add, and git commit --amend. That typo you made in line 4 - edit and amend. I have shell aliases for amend, rbs (git rebase -i) and rbc (git rebase --continue), and almost every commit goes through at least 3 amends (usually one because I missed something, one for that typo, one for commit log message editing). Importantly: it doesn't matter how often you amend. Really. This is local only, no-one cares. The important thing is that you get to a good patch set, not that you get there with one commit.

git commit --amend only modifies the last commit, to go back and edit the past, you need to rebase. So, you need to

Learn how to rebase

Not just the normal git rebase, the tutorials cover that. Make sure you know how to use git rebase --interactive. Make sure you know how to change the ordering of a commit, how to delete commits, how to abort a rebase. Make sure you know how to squash two commits together and what the difference is between squash and fixup. I'm not going to write a tutorial on that, because you can find the documentation is easy enough to find. Simply take this as a hint that the time you spend learning how to rebase pays off. Also, you may find git squash interesting.

And remember: even if a rebase goes bad, the previous state is still in the reflog. Which brings me to:

Learn how to use the reflog

The git reflog is the list of changes in reverse chronological order of how they were applied to the repository, regardless what branch you're on. So HEAD@{0} is always "whatever we have now", HEAD@{1} is always "the repository before the last command". This doesn't just mean commits, it remembers any change. So if you switch from branch A to branch B, commit something, then switch to branch C, HEAD@{3} is A. git reflog helpfully annotates everything with the type, so you know what actually happened. So for example, if you accidentally dropped a patch during a rebase, you can look at the reflog, figure out when the rebase started. Then you either reset to that commit, or you just tig it and cherry-pick the missing commits back onto the current branch. Create yourself up with a test git repository and learn how to do exactly that now, it'll save you some time in the future.

Note that the reflog is local only. And remember, if it hasn't been committed, it's not in the reflog.

Use a git push hook

Repeat after me: echo make > .git/hooks/pre-push. And no more embarrassment for pushing patches that don't compile. I've made that mistake too many times, so now I even use my own git patch-set command that will run a hook for me when I'm generating a patch set to send to a list. You might want to make the hooks executable btw.

The simple things in life
I've spent a while trying to make GPU switching work more reliably on Apple hardware, and got to the point where my Retina MBP now comes up with X running on Intel regardless of what the firmware wanted to do (test patches here. But in the process I'd introduced an additional GPU switch which added another layer of flicker to the boot process. I spent some time staring at driver code and poking registers trying to figure out how I could let i915 probe EDID from the panel without switching the display over and made no progress at all.

And then I realised that nouveau already has all the information that i915 wants, and maybe we could just have the Switcheroo code hand that over instead of forcing i915 to probe again. Sigh.

comment count unavailable comments

March 05, 2014

I'm finding it a fairly slow process to find the last 175 dialogs/tabpages in the LibreOffice UI. I have the full list of identifiers, and it's generally not a difficult process to find how to activate any individual dialog from the UI but in the aggregate it takes time and breaks the development flow if I have to stop/start to find the next one to convert.

So, you can help :-) This FindDialogs wiki page contains the list of identifiers and some hints as to how to find them in the source and in the help documentation. That's generally sufficient to determine what dialog they describe and how to launch it.

Filling out those remaining blank "how to activate" entries will speed up the conversion process.

March 04, 2014

Spin Class

via Roadsidepictures (CC BY-NC 2.0)

We’ve recently binged on making NetworkManager work better in more places, mostly enterprise and virtualization related.  But one thing we’ve wanted to do for a long time was make things more modular.  And that got landed this week via the dev-plugins branch, which makes ATM, Bluetooth, and WWAN into shared libraries loaded optionally at startup.

Distro packagers can now create separate NetworkManager-atm, NetworkManager-bluetooth, and NetworkManager-wwan packages, each with their own dependencies, while the NetworkManager core goes forward a slimmer, smaller, more efficient version of its previous self.  If you’re installing NetworkManager into minimal environments, you can just ignore or remove these plugins and revel in your newfound minimalism.

The core NM binary is now about 15% smaller, and there’s a corresponding 7.5% RSS reduction at runtime when no plugins are loaded.  What’s next?  Possibly WiFi, which would save about 6 – 8% of the core binary size.

March 02, 2014

First Class Goto

Oort is an experimental programming language I have been working on, on and off (mostly off), since 2007. It is a statically typed, object-oriented, imperative language, where classes, functions and methods can be nested arbitrarily, and where functions and methods are full closures, ie., they can be stored in variables and returned from functions. The control structures are the usual ones: if, for, while, do, goto, etc.

It also has an unusual feature: goto labels are first class.

What does it mean for labels to be first class? It means two things: (1) they are lexically scoped so that they are visible from inside nested functions. This makes it possible to jump from any point in the program to any other location that is visible from that point, even if that location is in another function. And (2) labels can be used as values: They can be passed to and returned from functions and methods, and they can be stored in data structures.

As a simple example, consider a data structure with a “foreach” method that takes a callback function and calls it for every item in the data structure. In Oort this might look like this:

table: array[person_t];

table.foreach (fn (p: person_t) -> void {
        print p.name;
        print p.age;

A note about syntax. In Oort, anonymous functions are defined like this:

fn (<arguments>) -> <return type> {

and variables and arguments are declared like this:

<name>: <type>

so the code above defines an anonymous function that prints the name and the age of person and passes that function to the foreach method of the table.

What if we want to stop the iteration? You could have the callback return true to stop, or you could have it throw an exception. However, both methods are a little clumsy: The first because the return value might be useful for other purposes, the second because stopping the iteration isn’t really an exceptional situation.

With lexically scoped labels there is a direct solution – just use goto to jump out of the callback:

  table.foreach (fn (p: person_t) -> void {
          print p.name;
          print p.age;

          if (p.age > 50)
              goto done;


Note what’s going on here: Once we find a person older than 50, we jump out of the anonymous callback and back into the enclosing function. The git tree has a running example.

Call/cc in terms of goto
In Scheme and some other languages there is a feature called call/cc, which is famous for being both powerful and mind-bending. What it does is, it takes the concept of “where we are in the program” and packages it up as a function. This function, called the continuation, is then passed to another, user-defined, function. If the user-defined function calls the continuation, the program will resume from the point where call/cc was invoked. The mind-bending part is that a continuation can be stored in data structures and called multiple times, which means the call/cc invocation can in effect return more than once.

Lexically scoped labels are at least as expressive as call/cc, because if you have them, you can write call/cc as a function:

call_cc (callback: fn (k: fn()->void)) -> void
    callback (fn() -> void { 
        goto current_continuation;


Let’s see what’s going on here. A function called call_cc() is defined:

call_cc (...) -> void

This function takes another function as argument:

callback: fn (...) -> void

And that function takes the continuation as an argument:

k: fn()->void

The body of call/cc calls the callback:

callback (...);

passing an anonymous function (the continuation):

    fn() -> void {
        goto current_continuation;


that just jumps to the point where call_cc returns. So when callback decides to invoke the continuation, execution will resume at the point where call_cc was invoked. Since there is nothing stopping callback from storing the continuation in a data structure or from invoking it multiple times, we have the full call/cc semantics.

Cooperative thread system
One of the examples on the Wikipedia page about call/cc is a cooperative thread system. With the call_cc function above, we could directly translate the Wikipedia code into Oort, but using the second aspect of the first-class-ness of labels – that they can be stored directly in data structures – makes it possible to write a more straightforward version:

run_list: list[label] = new list[label]();

thread_fork (child: fn() -> void)
    run_list.append (me);
    goto run_list.pop_head();

    run_list.append (me);
    goto run_list.pop_head ();

    if (!run_list.is_empty())
        goto run_list.pop_head();

The run_list variable is a list of labels containing the current positions of all the active threads. The keyword label in Oort is simply a type specifier similar to string.

To create a new thread, thread_fork first saves the position of the current thread on the list, and then it calls the child function. Similarly, thread_yield yields to another thread by saving the position of the current thread and jumping to the first label on the list. Exiting a thread consists of jumping to the first thread if there is one, and exiting the process if there isn’t.

The code above doesn’t actually run because the current Oort implementation doesn’t support genericity, but here is a somewhat uglier version that actually runs, while still demonstrating the principle.

February 28, 2014

Getting the details right

I’ve recently explained how GTK+ does quite a few things for you out of the box.

  • Theming ? You got it.
  • Accessibility ? You’re covered.
  • Keynav ? Sure.

But as it turns out, default implementations can’t always provide the optimum. To go from an application that works ok to one that is gets the details just right, some fine-tuning may be required.

Today, I want to take a look at a few examples of such fine-tuning for keyboard navigation, in particular around lists. I hope this also shows how you can learn tricks and borrow well-working code from other applications. If you ask yourself

How did they do this, and why does my app not do this ?’

look at the source! We all do it, and don’t feel bad about it.

Why lists ? They come in all sizes and shapes, from straightforward and simple to interactive and complex. It is no wonder that GtkTreeView with its supporting classes has around 40000 lines of code.

Connected lists

My first example is about segmented lists of controls. These have become more common in gnome-control-center panels. Here is the accessibility panel:

Accessibility panelWhen you use the arrow keys to navigate among the buttons, the default behavior of GTK+ is to stop when you come to the edge of the container. But in the situation above, we all would expect the focus to jump from the first list to the second.

Thankfully, GTK+ emits a ::keynav-failed signal when you use the arrow keys to go beyond the end of a container, and we can use this to our advantage:

static gboolean
keynav_failed (GtkWidget        *list,
               GtkDirectionType  direction,
               CcUaPanel        *self)
  CcUaPanelPrivate *priv = self->priv;
  GList *item, *sections;
  gdouble value, lower, upper, page;

  /* Find the list in the list of GtkListBoxes */
  if (direction == GTK_DIR_DOWN)
    sections = priv->sections;
    sections = priv->sections_reverse;

  item = g_list_find (sections, list);
  g_assert (item);
  if (item->next)
      gtk_widget_child_focus (GTK_WIDGET (item->next->data), direction);
      return TRUE;

We use this signal handler on every list:

g_signal_connect (list, "keynav-failed",
                  G_CALLBACK (keynav_failed),

And thats all! Here is a quick video of this in action (I’m repeatedly using the Down arrow key):

<video class="wp-video-shortcode" controls="controls" height="266" id="video-946-1" preload="metadata" width="474"><source src="http://blogs.gnome.org/mclasen/files/2014/02/Screencast-from-02-27-2014-081023-PM.webm" type="video/webm">http://blogs.gnome.org/mclasen/files/2014/02/Screencast-from-02-27-2014-081023-PM.webm</video>

If you watch closely, you’ll notice another fine point of this example – we scroll the panel to keep the focus location visible. This functionality is built into GTK+’s container widgets, and we activate it by setting a focus adjustment on the box that contains all the lists:

adjustment = gtk_scrolled_window_get_vadjustment (GTK_SCROLLED_WINDOW (panel));
gtk_container_set_focus_vadjustment (GTK_CONTAINER (content), adjustment);

These code examples were taken from cc-ua-panel.c in gnome-control-center.

The same trick is also used in the gnome-control-center overview to allow arrow keys to move between several icon views.

Tabbing out

GTK+ uses the Tab key to connect all active UI elements into a focus chain.  The default behavior of GtkListBox is to put all rows into the focus chain – that makes a lot of sense for the previous example where each row contains controls such as buttons, or brings up a dialog when activated.

Sometimes, it is more natural to treat a list as a single item in the focus chain, so that the next Tab key press takes you out of the list. The list content will still be keyboard-accessible with the arrow keys.

A sidebar like in gnome-logs is an example where this makes sense:

A sidebar listTo achieve this behavior, we can override the focus vfunc of our GtkListBox subclass:

widget_class->focus = gl_category_list_focus;

with a function that special-cases Tab key presses:

static gboolean
gl_category_list_focus (GtkWidget *listbox, 
                        GtkDirectionType direction)
  switch (direction)
      if (gtk_container_get_focus_child (GTK_CONTAINER (listbox)))
          /* Force tab events which jump to
           * another child to jump out of the
           * category list.
          return FALSE;

This code example was adapted from gl-categorylist.c

A back button

The last example does not involve lists, but a simple Back button. For example, gnome-software has one:

A back buttonYou will probably add a mnemonic to the button label, so it can be activated using the Alt-B shortcut. But your users will also expect the Back key on their keyboard to work, and many will probably try Alt-Left as well, since that is what they use in their web browser.

Key events in GTK+ bubble up from the focus widget, and until they are definitively handled by one of the intermediate containers, they eventually reach the toplevel GtkWindow. Therefore, to make the Back key work regardless where the focus currently is, we can override the key_press vfunc of the window:

static gboolean
window_key_press_event (GtkWidget *win,
                        GdkEventKey *event,
                        GsShell *shell)
  state = event->state & 
       gtk_accelerator_get_default_mod_mask ();
  is_rtl = gtk_widget_get_direction (button) == GTK_TEXT_DIR_RTL;

  if ((!is_rtl && state == GDK_MOD1_MASK &&
        event->keyval == GDK_KEY_Left) ||
      (is_rtl && state == GDK_MOD1_MASK && 
        event->keyval == GDK_KEY_Right) ||
      event->keyval == GDK_KEY_Back)
      gtk_widget_activate (button);
      return GDK_EVENT_STOP;


If you pay attention to detail, you’ll notice that we use Alt-Left or Alt-Right, depending on the text direction — your Hebrew-speaking users will appreciate.

This code example was taken from gs-shell.c

February 26, 2014

Popovers & You

GTK+ has been getting some really nice new features in recent times. Over the past few releases the list new of widgets has come to include things like ListBoxes and FlowBoxes, stacks and stack switchers, revealers and header bars. Now, in the upcoming 3.12 release, there will be another new widget: popovers. This is something that those of us who work on GNOME design have wanted for a while, and it’s exciting to finally have them.

Of course, once you have a new interface widget, you need to know what to do with it, so I thought I’d write a bit about how to design with popovers. If you’re an application developer or designer and you’re not sure what popovers are for or how to use them, then this post is for you.

What is a popover, anyway?


Popovers are containers that appear over a parent window. They have some significant characteristics:

  • They are generic containers, meaning that they can contain a variety of widget types (just like a dialog).
  • They have arrow points which are always directed at a specific interface element. Often, this is a button, icon or thumbnail, and the popover appears when this is pressed. (This isn’t always the case though, as one of the later examples indicates.)
  • They cannot be moved and have a fixed position.

Popovers are used to show additional controls or information. As such, they are an example of a widget that allows you to practice “progressive disclosure” in your application. This essentially means hiding non-essential or infrequently used interface elements slightly out of the way. Progress disclosure helps to keep your UI focused by giving more attention to the most important elements. As my favourite part of the HIG states:

“Every extra piece of information or interface control competes with the truly relevant bits of information and distracts the user from important information. Hence, don’t clutter your interface, and don’t overload the user with buttons, menu options, icons, or irrelevant information. Instead, use progressive disclosure and other techniques to limit what the user sees at any given moment.”

The next time you are adding extra controls or information to a UI, you might want to think: “maybe I should put this in a popover”.

When to use popovers

Popovers are one of a number of ways to achieve progressive disclosure, and there are a number of other widgets that can be used in a similar way. In particular, they are similar to dialog windows, so you might find yourself being uncertain about whether to use a dialog or a popover. There are a few things to consider here.

  1. What is the size and complexity of the content you want to display? A popover should generally be small and simple, so if you have a lot of information or controls to disclose, a dialog window is often better: they are nicer than popovers at larger sizes, and you can use tabs to break them up into sections.
  2. Is there a specific element that can act as the source of the popover? If the answer is no, you should use a dialog, since they don’t have to point to something.
  3. Do any of the established conventions for dialogs apply in this case? There are certain interaction patterns where dialogs are the established convention, such as presenting confirmation checks or application preferences. It is best not to abandon the conventions that people are familiar with for dialogs, as this will help your users understand what is happening.

If you can answer these questions and a popover still seems like a good idea, you may well want to use one. In fact, popovers have a number of advantages over dialogs when used correctly. They aren’t as disruptive, since dialogs require a bigger focus shift and give the users more new UI to interpret. In contrast, popovers don’t change the frame in which the user is working, and are generally a more subtle visual presence. This involves less severe context switches and a smoother user experience.


We’ve been utilising popovers in our application designs for some time, so I thought that it might be instructive to end this post with some examples from our mockups. Hopefully this will give you a better idea about some of the possibilities they present.

A filter menu

This is taken from our mockups for a new character map application. Here a popover is used to allow a filter to be selected for the view (in this case, selecting a font). In the past we would have had to have used a combobox or a dialog for this. A popover is better than either option: it is easier to scroll and search than a combobox, and less disruptive than a dialog.


Note that the popover header shown in this mockup currently isn’t possible, although you could implement something quite similar without it.

Gear menus

Gear menus are a common pattern in GNOME applications. Previously we have used a menu that is activated by a button for this. Popovers are a much nicer way to present a button menu though. Not only can popovers be used as a simple replacement for button menus, but you can also supplement the menu with other controls. In this example, taken from our latest Nautilus mockups, the menu has been prefaced with a pair of buttons and a slider. This makes the menu more compact and interesting, and the slider is a more appropriate control for setting the zoom level than menu items.


A word of caution when using popovers in this way: be careful not to make the popover too complex by loading them with lots of different types of widget. As a rule of thumb, use no more than three different widget types.

Editing Selections

One really nice way to use popovers is for editing controls. Here, the popover can appear in relation to the selection. The great thing about this is that it avoids showing controls until they are needed, so you don’t have toolbars full of insensitive buttons. It also emphasises the context specific nature of the controls that are being presented.

You can already see how this can work in the Notes app, which has had its own popovers implementation for a while. Here’s one of our early mockups for that:


And here is a similar pattern for Sudoku controls:


Have Fun!

Popovers are an extremely flexible widget, which means that they lend themselves to creative design. They are a really nice way to inject interest and originality into applications, I’m really looking forward to seeing how people end up using them. If anyone has any questions about how to design with popovers, I’d be happy to offer advice.

February 25, 2014

Extend GNOME Videos with Lua
As you've probably seen in my previous post, the new Videos UI has part of its interface focused on various channels from online sources, such as the Blip.tv, or videos from the Guardian.

Grilo recently grew support for Lua sources, which means you can write about 100 lines of lua, and integrate videos from an online source into Videos easily.

The support isn't restricted to videos, GNOME Music and GNOME Photos and a number of other applications will also be able to use this to be extended.

Small tutorial by example

Our example is that of a source that would fetch the list of Ogg Theora streams from Xiph.org's streaming directory.

First, define the "source": the name is what will show up in the interface, supported_keys lists the metadata fields that we'll be filling in for each media item, and supported_media mentions we only show videos, so the source can be skipped in music players.

source = { 
id = 'grl-xiph-example',
name = 'Xiph Example',
supported_keys = { 'id', 'title', 'url', 'type' },
supported_media = 'video',

We'll then implement one of the source methods, to browse/list items in that source. First, we cheat a bit and tell the caller that we don't have any more items if you need to skip some. This is usual for sources with few items as the front-end is unlikely to list items 2 by 2. If that's not the case, we fetch the page on the Xiph website and wait for the callback in fetch_cb

function grl_source_browse(media_id)
if grl.get_options("skip") > 0 then
grl.fetch('http://dir.xiph.org/by_format/Ogg_Theora', 'fetch_cb')

Here's the meat of the script, where we parse the web page into media items. Lua doesn't use regular expressions, but patterns. They're different, and I find them easier to grasp. Remember that the minus sign/dash is a reserved character, '%' is the escape character, and '()' enclose the match.

We create a new table called for each one of the streams in the HTML we scraped, with the metadata we said we'd give in the source definition, and send it back to Grilo. The '-1' there is the number of items remaining in the list.

Finally, we call grl.callback() without any arguments to tell it we're done fetching the items.

function fetch_cb(results)
if not results then

for stream in results:gmatch('<p class="stream%-name">(.-)</p>') do
media = {}
media.url = stream:match('href="(.-)" ')
media.id = media.url
media['type'] = 'video'
media.title = stream:match('<a href.->(.-)</a>')

grl.callback(media, -1)


We're done! You just need to drop this file in ~/.local/share/grilo-plugins/grl-lua-factory, and you can launch Videos or the test application grilo-test-ui-0.2 to see your source in action.

Why Lua?

This screen scraping is what Lua is good at, with its powerful yet simple pattern matching. Lua is also easily embeddable, with very few built-in functions which means we can have better control over the API plugins use, a low foot-print, and all the benefits of an interpreted garbage-collected language.

I hear heckles of "Javascript" in the background, so I guess I better address those as well. I think Lua's pattern matching is better than Javascript regexes, and more importantly, Lua is more easily embeddable in big applications, because of its simplicity as a language and a VM. Basically, Javascript (and the gjs implementation we'd likely have used in particular) is too powerful for our use case.

Better sources

It's obviously possible to avoid this screen scraping when the online source provides data in an easily parseable format (such as Json for which we have Lua bindings). That will be the case of the Guardian videos source (once we've figured out a minor niggle with the 50 items query limit) thanks to the Guardian's Open Data work.

Hopefully it means that we'll have sources for the Wiki Commons Picture of the day (as requested by GNOME alumni Luis Villa) for use in the Background settings, or for Mediagoblin installations.

Videos sidenote

An aside, for those of you who have videos on a separate network storage, or not indexed by Tracker, there's a hidden configuration to show specific paths in the Channels tab in Videos.

gsettings set org.gnome.totem filesystem-paths "['smb://myserver.local/videos/']"


I'm looking forward to seeing more Grilo sources. I feel that this Lua source lowers the barrier to entry, enabling the evening hacker to integrate their favourite media source into GNOME, which probably means that we'll need to think about parental controls soon! ;)

Thanks to Victor Toso for his work on Lua sources, both during and after the Summer of Code, and Juan Suarez for his mentoring and patch reviewing.

February 24, 2014

GNOME 3.12 on Fedora 20

I’ve finished building the packages for GNOME 3.11.90. I’ve done this as a Fedora 20 COPR. It’s probably a really good idea to test this in a VM rather than your production systems as it’s only had a small amount of testing.

If it breaks, you get to keep all 132 pieces. It’s probably also not a good idea to be asking fedora-devel or fedoraforums for help when using these packages. If you don’t know how to install a yum repo these packages are not for you.

Comments and suggestions, welcome. Thanks.

The importance of a community-focused mindset
Piston, an Openstack-in-a-box vendor[1] are a sponsor of the Red Hat[2] Summit this year. Last week they briefly ceased to be for no publicly stated reason, although it's been sugggested that this was in response to Piston winning a contract that Red Hat was also bidding on. This situation didn't last for long - Red Hat's CTO tweeted that this was an error and that Red Hat would pay Piston's sponsorship fee for them.

To Red Hat's credit, having the CTO immediately and publicly accept responsibility and offer reparations seems like the best thing they could possibly do in the situation and demonstrates that there are members of senior management who clearly understand the importance of community collaboration to Red Hat's success. But that leaves open the question of how this happened in the first place.

Red Hat is big on collaboration. Workers get copies of the Red Hat Brand Book, an amazingly well-written description of how Red Hat depends on the wider community. New hire induction sessions stress the importance of open source and collaboration. Red Hat staff are at the heart of many vital free software projects. As far as fundamentally Getting It is concerned, Red Hat are a standard to aspire to.

Which is why something like this is somewhat unexpected. Someone in Red Hat made a deliberate choice to exclude Piston from the Summit. If the suggestion that this was because of commercial concerns is true, it's antithetical to the Red Hat Way. Piston are a contributor to upstream Openstack, just as Red Hat are. If Piston can do a better job of selling that code than Red Hat can, the lesson that Red Hat should take away is that they need to do a better job - not punish someone else for doing so.

However, it's not entirely without precedent. The most obvious example is the change to kernel packaging that happened during the RHEL 6 development cycle. Previous releases had included each individual modification that Red Hat made to the kernel as a separate patch. From RHEL 6 onward, all these patches are merged into one giant patch. This was intended to make it harder for vendors like Oracle to compete with RHEL by taking patches from upcoming RHEL point releases, backporting them to older ones and then selling that to Red Hat customers. It obviously also had the effect of hurting other distributions such as Debian who were shipping 2.6.32-based kernels - bugs that were fixed in RHEL had to be separately fixed in Debian, despite Red Hat continuing to benefit from the work Debian put into the stable 2.6.32 point releases.

It's almost three years since that argument erupted, and by and large the community seems to have accepted that the harm Oracle were doing to Red Hat (while giving almost nothing back in return) justified the change. The parallel argument in the Piston case might be that there's no reason for Red Hat to give advertising space to a company that's doing a better job of selling Red Hat's code than Red Hat are. But the two cases aren't really equal - Oracle are a massively larger vendor who take significantly more from the Linux community than they contribute back. Piston aren't.

Which brings us back to how this could have happened in the first place. The Red Hat company culture is supposed to prevent people from thinking that this kind of thing is acceptable, but in this case someone obviously did. Years of Red Hat already having strong standing in a range of open source communities may have engendered some degree of complacency and allowed some within the company to lose track of how important Red Hat's community interactions are in perpetuating that standing. This specific case may have been resolved without any further fallout, but it should really trigger an examination of whether the reality of the company culture still matches the theory. The alternative is that this kind of event becomes the norm rather than the exception, and it takes far less time to lose community goodwill than it takes to build it in the first place.

[1] And, in the spirit of full disclosure, a competitor to my current employer
[2] Furthering the spirit of full disclosure, a former employer

comment count unavailable comments

February 19, 2014

On portability

There has been a lot of hand-wringing lately about how GNOME is supposedly forcing everybody to use systemd, and is not caring about portability.

That is of course not true at all.


Portability has been a recurring topic in the discussions of the GNOME release team (of which I am a member). We don’t make it a secret that modern Linux is the primary target that we are developing on and for. And we are aiming to use technologies that are best suited for the task at hand – which has led us to use more of the services implemented by systemd (some of which are direct descendants of earlier mechanisms shipped with gnome-settings-daemon).

But we are happy for everybody who wants to try GNOME on other platforms, and we have a strong interest in making GNOME work well there.

Reasonable patches to help with this are always welcome. The shape of those patches can vary from case to case: it could be reduced functionality, alternative backends, or a shim that puts systemd-like interfaces in place.


To show that these are not just empty words, here is a screenshot of GNOME running on FreeBSD:

GNOME on FreeBSDThe screenshot was provided by Ryan Lortie, who has done a lot of work on making jhbuild work on FreeBSD.

And here is a video showing GNOME running on OpenBSD, courtesy of Antoine Jacoutot:



Go here  to read more about the release team position on portability.

If you are interested in helping out with making and keeping GNOME running on more platforms, this page is another good place to go. It lists platforms that are well supported by GLib.