Posts
| « | November 2024 | » | ||||
|---|---|---|---|---|---|---|
| 1 | 2 | |||||
| 3 | 4 | 5 | 6 | 7 | 8 | 9 |
| 10 | 11 | 12 | 13 | 14 | 15 | 16 |
| 17 | 18 | 19 | 20 | 21 | 22 | 23 |
| 24 | 25 | 26 | 27 | 28 | 29 | 30 |
| « | November 2024 | » | ||||
|---|---|---|---|---|---|---|
| 1 | 2 | |||||
| 3 | 4 | 5 | 6 | 7 | 8 | 9 |
| 10 | 11 | 12 | 13 | 14 | 15 | 16 |
| 17 | 18 | 19 | 20 | 21 | 22 | 23 |
| 24 | 25 | 26 | 27 | 28 | 29 | 30 |
Just dropping a quick note here because Independence Week (I took a few extra vacation days after the 4th) gave me time to start more projects than I normally do. And now I'm really busy from them. So in that vein, three quick notes.
The first is a really cool patch to GTK+ which is the beginnings Inplace-Tooltips — the next generation version of my TreeViewHints hack. I have an Ogg Theora screencast of what it can already do in Ex Falso (with TreeViewHints disabled), and am hoping to make a MNG in case it's smaller. I think the hard parts of this battle are making sure the behavior is perfect, that it doesn't leak references, and getting the decision makers to recognize its importance.
The second is a request to those holding copyright interests in GPL-licensed GStreamer-using projects Fluendo has asked to consider relicensing under terms compatible with plugins with which they wouldn't otherwise be compatible. Don't. You chose to license your code under the GPL for a reason. You are being asked to weaken your support for Free Software for commercial reasons, and being offered only the opportunity to be distributed with a particular Linux distribution. You are being asked to trade your belief in Freedom for an imaginary increase in your user base. Freedom is worth much more than that. Stand up for your beliefs and don't worry about people using encumbered formats in distributions that can't handle this a better way.
The last is I've started learning Cairo and have put together the beginnings of a Go client (so far it's just the board display). Cairo is a real treat to work with, but there's not any good introductory documentation. I received a great explanation of the basics just last night from Carl Worth and I hope to put it and the rest of what I've learned into a series of blog posts, and later perhaps a three-in-one tutorial of PyGTK gobjects and cairo.
) (0 Trackbacks)
cairo
gtk+
licensing
The last couple days I went through one of the more painful debugging experiences of my life. The symptom was that one of our OggFlac unit tests in Mutagen was failing. But not on x86; that would be too easy. Instead it was only failing on AMD64 systems, which is reminiscent of a bug I covered before. Thankfully Pete was able to provide direct network access to an AMD64 machine for me to test on; otherwise we'd undoubtedly still be fighting this bug. As it was, we spent at least four hours each on Thursday and Friday.
So knowing that this was a painful bug, how do we debug such symptoms?
First things first, try to isolate the problem as much as possible. The failing test did a small sequence of things, starting from a nearly blank file, adding some stuff, deleting it, adding other stuff, and then failing the test on line 39. So pare it down. As it turns out all we need are the three lines 37–39: add, save, test.
Now that we know which methods reveal the bug, we can look into the code behind them. This can help find many kinds of errors. I think it helped us tighten our code a bit. But after many hours of staring at it and trying various things it was still no help.
We had a passing version of the same test on x86. Was the problem in our code, or in the crosscheck we were performing? Taking the output produced on x86 and testing it with flac on x64 (and vice versa) showed it was a problem in our code, and not the tool. Drat. But hey, the file is the same up until that last step. That confirms everything else so far is good.
Since looking at the file gave us no good clues, other than the broken file in question looked like garbage, it was time to delve deeper. I coded up a tracing wrapper, which injects tracing primitives into python code. I invoked this from the OggFLAC tests, and traced ogg, oggvorbis, and oggflac modules under mutagen.
Note: the linked version is the final version. Earlier versions lacked tracefile, function return values, and indentation.
The only differences for the longest time, after filtering out inconsequential differences, were at the failure spots themselves. The passing code would continue; the failing code would report an exception. I would add more tracing. The same. What else can I trace? There has to be a difference somewhere. So I added tracefile.
And the size of the tracelog exploded. I spent time coping with that when I should have been realizing that there is no longer any difference. The good and bad trace were too different (because of our fallback code for the AMD64 mmap problems in python) so make them more similar. Somehow the failing code started working. What did I do? I added tracefile. Huh? Great...now I can't even observe my bug without it morphing.
In concurrent programming, or code with extreme speed needs, heisenbugs can be a real problem. Fortunately Mutagen is neither.
The thing about the final push is you never know when it will hit. When you first start debugging you're sure it's right around the corner. Then the more you work at it without solving it, the more convinced you become that something insane is wrong with your tools. So here's a reconstructed stream of consciousness from when I first saw the heisenbug.
Take all tracing out, but leave in tracefile. Okay, it's still working. Take out tracefile. Okay, it fails. Hey, it fails on x86 too. Hallelujah! Wait. Huh? Pare down tracefile. Use it and take out #read. It fails. Put #read back and take out #write. It works. Leave in #read but simplify it to just recurse the call. It fails. What's left?
I fiddle with it and find the only piece I need is self.tell() before calling self.read. So I mention my bewilderment to Joe and he finds a gem: this bug has been reported to Python before. And denied. Apparently rightfully so. Six month old tested code is to blame. I fix it up by using explicit seek calls, and everything passes.
There's always another dark corner of your toolchain left to be learned, often the hard way. In our case it was the fact that interleaved read and write operations on a file pointer, without explicitly resetting the stream position, are undefined. They work most of the time on the implementations we run on, but they fail sometimes. I feel we did an okay job of challenging and proving our assumptions right or wrong, but it was really hard to make the leap that let us trust the new barely tested code that was revealing the failure, and stop trusting the old tested code that was causing it. I'd like to do better about that in the future.
In the meantime, Mutagen 1.4, with corrected file handling code on 64-bit systems, here we come!
I need a trick. I want to build a plugin system in C# that supports internationalization. I want to be able to create various interfaces (called IPlugin here for simplicity) which classes in the plugins can implement so the framework knows how to load them. The framework must also able to get metadata from them in order to describe them to the user. What are my options?
Here's what I've come up with so far. I can create two interfaces, IPluginFactory and IPlugin when I only really want one. This is potentially useful anyway for a number of reasons so I might go with it. It would look something like this:
interface IPluginFactory
{
string Name { get; }
string Description { get; }
IPlugin Load(string filename);
IPlugin New(string filename);
...
}
interface IPlugin
{
string PrettyName { get; }
void Save();
...
}Here IPluginFactory is implemented by a thin wrapper class which gives me data about the class implementing IPlugin which it knows how to manage. In the Name get method I can easly load and return Strings.MyString to retrieve a localized value. And all the meat can go into the IPlugin class. Thus it's cheap to instantiate all IPluginFactory classes, and put off instantiating a given IPlugin until I need it.
But isn't this sort of static metadata and creation needs exactly what attributes and static factory functions are for? Rather than having to implement two classes for every IPlugin, it would be nicer to just attribute one class.
[PluginInfo(Strings.PluginName, Strings.PluginDesc)]
class MyPlugin : IPlugin
{
static MyPlugin Load(string filename) { ... }
static MyPlugin New(string filename) { ... }
string PrettyName { get { ... } }
void Save() { ... }
}But there are two problems with this approach:
Between the two it means I can't easily enforce (compile-time) an implementation of the static methods, and I can't put localized strings into Attributes at all, as they all require runtime resource lookup. Because I can't put static properties in an interface, I also can't simply work around the second by putting the strings into static properties, although since unattributed classes aren't errors either, it's less of a compile time issue.
What ideas am I missing? I can use, but don't particularly like, the two interface approach. An inventive developer could probably implement both interfaces on one class, although it would only rarely be worth it, and would miss the point of having a thin wrapper class. I really like the look of the attributed single interface sample code. How close can I get?
I've been playing with C# lately with an eye towards using it at work instead of the nasty legacy C++ codebase. And one of the pieces I'm still struggling with is internationalization. On unix we use the infamous gettext _() macro, which works in the source as a marker that its string needs to be translated, and at runtime as a function which looks up the translated string. In MFC we use numeric resources and utility functions like CString::LoadString(IDS_MYSTRING) or constructors like CString(MAKEINTRESOURCE(IDS_MYSTRING)). In C#, at least as presented in Visual Studio 2005, the idea is similar to the latter.
So I tried to use it. You create a resource file, add some strings to it, and then it's time to use them. You can try something like the following (which I'm not sure is correct):
ResourceManager rm = new ResourceManager();
string mystring = rm.GetString("MyString");But there's one major quirk here. Unlike the unix or C++ methods, this is using a raw literal which is not a desired value. If you mess up the value passed to GetString(), you'll catch it at runtime. Clearly that's not the right way. You could make constants for these lookup strings, but that doubles the work. Turns out there's a better way.
Whatever you named your .resx resource file, there's an automatic type-safe class generated behind the scenes for you with static property accessors. So if I called it Strings.resx and had a string MyString, there's an accessor already set up so all I need is:
string mystring = Strings.MyString;
I'm new to C# in .NET 2.0, and the documentation isn't quite clear on this point, but I get the feeling this support is new to VS2005. It's definitely a nice experience for resource-style internationalization.
So what does ''.join(chr(sum(((val >> i) & 1) << (7-i) for i in range(8))) for val in range(256)) do? If you run it on python2.4 you get an ugly string. Specifically it's a list of 256 bytes with their bits swapped in order of significance: 01 becomes 80 and so forth.
But why is this useful? Thanks to Joe's work it's now part of Mutagen, and is being used as part of a scheme to calculate the kind of CRC32 that the Ogg container (of Ogg Vorbis) requires. Why a scheme? Speed. As an interpreted language, python isn't well suited to small bit calculations on large sets of numbers. A standard reduce(lambda x, y: table[(x>>24) ^ y] ^ (x << 8), data, 0) scheme is too slow for comfort. However the table used in the existing C implementations zlib.crc32 and binascii.crc32 is from the bitwise reversed generator polynomial of the one used for Ogg.
Peter Johnson came to the rescue and figured out we could get the same effect by bitswapping each byte of the data and then bitswapping the final 32-bit result. Thanks to str.translate, we hoist most of the work into C, and the above puzzler code runs once at module import to generate the translation table.
And yes, Joe, it's good for confusing you at 1AM. :)
What is the following useful for?
''.join(chr(sum(((val >> i) & 1) << (7-i) for i in range(8))) for val in range(256))
I'm sure anyone reading this is aware of the Google sponsored Summer of Code, as an interesting opportunity for Free Software and Open Source Software projects to attract new contributions (and ideally contributors) through a summer stipend paid by Google. What's less well known is the process of choosing applicants.
I applied to mentor with the PSF, but now I'm torn. I think Google is doing a good thing, but the application process has me questioning my interest. Between limitations in the web site project ranking system, and dealing with unknowns (did this student submit applications to other groups, or more than one to us?) it runs like a bad interview puzzle question:
You have 30 chickens, 30 foxes, and 30 bags of corn, one canoe, and one river. You can only fit one with you in your canoe, and wish to take them all to the other side of the river. If you ever leave a chicken with corn, or a fox with a chicken, the latter will be eaten. A bandit might come by and steal up to one item per river crossing. In what order do you take items across to maximize the results of Google's sponsorship?
And I hear it's much better than last year...
I came home from work today to find I'd missed quite an afternoon. The summary is best shown as a single message from Joe with whom I develop Quod Libet: We got ripped off.
He was tipped off early this afternoon after reading Ross's post about Listen, looking at the screenshot and recognizing the appearance of the OSD. When he looked, he found in some places our copyright was still in place, untouched; in others cut and paste jobs had completely obliterated it. Apparently it's an established project that uses our code, but one without any public links or references to Quod Libet. I got back too late to see the code or the website in question, so I'm not sure what I think of this mess. Hence I'll talk about this in a general fashion, considering this case only as an example.
A lot of people are calling Joe the bad guy for cutting this off with his brusque open letter. There seems to be a belief shared by several anonymous commenters that pointing out in public that someone else did something wrong is a bad thing. I disagree: how else will anyone outside the infringer learn? Especially with a mistake as core to the Free Software community as copyright infringement, everyone needs to be aware of mistakes. More importantly they need to know how to avoid missteps and follow licensing terms correctly. I do find Joe's choice of language regrettable, but his core message is spot on.
We need to take more care of observing copyright and licensing requirements in the community. Anyone writing software and licensing it under any version of the GNU General Public License (GPL) is likely to be trying to contribute to the community so the damage from this scenario (stripping copyrights off GPLv2 and representing them as GPLv2 or later, and as your own work) is small. Assuming the infringer cleans up his mistakes, we will allow him to partake of the code as we would any new licensee.
What if this was darker? What if amongst the code being put into a GPL project there was code that didn't offer open terms at all, much less Free terms like the GPL? What if the aggregated work was being offered as closed source pay-only? Either scenario is anathema to the Free Software community: the first because it weakens us; the second because it takes advantage of us. Should our reactions to these scenarios be so different from the reactions to Joe's letter? I say they shouldn't, but you know everyone would be rooting for the little guy for sticking it to the evil empire.
License intentions are important—it's what ties us together as a community—but carrying through and actually following the licenses is crucial for the survival of our community. I look forward to a validly attributed and licensed Listen appearing soon; let's keep that sharing part of this community working!
Continuing along the theme of trivial problems that go unfixed for a long time, with an occasional personal favorite, here's some problems with PyGTK.
Where I treated Mozilla Firefox as a user application, I treat PyGTK mostly as the development library/toolkit that it is. For quite a while GTK+ and GNOME were the poster children of the Free Software Movement. They handle some things really well, but there are still a lot of gaps. Here's some of the gaps I particularly want to see filled.
Real tooltips can only be single paragraphs of text. No images, no formatting, no nothing. At two and a half, and five years respectively, these strike me as a perfect example of the if we put this off until we do such-and-such (a hard thing), then this will be really easy degenerate case
mentality. Since GTK+ is much more concerned about backwards compatibility than Mozilla Firefox seems to be, I can give them some kudos for avoiding ill-considered changes. However the status quo is not good enough, and any tooltip reimplementations to acquire the different capabilities ruin the point of having a widget in the first place.
There is no builtin way to set tooltips on a per-column, per-row, or per-cell basis (three and a half years old). This ties into the above five-year-old bug, potentially as another easy degenerate case. Well, I'm still waiting.
Dragging is hard enough, but dragging multiple items is really hard. By default you need to start a drag on the same button-press that selects or unselects a row. I can't find a bugzilla bug on this one, but Quod Libet uses class MultiDragTreeView which implements this in a saner fashion.
There is no builtin way to show the contents of an ellipsized or otherwise truncated text cell. I also can't find a bugzilla bug on this one, but Quod Libet uses class TreeViewHints which does most of what I need. The idea is when you're hovering over an ellipsized cell in a TreeView, a tooltip-like window will show up exactly in place to show you the parts you can't see. Windows MFC has this built in, and that one even works if the contents are hidden due to scroll. TreeViewHints only handles a small subset of ellipsized cases, so I'd rather see something real implemented in C.
There's no right way to set a right-click menu on a column header. Apparently there are some hacks in C, but they are not available from PyGTK. This makes it practically impossible for us to do the user-friendly configuration of Quod Libet's column headers that we've wanted to for over a year.
Free software has a major weakness. It's not one that other people can worsen or exploit, other than by working harder themselves. So this weakness doesn't spell the end of Free software. But it consistently acts as a limiting factor in the greatness of Free software. To be fair plenty of commercial software suffers from the same problem, if for different reasons, so it's not even a weakness specific to Free software. But it's a weakness I want to see addressed and mitigated.
The weakness I refer to is in small but important features: they constantly slip through the cracks. What makes it infuriating is the number of wonderful, obviously-difficult, and often polished features that are written in the same timespan. Sure it's not a straight tradeoff, so lamenting if we had time for this, certainly we had time for that
doesn't help—different contributors are interested in writing different things. Official maintainers are interested in accepting different things. But the result is the same: something simple remains hard or impossible for years, and things you could do in the previous version cannot always be done in the next version.
It's not hard to find examples. I don't pick any given example to make fun of the project which lacks the feature; I do it to highlight a change which I would love to see make my life that much better. That said the more I use something, the more gaps I'm likely to notice—in that sense perhaps it's a compliment if I list a lot of gaps! Regardless it's probably best to treat this as my personal rant list. More examples coming later.
What makes the first two Mozilla gaps so annoying is that they've both been present in previous versions of the product. Now that's progress!
Back in the bowels of time the first well-known web browser could view just about anything as if it were served with content-type text/plain. This was a great thing, although its importance didn't crop up until much much later. Recently web servers have started sending source code under other content types like text/x-csrc or text/x-py. Since it became reborn as Mozilla, rather than always supporting an in-browser fallback, unknown content types are only supported by external applications. This means when I go to view plain ascii text files on certain web servers I have to load up a text editor. Insanity!
And I'm far from the only one to think so. This was reported on bugzilla five years ago.
Firefox before version 1.5 offered a "Manage folder" option in the right click menu of each bookmark toolbar folder. In 1.5 they removed this, apparently because some people thought others might find the expand option confusing, and because some functionality didn't work right in the subfolder-only view. The result? I and others can no longer easily access and edit bookmarks in a single folder. I have way too many bookmarks to view the whole set; managing a single folder was very important to me.
When a site offers an RSS feed, you get a nice icon letting you know this. Click on the icon to subscribe (create a live bookmark). However neither in this dialog nor on a right click can you find out what the URL is for that live feed. The only way to do that is to view the original source, or to create the bookmark and view its properties. This makes it much harder to verify that you're setting up your RSS rel links correctly on a page, or to directly visit the feed in your browser. I couldn't find a bug in buzilla about this limitation.