First off, there’s a release coming any day now. A number of things were screwed up with the September 2011 release, mostly my fault, and I’ve been busy patching them up. So there’s a new December 2011 release around the corner, and 4.0 runtimes will be available right at the start. I do want to point out, though, that the runtimes are strictly for end-users (non-developers) who are consuming SlimDX apps. You don’t need them to develop, and for that matter you probably don’t need them at all if you’re at a game development company. They install the DX runtimes, VC runtimes, and SlimDX itself. Given that both runtimes are now well over a year old, odds are you already have this stuff. While that doesn’t excuse my personal failures in getting this stuff out in a timely fashion, there is almost certainly no need to worry over it for 90% of you.

We’ve been promising a SlimDX 2.0 release for some time now, with substantially revised architecture. The redesign is based around many of the same concepts driving another wrapper library called SharpDX by Alexandre Mutel. Alex was working with us for a while but we split up over some mutual differences and went our separate ways. I’ve decided to withhold any comments on his work one way or the other. As far as our work… we need help. The three of us (Josh Petrie, Mike Popoloski, and myself) have been working on the library for something like five years, and things are pretty stable at this point. Sure there are bug fixes that we’re shipping out, but especially now that the DirectX SDK updates have stopped, the current codebase is largely good to go. The new codebase for 2.0 is really a prototype, and the simple fact is that it needs a lot of work and none of us has the time anymore.

I repeat: We need new people to help develop SlimDX. If that doesn’t happen then we’re likely stuck in place, which might not be that big of a problem except for one thing: Windows 8. SlimDX 2.0 is based on a code generation system that should allow us to target C++/CLI as well as the new C++/CX language. With CX support we get not only .NET but also JavaScript and native code support to interop with Metro apps. Not only that, but it also means ARM support and SlimDX on tablets in the coming years. I think that’s a big deal, if we can pull it off.

When I first wrote SlimDX in 2006, I believed that automated codegen like SWIG was not well suited to creating a simple, usable wrapper. SlimDX was hand written from the ground up to make using DirectX as painless as possible, and also to reshape the DirectX API into something that made sense as a .NET API. That was directly in the footsteps of Managed DirectX which Tom Miller had created, though we took the model a lot farther in that direct path. Alex came to us with an approach for code-gen which we felt really has potential, but there’s still a lot of rough edges and a lot of work in getting it to the standard which we really want it to be at.

So, who can help? You’ll need to have a working familiarity with C++ and C#, and DirectX of course. It is not, contrary to popular belief, necessary to really deeply understand any of these things. Working on SlimDX is an adventure in quirks and details of interop that I can almost guarantee you have not seen. Don’t worry about experience if you’re looking to help out. There will be a lot to learn of course, and you’re going to need a lot of free time to commit to this, but we’ve spent a long time building SlimDX and have a pretty solid handle on what’s going on. The only other requirement is the understanding that what you get out of this is experience, an excellent resume item, and skills that are fairly rare. Money is very unlikely to appear directly unless donations take a serious uptick.

If you’re interested in helping out, please post here, or ping us via Twitter or IRC or e-mail or GameDev or whatever. I really do need one or two people to join as regular developers, otherwise DirectX and Windows may well move forward without a SlimDX to help glue the bits together.

Money and power are and always have been centered around a singular point: control. In order to protect an oppressive government, or an oppressive business model, you must control the basic pathways and communication channels. The methods have changed over the course of centuries but the ideas have not. The Internet and the Web represent largely uncontrolled systems of communication. As a result, it’s been a continued thorn in the side of governments and corporations for many years. From Napster to PirateBay and WikiLeaks, and far more reprehensible things (eg child porn), there’s been a constant struggle between freedom and control. That struggle has been largely random and without direction, because nobody really knew how to police the internet. The system was designed to be resilient, and there are many, many ways in which blocks by oppressive regimes have proven ineffective.

Now we’re seeing the next phase, which is to target the gate-keepers. The internet is resilient, but it is not resilient enough. Search engines and link accumulators were targeted first. Coupled with DMCA provisions, sites are vanished from Google and Bing and once that happens the site may as well not exist. Discovery becomes nearly impossible. This has been done to protect “copyright holders” and “intellectual property”, but that is merely a proxy for ANY information that any party or any government (primarily the US) does not want in the wild. You only need to observe Universal’s assault on the MegaUpload video to understand that. Making somebody invisible, even temporarily, is an enormously powerful ability.

The next target, possibly the crucial one, is the Domain Name System (DNS). DNS is responsible for translating a domain name like “google.com” into an IP like “10.11.12.13″. The US Department of Homeland Security has gleefully pursued sites by revoking their domain names without anything resembling due process and without available recourse. And without actual authority, for that matter. The results were predictable: a technical workaround which got the government mad, and a bogus seizure that made the whole program look corrupt, which it is.

The last gatekeeper is the ISP, the guys who hold the actual physical connection between us and the internet. They are under assault too. It’s the same story over and over again, but in the end the ISPs will cave because it will be difficult or illegal for them to hold out.

SOPA might be the greatest ever attack on Internet freedom, but it’s also a dead-on logical expansion of a war that has unfolded continuously over the past decade or more. It’s possible that this particular measure will be defeated. The trouble is that it doesn’t matter. There is far, far too much at stake for the corporations and governments to let this go so easily. They will learn from their mistakes here, tweak and tune the language and the pitch, and come back with armies of lobbyists time and again until the chaotic political winds line up in their favor. That WILL happen, and things will start to crumble for those who value freedom.

Ultimately Hollywood wants the same thing that the government wants: the ability to control and restrict what happens on the Internet and how. They are on the same side, and all the calls in the world to your Representative will only delay what’s coming. It’s useful to buy time, but at the end of it all there is only one choice that will work: the Internet and the World Wide Web must be made entirely immune to censorship at a fundamental technical level. It must be redesigned so that no amount of legal threat is capable of affecting it at all.

From a technical point of view, that means a few things. First, the DNS system must be secured against the whims of any government. There are two options for doing that. One is to secure the DNS system so that every country controls its own TLDs and cannot affect any others. I believe this is doable with a widespread rollout of DNSSEC. The US could still revoke domains, but only those hosted as COM/ORG/NET/US/etc which are ostensibly subject to their legal control anyway. Just pick a country where whatever you’re doing is legal and sign up with them. The other option is rather extreme, and involves replacing the DNS system entirely with a new naming system that is not under anybody’s control at all. There is work along these lines, but it’s difficult to see potential for mainstream adoption. (On the other hand, it could thrive in environments like P2P networks if the tech details are hacked out.)

Then there are the ISPs. There’s no point locking the overall system down if your personal uplink still says “hey, no PirateBay for you no matter how you’re trying to get there.” That requires end-to-end encryption of your sensitive traffic. We have a system for that called Tor, but it’s possibly extreme. The ability to perform encrypted DNS queries locally (this is different from DNSSEC), plus secure HTTPS connections, achieves nearly everything we need. The latter has already become commonplace on major sites, which only leaves us to solve encrypted DNS queries. Luckily we’ve got that too.

That leaves us with the visibility problem in search engines, social networks, and similar services controlled by a single entity. I’m less concerned about this, because the steps I’ve discussed so far open the door for somebody in a more open country to build systems that are not subject to government or corporate whims. There is work on a decentralized search engine that isn’t subject to any control at all, but it’s unclear whether such a system is actually workable. Similar efforts are underway to replace centralized services such as Facebook, Twitter, and even semi-centralized mechanisms like OpenID. There is a core belief here that any system that is centralized is necessarily a threat, and cannot be trusted. I don’t know if that’s the case, but the more research we have in building completely distributed tools the better.

To try and win true freedom for the Internet on political and policy grounds is an eternal battle which we will likely lose. There is too much at stake for the power players to give up what we are asking of them. If we’re lucky, Google and all the other internet companies will remember to sink millions of dollars into R&D into making the Internet unbreakable, instead of simply lobbying the government not to do it. Once we make it indestructible on a technical level, governments and corporations will be forced to adapt to the new order, instead of trying to stop it. That’s our only chance to preserve what we’ve built and earned in the last forty-odd years: a completely free communication system that is equal to everyone.

Effective immediately, I am shifting all domains away from GoDaddy. Because I’m rather new to this process, I don’t know what will happen to DNS and email during the transition. SlimDX or @slimdx.org addresses may become inaccessible for a short period while I sort things out. Please bear with me.

If you are interested in moving your own domains, I found out that NameCheap is running a promotion with code “SOPAsucks”. Their pricing is not quite as aggressive as GoDaddy but it appears that they do offer very competitive pricing ($2 specials) nonetheless. Transfers with the code cost $6.99 per domain, which includes a year renewal of the domain. I am sure there are other anti-SOPA registrars but this one is mine.

Something on the personal front: I got a few comments, directly and indirectly, about being an MS or DirectX “hater”. Good lord no! I adore DirectX, XNA, Windows, and Microsoft. I criticize because I want to see these technologies thrive and succeed. I’ve been doing a lot of iOS/Mac/OpenGL work lately and from a technical standpoint it’s absolutely miserable. I miss the wonderful Microsoft world of development. But a lot of what I’m hearing in public and private worries me. My alarmist approach is designed to bring attention to these things, because oftentimes the development teams live in a bubble separated from their users. (Hell, I can barely get in touch with people using SlimDX.) The XNA team, for example, are terrible at communication. It’s exasperating, because the direction, plans, and schedules are completely opaque — even to those of us who have signed an MS NDA and are ostensibly supposed to see this information early. (We don’t see jack shit, by the way. Exasperating.)

Second, no I do not think DirectX is dead. No I do not think everybody should switch to OpenGL. From a platform and technical standpoint, we’re probably as much or even more of a commitment than in the past. What’s bothering me is the pathetic way that community, documentation, etc is being handled. Look: Where is the DirectX SDK? I don’t know when that was published, but it appears that nobody noticed it until very recently. How would we? I don’t wander MSDN online at random. And it’s been placed right next to this comically worthless page. This is the kind of developer support I’m complaining about. Nobody outside MS understands what is going on in Bellevue, and I’m getting this worrisome feeling that nobody inside understands either. The DirectX SDK wasn’t just a diverged way to deliver support for a core platform technology, which is what seems to be driving the current decision making process. It represented half a gigabyte of commitment to the developers who arguably make the entire Windows ecosystem compelling to a consumer. “Developers developers developers!” “Yeah, what’s up?” “Uhhh, hi?” That’s what it feels like. MS wants developers around, but they forgot to put any thought into why. (Here’s a hint guys, ask DevDiv. They seem to still have a clue, C++ team aside.)

And in the other corner, we’ve got XNA. Whoo boy. There’s no point to sugarcoating this, although I’ll probably ruffle some feathers: XNA 4.0 is garbage. It exists to support two things: XBLIG and WP7. XBLIG is a joke, so really the only productive arena remaining is WP7. It’s sad because XNA 3.x was actually an excellent way to do managed development on PC. 4.0 introduces a profiles mechanism that is focused specifically around the Xbox and WP7, and produces a stunningly foolish situation on PC where DX10+ hardware is required but none of the new features are supported. Worse still, that was a few years ago. Now we’ve got DirectX 11 with Metro coming down the pipeline and XNA staring blankly back like the whole thing is a complete surprise. It tends to raise some questions like, is XNA 5 coming? Will there be DX11 or Metro or Windows 8 support? Is anybody even listening? And the answer we got was, and I quote: “We’re definitely sensitive to this uncertainty, but unfortunately have nothing we can announce at this time.”

My judgement on that message is that XNA does not exist on PC. I’ll say the same thing I said in 2006 when XNA was first revealed. Treat it as an API that happens to run on Windows as a development convenience, not as something it’s actually meant to do. The computing world has moved on, and if Microsoft can’t be bothered to bring XNA along then that’s just something we have to work with. If and when XNA 5 is announced, then we can go back and take a look at the new landscape.

Let’s start with the DirectX SDK, which you may have noticed was last updated in June of 2010. That’s about a year and a half now, which is a bit of a lag for a product which has — sorry, had — scheduled quarterly releases. Unless of course that product is canceled, and it is. You heard me right: there is no more DirectX SDK. Its various useful components have been spun out into a hodge-podge of other places, and some pieces are simply discontinued. Everything outside DirectX Graphics is of course gone, and has been for several years now. That should not be a surprise. The graphics pieces and documentation, though, are being folded into the Windows SDK. D3DX is entirely gone. The math library was released as XNA Math (essentially a port from Xbox), then renamed to DirectXMath. It was a separate download for a while but I think it might be part of Windows SDK from Windows 8 also. I haven’t checked. The FX compiler has been spun off/abandoned as an open source block of code that is in the June 2010 SDK. There are no official patches for a wide range of known bugs, and I’m not aware of a central location for indie patches. Most of the remaining bits and pieces live on Chuck Walbourn’s blog. Yeah, I know.

In case it’s not obvious, this means that the DirectX release schedule is now the same as the Windows SDK, which always corresponds with major OS updates (service packs and full new versions). Don’t hold your breath on bug fixes. Last I heard, there’s only one person still working on the HLSL compiler. Maybe they’ve hired someone, or I assume they have a job opening on that ‘team’ at least. What I do know is that for all practical purposes, DirectX has been demoted to a standard, uninteresting Windows API just like all the others. I imagine there won’t be a lot more samples coming from Microsoft, especially big cool ones like the SDK used to have. Probably have to rely on AMD and NVIDIA for that stuff moving forward.

That covers the native side. What about managed? Well the Windows API Code Pack hasn’t been updated in a year and a half so we won’t worry about that. On the XNA front, two things are becoming very clear:
* XNA is not invited to Windows 8.
* XBLIG is not a serious effort.
The point about XBLIG has been known by most of us MVP guys for a while now. Microsoft promised a lot of interesting news out of this past //BUILD/ conference, which I suppose was true. However you may have noticed that XNA was not mentioned at any point. That’s because XNA isn’t invited. All of that fancy new Metro stuff? None of it will work with XNA, at all, in any fashion. (Win8 will run XNA just like any other ‘classic’ app.) That also implies pretty minimal involvement with the Windows app store. Combined with the fact that XBLIG has never been a serious effort to begin with, I’m dubious about tablet support for managed games. XNA does work on WinPhone7 and Win8 does support Win7 apps, so it ought to work in principle. Maybe. Given the niche status of Windows Phone 7 at the moment, and major losses of tech like UDK and Unity from that ecosystem, I’m also expecting WinPhone8 to be much more friendly to native code. (If not, I expect that platform to fail entirely, and take Nokia with it.)

I’m also looking at this e-mail right now in my box, which starts as follows:

As you know, the 2012 Microsoft MVP Summit is Feb 28-Mar 2, 2012. We wanted to inform you that DirectX and XNA technologies will not be hosting sessions at the Summit. As MVPs, you are still encouraged to attend and be a part of the global MVP community, and you’ll have the ability to attend technical sessions offered by other product groups.

Really? No DX/XNA sessions at all? I don’t think DirectX is fading into the sunset because it’s a core technology. XNA will most likely disappear. For some reason, Microsoft is getting out of the business of helping developers produce games and other 3D applications at the exact same time that they’re adding core support for it. Yes, Visual Studio 11 has model and texture visualizing and editing. It even has a visually based shader editor. I’ll let you take a wild guess on how well that will work with XNA code.

I don’t know exactly what Microsoft is going for here, but every couple of days somebody asks me when the next DirectX SDK is going to be released and I think I’m just explicitly stating what Microsoft has vaguely hinted for a while now. There is no new DirectX SDK, soon or ever. And I’m not holding my breath for any XNA updates either. I am told there is an XNA 5, but they won’t be at the summit apparently. If there is some actual future in XNA, feel free to make yourselves heard on the mailing lists because right now it’s really difficult to understand why anyone would bother.

Important clarification: I do NOT think DirectX is being deprecated or vanished or even dramatically changed at the core. (I’m less confident in XNA.) The SDK is being vanished, subsumed into Windows SDK as a component just like all the rest of Windows. So coding DX is no more special than coding GDI or Winsock. There will still be 11.1 and 12 and all that, probably delivered with OS releases and service packs. “DirectX 11.1″ will actually be synonymous with “Win7 SP2″. That has been the case for a while, actually. What’s going away is the wonderful developer support we’ve enjoyed as long as I can remember. Compare the D3DX libraries across 9, 10, and 11. Is this really a surprise? It’s been slowly happening for years.

This post is the first of a couple, dealing with overall eyeglass selection and particularly the question of whether to buy in-store or online. Later on, I’ll discuss lenses, coatings, etc. But for now, let’s just tackle the overall problem of where to get your glasses.

Online or brick and mortar store? In most cases, the person doing your eye exam rents space from or is outright affiliated with a retailer. Once your prescription’s been established, they hand it straight off to the shop, and you pick out your frames. Nice, easy, and totally ignorant. That’s not to say you’re getting a bad product, but you are probably paying a lot for the convenience. How much is a lot? The typical margin on frames varies from 100% to 1000%. Yes, that’s 10x in pure profit. Those $300 designer frames cost probably $30 to manufacture, and they’re really not precision equipment. I don’t have good numbers on lens markup, but it’s not subtle either. Cursory exploration suggests it’s at least 100%. For your money, you get free adjustments and maybe repairs from the people who sold you the glasses. Odds are you’ll walk out with a pair of comfy glasses that look pretty decent on you and assurance that if something goes wrong, you’ll have someone to yell at.

So where do you go in the brick and mortar world? Walmart and Target are, well, pretty much what you’d expect. Sort of vaguely competent but minimal at best. Big name chains like Lenscrafters are popular and prevalent in malls all over, but it’s turned out time and again that those guys over-charge and under-deliver. Shoddy lenses, shoddy coatings, even shoddy frames sometimes. Consumer Reports tells me that Costco is stand-out in quality and price, and as a matter of fact that’s where I got my glasses many times. If I were looking for a quality pair of reasonably priced glasses to try on before I buy, that’s where I would go.

That was the easy part — now for online retailers. The first challenge is even knowing what to buy. You’ll need your measurements, all of them. What size frames are good for you, your prescription, and also an obnoxious number called the pupillary distance (PD). PD is the straight line separation between your pupils when looking straight forward, and is used to place the optical centers of the lenses correctly. It’s also not officially part of your prescription, and difficult to measure reliably on your own. All opticians are equipped to take this measurement in order to sell you glasses. The only reason you could possibly need this number yourself is to order online, and that is why pretty much no optician anywhere will give you that number. It is not legally part of your prescription. Many online retailers tell you how to measure the PD by yourself. Don’t do this. I got mine by asking Costco to give me the stats on the last frames I bought from them. I’m due for a new prescription any day now, and I intend to find an optometrist (not an optician) who will give me the number properly as part of the exam. You’ll also need to know the ballpark for what frames fit you, best established by finding frames that already fit you and getting the measurements off them. Whether it’s ethical to do that by trying glasses in a store you won’t buy from, I leave for you to decide.

So where do you buy from? Pretty much everything you need to know is at GlassyEyes. I believe Zenni Optical is the largest and most popular of the online retailers, and I’ve been fairly happy with them myself. Just keep in mind that you’re getting made-to-a-price Chinese frames with Chinese lenses and Chinese coatings shipped from China, with everything that implies. Anti-reflective coatings have a substantial mark-up here, but Zenni charges $5 and corners are getting cut somewhere. Support and returns are also about as easy as you’d expect with any of these companies, which is to say dismal. If you have a bad experience, you’re not likely to find a good resolution. But for the price of a reasonably decent Costco pair, I can easily order four or five pairs from Zenni with all of the relevant coatings. The Costco will be better, though, so it’s still not clear if things stack up in your favor. At the very least, online is a great way to get backup pairs, prescription sunglasses, or costume glasses. That’s assuming you’re not in a hurry, because these things take 2-4 weeks and there are horror stories out there about things going completely awry.

So that’s the run-down. Retailers = reliable and safe but horrifically expensive. Online = enormous cash savings and super sketchy. Actually fairly typical. So which one to pick? If this is for a first time purchase, go to a retailer. Seriously. First-time wearers have absolutely no business buying online. If you have a very complex prescription or health problems with the eyes, you’re probably better off with the retailers. The online places are just too likely to screw it up, and you need someone who is equipped to CHECK the lenses you get back from the lab. (Some optometrists will do this for ordered glasses though.) If you’re poor/broke or just looking for backup pairs, online is a great way to get them. I have a couple lying around from online and they do the job. The Zenni coating is clearly trash, though, and I suspect some of these lenses from other online shops won’t be coated in 6-12 months. But at $20 per, it’s difficult to care.

If you’re looking for a primary pair though, especially if you’re not quite certain about the necessary measurements, then it gets tricky. I ordered four pairs recently to try out. One is great, one is workable, one got returned due to incredibly poor fit, and one is being donated because it looks laughably terrible on me and returning it would net me nothing after shipping. That’s a risk I took, and I’m a little irritated about the losses on the bad ones, but my new sunglasses from Zenni are really nice, and these rimless from Goggles4U are adequate too. But I don’t love them; they fit great and look nice but they’re made poorly and it’s kind of a hassle. The people over at Optiboard would gloat; that’s a forum specifically for people in the optical industry, and their attitude about online purchasing is exactly what you’d expect, turned up to 11 in some cases. (By the way guys, online optical shops may be a lot of negative things but ‘criminal’ isn’t among them.) Remember that online optical, though in its infancy, represents an existential threat to these people’s careers.

All the same, I still need a primary pair of glasses. I used a pair from Zenni as my primary for about a year but they were never quite right and the coatings just plain rubbed off. They’re scratched up badly. So I thought to myself, this one time I’m going to do it right. I’m going to visit an actual store, pay for real frames and real lenses, and have a real professional set them up just so. Pretensions being what they are, I wasn’t willing to buy ‘good’ frames unless they were Oakley, and the local Oakley distributor optician guy was really incredibly nice and incredibly helpful. I was pretty much ready to buy from him until I got the price tag: $550. Ouch. I know there’s about $200 in markup on the lenses alone there. I can get a Leica 14 element or Zeiss 7 element camera lens for not much more money, and corrective lenses are not in the same league of quality as those beauties.

Right now, I’m researching some alternatives on how to get real brand name, good quality glasses. The Internet is here, after all, and I’m all for ruthless global competition. (That’s a nod to you libertarians, as long as we don’t have VAT in the US.) One of the big problems with online is the lack of professional adjustments, but it’s not like you always have to take your glasses to the person who sold them to you. Of course a glasses store is hoping to get your business with services like free adjustments, but I don’t mind adding an extra fifteen bucks on top for fifteen minutes of that person’s time, and that’s more than I’m paid. I’m also going to find an optometrist who doesn’t have skin in the retail game and is willing to help me see better, regardless of who I pay for the glasses themselves.

The seminal work driving this effort is really the findings by Daphne Bevalier at the University of Rochester. All of the papers are available for download as PDF, if you are so inclined. I imagine some background in psychology, cognitive science, neurology is helpful to follow everything that’s going on. The basic take-away, though, is that video games can have dramatic and long-lasting positive effects on our cognitive and perceptual abilities. Here’s an NPR article that is probably more helpful to follow as a lay-person with no background. One highlight:

Bavelier recruited non-gamers and trained them for a few weeks to play action video games. [...] Bavelier found that their vision remained improved, even without further practice on action video games. “We looked at the effect of playing action games on this visual skill of contrast sensitivity, and we’ve seen effects that last up to two years.”

Another rather interesting bit:

Brain researcher Jay Pratt, professor of psychology at the University of Toronto, has studied the differences between men and women in their ability to mentally manipulate 3-D figures. This skill is called spatial cognition, and it’s an essential mental skill for math and engineering. Typically, Pratt says, women test significantly worse than men on tests of spatial cognition.

But Pratt found in his studies that when women who’d had little gaming experience were trained on action video games, the gender difference nearly disappeared.

As it happens, I’ve wound up involved in this field as well. I had the good fortune to meet a doctor at the Johns Hopkins medical center/hospital who is interested in doing similar research. The existing work in the field is largely focused on cognition and perception; we’ll be studying motor skills. Probably lots of work with iPads, Kinect, Wii, PS Move, and maybe more exotic control devices as well. There’s a lot of potential applications, but one early angle will be helping stroke patients to recover basic motor ability more quickly and more permanently.

There’s an added component as to why we’re doing this research. My team believes that by studying the underlying neurology and psychology that drives (and is driven by) video games, we can actually turn the research around and use it to produce games that are more engaging, more interactive, more addictive, and just more fun. That’s our big gambit, and if it pans out we’ll be able to apply a more scientific and precise eye to the largely intuitive problem of making a good game. Of course the research is important for it’s own sake and will hopefully lead to a lot of good results, but I went into games and not neurology for a reason

Do not install ANY ad blocking add-in. It isn’t necessary. IE has a facility called Tracking Protection Lists that will do the job. You just need the right setting…

From IE9, visit Privacy Choice, who maintain block lists. Click either giant button, and Internet Explorer will offer to install it. That’s all you need to do. Most ads will now be blocked.

The downside is that, as far as I can tell, you can’t really configure this with custom filters like Firefox’s popular AdBlock Plus extension. Google Chrome’s available extensions offer custom filters but they don’t work correctly. Still, this does clean up nearly everything you’re likely to encounter in most browsing.

SVN is a long time stalwart as version control systems go, created to patch up the idiocies of CVS. It’s a mature, well understood system that has been and continues to be used in a vast variety of production projects, open and closed source, across widely divergent team sizes and workflows. Nevermind the hyperbole, SVN is good by practically any real world measure. And like any real world production system, it has a lot of flaws in nearly every respect. A perfect product is a product no one uses, after all. It’s important to understand what the flaws are, and in particular I want to discuss them without advocating for any alternative. I don’t want to compare to git or explain why it fixes the problems, because that has the effect of lensing the actual problems and additionally the problem of implying that distributed version control is the solution. It can be a solution, but the problems reach a bit beyond that.

Committing vs publishing
Fundamentally, a commit creates a revision, and a revision is something we want as part of the permanent record of a file. However, a lot of those revisions are not really meant for public consumption. When I’m working on something complex, there are a lot of points where I want to freeze frame without actually telling the world about my work. Subversion understands this perfectly well, and the mechanism for doing so is branches. The caveat is that this always requires server round-trips, which is okay as long as you’re in a high availability environment with a fast server. This is fine as long as you’re in the office, but it fails the moment you’re traveling or your connection to the server fails for whatever reason. Subversion cannot queue up revisions locally. It has exactly two reference points: the revision you started with and the working copy.

In general though, we are working on high availability environments and making a round trip to the server is not a big deal. Private branches are supposed to be the solution to this problem of work-in-progress revisions. Do everything you need, with as many revisions as you want, and then merge to trunk. Simple as that! If only merges actually worked.

SVN merges are broken
Yes, they’re broken. Everybody knows merges are broken in Subversion and that they work great in distributed systems. What tends to happen is people gloss over why they’re broken. There are essentially two problems in merges: the actual merge process, and the metadata about the merge. Neither works in SVN. The fatal mistake in the merge process is one I didn’t fully understand until reading HgInit (several times). Subversion’s world revolves around revisions, which are snapshots of the whole project. Merges basically take diffs from the common root and smash the results together. But the merged files didn’t magically drop from the sky — we made a whole series of changes to get them where they are. There’s a lot of contextual information in those changes which SVN has completely and utterly forgotten. Not only that, but the new revision it spits out necessarily has to jam a potentially complicated history into a property field, and naturally it doesn’t work.

For added impact, this context problem shows up without branches if two people happen to make more than trivial unrelated changes to the same trunk file. So not only does the branch approach not work, you get hit by the same bug even if you eschew it entirely! And invariably the reason this shows up is because you don’t want to make small changes to trunk. Damned if you do, damned if you don’t.

Newer version control systems are typically designed around changes rather than revisions. (Critically, this has nothing at all to do with decentralization.) By defining a particular ‘version’ of a file as a directed graph of changes resulting in a particular result, there’s a ton of context about where things came from and how they got there. Unfortunately the complex history tends to make assignment of revision numbers complicated (and in fact impossible in distributed systems), so you are no longer able to point people to r3359 for their bug fix. Instead it’s a graph node, probably assigned some arcane unique identifier like a GUID or hash.

File system headaches
.svn. This stupid little folder is the cause of so many headaches. Essentially it contains all of the metadata from the repository about whatever you synced, including the undamaged versions of files. But if you forget to copy it (because it’s hidden), Subversion suddenly forgets all about what you were doing. You just lost its tracking information, after all. Now you get to do a clean update and a hand merge. Overwrite it by accident, and now Subversion will get confused. And here’s the one that gets me every time with externals like boost — copy folders from a different repository, and all of a sudden Subversion sees folders from something else entirely and will refuse to touch them at all until you go through and nuke the folders by hand. Nope, you were supposed to SVN export it, nevermind that the offending files are marked hidden.

And of course because there’s no understanding of the native file system, move/copy/delete operations are all deeply confusing to Subversion unless it’s the one who handles those changes. If you’re working with an IDE that isn’t integrated into source control, you have another headache coming because IDEs are usually built for rearranging files. (In fact I think this is probably the only good reason to install IDE integration.)

It’s not clear to me if there’s any productive way to handle this particular issue, especially cross platform. I can imagine a particular set of rules — copying or moving files within a working copy does the same to the version control, moving them out is equivalent to delete. (What happens if they come back?) This tends to suggest integration at the filesystem layer, and our best bet for that is probably a FUSE implementation for the client. FUSE isn’t available on Windows, though apparently a similar tool called Dokan is. Its maturity level is unclear.

Changelists are missing
Okay, this one is straight out of Perforce. There’s a client side and a server side to this, and I actually have the client side via my favorite client SmartSVN. The idea on the client is that you group changed files together into changelists, and send them off all at once. It’s basically a queued commit you can use to stage. Perforce adds a server side, where pending changelists actually exist on the server, you can see what people are working on (and a description of what they’re doing!), and so forth. Subversion has no idea of anything except when files are different from their copies living in the .svn shadow directory, and that’s only on the client. If you have a couple different live streams of work, separating them out is a bit of a hassle. Branches are no solution at all, since it isn’t always clear upfront what goes in which branch. Changelists are much more flexible.

Locking is useless
The point of a lock in version control systems is to signal that it’s not safe to change a file. The most common use is for binary files that can’t be merged, but there are other useful situations too. Here’s the catch: Subversion checks locks when you attempt to commit. That’s how it has to work. In other words, by the time you find out there’s a lock on a file, you’ve already gone and started working on it, unless you obsessively check repository status for files. There’s also no way to know if you’re putting a lock on a file somebody has pending edits to.

The long and short of it is if you’re going to use a server, really use it. Perforce does. There’s no need to have the drawbacks of both centralized and distributed systems at once.

I think that’s everything that bothers me about Subversion. What about you?