Windows API Code Pack — Is It Any Good?

Just to be clear, I’m restricting my comments to the DirectX section of the codepack. I’ll probably be integrating some of the other bits (jump list support etc) into SlimTune, so we’ll see how that goes later. But around a week ago, they finally released 1.0. Same day as Windows 7 was out on MSDN actually — I’m pretty sure that’s not a coincidence. SlimDX’s release with Direct3D 11, Direct2D, and DirectWrite should come later this month, if all goes well. Now, the code pack does cover a few things we don’t, like the Windows Imaging Component.

So it’s not beta anymore. Now, I’m fairly sure that they haven’t looked at our code for legal reasons, but I did make some harsh comments about their work. They’ve made some changes that seem to follow directly from those comments. I’m about to make some more. I’ve been perusing the release and frankly, it’s just not any good. They seem to have spent most of their time implementing equally shoddy support for D3D 10 than fixing the actual problems. I’m going to run through all the reasons I see that this thing is not well done.

Okay, so they added a math library. I very pointedly slammed them for not having one in the 0.90 release, so let’s start with the bread and butter of graphics — Vector3F, as it’s called in the Code Pack. It offers X, Y, Z, Normalize, NormalizeInPlace, static Dot, static Cross, operator +, operator -, and (in)equality comparisons. Yes, that’s the entire class. No ref overloads, which are important for performance. No other helper methods of any kind. No PropertyGrid or System.ComponentModel compatibility. Matrix is even sadder — it has operator * (by-value only) and Identity. That’s it. Compare to ours, or XNA.

Functions that return object references still create brand new instances, which then not only have to be garbage collected, but if you don’t remember to Dispose them, they’ll be queued for finalization too. (These USED to be properties…it’s an improvement, I guess.) This is a similar effect to the original MDX’s event problems, just smeared out over time and difficult to track. There’s certainly no leak tracking functionality like SlimDX has. (OTOH they will be released eventually, which SlimDX does not promise.) These are lots of small allocations, which the .NET GC is good at handling, but if you don’t remember to Dispose them, or have a lot of them in general, this could really sour your day. It’s a problem that just doesn’t exist in SlimDX.

As for 64 bit support, it’s simply not configured at all in the solution (remember, the code pack is source code only, no binaries). I set up and ran an x64 build that went off without a hitch, and there’s no inherent reason for x64 to not work. I haven’t tested it though, and neither has anyone else apparently.

Lastly, even though they’ve added D3D 10 support, there’s no D3DX support in here at all. They basically invite you to go ahead and write what you need yourself, but none of it is done for you. For something that’s intended to make your job easier by letting you use managed code, this is another odd omission.

The 1.0 of the code pack IS dramatically improved in several respects — 0.90 had no math code at all, the memory situation was far worse, etc. Even so, this really doesn’t inspire a lot of confidence. Although the core APIs are wrapped, the support code is basically non-existent. There’s no binary distribution or redistributable, so you’re on your own there. I know this is probably a small team at Microsoft with nowhere near the level of resources it needs, and I’m sorry that I’m continually trashing your work. But if this is what constitutes the successor to Managed DirectX, I don’t think SlimDX is in any danger and I can’t say I mind.

SlimTune Profiler 0.1.5 Released!

Let’s recap. For about two months now, I’ve been working on a brand new profiling tool for .NET, C#, CLR, and all that jazz. It’s open source, completely free, and supports frameworks 2.0 and later (no 1.x, sorry). Some of the notable features include remote profiling, real time results analysis, and multiple visualizers. Today, the first public release, version 0.1.5, is available to the public.

Project Homepage
Direct Link to Installer

Although this is still an early version, it is already quite capable. It supports sampling mode profiling for both x86 and x64 applications, and provides views that will be familiar to users of NProf or dotTrace. Speaking of NProf, it’s my belief that this completely replaces it for .NET 2.0+, with a better UI and more features too. (And a far more lenient source code license as well.) There is still a lot to come, of course, but with this release I finally feel that this is ready for the general public.

I’m looking forward to getting lots of feedback, both positive and negative, and I hope that this is a useful tool for everyone.

(P.S. If you want to build from source, you’ll need to do it with a non-express version of VC++ 2008 SP1 and VC# 08, with a full boost installation. Also install the SQL Server Compact redist, which is in the repository under trunk\install\ExtraFiles.)

SlimTune’s Hybrid Mode

I decided to try out the dotTrace Profiler, which runs $200 for a personal license and $500 per developer for organizations. IOW, it’s expensive. That $200 license makes it the cheapest of the commercial options, and I ran the trial on one of my games. They have some nice UI touches I like. The data is valuable as well — I would not have guessed that my MainGame.Update function takes five billion percent of total program time.

---

Generally speaking, profilers operate in one of two modes: sampling (statistical), and tracing (instrumenting). Sampling operates by suspending the process at high frequency and examining the program state. It converges to a decent overview of where your code is spending time, but it doesn’t produce meaningful timing information. The frequency simply isn’t high enough. Tracing injects calls to the profiler every time a function is entered or exited, allowing it to monitor the complete progression of your code. You get accurate results with fairly reliable timings, but it’s incredibly slow. (Oh, and it crashes dotTrace. People pay for this?)

I’ve been looking at doing a hybrid mode since the beginning of the project; I finally came up with a concrete approach when a friend gave me a rough overview of SN Tuner, the profiler for the PS3. The idea is fairly simple: you don’t generally want tracing-level accuracy for the vast majority of the code. All you really need is an overview, which sampling does a good job of, and then tracing when you’re focusing on one specific piece. You also don’t really need detailed profiling of framework code (everything in System, for example). Although I’m still working on the front-end, SlimTune is now able to do this type of selective instrumentation at runtime.

Using it is pretty simple. When you start up the target, select Hybrid in the SlimTune UI. This will cause the program to run in sampling mode, and you’ll get your overview results. Then, you can select a function from the overview and ask it to be traced, and then results will flow in from that function and its children only. You can also turn it off again, and you can ask for entire namespaces to be skipped in either tracing or hybrid mode. Hybrid mode is a little slower than sampling overall, but it allows you to get very detailed results without the huge performance hit that normally accompanies that level of detail.

Internally, it’s a little tricky to pull this off but it’s not too bad. I discovered early on that taking a lock on the function hooks is hideously expensive, even at zero contention. I use a few lockfree tricks to get the necessary data much faster. It’s also very important not to let the sampling profiler attempt to sample inside the hooks, as this leads to some nasty deadlocks; again, lockfree code is used to lay out some unsafe zones that the sampler can detect and avoid. SuspendThread is one messy son of a bitch.

So there are at least three features I’m giving you for free which a five hundred dollar competitor doesn’t have. Sure they have a much cleaner interface, VS integration, memory profiling, and so forth…but I’ve only been doing this for a month. Kinda makes me wonder. Oh, and guess what I spent the last day or so doing…

Cleaned up version of dotTrace style visualizer

More SlimTune Pictures

Hey, why not?

More pies, courtesy of my other monitor.

Prototype of the connections window

SlimTune’s NProf Style and Pie Visualizers

SlimTune’s UI supports pluggable visualizers. What I realized was people were going to want to see their data sliced in different ways, and there would be no sane way to anticipate all those needs, let alone fuse them into a single viewing style. You’ll actually be able to drop in .NET assemblies of your own as plugins and have the option to view the profiling data using YOUR plugins, or the ones I ship. Multiple plugins at once on the same data? Check. Real-time view? As long as the plugin supports it. Remote profiling? Absolutely check.

I spent today working on the support for visualizers, and threw together an NProf style visualizer. It’s sampling-only, doesn’t handle real time yet, and generally a bit rough around the edges. But seeing as how I was typing in SQL by hand before, this is a pretty useful step up. It looks pretty decent, I think:

NProf-style visualizer

And then there’s this thing that took two or three hours, is a pain in the ass to navigate, but does update in real time:

Pie chart visualizer

A New NProf Release?!

I guess what I posted about SlimTune must have made the rounds or something, cause three days after I said what I was doing, a new NProf was released. Here I went to all this trouble of creating a new profiler, and all I really had to do was complain

I haven’t played with the new iteration in detail, but it does seem to have some nice new touches. I don’t actually believe it’ll be a competitor for me once I’m done — but hey, his is functional. Mine’s just a toy right now. I’m willing to concede at least that much. It’ll do the job for most people.

On a side note, after discussing with friends, I’m decided to use MIT license after all. I think it’s more advantageous to me to relieve any legal concerns that anybody may have than to maintain some vague semblance of control over the code. I’ve also more or less confirmed that SlimTune will be able to profile fully native apps eventually. It won’t be a feature in the initial release, most likely, but it’s coming.

Moar SlimTune

Primarily due to Washu request!

Mappings, native functions included!

It's...an actual profile!

Another run, slightly improved SQL.

Tracking constructor calls!

Remote Profiling will NOT be secure in SlimTune

At least, not to begin with. There are some drawbacks to not being a security professional, one of which is that I have neither the qualifications nor the experience to do a proper security analysis of the profiler backend. Since I can’t audit the backend for security, it will be considered insecure, and that’s that.

The practical result of this is that allowing uncontrolled remote connections to the profiler will be incredibly dangerous. I am planning to include a setting that disallows connections except from localhost. However, if you are actually using remote profiling on something that might be attacked, it’s critical to make sure it is behind a firewall that will not allow arbitrary connections.

Eventually it should probably allow you to set a username and password for connections, but that’s again something that takes some care to implement properly and I’d rather not be the one doing it. In any case, that’s functionality which will come much later. Sorry if secure remote profiling is high on your list.

And so it begins…

Mappings test, July 15

SlimTune Profiler for .NET

I basically took last week off from blogging. Time to try and get some new entries out! Things have been very SlimDX focused, but what did you expect? It’s what I do. Maybe today’s will inspire a bit more general interest.

---

As a creature of GameDev.Net, I get to see lots and lots of discussions questioning whether or not C# and .NET are “fast enough” for games. What I don’t much is people working on actually analyzing and tuning the performance of their .NET code to see what’s going on. I’m not sure why this is, but I have a theory that it’s partly because of the sorry state of available performance tools. The only version of VS that has a profiler is Team Edition, which damned near nobody has. Other commercial offerings are also seriously expensive. There are only two free profiling tools that are really available for use: CLR Profiler and NProf. (I’ve seen a few other tools, but it’s clear that they’re fringe tools that aren’t well supported.)

CLR Profiler is written by Microsoft, and it’s a pretty good tool. They’ve even released the source code, although the licensing is vague. It has a few drawbacks though. First of all, it only does memory profile analysis. It does a very good job of tracking allocations and garbage collections, and the visualizations are very well done too. But that’s all you get — no timings of any kind, let alone a breakdown of where time is being spent. Also, it hasn’t been updated since late 2005.

Then there’s NProf. Oh dear. The good news is it works, barely. That bad news is that’s all the favorable comments I have about it. It does simple sampling based profiling only, and will show you a simple tree based breakdown of time spent. It’s not that NProf is useless; I’ve done lots of good performance tuning with it. But this is literally all it can do, and there’s a lot more you want from a profiler. The last release was December 2006, and there’s some scattered SVN traffic since then but it’s basically dead. Support for x64 is apparently doable if you compile from source. I looked at the source, which is also poorly written. I decided immediately that I could do better than this toy, and now I’m putting my money where my mouth is.

I’m working on a new open source profiler tool right now called the SlimTune Profiler. It will probably release in early September, and the initial feature set is taking direct aim at NProf. The initial version will support sampling and instrumentation profiling for .NET 2.0 and above on local and remote machines. A little later on, you’ll be able to profile-enable a long running process at zero performance cost, and then profile it in real time for short periods. Imagine running a production server, and actually connecting with the profiler while it’s serving real requests to see what’s happening.

On the front-end, data will be collected from the profiling backend and dropped into an embedded relational database. There will be some preset views of the data, but the idea here is that you should be able to apply your own queries to the data and get results that are useful to you. Reporting is not expected for the initial version, but it will be supported eventually as well. I imagine you’ll be able to create various tables, graphs, etc and export them, although I’m not sure exactly what format that’ll be in. PNG and Excel seem reasonable. I’m hoping that you’ll be able to combine results from multiple runs, which would allow you to make all kinds of snappy graphs to show off to your boss.

It’s been my plan for some time now to expand beyond SlimDX, and create a suite of Slim software. We’ve got a good reputation and lots of respect for our work, and I’m looking to build on that. SlimTune is the first step. It probably won’t be able to compete with the commercial offerings — but RedGate ANTS runs $400 or more per license. SlimTune will blow NProf out of the water in a scant two months, and it won’t cost you a dime. The feature set is pretty well specified, and the profiler already works in prototype form. The work over the coming weeks is in building a product instead of a project.

And yes, I know I’m a tease. It’ll be worth the wait.