On the .NET side, there's a namespace called System.Windows.Forms where most of the non-web UI library is found. There's some other usefull stuff, such as System.Drawing and System.Text, but mostly in the System.Windows.Forms.
As to how it compares to Swing, of course it depends 99.6% on who you ask. Ask a Java coder, and it sucks. Ask a .NET coder, and it rules.
I personally code in .NET (some Flash as well), so I'm very comfortable w/ WinForms. I'm very productive, and the .NET framework is very capable and flexible. The Visual Studio environment is well rounded and capable (and well reviewed). Debugging is outstanding.
.NET is also supported by a fairly vast framework besides what's in System.Windows.Forms. Things like web services are simple to implement, great XML support, great database access support.
For custom control development, there's a very usable library for native 2D graphics. The classes in the framework can be overridden as you would expect to do just about anything you can imagine, or you can write completely from scratch.
Performance is very good, close to native C++ on the Win32 platform.
Documentation is well integrated into the development environment and is quite complete.
C# has the ability to declare "unsafe" code sections, so if you're in a controlled environment where you can declare elevated privs to a piece of code, you can do pointer manipulation as you can in C++. This type of thing is handy in image processing for example. (you don't want to use any object hierarchy to loop through 6,000,000 pixels one by one...)
In addition to the native libraries, DirectX 9 now has a managed interface (the term they use for native .NET interface). Which makes it conceivable that you could write 3D apps / games in .NET. I've cracked open the DirectX 9 SDK, but need to brush up on matricies and transforms before I get anything accomplished in 3D :)
All in all, if you're open to new experiences, and non-religious about platform you will certainly enjoy working with C# on the Win32 platform. It's a good language, not too far from Java or C++, but a reasonable evolution of both.
The Framework is well rounded for an early version and you can get most everything you need done and done well. Much of it is actually very cool.
If you're hopelessly tilted against Microsoft, you may not be able to deal with it; this would be for no technical reason but rather for philosophical ones.