Reality is an illusion – .NET OSS is hard

So lots of posts over the last few days about that old chestnut, OSS and .NET:

Those are all good, well-written posts, and they all make very good points, and I don’t want to debate any of them, but I had a tangential thought on the subject last night while travelling home from the Dot Net Rocks UK Tour event in London. I may be repeating things other people have said, in which case you should definitely tweet me a link to their post with a rude comment about how they said it better.

TL;DR

  • Ruby, Node, etc. are text-based; text-based is easier to extend.
  • .NET is Visual Studio-based; Visual Studio is harder to extend.

.NET is an unusual ecosystem

When groups of developers come together to talk about the state of OSS in .NET, the comparisons were traditionally drawn with Ruby; more recently Node.js has also become a focus for analogy. Both those ecosystems are almost entirely open-source, with lots of healthy competition and innovation, and a genuine sense that if somebody builds a better mousetrap, people will switch to it. “Why isn’t the .NET world more like that?” people ask, and they talk about whether it’s Microsoft’s fault and what they could do to help. What I rarely hear mentioned is the humungous elephant in the room: Visual Studio.

Ruby, Node, Python and most of the other OSS darlings are predominantly CLI-based. There are excellent tools for scaffolding MVC sites, running builds and tests, managing version control or migrating databases; and the thing they all have in common is that they are command-line tools. That is what developers in that ecosystem expect (and, incidentally, is why they prefer Linux and OSX, because they have bash and decent Terminal applications built-in). Text editors are commonplace, from Vim to Sublime Text, and although excellent IDEs are available (thanks mostly to JetBrains) they are luxury accessories rather than fundamental components. I have no idea what the ratio is of Vim users to, say, WebStorm users in the Node.js world, but I wouldn’t be surprised if it came down fairly heavily on the side of Vim, or at least Sublime Text (with or without the excellent Vintageous plug-in).

In the land of .NET, Visual Studio is king. It scaffolds, it builds, it runs tests, it manages version control and it does this really bizarre thing which is instead of migrating databases. You don’t need a CLI, which is lucky, because even though there’s PowerShell now, the Console window it runs in is still a 20-year-old piece of shit.

Disclaimer: I don’t really know about Java, and I don’t really want to, but I suspect it’s different because there are, like, nine different IDEs and even though only one of them doesn’t suck it’s probably still easier to write command line stuff (and also code completion in Java is really just a tiny floating window into a yawning abyss of despair).

The Visual Studio experience

So the majority .NET developers are, first and foremost, Visual Studio users. And Visual Studio with a decent productivity add-on is the best IDE ever, by a considerable margin; yes, it is, and if you try to argue with that in the comments I won’t approve it so don’t even bother. It holds your hand and guides you through things, and it doesn’t demand that you actually remember a whole bunch of stuff, like, in your head. Scaffolding is done with wizards; builds and tests are run at a key-press with colourful graphical output; version control is integrated into the Solution explorer; the database design stuff is all drop-downs and pop-ups.

People complain that Microsoft built Entity Framework instead of embracing NHibernate, but NHibernate never had an Entity Model Designer integrated into Visual Studio (correct me if I’m wrong (Update: I was wrong; MindscapeHQ do an NHibernate designer, albeit a commercial product)). The OSS section of the stands may boo and hiss at the very idea of the Entity Model Designer, let alone the horrible, horrible code that it generated, but it made the ORM concept approachable for a vast swathe of developers who would otherwise have carried on blithely binding web forms to DataSets.

I don’t know whether it would have been realistic for Microsoft to add tooling for NHibernate to Visual Studio than to write their own ORM from scratch, but I suspect writing their own and creating the tooling as they went along was easier.

The same can’t really be said for MSTest, which was a hazardous by-product of the NIH mentality of the time, but it did allow integration with Visual Studio and various frameworks (think “Generate Unit Tests”) in a way that blessing NUnit probably wouldn’t have, at least not without Microsoft developers contributing to that project on company time.

And so on and so on. ASP.NET MVC quietly adds a bunch of features to the IDE, such as “Strongly-typed Razor View with Layout” in the New Items dialog and the “Add Controller…” option on the Controllers folder context-menu. NuGet gets more and more integrated and powerful with each point-release.

How does this affect OSS?

It makes it difficult. It raises the bar, and demands skills and investments of time that most OSS framework or library projects, especially fledgling ones, are unlikely to be willing or able to muster.

In a text-based world where pretty much every tool is a script, it’s easy to compete, because scripts are (relatively) easy to write.

In an IDE world, especially where the IDE’s SDK is baroque and complex and poorly-documented with few good examples available, it’s hard to compete, especially when the people you’re competing with are the people who make that IDE.

Does this mean that .NET OSS is doomed to edge status, an underground movement of which the majority of surface-dwellers are blissfully unaware? Maybe. But there are things that OSS projects could do to improve their chances; things which some are starting to do, in fact.

Things your OSS project should bring to the party

If you’re an application framework, you should at the very least provide Project templates and maybe Item templates. Bonus points for injecting context-specific items into the Solution Explorer menus. If you’re a web framework that supports Razor, find a way to make the IntelliSense work properly in .cshtml files, or use a .razor extension and write your own IntelliSense provider for it. Related: if you’ve invented your own View Engine, Item templates and IntelliSense are expected.

BTW, framework projects: you still shouldn’t expect to gain a huge amount of traction as a percentage of the whole. The overwhelming majority of .NET web development in medium-to-large enterprises still uses ASP.NET Web Forms, and always will, because drag-and-drop and Infrabloodygistics.

If you’re a testing framework, you need to integrate with the built-in Unit Test explorer, as well as ReSharper and other third-party tools. If you can find more ways to provide a lovely hand-holding experience, like SpecFlow’s editor integration and test class generator, then even better.

If you’re a database access library that uses dynamic trickery to do your thing, you still need to make it work with bloody IntelliSense.

And so on.

(Also: documentation, but I’m going to be announcing something about that soon.)

Things Microsoft could do to help

Improve the Visual Studio extensibility story, both in terms of the SDK and the documentation of the SDK. Provide more, and more relevant, examples. And open source the Visual Studio components from your own frameworks for OSS authors to use as a reference. (A good example here is TypeScript; the language, compiler and so on is completely open-source, but the Visual Studio package is closed-source, and I don’t understand why.)

Kudos at this point to Mads Kristensen, a PM on the Microsoft Web Platform team and author of the Web Essentials extension, for open-sourcing that project. The source code for that is an excellent reference for anyone looking to extend Visual Studio at the editor and Solution Explorer levels.

Also, provide MSDN Ultimate subscriptions for established OSS projects, applying the same criteria and license-restrictions as companies like JetBrains and Telerik do for their OSS-supporting licenses. And when the creators of a good OSS project do provide a decent level of integration with Visual Studio, make them Visual Studio MVPs and Insiders so they can access more information about the thing they’re trying to work with.

 

Update: given some of the comments, I think I should clearly state that I am an MVP (though not for any of my open source stuff) and also a member of the BizSpark+ program, and thus have more MSDN Ultimate subscriptions than I can actually use. I am not advocating this for my own personal benefit, but for the benefit of other open source project maintainers who are not so well-resourced as I am.

Comments

  1. Your opinionated rants are far too reasonable sir, you make the rest of us sound like arguing drunks.

  2. The question here is whether OSS is really a necessity. In the Ruby, NodeJS and to a smaller extent the JAVA-world the situation is completely different. Those environments provide a bare minimum to start with and then count on OSS to complete the platform (environment, framework, however you want to call it).

    In the .NET-world, a lot of this comes in the box. Of course, pieces are missing, and that’s exactly the reason why there is some open source. For example, like you said, Visual Studio is the best IDE ever. Who’d want to spend effort in creating an open-source IDE? In Java, apart from IntelliJ, I haven’t seen a lot of good editors, so the market is more open.

    We think of OSS as inherently good and desirable, but maybe .NET just doesn’t need as much of it.

    • Visual studio may be the best IDE ever in terms of features, but I’d love to turn many of them off that I don’t use to get some sort of speed improvement. I suspect that by doing that, many of the conflicts between components would also go away. I’ve lost count of the number of times VS freezes on me. I suspect is because of two or more parts competing to update themselves in the wake of a fairly basic change that I’ve just made (e.g. removing a file from the project).

  3. One of the big problems, as you alluded, is that there’s a significant barrier for OSS libraries; about $500 or however much the cheapest non-free VS version costs. This means that if you’re an OSS developer and want to make a nice VS integration you have to have VS already (from other projects or work), which limits the potential market of developers.

    Giving the Ultimate version to big OSS projects poses a chicken and egg problem. Wouldn’t smaller projects need it the most so that they could compete with big OSS projects, that could get away without VS integration anyway?

    The solution should be, at a minimum, to allow the free version to support add-ins. This way an OSS project could consider spending time on VS integration without having to invest in VS and also, very important, without asking their users to do so as well. OSS is about donating your time for free so that others could benefit. Having the price tag of VS attached to that defeats the purpose.

  4. Slightly tangentially, I remember John Lam showing off a VS plugin for writing VS plugins in IronRuby, back when that was something Microsoft cared about. It never saw the light of day – I think it was barely prototype-grade, to be honest – which is a crying shame, because making that API scriptable would have been *incredibly* useful.

  5. Great view on OSS and well written. As a start-up you can have 3 years of free Visual Studio. http://www.microsoft.com/bizspark/

  6. I mainly agree but I think this is no ones fault. Writing (or extending) an IDE is a very complex programming exercise. Memory and execution speed performance, asynchronous events, backwards compatibility, complex plugin dependencies — you will get these into your face immediately after stepping out of the demo-project level. And the users are the most critical ones… (see the story about the infamous Issue #128: http://bit.ly/15wis3Z). SpecFlow is a fairly large OSS project, but I have hardly received any contribution to the VS integration part… and I can fully understand that. If you want to see a complex code, just look at the VS Test Window implementation (with dotPeek ;-), I have never seen such an extensive usage of async calls…
    One thing is absolutely true: Microsoft (as a company – there are quite some good guys at MS who help a lot) could be more friendly to OSS projects. Not with free VS licenses (that you anyway have already), but with more active promotion of good tools, inviting them to conferences, maybe even product strategy plannings, etc.

  7. You absolutely nailed it: .NET = VS and that’s the biggest difference with other OSS ecosystems…
    But why isn’t there more momentum around {Mono|Sharp}Develop, then? They should be the default choice for OSS developers and the obvious way to break the link with VS.

    • Tom Miller says:

      I think it is dangerous, for lack of a better description, to refer to .Net == Visual Studio (or even vice versa). They are not one in the same; Traditional C/C++ is still an option in VS without .Net…it is simply an IDE. So, .Net != VS. I call this out because it is a common misunderstanding among those who aren’t familiar with VS and the .Net Framework in general. Either way, there are options, but seeing as MS offers the free Express versions of VS, this in and of itself makes it difficult to compete in the “free as in beer” IDE space, aside from the fact that VS Express versions are not OSS, and don’t allow extensibility, from what I understand. IMO, there really needs to be a distinction between .Net and VS when having the discussion around OSS–are we talking about extending the IDE, or are we talking about developing OSS using the languages and technologies supported by the .Net Framework and Visual Studio? This is really more of a cultural phenomenon vs availability of tools.

  8. I remember an interesting discussion with some jetbrains employes and their friends I overheard on some conference some time ago. Topic was if JetBrains will ever release IDE for C# – all people who are not from jetBrains loudly asked for it, and all people from JetBrains repeat and repeat that it will never happen… It is somewhat interesting why?

    — s.

  9. Good post! Vim is a pretty decent tool for C# too when using OmniSharp https://github.com/nosami/Omnisharp . I wrote it to help break free from the VS shackle.

  10. Interestingly the F# OSS world is a little more vibrant than the rest of the .NET world- think FSharp.Data, Foq, TickSpec, FsCheck, FsUnit, FParsec. This could be down to a couple of things –

    1: F# itself is OSS. Being open from the top down probably helps the ecosystem in a general sense
    2: It has CLI based tooling – from F# Interactive, to the sane command line compiler.
    3: F# community is trying to grow from under the C# shadow and providing F# specific (or F# friendly) solutions is a good way to help build and grow a community.

    F# also provides a great story for MS – it’s a first class citizen of VS with the usual great tooling, but can be installed without VS and used from the command line.

    C# feels like the “old” microsoft, and F# the “new” microsoft. Unfortunately people still peg MS as evil and closed and monopolistic, despite their recent open efforts, which feels like it hurts .NET OSS in general.

  11. Andrew Gilbert says:

    This is a great post, thankful to have discovered it (through your projects). It isn’t just the VS to .NET problem, but also the lack of natural selection on the library / API level, and being too stuck in the “statically typed / compiled / massive framework” world that are core problems. I had to come over from (happy) Python/Ruby/CLI land to .NET/VS land recently and the experience has been interesting. Some okay moments, but a lot of “you have to be kidding me, really?”. I found my way to this site because of Simple.Data, which was motivated solely by my horror of how unobvious it was to do simple database input/output say for utility or data migration work. EF certainly was a bad choice. Linq to SQL was a bad choice. ADO.NET should be the right choice, but it is really kinda of a whacky set of assumptions and abstractions, as compared to say something like Python or Ruby direct db access api’s.

    In the end, it would be “nice” if MS could compete better. But I have to say, it doesn’t seem encouraging. Viewing development tools as a revenue stream is shooting yourself in the foot. Assuming you (the company) can write better API’s, tools and abstractions than an open competitive and vibrant community is old school (think DEC) and also shooting yourself in the foot. And the previous two issues are making it very hard to adapt to the ever more rapid realities going on around you. A good example being that not everything needs to be a statically type, compiled, deeply abstracted pile of obtuse confusion. That is not a “professional” vs amateur question. It is “unproductive and uncompetitive” versus quick, efficient and successful. Alas I think company wise there is a lot of “interests” tied to the model and revenue stream. And I fear “community” wise there are also probably a lot of .NET/MVP professionals that view their knowledge of the obtuse as differentiating, and letting that go is an issue (barrier to change) as well.

Trackbacks

  1. Noticias 09-10-2013

    Valve presenta un Linux para jugar, un mando "hackeable" y unas specs para PCs de gaming .

  2. […] Rendle, quien ya en una ocasión había hablado sobre el oss en .net, ha publicado en su blog un articulo en el que habla sobre algunas nuevas características […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 4,057 other followers

%d bloggers like this: