On open-source motivations and commitment

April 27, 2013 By 1 Comment

Last night I took part in a panel on Microsoft and OSS as part of dotNetConf, and I was trying to keep one eye on the Jabbr room while also being a useful part of the panel, which was fun. Anyway, at one point I said of Simple.Web something like: “I’m writing it for me and making it do the stuff I need,” and I noticed a comment in Jabbr that seemed to me, with my half-paid attention, to be questioning whether one could wisely adopt an open-source framework created with that motivation, and wondering what would happen when the author got bored of it. I wanted to answer that, and while I’m at it, give a status update on the things I’m involved with, so here goes.

The motivation for Simple.Web

I’m absolutely not ashamed to say that I am writing Simple.Web for myself. I started writing it because Web API went in a different direction than I needed, and because Nancy didn’t, at the time, support async handlers and content negotiation. There were a couple of other options, but nothing that had exactly the right mix for me, so I made one that did. And I’m still making it, and it’s what Zudio, my commercial start-up project, is running on. I’ve been heavily focused on that for the last few months, and most of the commits I’ve made to Simple.Web in that time have been things I needed for Zudio; I recently added the first part of CORS support because I needed it immediately, and I’ll be adding the rest of it pretty soon because I’m going to need the rest of it pretty soon. There have been other things that I haven’t needed, such as Razor support, or indeed any server-side rendering because Zudio doesn’t use that. Somebody else from the .NET community (hello @Cranialstrain) did need Razor support, and also needed the framework to support Mono, so he worked on that and sent me pull requests and I gave him commit rights on the main Github repository and he’s basically fixed the Razor support and it runs on Mono and he’s put an epic CI system in place to boot, and I think he’s finally starting the project he wanted to do with it: he’s added the stuff that he needs. See how this works?

The commitment to Simple.Web

What does this mean for my commitment to Simple.Web? I’d say it should inspire faith, not doubt. If my start-up is successful, I’m not going to suddenly port all the code to ASP.NET MVC, am I? No. I’m going to build on Zudio, add new features, try to improve performance, and a big chunk of the effort that goes into that will go into Simple.Web, which will remain completely open-source, MIT-licensed and open to pull requests. As long as they don’t break Zudio.

I imagine the same is roughly true for Ian, whatever it is he’s secretly working on.

Now, this does mean that this is likely to be one of those open-source projects where, if you ask for a feature that none of the current maintainers need for themselves, you might have a wait on your hands, but that goes with the territory, and rolling up your sleeves and trying to add that feature yourself will do you far more good than harm, I promise. Apart from a couple of slightly scary dynamic-code-generating rabbit-holes, the code is not that complicated.

What about Simple.Data?

The motivation for Simple.Data was different, in that it was far more experimental and driven first by “look, this works” and then by “ooh, people are liking the thing I made, I’ll make it some more”. I do use it; I use it on as many projects as people will let me use it on, because I really like it. But that’s not my motivation for supporting it and continuing to work on it. I’ve gotten a lot of other intangible benefits as direct or indirect results of building Simple.Data, and the fact that people seem to really like it (enough to talk about it on podcasts in foreign languages (that I listen to all the way through for the times they say things like “SQLite” or “InMemoryAdapter” which are the same in Swedish as in English)) still makes me very happy.

I didn’t get many hugs as a child. That probably explains it.

So, it’s not going away, I promise.

But it is going through something of a development lacuna, because I’m busy. Like, crazy busy. Things have to give, and Simple.Data is pretty much stable for most things, so it’s been dialled back. What’s actually happening is that Dan Maharry is working on the docs and finding bugs as he goes along, and when things calm down a bit I’ll fix those bugs and he’ll finish the docs and that will be 1.0 final. I’m really hoping that happens in early summer, because I want to start on 2.0, for which the headline feature is async*. It riles me that EF has that before Simple.Data. That’s the wrong damn way round. Must fix!

*Oh. And IntelliSense. :)

Anyway…

My current dream is that Zudio will do well enough to support me and a few other developers working on it full-time, and I’ll have more control over how I spend my time, with a good balance of happy open-sourcery and money-making entrepreneur stuff. That would be nice.

Filed Under: Uncategorized

Don’t unit test trivial code

March 9, 2013 By

Hopefully just a brief post in response to one I consider to be spectacularly wrong.

TL;DR

  • Don’t test other people’s code.
  • Don’t test implementation details.
  • Don’t pad your test suite with pointless tests.
  • Don’t have code which is only ever called by tests.
  • TDD is awesome; don’t make it look bad.

What’s all this about then?

The author of the above-linked post knows a hell of a lot about TDD and the various disciplines that go along with it, and I’ve generally got a lot of respect for him and his work, but when you’re wrong, you’re wrong, and when you’re an authority figure and you’re wrong it needs to be pointed out in case you make a lot of other people wrong too.

The assertion (no pun intended) at issue is that this line of code requires a unit test to cover it:

public int Year { get; set; }

The proposed tests for this single line of code are:

[Fact]
public void GetYearReturnsAssignedValue()
{
    var sut = new DateViewModel();
    sut.Year = 2013;
    Assert.Equal(2013, sut.Year);
}

[Fact]
public void GetYearReturnsAssignedValue2()
{
    var sut = new DateViewModel();
    sut.Year = 2010;
    Assert.Equal(2010, sut.Year);
}

The post goes on to discuss using parameterized tests to “save time”, but never thinks to suggest writing some reflection code that finds all the public types in an assembly, then all the public properties on all of those types, and tests setting them to all the possible values to make sure it always works, if you will excuse the reductio ad absurdum (of course, this wouldn’t show up in your code coverage, so you’d probably end up using T4 templates to generate parameterized tests and then check them into your test suite…).

Anyway, here are a few reasons why it’s wrong.

1. It’s testing the compiler, not your code

The line of code being tested is syntactic sugar for a property with a backing field. The tests are checking only that the compiler has done its job properly, and that’s already been extensively tested by the people who wrote the compiler. You don’t need to test it again. This applies to all language features, standard library functions and classes, and public APIs. You don’t test the .NET StreamWriter class to make sure it writes to a stream; you test the code you’ve written to make sure it writes the right thing to the stream.

2. It’s test-driving an implementation detail, not application logic

This Year property is clearly used by some application logic, otherwise it is redundant and should be excised. Both the getter and the setter for the property will be covered by the tests which test whatever the class with the year property actually does, and hopefully with a representative range of valid values.

There is no user story on the board which says “As a developer using the DateViewModel, I want to set the Year property to a positive integer, so that I can get it back again later”.

There may be a user story that says “As a user, I want to be prevented from accidentally adding events in the past”, in which case there should be some validation code that happens in the view-model when the Year property changes, and that should be tested.

In the paragraph headed “Causality”, the author makes the argument that “the tests drive the implementation”. This is true, but what you are implementing here is not a Year property; it’s some kind of higher-level date-related functionality that is required by your application. The Year property is a part of that implementation; it is not the implementation itself.

3. Pointless tests are worse than no tests at all

Using the suggested approach to testing, it would probably be possible to achieve 100% coverage of all public properties and methods in a project without actually testing 100% of the application logic. Now you have a metric that tells you you’ve done everything right, even though you’ve done everything wrong.

For example, let’s say our hypothetical DateViewModel also has Day and Month properties, and a Date property, all of which have getters and setters. Setting any of Day, Month or Year should change the Date property, and setting the Date property should change all the others. But with tests that test each property in isolation, you have 100% code coverage, and no useful information at all. Conversely, if you have a test for the Year property that asserts against the Date property, and a test for the Date property that asserts against the Year property, then the other tests are redundant, and redundant tests serve only to make the test suite look better than it really is.

4. It hides code redundancy

Code coverage is not just a metric for making sure you’re testing properly, it’s an excellent way of detecting redundant code. If you have worthless tests covering essentially non-functional code, then neither the compiler nor your coverage tool will ever be able to say “this property doesn’t appear to be used by anything”.

5. It presents TDD in a terrible light

The author of the post suggests that Bob Martin’s statement that “you don’t need to test-drive trivial code” is “horrible advice for novices”. I would submit that the counter-advice he goes on to give is far more horrible, since it makes TDD look unnecessarily time-consuming, pedantic and pointless. Telling people that they can’t apply judgement, common sense and pragmatism to TDD is going to result in them writing the whole thing off as a bad job, which is a far worse outcome than them maybe skipping the odd test here and there (which they will learn not to do over time, as those missed tests come back and bite them in the backside).

2013-03-12: The comments have descended into abusive nonsense, and been disabled.

Filed Under: testing Tagged With: ,

Saving Git credentials for HTTPS remotes in Windows

February 14, 2013 By 1 Comment

Very quick post

I just started using the new Git support in Team Foundation Service (which is also to say “I just started using Team Foundation Service”). It’s great; it’s proper Git and it works with the msysgit command line, but (as yet) TFS Git doesn’t support SSH authentication. After a few prompts for my user name and password when doing a git push origin, I decided I’d had enough and looked for an easier way. I found one.

Andrew Nurse, smart chap who works for Microsoft, has created git-credential-winstore, one of those apps that is unbelievably tiny and amazingly useful. Download it, run it, and the next time you push to a remote repository, up pops a Windows dialog instead. Enter the credentials in there, and on subsequent pushes to that repository, no prompt, no dialog, it just works. I love stuff that just works.

Get git-credential-winstore here.

Filed Under: git Tagged With: , ,

Post-mature optimization

January 31, 2013 By 8 Comments

I leave optimizations to the end, as much as possible. Get the code working, then profile it and optimize the problem areas. It was not always thus, but obviously one of the occasions when I found myself just deleting code that I had spent hours tweaking and perfecting, the penny dropped and I went for pragmatism and patience.

Of course, it’s possible to go too far the other way, and for too long put off optimizations which are [a] blindingly obvious and [b] not that difficult to apply.

Yesterday, my attention was drawn to the performance of Simple.Web, compared to ASP.NET MVC and Web API. Mattias Nordberg, an application performance consultant working in London, sent me a pull request. Not a huge change, Github shows it at 37 additions and 18 deletions. But it increased the requests-per-second of Simple.Web from 600 to 4,500, putting it roughly on par with ASP.NET MVC and 1,000 reqs/sec slower than Web API.

I ended up in a Skype chat with Mattias and one of his colleagues, Tomas. They said they were evaluating frameworks and performance was the main issue for them. Obviously they’d taken a pretty good look at the code, run it through a profiler and so on. The other thing they were worried about, they said, was the routing implementation. That was completely understandable: I threw together the original routing code in a bit of a hurry, if I’m honest, and I’d always intended to go back and tweak it, but I wasn’t sure how much of a block it really was, and how much I could improve it.

How not to do URI routing

  1. Turn every URI template into a regular expression with captures for the parameter names.
  2. Every time a request runs, run all the regular expressions.
  3. If more than one result is found, filter based on media types.

Seriously, that’s wrong, don’t do it. It’s slow and it absolutely devours CPU time.

In the Skype chat, Mattias mentioned something called the Rete algorithm. I was not familiar with it, and he admitted that he’d only just learned about it himself. I read the Wikipedia page and it sounded similar to the “tweak” I had in mind for the routing table, so I went ahead and had a go at implementing it.

I copied the regex-based Simple.Web routing classes into a console app, set up a routing table with 4,096 entries, and timed how long it took a single thread to run 4,096 lookups: 2.8 seconds. Slow. Much slower than I’d expected, but when I thought about it, not really surprising.

A better way to do URI routing

  1. Break each URI template down by its delimiter (i.e., forward slash).
  2. Construct a tree using the unique values at each level of the templates.
  3. Populate the tree with “matcher” objects that only check the relevant part of a URL:
  4. Every time a request runs, break its URI down and run it through the tree.

I think this is at least a nod in the general direction of the Rete algorithm, although in this case it’s been heavily adapted to the specific task of matching URIs to templates. I don’t know if I’ve made it sounds more or less complicated than it is, but you can look at the code if you’re interested. (One of the things that surprised me a little was that it didn’t actually need any regular expressions at all to reproduce the existing functionality.)

Anyway, initial implementation in a spike project took under an hour, and I ran the same performance test, 4,096 entries, 4,096 lookups: 0.04 seconds. 70 times faster. I reported back to Tomas and Mattias, who immediately asked “and with 10,000 routes?” My test harness was working in multiples of eight, so I bumped it up to 32,768 routes, and doing 32,768 lookups.

The old code took 19.5 seconds.

The new code takes 0.27 seconds.

When a route matched to the last template added to the routing table, the old code took 0.2 seconds. The new code takes 0.0005 seconds.

Oh, this is all still on a single thread, of course.

I integrated this new algorithm into Simple.Web last night, and the demands of reality meant that things slowed down a little, but a single lookup against a 32,768 route table still only takes a shade under 1.5 milliseconds, on an Ultrabook with a ULV Core i7 CPU.

What a difference a day makes

In summary, then, when I woke up yesterday, Simple.Web was an epic performance failure, managing 600 reqs/sec. Out of the blue I get a pull request which increases that almost by a factor of 8. Inspired, I finally get round to fixing the other performance bottleneck, with a speed boost somewhere between one and two orders of magnitude depending on how many routes you have. End result, it’s probably a good 10 times faster than it was yesterday, and it won’t slow down as the number of routes increases.

I’ve got some profiling software installed now, and weighttp, and I plan to run a few more tests and see if I can identify any other hideous bottlenecks that can be optimized without too much effort. I want to be faster than Web API.

I’ll be pushing new packages to NuGet in a day or two, but if you want the CI builds, you can add http://teamcity.cloudapp.net/guestAuth/app/nuget/v1/FeedService.svc/ to your package sources and install them direct from the TeamCity server.

Filed Under: Uncategorized

Building web apps in 2013

January 4, 2013 By 12 Comments

A Very Short Post:

I’ve lately come to the opinion that desktop application development is a moribund activity, which suits me quite nicely because I’ve never really liked any of the ways people invented to do it. So I’m getting my JavaScript on and building web applications, and more specifically, Single Page Applications.

To do this, I’m using AngularJS and TypeScript for the client-side development, and Simple.Web for the server side, and I’ve discovered this amazing thing: you don’t need server-side page rendering of any kind. You can serve purely static HTML files, and all the things you might do with Razor on the server can be done with JavaScript MVx frameworks on the client. You can even do Windows Live authentication with a static HTML callback page, and then pass the access token to the server using AJAX.

My favourite thing about this is, I can stick all the HTML, JavaScript, CSS and images that make up my UI into a CDN with a custom domain name, and set up CORS on my service servers so they’ll accept AJAX requests from that domain. Which makes serving the app lightning-fast, and reduces the load on expensive servers which have better things to be doing than shovelling static files.

The only fly in this ointment is that all the AJAX calls can be done to SSL endpoints, but unless the static files are served over SSL, there’s no reassuring padlock symbol to tell the user that their data is being transferred securely. So you end up pointlessly HTTPS-ing your static content just to keep the padlock there. If anyone knows of a solution to that problem, please let me know.

Filed Under: web development

It’s gonna be the future soon

January 2, 2013 By 3 Comments

I wrote a few posts last year which talked about the near(ish) future of computing, devices and so on, but I never really gave a detailed description of where I think the hardware side of things is going. A couple of things I’ve seen recently supported my personal theory on this, and I read an article that I think misses the point, so I thought I’d take some time to share it.

The things I’ve seen

1. PlayStation Move WonderBook of Spells

Santa delivered this as a joint present for me and my six-year-old daughter. Of the games console motion-control offerings, the PS3 Move add-on has been the least compelling to me, but the WonderBook system is very impressive. It’s "just" an augmented reality application, where the PS3 Eye camera sees the book with its various AR tags printed on the various pages, and overlays it in the video displayed on your TV with an animated book. It also overlays the Move controller with a magic wand. The thing is, it does it amazingly well: the pages of the on-screen book turn with the pages of the physical book, and the wand is consistently overlaid on the controller, even showing your fingers wrapped around it. I tried to get it to glitch, to show the real book or controller, but it resolutely maintained the illusion.

2. Project Glass

If you’re reading my blog, then you’re almost certainly the type of person who knows what this is, so I won’t patronise you by explaining it. Google’s project is the one that is the most publicly visible, but Microsoft are also (probably) working on "Kinect Shades", and I would be amazed if Apple weren’t engaged in some R&D in this domain too.

3. The article that misses the point

Wired.com posted this article about what they predicted for 2013 ten years ago, which includes new or updated predictions for tech in 2023. In it, they predict variable-size displays using technology like Samsung’s foldable OLED prototype. The funny thing is, they also mention smart-glasses. They’ve just failed to put two and two together.

What I reckon, right, is…

Technology moves at an incredible pace. In 2003, smartphones were hideous, bulky things with resistive screens and styluses and the first version of Windows Mobile. Ten years later we have devices that easily out-perform the high-end gaming PCs of 2003.

So in ten more years, we’ll have had several iterations of the smart-glasses concept, and it seems reasonable to predict that what they have become in 2023 will make Project Glass look like Windows Mobile 2003. Smart-glasses will be affordable, common-place, and very, very powerful. They’ll have built-in cameras; motion-tracking; eye-tracking; they’ll probably be tracking things we haven’t even thought of tracking yet. They will also have extremely high-resolution displays, and they’ll be able to render 3D objects, and hold them steady in the user’s perception of what’s in front of them, so that they appear to be real. And the motion-tracking will allow the user to interact with these objects.

The final piece in the puzzle will be pervasive, high-speed, wireless internet access, with bandwidths and latencies that are to LTE what Google Fiber is to ADSL. The only place on earth you won’t be able to access the internet wirelessly will be my house, if you’re on O2.

Augmented reality – proper augmented reality – will just be reality, at least for the majority of western society.

OLED – the O is for "obsolescent"*

In this world, there is no need for foldable display technology, nor indeed for "devices" of any kind. What point is there in cramming all that technology into a phone or a tablet when you can just pick up any piece of paper or card and have your smart-glasses overlay anything you want on it, and track your interactions with it? And even that is really just skeuomorphism; the display could be projected in empty space, but it’s easier to interact with something you can physically touch and feel. We may still have physical keyboards for writing and programming and so on, but they’ll be empty shells, used only as a reference point by our motion-trackers, and the displays floating above them won’t really be there.

You know in Iron Man, in Tony Stark’s lab, he pulls up all those floating 3D schematics and throws them around? That’s impossible, unless the entire lab is an insanely advanced volumetric display, or something. But stick a pair of smart-glasses on our hero and make it clear that we’re seeing what he sees, and suddenly you’ve got something that’s just a few easy, obvious steps from where we are already.

tony_stark_hologram

If you get me talking about this in the pub or at a conference or something, I’ll go rambling on about how this image can be made to appear as if it’s in the same place for multiple people in the same room, but actually, they don’t even need to be in the same room, they can be anywhere they want and see each other and the "hologram", and you’ll be able to make people look however you want them to look whether they’re there or not, or maybe they’d choose how they look and I’d have a whole Statler avatar, and I want to write an app which makes the entire world and everything in it look like a cartoon, which would really just be some very simple image processing, and you’d get those ads like in Minority Report, and holodecks and it’s going to be awesome, and at around this point you’d probably start saying "anyway" and trying to find an exit. Fair enough. But when you start thinking about it, the possibilities are pretty exciting. I wonder if we’ll still be building the software for this stuff using C# and JavaScript…

* I suppose OLED technology might be used in the smart-glasses lenses. But you know what I mean.

TL;DR

By 2023, smart-glasses will have rendered all other types of display obsolete, there’ll be virtual 3D stuff everywhere, and your computer keyboard will be hollow.

Filed Under: futurology

Surface First Impressions

October 29, 2012 By 4 Comments

So I went and queued for 3 hours at the Microsoft Store in Orlando, Florida and paid $700 for a 64GB Surface with a Touch Cover. I’m going to spend at least a week using it instead of my iPad for as much stuff as possible; I’ll drop back to that for watching iTunes videos, but I’m going to forgo the games since part of this whole deal has to be sticking to the Windows Store for apps and such.

I’ll post more as my experience unfolds, but I wanted to get some initial thoughts down after a couple of days with the device.

Hardware: The tablet

The Surface is a very, very nice piece of hardware. Whatever this metal is that it’s made from, it’s good stuff. This thing is more rigid than any other tablet I own, including the iPad 2, but the weight is about the same. The screen is good, with decent viewing angles and contrast. When I first found out it only had a 1366×768 resolution, I was disappointed, but in actual use it is absolutely fine. The only area where the rendering of text is noticeably “jaggy” is in the browser.

There is a single capacitive button on the front, just the Windows logo, which takes you to the Start screen. It also wakes the Surface from sleep, which is nice. It’s at the “bottom” of the screen in landscape orientation, and highlights the fact that you’re expected to use this device primarily in landscape mode. Above the screen is the front-facing camera, which I haven’t tried yet.

The top side has just a power button and two small speaker grilles. The sound from them is not bad, I’ve happily watched a Channel 9 video without headphones and had no problems hearing it. There are additional speaker grilles at the top of each side too, possibly to improve stereo reproduction. More on that in a later post when I’ve tried watching a movie.

On the left side there is also a 3.5mm headphone jack and the volume rocker. On the right, there’s a micro-HDMI port (not micro-USB, thank you Erno for pointing that out), a full-size USB port, and the power connector port, which is magnetic; I wonder if Microsoft have come to some kind of patent arrangement with Apple.

The USB port is a killer feature. It’s ridiculous to say that, because it’s such an easy thing to add to a device, but this is the first thing I’ve owned other than a full-blown PC that has one. Currently I’ve got my Arc Touch Mouse plugged into it, just for the hell of it.

The bottom has just the connector port for the keyboard covers (and possibly other accessories in future, like docks?). The clunk-click you get when attaching the keyboard is very satisfying.

On the back, there is just the rear-facing camera, a very subtle Windows logo, and the kickstand. This has just the one angle, and I’d have preferred a couple more to be honest. Right now I’m sat typing this with the Surface resting on the very wide arm of a hotel sofa, and it’s OK, but when working at the table, it’s a bit too vertical.

Hardware: The Touch Cover

I’m surprised by how good this is. I’m typing this post on it in MarkPad, and while it’s not as good as an actual keyboard, it’s pretty close. I don’t quite hit the keys every time, but it’s getting better as I get used to it. You still get the same sound feedback as from the on-screen keyboard, which is useful: the absence of a key sound is a good indicator that you missed it.

The thing that’s most surprising about the keyboard is how many keys it has. It’s missing the right control key, but pretty much everything else seems to be here. The top row has volume, play/pause, Charms, and Home/End/PgUp/PgDn, which are also F1-F12 if you use the Fn modifier key, although they don’t say that on them.

I’ve seen a couple of posts or articles suggesting that a physical keyboard on a tablet is redundant, but I have to disagree. However good an onscreen keyboard is, it’s taking up screen, which means less space to see what you’re working on. That makes my iPad, for example, pretty much unusable for writing anything of any length without the Bluetooth keyboard.

Software

I’m not going to cover the software in depth in this post, because it could go on for ever, but I’d like to address a couple of points that have been raised elsewhere.

First up, Windows 8 itself boots up very quickly, certainly under half a minute, and the wake-from-sleep is instant. Everything about the OS seems to work much as it has on my Samsung Series 7 from last year’s BUILD since I installed the RTM release on that.

I haven’t experienced any real problems with the Mail app, other than that it doesn’t automatically turn URLs into links, which is annoying. I’ve added a Google Apps Gmail account and an Office 365 Exchange account to it with no problems at all.

Office RT and Windows Update

There have been a few mentions made of the typing lag problem in Microsoft Word RT, and the first time I tried it, I had these problems too. It was fixed by installing the update to the final version, which was available at launch, but here’s the thing: I knew the update was available, but it still took me a while to find it, and I’ve been using Windows 8 for over a year now. I can’t imagine that many regular users are going to track it down without help.

You see, Windows 8 RT has three places it can download updates. There’s the Modern UI Windows Update which you access through settings, but the Office update wasn’t there. Then there’s the Store updates procedure, but the Office update wasn’t there, either. Finally, and very well hidden unless you know to look for it, there’s the Classic UI Control-Panel-based “Install Optional Updates”, and that’s where the Office update is hiding.

This is close to being a PR disaster. One of the major selling points of the Surface RT is that it comes with Microsoft Office. The version of Office that is pre-installed has this massive issue that makes the whole device look slow (which it isn’t). And getting from that version to the final version (which makes the device look great) is an enormous pain, involving a part of Windows which frankly shouldn’t even be there on the RT, and just makes Microsoft look like they haven’t really done the whole job.

In my opinion, it would have been better if Microsoft had sold me the device without Office pre-installed, with an installer app that would notify users when the final version was available and install it for them. It would also be better if they had found some way to make the Office apps look and act the same as all the other apps, even if they had to fake it. You know, run them maximized, hide the window chrome, hide the taskbar, just pretend that they’re Store apps. It’s great being able to switch to the desktop on Windows Pro, but it just flat-out shouldn’t be there at all on this RT tablet.

Anyway, that horse has bolted, and I can only hope that the devices that are shipping to the non-enthusiasts this holiday season have the final version of Office pre-installed, and that Microsoft fix the update process for it. Since there are obviously blockers that prevent it from being updated through either the Store or the main Windows Update, I’d suggest building it into Office itself.

//TODO

I’ll do another post in a couple of days to talk about the default apps and some that I’ve downloaded. The other thing I intend to try soon is using Cloud9 with the Touch cover and a mouse, because if that works, that’ll be awesome.

I’m not going to give you a “buy/don’t buy” recommendation just yet. Suffice to say, I will not be returning my Surface RT.

Filed Under: Uncategorized

HOWTO: Dial up the static on Simple.Data

October 12, 2012 By 1 Comment

Yo dawg, I heard you like IntelliSense, so I put an interface in your dynamic…

One of the big problems people tend to have with Simple.Data is that it’s dynamic, which means you don’t get IntelliSense when you’re working with it. It’s common for people to wrap Simple.Data in a repository which has almost the exact same signatures as the Simple.Data calls, like this:

public class CustomerRepository : ICustomerRepository
{
    private readonly dynamic _db = Database.Open();
    public Customer Get(int id)
    {
        return _db.Customers.Get(id);
    }
}

Then a team can use the repository class with all the IntelliSense and sanity checking that goes along with that. I’ve used this pattern myself. One day I started wondering if there might be a way to automatically generate proxy classes around Simple.Data’s dynamic objects to implement the interfaces, and during the course of my investigations, I found a project called Impromptu-Interface.

Impromptu-Interface

This project exists to address the lack of structural typing in C#. Remember in the TypeScript post, I mentioned how that language includes structural typing, so if an object has all the required members of an interface it is considered to implement that interface implicitly. C# doesn’t do that, but with Impromptu-Interface, you can explicitly say that object O implements interface I by wrapping a proxy around it. And you can do this with dynamic objects as well as statically-typed ones.

When I first discovered this project about a year ago, it wouldn’t recursively apply interfaces, but there was a feature request in for that and I recently checked again and it’s done, so I had a go at using it to apply interfaces to Simple.Data objects. It works really well. Take these interfaces, for example:

Lots of interfaces there. The IDatabase interface is to be applied to the dynamic database object, and includes properties for the Customers and Employees tables. Each of those has its own interface, with declarations for methods that are implemented by Simple.Data’s runtime jiggery-pokery. Note the UseNamedArgument attribute on the Insert methods; that’s an Impromptu attribute that tells it to include the argument names when it invokes the dynamic method.

I’ve also created interfaces for the Customer and Employee data objects; these interfaces will be applied to the SimpleRecord objects that are returned from queries, inserts, and so on.

Using Impromptu-Interface to apply these interfaces is embarrassingly easy:

It’s just that one call to ActsLike, and you’re done.

To illustrate how this works, here are some tests:

Neat stuff

Lazy-loading

I think my favourite thing is that lazy-loading still works with the IList<IEmployee> Employees property on ICustomer, and the ICustomer Employer property on IEmployee. When Simple.Data materializes statically-typed instances of DTOs or entity classes, you lose any lazy-loading that you would have got from the SimpleRecord object.

Inversion of Control

The other nice thing that this facilitates is dependency injection. You can take a dependency on ICustomers in your code, and configure your IoC container to return an Impromptu proxy. When you add a new method to the ICustomers interface, as long as it matches a Simple.Data method, it’s just there, you don’t need to change any other code.

No additional dependencies

A lot of projects that do stuff like this have a bunch of external dependencies on things like Castle Windsor. Impromptu-Interface is a single assembly and the NuGet package has no dependencies, so you can drop it into your project without worrying about bloat or conflicts or any ot that stuff.

Limitations

So this is still not going to run any kind of compile-time check over the interfaces to make sure that you’ve got the method and property signatures right, but it will catch any errors people make consuming those interfaces, and it gives you a nice neat bundle of things you can write some tests against.

Think of this as a way of defining a canonical set of Simple.Data method calls that can be used in your project, with all the tooling help you expect from your IDE.

ImpromptuInterface is available on NuGet. Make sure to update to the latest Simple.Data (0.18.2.1 at time of writing), because I had to tweak a couple of things to optimize the experience.

Filed Under: Uncategorized

Running TeamCity in Windows Azure

October 9, 2012 By 11 Comments

I’ve used JetBrains’ TeamCity continuous integration server on a couple of projects that I’ve worked on recently, and I really like it. It’s more approachable and flexible than the TFS Build system, which is kind of hung up on the whole MSBuild thing, and to be honest, if I wanted to write all my tasks as mind-bogglingly verbose XML, I’d use NAnt. TeamCity plays nice with all manner of build and test tools, version control systems and so on. It’s also free to use the Professional Edition; you only need to buy a license when your requirements exceed the generous limits (20 build configurations etc.). The only problem is that there is no hosted, as-a-Service provider as yet, so I decided to see if I could get it running on one of the new Windows Azure Infrastructure-as-a-Service VMs.

Initial setup

Getting things started was easy enough. I went to the Azure portal and created a new VM from the default Windows Server 2012 image; I used a small instance, with a single core and 1.7GB RAM. After waiting a few minutes while it was provisioned, I logged in via Remote Desktop, disabled Internet Explorer Enhanced Security Configuration, downloaded the TeamCity 7.1 installer and ran it. I set up the build server and a build agent on the same machine, just to see how it would cope with the limited resources. To make it accessible from the internet, I configured it to run the server on port 80; I had to open that port in the Windows Azure management portal and in Windows Firewall, as all ports except the RDP one are closed by default on Azure VMs.

It ran right away with no problems, but using its local storage engine, which is not recommended for production; you get a message box telling you to configure a proper database. Here begins the fun.

Database setup

TeamCity supports MySQL, PostgreSQL, Oracle and SQL Server. That’s the order of preference: the documentation recommends using MySQL unless you absolutely can’t for some reason. I guess this is due to its Java heritage. I could have set up another VM running Linux and MySQL, but that’s getting kind of expensive, so I decided to have a go at getting it working with a Windows Azure SQL Database (formerly known as SQL Azure). This is completely unsupported by JetBrains, but as we all know, unsupported doesn’t mean impossible.

I created a 1GB SQL Database through the portal, and followed the instructions for setting up an external database. The JTDS driver didn’t seem to get on with Azure for some reason, so I used Microsoft’s native JDBC driver. That allowed TeamCity to connect to the database, but when it tried to run its migration to set up the schema, incompatibility disaster struck. Windows Azure SQL Database requires a clustered index on every table. Most databases have a primary key on every table, which includes a clustered index, but the TeamCity database has a couple of tables with no primary key. The database setup appears to be compiled into the application somewhere; I couldn’t find any SQL scripts in the installation, so I couldn’t tweak anything.

I tried manually modifying the database to include clustered indexes and continuing the setup, but it wasn’t having it, so I went to plan B. I installed TeamCity on my workstation and set it up with a database on my local SQL 2012 instance, which worked without any problem. Then I used the SQL Azure Migration Wizard to copy that local database up to Azure. The Migration Wizard is a fantastic tool which automatically makes the necessary changes to schema as it copies the database, including creating clustered indexes. It also copies data using BCP, so all the standard configuration and user data was copied too.

Collation hell

Once I’d copied the database up, I tried again with the Azure-hosted installation, and it mostly worked, but a couple of features kept crashing. Checking the logs revealed that old chestnut, the collation error. I’d created the Azure database with the default collation, SQL_Latin1_General_CI_AS, but TeamCity wanted Latin1_General_CI_AS and wouldn’t budge. So I dropped the database and recreated it with that collation, and ran the Migration Wizard again. Happy with the new collation, TeamCity started working perfectly.

I hope that JetBrains will consider making the few changes necessary to support Windows Azure SQL Database as a database server in the future. There is an issue for it on the TeamCity YouTrack, but currently there appear to be no plans to address it.

I may put a script-dump of the database with the relevant changes somewhere for people to use to avoid this slightly tortuous route. The only issue is that you have to create the user account on the local database, but I could create one with a default administrator username and password. Let me know in the comments if you’d find such a thing useful.

Setting up a build

I wanted to test the system with a proper project with a good number of unit tests, so I used Simple.Data. TeamCity will pull from GitHub, and you can configure it to check for updates regularly. It will connect using HTTPS with your username and password, but I prefer using SSH, so I installed msysGit on the Azure VM and configured it with an SSH key. I’d set TeamCity up to run with a local user account, so I just copied the .ssh folder into that user’s home folder. With that set up, it was able to pull the Simple.Data repository with no problem.

The main Simple.Data project has a bunch of unit tests and integration tests that all use NUnit, for which support is built-in to TeamCity. The unit tests all ran fine, but the integration tests for SQL Server and SQL Compact 4.0 required some additional setup. For the SQL Compact tests to run, I had to install the redistributable on the VM, which was straightforward enough. For the SQL Server tests, I created another Azure SQL Database and tweaked the database creation script (which runs for every test run) to work; again, mainly to do with primary keys and clustered indexes, but also removing references to file groups and options that are not valid in the Azure environment.

The last problem was managing the connection string. I don’t want to leave a valid connection to an Azure database in the source code of an open source project. Fortunately, TeamCity can set environment variables for the duration of a build/test cycle, so I tweaked the test code to check for an environment variable and use that as a connection string if it found it. That meant I could put the connection string in the build configuration. I also set one test to conditionally ignore if the environment variable was detected, because it was to do with named connections and wouldn’t work.

With that done, I had all but three tests passing; the three failures were due to the Microsoft SQL Server CLR types assembly not being in the GAC on the VM. I copied the relevant assembly into the project and set the reference to Copy Local=true, and voila! All 800 tests pass!

Performance

Considering that the spec of the Azure VM is considerably below the recommendation for TeamCity, and that the server and a build agent are running on the same box, the performance is surprisingly good. A build takes less than a minute and a half, and a good chunk of that is running the integration tests. In terms of using the TeamCity web application, the performance is fine; having the database off the server probably helps there. I’ve enabled guest access, so you can take a look for yourself: teamcity.cloudapp.net (there’s a "Login as a Guest User" link at the bottom of the login window).

Production use

I’m intending to run TeamCity in Azure for proper production use, including continuous deployment to an Azure hosted service, in the near future. There’s only one change I intend to make to the setup I’ve got now, and that’s to have a separate build agent, which shouldn’t be very difficult to set up.

When I shared my experience with an Azure mailing list I’m on, someone else said they had got TeamCity running on an Ubuntu Linux VM with it’s own MySQL database, and a Windows build agent, so that’s another option, and of course, if your project is entirely Linux-based, then you can have a Linux build agent too.

Filed Under: azure, continuous integration, Simple.Data

Simple.Data change: FindBy is deprecated for 1.0

October 3, 2012 By 5 Comments

When the guy who’s writing your documentation is getting confused by your API, it’s time to do something about it.

What’s changing?

The first method that Simple.Data ever had was FindByXyz, and all it did was blindly generate a SQL Server compatible “SELECT * FROM” statement and return the first record it found as a dynamic object. At the time, I really hadn’t put any proper thought into the design of the API, because, well, it wasn’t an API, it was just a proof-of-concept.

Since then, about half the questions I answer are along the lines of “why doesn’t this work”:

var customers = db.Customer.FindByRegion("NW").ToList();



The answer, of course, is that FindBy returns a single record; you want FindAllBy.


The problem is that even I get it wrong. If I had a magic wand, and could change one thing in Simple.Data without breaking anybody’s code, I’d make FindBy return a query and get rid of FindAllBy. Sadly, I can’t do that. The best I can do, for now, is to make this announcement that FindBy is deprecated, and make it write a Trace Warning if you use it in your code.


So how do I find a single record?


Much like you would using LINQ:


var justOneCustomer = db.Customer.FindAllByName("Phronesis").FirstOrDefault();



Simple.Data will use that FirstOrDefault to intelligently limit the number of returned results in the generated SQL (for example, using TOP 1 in SQL Server) so you won't get enormous result sets being generated.


Of course, if you have a primary key, you can still use Get which will continue to return a single record:


var justOneCustomer = db.Customer.Get(42);



Define "deprecated"


Deprecated means that the FindBy method will continue to work in Simple.Data 1.0 and all subsequent 1.x maintenance releases.


In Simple.Data 2.0, which is already in development, FindBy won't work at all, and if you try to use it you'll get a very specific Exception telling you not to.


At some point in the Simple.Data 2.x lifecycle, FindBy will become an alias for FindAllBy, and that will be an end to it.
Filed Under: Uncategorized Tagged With:
« Older Posts
Follow

Get every new post delivered to your Inbox.

Join 36 other followers