Teun.ToString()

Yesterday, Google announced it's entry in the cloud computing game. Amazon Web Services has long been the only serious player (with S3, EC2 and SimpleDB), but now the other large online powers are lining up. Microsoft has unveiled a number of online components that would fit into a comparable image (SSDS, SkyDrive, BizTalk services). Expectations were that Google would open up it's BigTable database for external users, but their offering turns out to be much more comprehensive. It is a fairly complete web application platform.

One size fits all?

Where Amazon offers separate, but complementary services that can be used for computing and storage in the broadest sense, Google focuses on web application hosting only. They basically offer to host your python code on their infrastructure. While Google insists that python is just the first supported language, it seems deeply involved in the whole system. For storing data you can either choose to use BigTable or use the Google File System for reading and writing files.

The upside of this very narrow approach (only websites, only python) is that setting up a website can be very easy and can include all of the scalability and monitoring tools that you would require. Downside, of course, is that python is not the language with the largest developer base. What Amazon offers is basically to run your virtual machine on their hardware, the Google App Engine offers way less freedom, but is obviously easier to get started.

Free for now

The pricing model of App engine is very friendly for starters. You will not pay any fee as long as your usage stays under certain thresholds (500MB storage, CPU and bandwidth amount to ca. 5M pageviews per month). Most web applications will actually never reach that level. But if your brilliant idea resonates with the crowds, your app will scale to many millions of users. You will pay only for the storage, cycles and bandwidth above the set limits. This model will surely attract many hobbyist developers creating the next big thing in their garage.

Lock in?

When you run your site on Amazon EC2, you can always take your virtual machine images elsewhere. Many Amazon customers run their base setup on owned hardware and used their Amazon environment only in periods of heavy use. With Google, you're bound to their platform. It will not be easy to run an application built for the Google App Engine on your own premises. So you're more or less stuck with them. Then again, by the time you App Engine bills become significant, your user base should allow for some profit. And, in your start--up period (and for most applications forever), you will be using the platform for free. Of course, once your site is successful, you can always re-engineer the code for another platform.

What about .NET?

I think Google is getting it right again. Easy to set up and free to start are powerful arguments for independent app builders. Some applications are just not fit for the platform (think data needing offline processing, workflow), some companies will not be willing to store their business information in Googles systems. The pricing for sites exceeding the thresholds is still unknown, but 'free for small sites' will make Amazon look very expensive. Also, what can we expect for programmers not fluent in python? One might expect PHP to be the next implemented language.

Personally, I would really like to have a comparable platform using the Microsoft stack: SQL Server Data Services, SkyDrive and a hosted scalable ASP.NET site engine. That would be awesome. And free for the small guys, of course. I hope that Microsoft have something like this up their sleeve. Steve Ballmer seemed to suggest this at MIX'08. Otherwise, I'll just learn python, I'm feeling a little dynamically typed already.

The current preview was open to only 10.000 developers and apparently it filled up quickly; I couldn't sign up. But you can download the SDK and check out the documentation. The SDK will allow you to develop and locally run your app as if it were on Google App Engine. Sooner or later, App Engine will be open for all.

Labels: Cloud computing

Just to keep myself from figuring this out every time again. ;)

When building a URL from a base URL (say: http://www.teuntostring.net/blog/2008) and a relative path (say: ../../blog/index.html), the .NET framework supports us nicely with the Uri class. Just say:

Uri baseUrl = new Uri("http://www.teuntostring.net/blog/2008"); Uri abs = new Uri(baseUrl, "../../blog/index.html"); This works nicely, but for file system paths ("c:\Temp\myfile.htm"), the Uri class isnot suitable. You can of course transcribe all file paths to Uri form (file:///c:/...), but the resulting Uri cannot be used to open the file. It would be nice to have a way to combine file paths in the way we can with URLs. At first I thought that the Path.Combine method would help me out, but alas: string abs = Path.Combine("C:\\temp\\", "..\\Projects"); // abs is now C:\temp\..\Projects Using Reflector on the mscorlib led me to the Path.GetFullPath method. It can normalize a path with relative parts into a clean absolute path. So the way appears to be: string abs = Path.GetFullPath(Path.Combine("C:\\temp\\", "..\\Projects")); // abs is now C:\\Projects I wonder if an easier and cleaner way exists. This solution doesn't feel like it was meant to be like this.

In dit stukje ga ik je overhalen om te solliciteren bij funda en lid te worden van ons team. Ik denk namelijk dat funda voor een ontwikkelaar één van de leukste, uitdagendste en inspirerendste plekken is om te werken.

Funda is het bedrijf achter enkele websites, waaronder funda.nl. Funda.nl is het soort website waarover iedereen een mening heeft en die voor veel Nederlanders op de short-list staat van sites waar ze regelmatig een kijkje nemen. En uiteraard een onmisbaar hulpmiddel voor kopers en verkopers van huizen.

Waarom wij nu developers zoeken? Dat heeft zeker iets te maken met de concurrentie die afgelopen jaar in de markt is ontstaan. We moeten stevig aande slag om voorop te blijven lopen, we moeten met afstand de beste huizen-site blijven. Binnen funda hebben we dan ook gekozen voor investeren in innovatie en dat is goed nieuws voor ontwikkelaars: het betekent coole, innovatieve projecten.

We hebben een front-end ontwikkelaar nodig. We nemen XHTML en CSS serieus en je moet voor deze functie dus echt dromen van stylesheets. Je bent geen designer, maar een software ontwikkelaar die client-side technologieën het leukst vindt.

We zoeken een back-end ontwikkelaar. Dat is iemand die jaren ervaring als .NET ontwikkelaar meebrengt. Zij begrijpt dat grote schaal alles anders maakt. 120.000 zoekopdrachten per uur is niet niks en je moet dan ook over iedere query en iedere regel code goed nadenken. Veel van onze code gaat niet over het zoeken in een database vol huizen, maar over het opbouwen van die hoog-kwalitatieve actuele data.

We zoeken een QA engineer/test coördinator. De gebruikers van funda vertrouwen erop dat de site het doet. Als er problemen zijn bellen ze teleurgesteld onze klantenservice plat. We vinden het dan ook super-belangrijk om de kwaliteit van onze websites te waarborgen. Innovatie kan alleen als we in staat zijn onze goede bestaande functionaliteit niet per ongeluk kapot te maken.

Even een stukje bla over werken bij funda (wel waar trouwens):

Funda is een jonge informele omgeving waar iedereen super-enthousiast bezig is met een mooi product. De nadruk ligt op resultaat en pragmatiek. En er mag gelachen worden.

Bij het opbouwen van ons team zijn we op zoek naar mensen die op hun eigen terrein de expert zijn, maar vooral teamplayer zijn. Die genieten van de wisselwerking met anderen die in hun vakgebied net zo gepassioneerd zijn. Die graag beter willen worden door het contact met anderen. De geen genoegen nemen met "best goed"; wat wij bouwen moet uitstekend zijn.

Dan alvast wat vragen en antwoorden:

Funda is toch al af? Lijkt me saai.

Think again! Funda is continu in beweging en we zijn de hele tijd aan het nadenken over wat beter kan (en dat zijn niet alleen kleine dingetjes). Het begint nu pas echt.

Waar zitten jullie eigenlijk?

In Amsterdam, naast het Amstelstation. Goed bereikbaar per auto en trein. Ik ga zelf altijd met de trein vanuit Utrecht: super-verbinding.

Heb je vragen? Mail mij gerust direct: teun [at] funda [punt] nl

Selenium is a really nice tool to do web (regression) testing. With Selenium Remote Control, you can run tests from your C# unit testing code and perform tests on a "remote controlled" browser instance. Very nice.

Recently one of my tests suddenly failed after adding ASP.NET AJAX UpdatePanels to the page. The problem turned out to be that after a Click, we normally issue a WaitForPageToLoad before continuing testing on the next page. This makes sure that the next page (after a postback) is fully loaded before we do the next test action. When using UpdatePanel postbacks, the page is never really submitted and no onLoad event ever happens in the browser. This makes WaitForPageToLoad time out.

Selenium is rather flexible and you can do a lot of custom tests in javascript, but you have to know how. It took me some time, but here it is: make Selenium wait until an UpdatePanel postback has finished:

selenium.WaitForCondition( "var busyFunc = selenium.browserbot.getCurrentWindow(). Sys.WebForms.PageRequestManager.get_inPostBack; busy = (busyFunc == null)?true:busy = busyFunc(); busy == false;", "10000");

Sometimes you just run into a piece of technology that is so cool, that you wish you could make up a way to use it in a professional setting. Exhibit from the Simile project at MIT is such a technology. It brings a certain class of web application to the masses. Displaying data in compelling user interfaces that allow for searching, sorting, filtering, etc. used to be the domain of database backed web applications. Not anymore.

Exhibit is basically a very nice javascript library. It can be used to create pages on the client based on data files and templates (nicely separated from each other). The UI is created by coding up templates in HTML.

The sample

The sample I made: last summer, I went on a bicycling tour with my girlfriend visiting all prehistoric dolmens in the Netherlands. There are 53 of them and then a small number of fakes. At each dolmen we made picture and we logged the location with our GPS receiver. Back home, I created a data file (this is JSON formatted) of all dolmens and whipped up an Exhibit view page for the set. Compared with the time this took (of course I spent some time reading docs), the results are very impressive.

Noteworthy details

This page is pure flat html and javascript. No server-side code, no database. No prgramming done by me.

Note that when you create a filter in the thumbnail view (say: show only dolmens rated ** or ***), and then you switch from the thumbnail view to the Google Maps view, you selection remains active.

Some properties cancontain multiple values. You don't have to specify this in advance, Exhibit just gracefully deals with the fact that multiple values are found in the data. For example: the dolmen D26 is near two villages, Borger and Drouwen. In the filter list, you'll see Drouwen listed as having 3 nearby dolmens and Borger as having 2. If you select them both, the view will display only 4 hits (because one is near both). In the 'Photos' view, you will see the property rendered as 'Drouwen and Borger'.

When a filter is applied on one property (i.e. 'Number of Stars'), you will see the available options and subtotals in the other filters (i.e. 'Close to')

Other demos

On the Simile site, a load of other demos (some extremely nice, showing graphical plots of data) can be viewed. Go check them out and read the documentation and tutorials.