Scalable web applications – Google App Engine and Amazon S3
Google announced today the beta release of Google App Engine. This application engine is designed to facilitate the quick development and deployment of new web apps. They only released it to the first 10,000 people to sign up, but I'm sure that will expand before too long. This service seems to be in competition with Amazon S3. I think these are really exciting developments, even though I'm not so sure I'll be using them anytime soon. Essentially, they both allow really cheap storage and processing for all your web applications, and they scale with you. At least, Google's will eventually...starting out they are offering 500Mb per month of storage space and 5 million page views for free with pay service and expandability to come at some point. Amazon S3 is a pay service from the start, but the prices are really quite cheap.
These developments could especially be important for GIS applications. I know a few other GIS people have blogged about the use of S3 for GIS web apps, and it could really be a boon for people who need to serve large amounts of data to a lot of people. I'm working on a project now that will require hundreds of gigabytes of storage (to start) and will need some serious computing horsepower if its to be served to more than a handful of people. I've started looking into S3 and Google App Engine today, because I've realized that there is a good chance that this is where we are headed.
Sure, I've still got my web hosting account and probably will for a long time to come, but for anybody developing web applications that are targeted at more than just family and friends should probably start looking into these types of services. It does require a bit of a shift in thinking when developing apps, especially from the storage side of things. For example, Google's storage is called "Big Table" and is just that. In my application where I had been planning to use MySQL with lots of tables and joins, I would probably just use one or two with Google's Big Table because of the way that it can efficiently handle huge amounts of data (and doesn't allow joins anyway). Same thing with S3. This is presenting a bit of a challenge for me as I start down the development of my application, because I have to decide how I'm going to store and access my data. I can do some abstraction, but there is only so much I can do when we're talking about completely different storage and access methods.
Anyway, hopefully I'll be able to post some more info on my project sometime soon!