Some of you may heard about A/B testing. I believe this became famous when Obama Digital team produced campaign site. Simply stated, different web pages will be displayed on the same URL. This is useful to test the hypothesis which page are more attractive. There is A site and B site for the same URL. Some users traffic goes to A site, and the rest goes to B site. The team carefully monitor the access to the site to see which site is more attractive.
I wondered how this can be done with Bluemix. Here is my experiment steps.
1. Create a single route
> cf create-route dev mybluemix.net -n amntabtesting
2. Create applications with no-route option.
>cf push --no-route -p A -b https://github.com/cloudfoundry-community/nginx-buildpack -m 64M amntatesting >cf push --no-route -p B -b https://github.com/cloudfoundry-community/nginx-buildpack -m 64M amntbtesting
3. Bind the both applications to the same URL.
> cf map-route amntatesting mybluemix.net -n amntabtesting > cf map-route amntbtesting mybluemix.net -n amntabtesting
4. Check the application URL
> cf apps amntatesting started 1/1 64M 1G amntabtesting.mybluemix.net amntbtesting started 1/1 64M 1G amntabtesting.mybluemix.net
As you can see the same URL is mapped to different applications. I tested the Web URL, and hit “reload” button several times. As I expected, I see different sites for the same URL.
So the Bluemix is not only provide horizontal scalability, but also provide A/B testing capability. I believe this may be useful for canary testing too. In this test, some of new features are added and deployed to Bluemix to see if the new features are working correctly. If not, developer can simply unbind the route to new application. When testing goes well, then developer can unbind old applications. I may explorer more on how to do canary testing on Bluemix.