Cloud Services Redone - Part 2 of 2

Anywhere Cloud Services - Thanks to Docker & Kubernetes

Cloud Services without the Lock In 

Part 2 of 2 

In part one of this two part post I wrote about how open source software will be delivered as services instead of as binaries. These open source services will deliver a free, open, flexible service in which engineers can use to help build their applications without everyone all trying to solve the same problem of figuring out how to keep the software up and running. If you haven't read that post, I encourage you to read it before proceeding with this post. 

These posts will mostly describe a vision I have but will also include some back story and some technical information. The attended audience is anyone building or using applications. 

Where are Cloud services Today? 

Today's application level cloud services like AWS's Kinesis or Google's Big Table provide engineers fully managed services in which you can build an application on top of that require no effort to maintain. These services are secure, autoscaled, monitored, logged, integrated and updated. Personally, I love these services. But...

• You are locked into these services via your code (ie, changing database backends would need a software change)
• By choosing a public cloud service, you also are forced to use their hardware. What if you run on Microsoft Azure but love Big Table?
• What if you want to use open source software but you don't want to operate, scale and secure open source software myself?

Enter anywhere cloud services. 

Model #2 - Anywhere Cloud Services

An anywhere cloud service is a fully managed service that can be delivered anywhere and should have these characteristics:

It is fully managed - The consumer doesn't need to worry about keeping the service online. This responsibility is owned by the service provider. 

It runs anywhere - The consumer chooses where the service lives. This could be any one of the major cloud providers or a private cloud with a standard API. There is a clear separation between the entity running the infrastructure and the service running on top of it. 

It is consumed as a service, not as software - It provides a function to the consumer in a clearly consumable method. Ie, a NoSQL service provides a way to insert data and retrieve data. The services heals itself. It scales when required. It provides metrics, logs and alerts to the consumer. It is highly available. The service is installed as easy as running a command or clicking on a button.

Give me some Examples...

To help illustrate this idea, here are a few examples that we might see in the future:

• Any of AWS's non-infrastructure services, like the NoSQL DynamoDB service, can be provisioned in all of the other cloud providers. Ie, Azure, GoogleCloud, Etc. 
• Google's non-infrastructure services, like their PaaS App Engine, can be deployed on any public cloud or even in on-premise data centers via openstack
• A startup goes after the Docker container space by building a service that will run your application/code on any public or private cloud. 
• A development team decides on a data stack of Kafka, Spark and Cassandra because they want the benefits of open source software. However, they don't want to pay consultants or do operations themselves. So, they use a new service made by a startup that specializes in delivering these software stacks in any location desired. 
• Creators of open source software will package their software up so that is can be easily deployed as a service in addition to supping the source code and binaries. Companies will also offer 'supported' versions of these services at a cost. 

All of these are possible, thanks to Anywhere Cloud Services

How do we get to Anywhere Cloud Services? 

Their were two big hurdles to Anywhere cloud services:

1. Standard and efficient way to package and deploy software services across the multiple cloud providers and on premise hardware
2. Efficient way to package, run, deploy, scale and secure your services (NoOps)

Enter Docker and Kubernetes!

Docker enables us to easily build, ship and run software by packaging it up in a way that will run on a wide range of systems. By using containers, cloud service providers can run their software in any environment. But, that isn't enough. We need a way to get that software installed, working and highly available. We need something like Kubernetes. 

Once the software is packaged in Docker containers we can turn their software into services by using kubernetes's features. Kubernetes's goal is to take a set of infrastructure (that lives anywhere) and provide an API to create services on top of that infrastructure. Kubernetes includes features like load balancing, scaling, rolling-updates, service discovery, name spaces, APIs, scheduling and configurable infrastructure threshold limits. Using these features, we can create declarative services and give control of the service to the consumer. Kubernetes provides the No in NoOps while providing developers a single API to launch into both public and private clouds. 

How will we consume Anywhere Cloud Services?

First, consumers will need to have one or more kubernetes clusters. Getting kubernetes up and running can take as little as 5 minutes or as long as a few days depending on your situation. Kubernetes supports most public clouds, openstack clouds and on-premise data centers. Consumers can also choose whether to run on top of VMs or physical machines. We can run kubernetes ourselves or we can use a managed option if we are looking for increased support. 

Notice that the consumer is in complete control here of where their applications will run as they will be freed from any lock-in. Thanks to the common API that lives in any infrastructure environment, consumers can use these clusters to deliver on a multiple cloud provider strategy or they can create a true hybrid strategy.

Second, consumers simply consume these services by having these services deploy their code into the customer's Kubernetes clusters. Either through an app-store like experience or through a developer friendly like experience, users will search for a service that meets their needs and click install. The service provider would than reach out to the Kubernetes cluster in question, either in the cloud or on-premise, and install the service. Developers could than immediately start using the service. The service provider's job is to ensure the service is running, healthy and scale it as required. 

In this new world, consumers will have more control of their application and the services that they depend on. They can move our applications between clouds or between private and public. Consumers can run their applications actively across multiple clusters or do active-standby. All this is sweetness.

In summary, Anywhere Cloud Services

With anywhere cloud services, developers will be able to build their applications the way they want while not having to worry about running and operating all their back end services. Tech execs get freedom from cloud lock-in and have a path to a true hybrid solution. Public cloud providers get access to customers on other clouds that don't want to move. Open source projects get a new path to revenue.. Entrepreneur's get a brand new gold rush as their are plenty of challenges and opportunities.

In summary - Anywhere and Open Source Services

In 5 years, cloud services will be just like software in that you go buy some and install it where you want. Pretty easy. 

Thanks
Chris Kleban
Twitter: @kleban

Cloud Services Redone - Part 1 of 2

Disruption in Open Source Delivery, the Open Source Service 

What if we took the positives of open sourced software and merged them with the positives of cloud services like DynamoDB, BigTable, Kinesis, Lambda and S3? We would have a disruptive service delivery model.

In a two part post, I'll talk about two new approaches to delivering software as a service so that engineers can build better applications. One model I will call Open Sourced Services (Part 1) and another model I will call Managed Anywhere Services (to-be-written Part 2).

These posts will mostly describe a vision I have but will also include some back story and some technical information. The attended audience is anyone building or using applications. 

Where are we today?

Today's Cloud Services 

Today's application level cloud services like AWS's Kinesis or Google's Big Table provide engineers fully managed services in which you can build an application on top of that require no effort to maintain. These services are secure, autoscaled, monitored, logged, integrated and updated. Engineers can focus on their particular applications versus worrying about things like mysql cluster management or file replication. Personally, I love these services. However, these services lock you into a single cloud vender and you loose control of the functionality of these services. What if we could get the benefit of services without the lock in?

Today's Open Source 

Open source software has many benefits. The source code is open, which gives you increased control. The software is free which helps drive adoption and potentially lowers cost. It gives back to the community by enabling the free re-use of software. And it frees you from cloud lock-in as you can easily more your solution anywhere. However, open source software is normally delivered in a software package in which someone needs to install, monitor, configure, scale and upgrade so that it can be used in production. Each company often does all this a different way and their work isn't reusable by others. Isn't that a shame? What if we could take the increased control, the re-use of code, freedom from cloud lock-in and the free price tag of open source software but deliver it as a service?

Well, what do we want? 

The three main items that stick out are:

• services
• control 
• portability (no lock in)

Below, I will talk about one way how we can get there. 

Model #1 - Open Sourced Services

An open source service is the packaging and delivery of open source software such that it is delivered as a service. An open source service should have these characteristics:

It is consumed as a service, not as software - It provides a function to the consumer in a clearly consumable method. Ie, a NoSQL service provides a way to insert data and retrieve data. The services heals itself. It scales when required. It provides metrics, logs and alerts to the consumer. It is highly available. The service is installed as easy as running a command or clicking on a button.

It is controlled as a service, not as software - The consumer takes action on the service itself versus it's subcomponents. They can initiate upgrades of the service. They can start, stop or restart the service as needed. The service should expose configuration knobs so that the consumer can control the level of redundancy, user access, scaling parameters (auto scaling, infrastructure thresholds, etc), capacity and user access.

It can run anywhere - The service should be able to be deployed anywhere. In a public cloud or in your company's data center.

It is open - The source code of the software and the declarative definition of how the software is wrapped as a service should both be open sourced. This will give the the consumer of the service the control they desire.

How do we get to Open Sourced Services? 

The creators of open source software have not had an easy and unified way to package up their software in a way that meets the above criteria until recently. Thanks to the disruptive technology of containers, such as docker, and orchestration systems (such as kubernetes, swarm and mesosphere), I believe we now have a platform in which we can build our open sourced services on top of.

Note: I'll be using Docker and Kubernetes in the rest of this article but you should know that there are alternatives (swarm, mesospahre, rocket, oci, etc). 

Docker enables us to easily build, ship and run software by packaging it up in a way that will run on a wide range of systems. If open source groups takes their code and packages it inside containers they will have the building blocks required for turning their software into a service (that can run anywhere).

Once the software is packaged in Docker containers we can turn their software into services by using kubernetes's features. Kubernetes's goal is to take a set of infrastructure (that lives anywhere) and provide an API to create services on top of that infrastructure. Kubernetes includes features like load balancing, scaling, rolling-updates, service discovery, name spaces, APIs, scheduling and configurable infrastructure threshold limits. Using these features, we can create declarative services and give control of the service to the consumer. Kubernetes helps groups deliver on the above characteristics for what makes up an open source service.

Creators of open source software now have a way to package their software up and deliver it as a service.  Sweet!!!! So, how would we consume these new services?

How will we consume open sourced services?

First, consumers will need to have one or more kubernetes clusters. Getting kubernetes up and running can take as little as 5 minutes or as long as a few days depending on your situation. Kubernetes supports most public clouds, openstack clouds and on-premise data centers. Consumers can also choose whether to run on top of VMs or physical machines. We can run kubernetes ourselves or we can use a managed option if we are looking for increased support. Notice that the consumer is in complete control here of where their applications will run as they will be freed from any lock-in. Thanks to the common API that lives in any infrastructure environment, consumers can use these clusters to deliver on a multiple cloud provider strategy or they can create a true hybrid strategy.

Second, consumers would need to choose and install some open source services . For example, let's say they want to use a noSQL service for their application. Previously, their choices where using something like AWS's DynamoDB (with AWS lock-in) or they could have engineered their our own Cassandra cluster with a bunch of blood and sweat. With open source services, all consumers will need to do is install the cassandra service on their kubernetes cluster of choice. They might do this with an easy command line tool ('kubernetes install service cassandra') or perhaps by logging into an app store and clicking on the cassandra service. Consumers will want to control the service by setting some parameters, like capacity limits and redundancy options, so that the service meets their needs. After that, the consumer simply starts using the service and keeps an eye on all the fun metrics and logs the service provides.

In this new world, consumers will have more control of their application and the services that they depend on. They can move our applications between clouds or between private and public. Consumers can run their applications actively across multiple clusters or do active-standby. All this is sweetness.

In summary

With open source services there are lots of winners: Developers will be able to build their applications the way they want to while getting the control they need. Tech execs get freedom from cloud lock-in and have a path to a true hybrid solution. Public cloud providers get to continue to sell infrastructure. Open source projects get a way to easily package their software as a service which increases their user's perceived value. Entrepreneur's get a brand new gold rush as their are plenty of challenges and opportunities.

Up next - Marriage #2 - Managed Anywhere Services

One of the down sides to open sourced services is that sometimes there is no one to call in case things go wrong. In my next post I'll write about an alternative approach that aims at solving that problem and opens up a new way for companies to make $$$ by delivering these services.

End

Thanks
Chris

--

Some useful links:

• Learn more about containers and kubernetes: http://kubernetes.io/v1.0/docs/whatisk8s.html
• A list of some early versions of open source software packaged up as services: https://github.com/kubernetes/kubernetes/tree/master/examples

Containers and Clusters - Disrupting how cloud services are delivered

The Problem

Cloud computing and the higher level cloud services that public cloud companies offer have changed our industry. They allow developers to focus on solving customer needs versus worrying about servers, databases, asynchronous messaging, analytics, storage, media encoding, graphic rendering, content delivery and so on. These cloud services enable engineers to create applications quicker and reduce their costs. However, one of the draw backs of using the higher level cloud services is cloud lock in. The SDKs and APIs developers need to use to interact with these services are, for the most part, not standardized. If you use these services and you want to run the same application on another cloud or on premise, you need to rewrite some code. I don't think that this lock in should exclude people from using these services. However, we need a model that allows for these companies to provide innovative high level cloud services while also allowing us the freedom and flexibility of true portability. 

Disruption is needed 

One approach could be to get all the cloud companies to standardize their services. Ha. Anyone see pigs flying? Another approach, and the one I believe in, would be for mainstream adoption of containers and clusters to provide a path for cloud services to run anywhere. While I don't think cloud providers will jump at this notion, I do think there is an ongoing technology change that might force their hands. If they don't jump on board, startups will create similar services and take away their market share. Heres how.

Containers

Containers will be the foundation of this revolution. Docker and the rest of the container technologies offer a way for an engineer to package their software, deliver it to a server and run the package anywhere. If you want to learn more about containers and docker you should google it or read this webpage. Docker Docker Docker.  It's all you here about these days. In time, containers will be main stream.

Clusters

In order to run a production service or application, you will need to build, run and manage many different containers. For redundancy, performance and scalability you will need to spread these containers across multiple servers and data centers. This is starting to sound complex. To make this easier, people have created clusters. 

Kubernetes, Apache Mesos and Docker Swarm are all clustering and scheduling software frameworks that allow organizations to create a logical collection of compute power called clusters. These clusters are made up of servers or VMs and enable engineers to deploy their containers across the infrastructure. In addition, the cluster software also provides container replication, auto-scaling, load balancing, monitoring, logging, scheduling, resource management and so on. The end result is that we can create clusters on the hardware of our choice: in the cloud or on premise. We package up our code and deploy it wherever we want. True utility compute. 

Operating systems for Clusters

To make things easier to manage, clustering software allows engineers to logically define groups of containers and various cluster attributes, like load balancing and security, into logically defined services. This allows engineers to manage services on their clusters, instead of a collection of containers. To make things easier for engineers, enter the concept of cluster operation systems. Two  exist today: Mesosphere's data center operating system (DCOS) which offers a web-portal and CLI and some might consider kubernetes CLI tool a cluster operating system as well. These cluster operating systems make it easier for engineers to deploy and manage  services on one or more clusters. They allow us to see the status of the cluster, the services, the containers and so on. All the things you need to run and maintain your service in production. Cluster operating systems will make deploying your services across multiple clusters a breeze. 

The app store for clusters 

It pretty obvious how this will enable us to do great things. But how will this change how cloud services are delivered? The answer will be a cluster service repository. Like an app-store for clusters. Want to run a HA database with your application? You will simply download the DB service you wish to integrate to your application. Want a web server? Go choose a web stack. Need caching? Need messaging? Simply pick your service. Then, you get to write code to utilize it, package it up and deploy it to whatever cluster you have running. 

Disruption in how services are delivered

Folks in the community will create and manage services in the services repo based on open source software packages. I see traditional software companies packaging their software into cluster services and perhaps charing for licensing. I see startups jumping at the chance to be first to offer new services that run on these clusters. And the kicker, I see cloud providers packaging their existing services so that you can run a copy of their service in your cluster, perhaps for a fee. Imagine running services like AWS Kinesis, Google machine learning, Oracle DB, azure's data warehouse or whatever, anywhere you want. Awesome.  

The end state unicorn

Containers, clusters, cluster operating systems, cluster services and a service repository will change the way we use data centers. It will change the way cloud services and open source software is packaged and delivered. This will have all the benefits of Platform as a Service. The control of doing things yourself. The dream of utility compute will be realized. Cloud provider lock in will be a thing of the past. Clustering and the service appstore will rock the cloud industry. 

What now

There is still a lot of work to do. Some of the things I described are here today and some are just visions that are being worked on. These technologies have huge momentum and I'm personally very interested in all this.  I believe in this disruption and I'll be investing in it one way or another. If you are in the cloud business you should consider this idea and decide if it's worth embracing. If you are an engineer or developer, keep an eye on these developments. 

Excited 

--Chris

8 Tips - Build a Highly Available Service

Working at AWS, Citrix, Register.com, Above.net and CenturyLink has taught me a lot about availability and scale. These are the lessons I've learned over time. From infrastructure as a service to web applications, these themes will apply.

Build for failures

Failures happen all the time. Hard drives fail, systems crash, performance decreases due to congestion, power outages, etc. Build a service that can handled failures at any level in the stack. No one server, app, database or employee should be able to cause your service to go offline. Test your failure modes and see how your system recovers. Better yet, use a chaos monkey to continuously test for failures.

Build across multiple failure domains

A failure domain consists of a set of infrastructure that can all go down due a single event. Data centers and public cloud availability zones are examples of failure domains as they can go down due to one event (fire, power, network, etc). Build your service so that it actively serves customers in multiple failure domains. Test it. A simple example is to use global load balancing to route customers to multiple failure domains.

Don't have real time dependencies across failure domains

Don't build a distributed system that relies on synchronous communication across failure domains to serve your customers. Instead, build systems that can independently service your customers completely within a failure domain and make any communication between failure domains asynchronous. Having inter-failure domain dependencies will increase the blast radius for any single outage and increases the overall likelihood of service impacting issues. Also, there are often network instabilities between failure domains that can cause variable performance and periods of slowness to your systems and your customers. One example of this is data replication. Don't require storage writes to be replicated accross failure domains before the client considers the data 'stored'. Rather, store it inside the failure domain and consider it committed. Handle any cross failure domain replication requirements asynchronously, IE, after the fact.

Reduce your blast radius

If a single change or failure can impact 100% of your customers, your blast radius is too large. Break your system up in some way so that any single issue only impacts a portion of your customers. User partitioning, using multiple failure domains (global load balancing), rolling deployments, separate control planes, SOA and A/B testing are a few ways to accomplish this. One example is using partitioning for an email sending service. Assign groups of customers to different groups of email sending servers. If any group of servers has an issue, only a portion of your customers are impacted versus all of them.

Reduce the level of impact

Having your service go completely down for a portion of your customers is much worse than only having a part of your service unavailable for a portion of your users. Break apart your system into smaller units. An example is user authentication. Consider having a scalable, read only, easily replicated system for user logins but have another system for account changes. If you need to bring down the account change system for whatever reason, your users will still be able to login to the service.

Humans make mistakes

Humans are the reason for most service impacts. Bad code deployments, network changes with unintended consciousness, copy/paste errors, unknown dependencies, typos and skill-set deficiencies are just a few examples. As the owner of a service it is critical that you apply the appropriate level of checks, balances, tools and frameworks for the people working on your system. Prevent the inevitable lessons learned of one individual from impacting the service. Peer reviews, change reviews, system health checks and tools that reduce the manual inputs required for 'making changes' can all help reduce service impacts due to human error. The important thing here is that the sum of the things you put in place to prevent human error can not make the overhead of making change so high that your velocity falls to unacceptable levels. Find the right balance.

Reduce complexity

I am not a fan of the band Kiss, but I do like to keep it simple stupid. A system that is too complex is hard to maintain. Dependency tracking becomes impossible. The human mind can only grasp so much. Don't require geniuses to make successful changes on your system.

Use the ownership model

IE, use the devops model. If you build it you should also own it end to end (uptime, operations, performance, etc). If a person feels the pain of of a broken system, that person will do the needed to stop the pain. This has the result of making uptime and system serviceability a priority versus an after thought.


Good luck

--chris

Infrastructure Internet Markets - Then, now and next.

The garden of markets

The internet is like a garden. Things grow, as long as you water them. In fact, the internet is really like a continuously expanding cluster of gardens, each with a different environment (aka market). Sales, marketing, finance, communication, and everything in between.

What I'll be talking about here are the infrastructure markets that grew to meet the needs of the internet.

Some new markets (gardens) of the past ...  

Connectivity, Dialup and beyond

First, there was the need for people and companies to be connected to the Internet. An untold amount of ISPs (internet service providers) were created to meet this need. They ranged from the size of your local mom and pop shop all the way to the size of AOL. Hundreds, thousands, hundreds of thousands of these companies were created world wide in order to meet the demand of humans wanting to communicate with one another. The ISP market will always exist and it will continue to grow and evolve. The speed of our connections will increase and along with it, what we do on the internet will change as well.

Web Hosting. Anyone remember geocities?

How many web hosting companies have existed in the last 15 years? A lot. If companies wanted to get on the internet, the most common way to do that at the time, was to pay a web hosting company to manage the server and internet connectivity for you. In this method, all you had to do was worry about creating and maintaining the content (which was a whole other market).

Today's new garden is... IaaS and PaaS

Infrastructure as a Service (Iaas)

Remember the ISP and web hosting markets exploding? Well, IaaS and PaaS is next! It is now!


When Amazon web services came out, it caused somewhat of a distribution in the industry. Why was this such a big event? Well, it was the first time internet infrastructure could be programatically controlled (can anyone say api). So simple, yet so game changing. Infrastructure as a service (IaaS) is really just web operations (data center, servers, storage, network) with an api in front of it. The benefits are huge to both internet based companies and tradition enterprise IT organizations.

We are now in a gold rush. Everyone under the sun wants to offer an IaaS solution. No one wants to be left behind. ISPs, Data center providers, enterprises, web hosting companies, governments, education, and even individuals are all getting on the boat. All of these people are interesting in providing IaaS features to their clients. Both publicly and privately.

There are now CMS (cloud management solutions) that allows these organizations to create an IaaS solution without the investment in writting their own code. Open Stack, Cloud stack, and eucalyptus seem to be the leading options available today. An organization can have an inhouse proof of concept up and running in a matter of hours. A production ready system can be ready in a few months.

Platform as a Service (Paas)

This is up a layer from IaaS. Instead of offering raw infrastructure, it offers a more of an application platform to its users. Someone only needs to upload their application, ie source code, and the PaaS will run and scale their solution as required.

Salesforce, Engine Yard, AppEngine, and Azure are a few of the leading public PaaS providers. Now, there are also PaaS software solutions that will allow people to create their own PaaS. Openshift and cloudfoundry are a couple in this category.

If I where you

I would get into this scene today. In 5 years from now, no one will be provisioning infrastructure manually anymore.

 

 

 What Next?

Obviously, more growth in IaaS and PaaS is coming. Software and solutions to help secure, monitor, provision, deploy all this stuff is currently being developed. But, what is the next garden? What is the next culture shift?

If I knew the answer to this, I probably wouldn't be talking publicly about it. The point is, some new internet garden will arise in which we will all see ourselves shifting onto.

Do you have a guess? Do you know?

Thanks
Chris