Save Cost with combinations of Runbooks and Reserved Instances

I have explained in my previous blog about saving cost using RI and how we can plan. Let’s looks at the combinations of Runbooks and Reserved instances in this blog to save the cost further. We should be careful to get the cost savings out of this model.

You can power off the VMs in non-production environment to save cost when not in use. You can use the runbooks to schedule the Power on and off of the VMs. The below table will provides an insight on what level of cost saving can be brought using runbooks. 

  If you power off your VMs after your working hours and holidays you would be saving almost up to 76%. I have placed the 3 VMs in different time duration to explain the RI calculation later in this blog.  I considered 22 days in a month which excludes weekends    

Runbook                     Table 1 – Saving using Runbooks

However, if you consider public holidays, it would be more savings for sure. The runbooks provide cost savings more than 1-year RI and 3 years RI which are 21% and 34% respectively. This proves that RIs do not suit for your non-production environment because runbooks give you more cost savings than RI. So, you may consider RIs only for the production environment unless you see good amount of usage in the non-production environment. Continue reading “Save Cost with combinations of Runbooks and Reserved Instances”

All About Azure Reservation – VMs

Azure Reservations help you save money by pre-paying for one-year or three-years or monthly but commitment for 1 or 3 years of virtual machines, SQL Database compute capacity, Azure Cosmos DB throughput, or other Azure resources. Pre-paying allows you to get a discount on the resources you use. Reservations can significantly reduce your virtual machine, SQL database compute, Azure Cosmos DB, or other resource costs up to 72% on pay-as-you-go prices.

I would like to talk about how best we can utilize reserved instances (RI) and other techniques (runbooks) to bring more cost savings. We will also talk about how we can decide whether we should go with RI or on Demand Virtual Machines (VMs).

Let’s look at the some of the terminologies and how is it being used in the buy the RI from Microsoft.

Purchasing options

  •     1 Year commitment – Paid upfront or monthly
  •     3 Years commitment – Paid upfront or monthly

Microsoft has recently announced monthly payment of RI price which is really a welcome move from Microsoft. You can buy new reservations with monthly payment frequency and you can convert the existing RIs when you renew it to get the bills monthly.

You get the recommendation from the Azure Advisor which is available in the Azure portal for all the subscriptions. It is based on your usage. However, it is good if we could plan to select the right VM SKUs. Will talk about it.

One thing that you must remember that reservation discount is ‘USE IT OR LOSE IT’. You can’t carry forward unused reserved hours.

Generally, you do not get any benefits from RI if the VMs are not utilized above 60-70%. But I will talk about this how we can bring additional benefits on such scenarios.

I will be talking only about VM RI in this blog.

To plan, you need to know few things.

Continue reading “All About Azure Reservation – VMs”

You don’t be that FROG in the well.

I remember I learned about a frog that lived in the well. Since frog was in the well entire his life he thought that the Well is the world and there is nothing beautiful than that until the water in the well dried and frog had to come out of it.

We should not be that FROG in the well.

Froginthewell

There are lot of things in the world that we have not learned and seen yet. We must be open to learn. You might be able perform well in your job today. But we need to think about tomorrow. The world is changing too fast, so we need to learn faster and act swiftly to survive in the world.

It is not just about learning, we also need to look around and check are we doing it properly. It this the best of doing? There are 100s of people in the world doing similar stuff differently. It is a challenge to get know about how others are doing. That is where you need to collaborate with others, engage yourself with others in the world, hear other, share your thoughts,  arranged hackathons to bring new ideas and encourage openness in the team to bring new thoughts. We never know, the ideas can come from junior in the team. Remember an idea can change your world !!!

Continue reading “You don’t be that FROG in the well.”

Satya’s Microsoft is making more friends, Hello Oracle !!!

Using ExpressRoute and FastConnect, customers can peer a virtual network in Azure with a virtual cloud network in Oracle Cloud infrastructure (OCI). This is probably pleasant and positive news I heard in this space. It is a game changer model they two Cloud Provider has done so far. It requires lot of courage at the leadership level to think about it and execute it. I have been thinking about such scenarios but was expecting a network provider to have this connectivity across the multiple clouds, but Azure has done it by themselves.

Azure AD synchronizes any changes in the directory with the corresponding Oracle directory and is used for single sign-on to cross-cloud Oracle solutions. I can only say ‘WOW’ here.

Oracle realized that we need more partners than enemies, I think leadership changes at the Oracle would have helped change their strategy. Under the leadership of Satya Microsoft is making more friends. It is like ‘an idea can change your like life, a leader can change a company’.

Oracle and azure connect

Looks at that picture above when you run the application tier in Azure and Database tier in Oracle Cloud. Good for the customer who is paying huge amount for Oracle license for running Azure VM. It must be cheaper in OCI. We still need to see how the network latency and cost behind this. This give Azure an advantage over AWS because Oracle lovers will opt for Azure and they connect to Oracle for those Oracle products feature like Real Application Cluster (RAC) which can not be run on Azure or AWS today. I am sure there will be challenges but would love to propose a design to our customers. It is too early to comments on the issues and challenges, but I am excited about it.

Oracle and azure connect3

Continue reading “Satya’s Microsoft is making more friends, Hello Oracle !!!”

Infra Folks, shouldn’t we change our approach with Public Cloud

All of us are familiar with below picture and can easily describe the IaaS, PaaS and SaaS. Are we thinking about how it is changing our way of working? It is affecting both Infra and App team as both the teams need to think about each other’s territories especially when somebody need to build DevOps culture.

PaaS

It is gone those days we just don’t care about any application or functionality what is running on the servers that we built. We used to provide number of servers based on the sizing and design. We did not have to know what functionality the application would perform or roles as well.

You can continue with the above if you are still thinking of riding on IaaS (Infrastructure As A Services) but not with PaaS (Platform As A Services) and SaaS (Software As A Services). Infrastructure team has little things to do now as most of those tasks will be performed by your Cloud Service Provider.

Everybody talks about serverless computing even though, there is none at the backend 😊. We should be able to think about moving away from monolithic architectures, moving to microservices and containerize wherever possible.

Continue reading “Infra Folks, shouldn’t we change our approach with Public Cloud”

All about Application Security Group

What is Application Security Groups?

ASGs enable you to define fine-grained network security policies based on workloads, centralized on applications, instead of explicit IP addresses. Implementing granular security traffic controls improves isolation of workloads and protects them individually. If a breach occurs, this technique limits the potential impact of lateral exploration of your networks from hackers.

You may find the details in the MS site more about this which I do not want to copy and paste it here. Let’s talk about the use case and how we can make use of his in better way.

Deny all the communication and open the specific communication using ASG. Yes, you can create a Deny All rule with lower priority within your vNET. Then you create specific ports to open but you will select ASG as source and destination. This will open the communication between those servers have the specific ASG configured. Looks at the below pictures (figure1&2) to understand this better.

You do not have any option to add a server in the ASG but you need to go and select the required ASG from the vNIC of the VMs. You can add this option in the ARM templates to configure when you create this VM. This will reduce number NSG changes you need to make every time you add a server rather you select required ASG while you create the VM.

You need to remember few things about ASG.

  • You cannot make any settings on ASG but you can only add tags.
  • You can only select one ASG as source or destination in every NSG rules.
  • You can select multiple ASGs for single VM.
  • Limitations
    • 3000 per subscription
    • 20 per vNIC
    • 4000 IP configuration per ASG
  • You can only assign ASG from the same subscription.
  • You cannot have VMs from different vNETs in one ASG.
  • Both source and destination ASGs in your NSG rules should be in same vNET.

Continue reading “All about Application Security Group”

Increase your IaaS Cloud Security in Azure

The security is the key pillar of designing the public cloud infrastructure. We must create the security principle to define the security design by bringing best practices.

There are different ways of securing your environment by implementing such as identity, automation in security, secured data at rest, transit etc. We would be looking at applying security layers in the Azure infrastructure. Microsoft is providing multiple options in Azure to secure your network like Network Security Groups (NSG), Application Security Grups (ASG), Azure Firewall, Web Application Firewall (WAF), Network Virtual Appliance (NVA), DDoS etc. too apply at different layer. Based on our requirements we can select each option available and or combinations of each those options.

 

It is important that we understand important of each of those to select right security options available. There is no doubt that security is important but at the

 same time we should not make things very complex by introducing everything in our Azure infrastructure. Let’s take some scenarios and discuss those each of my coming blog.

Let’s looks at the above scenario when we have Hub and Spoke model Azure vNET implement. Hub and Spoke vNET implementation are recommend by Microsoft.

Let’s identify our building block of our core design and will do one by one. Let’s create some assumptions for our design.

vNET – We will use 4x vNETs.

Express Route – Express route will connect to the Hub vNET

                1x Shared vNET called Hub vNET

                1 spoke vNET each for segregating each environment required.

Site to Site – this can be used alone of combinations of both ER and S2S.

1x NSG per vNET – 4 NSGs in total for 4 Spoke vNET. Or you may prefer to use per subnet.

ASG will be create for each tier for the specific security zones or specific applications.

2x node checkpoint NVAs in scale set mode. This includes 1x external and 1x internal load balancers.

No public IP on the NVA.

Public IP will be configured on External Load balancers only.

Application Gateway (WAF) with private IP in each vNETs.

We will talk about different benefits and use cases when you combine all of these in my coming blogs. Please join me interacting with my blog posts.

Old is not bad as you think…

I had a train ticket booked from Bangalore to Chennai after a training. I planned and started from office on time (3pm) to catch the Shatabdi train (4.25pm) and reached Cantonment by Uber taxi in 50 minutes ahead of the train time. I got a call from my wife by 3.58 pm and said that Shatabti won’t stop at Cantonment, Bang !!!.  Just over 25 minutes left, and I need to reach Bangalore City Railway station from Cantonment in the hectic Bangalore traffic, I literally gave up and told her to look for bus ticket.

Old vs New

However, I decided to give it a try, otherwise I wasn’t sure whether I would end up going back to the hotel and may have to delay my travel further, during the festival season.  My natural choice would have been booking Uber or Ola cab, but I decided to hire an auto which was right in front of me, that saved me minimum of 5 minutes of waiting time for OLA/Uber and also auto guys could beat the traffic better than taxi. I checked my google map and it showed me 4.30 pm as ETA but I wasn’t sure with the number of signal the auto should cross. However, auto guy managed to get me closer but nearly 1 KM away from the station. And then we were stuck at the signal with more than 300 meters long traffic. I got down and walked around 400 meter and got in another auto just for another 400 meters and reached railway station. Then I took the help of carriers(porter) to run with me with my travel bag and also to locate the train as I was not familiar with that station, finally I got in the train by 4.35 pm and the train started by 4.36pm. Thanks to Railways (train was late) and my luck J. I would have missed the train if I had opted for App taxi or stayed in the first auto or had not used those carrier guys for sure. Continue reading “Old is not bad as you think…”

Don’t hurry up. Wait. Prepare first then make slow cloud move.

Most of the organizations are keen on moving their workload to cloud today for several reasons like their IT vision, reduce the spend on hardware refreshes, data center consolidations etc.

Are they ready move into the Cloud? It is an important question that every organization should ask again and again before taking the decision to move in with big bang. We see a trend with many customers to move their existing legacy applications ‘as is’ to the cloud.  Shouldn’t we move into the cloud and utilize those benefits, or we just move in and I don’t care about those cloud features?

Let me start with an example here. Let’s take a case of four webservers and two database servers clustered available 24/7 with environments like Dev, Test and Prod. And you wanted to move this workload to cloud ‘as is’.  My question is, what is the objectives are you trying to achieve? If the answer is, our organization wanted to move all the workload to cloud for cost saving, changing from Capex to Opex model etc.  Guys, hold on… Let’s calm down, think, look around and plan again.

Lift and shift should not be our strategy for cloud migration. We should make our application to live smartly in the cloud to utilize the cloud benefits and reduce the cost. Let’s use the above example to explore this further.

  • Can we make this application horizontally scalable?Cloudthink
  • Can we make this application to use cloud native authentication?
  • Can we make this application to work stateless?
  • Can we make the applications to use distributed data storage?

Continue reading “Don’t hurry up. Wait. Prepare first then make slow cloud move.”

ExploreExplore the technology to Upskill, Reskill and CrossSkill the resources.

Yes, most of us work in Technology companies those advise many organizations globally for taking technological decision and encourage them to use the latest and advanced technologies to automate their operations. However, how many of those companies are using technology within their organizations and drive innovations for their internal requirements. Both the employees and employers are suffering because a proper system is not in place for solving the problems like Lack of effective appraisal system, identifying the right resources for the projects, unable to stop the talent leaving the organization etc. Let’s look at those and how we can try to solve using the technologies like Machine Learning(ML) and Artificial Intelligence(AI). Yes, we are in the world of Machine Learning(ML) and Artificial Intelligence (AI) and we must now start thinking about using them effectively.

upskill2

Continue reading “ExploreExplore the technology to Upskill, Reskill and CrossSkill the resources.”

Machine Learning: I do it, you do it everyday without a computer

Machine learning is ‘predictive analysis’ in very simple terms, agree? Arrive into a conclusion by analyzing the data. Can this possible only with computers?  We, human beings do it on a daily to basis, to catch a bus, to drive a car, to shop and what not.

Machine learning Fig1

Don’t be confused. Let me try to explain and don’t blame me at the end if that doesn’t work, a pre-bail has been taken :-).

Let me take the example of how we learn driving. Excuse me for those who never tried it 🙂.

The driving instructor gives you first set of data like usage of steering wheel, gear shifting, clutch, brake, accelerator etc. Initially, with those basic data you mess with clutch, gear, accelerator and brake often. Then you slowly correct the mistakes by practicing which means you are learning by feeding your brain with additional data on how to use it effectively.  Eventually you get it correct when you have more and more data; and create your own algorithm to drive your car.  Result of this you start applying the break softer, start shifting the gear smoothly.  That is exactly the machine learning does with the help of your intelligence (it is currently being replaced with Artificial Intelligence –AI 🙂).  Continue reading “Machine Learning: I do it, you do it everyday without a computer”

Availability Zones for Azure, like in AWS.

Microsoft has now announced its long pending Availability Zones in each region. It is currently on preview and recommended only for non-critical workload as Micorsoft does not provide any SLA now. With this, you can now provision your workload from different data centers in the same regions for resilience as you will have options to select between minimum of 3 AZs in each region with GA. However, it is now available only on East US2 and West Europe for Preview.

AZ Fig01.png

AWS currently operate 44 AZs across 16 Regions and 14 more AZs are already planned (44+14 =58). Microsoft currently operate in 36 regions and 6 more to come. If you assume that Microsoft will bring 3 AZs minimums at each of these regions, Microsoft would have (36+6) *3= 126 which itself is more than double the size of AZs across the globe. I agree, it does not make much sense to just play with the numbers so Microsoft need to bring the services that would help customers to make use availability zones and add value to their workloads hosted in Azure services. AWS currently offers multiple PaaS services for their Multi-AZ deployment model so Microsoft still need do good job on making sure that more services are available for Multiple AZ deployments.

Continue reading “Availability Zones for Azure, like in AWS.”

Bring Home Azure with Azure Stack

It is not just the public cloud today, but it is Hybrid Cloud.

Microsoft is working on making our hybrid life less difficult by introducing Azure stack. We all know the pain of getting Microsoft System Center integrated and working on-premises to enable private cloud. Yes, I agree with you SC is not a candidate for comparing it with Azure Stack. However, I believe Azure Stack will be solving these issues and bringing cloud to your data center with ‘Pay as You Use’ Pricing model.

What is Azure stack as per Microsoft?

Microsoft Azure Stack is a hybrid cloud platform that lets you deliver Azure services from your organization’s datacenter. Bring the agility and fast-paced innovation of cloud computing to your on-premises environment with Azure Stack. This extension of Azure allows you to modernize your applications across hybrid cloud environments, balancing flexibility and control. Plus, developers can build applications using a consistent set of Azure services and DevOps processes and tools, then collaborate with operations to deploy to the location that best meets your business, technical and regulatory requirements.

Azure Stack

Continue reading “Bring Home Azure with Azure Stack”

Amazon EC2 Systems Manager – Key Points We need to know

Amazon EC2 Systems Manager is a flexible and easy to use management service that enables enterprises to securely manage and administer their workloads, running on-premises or in AWS, using a single unified AWS experience. EC2 Systems Manager is designed to be highly automation focused to enable configuration and management of instances at a large scale, while making it really simple to write and maintain automation artifacts.

It is another service which going to kill slowly others by extending support to on-premises servers. The beauty of this service is, it is absolutely free as long as you meet the pre-requisites.  It covers all we need like patching, compliance management, automation, inventory etc. Interesting one is Parameter Store using which you can store your password encrypted reducing the manual overhead of storing and managing it in configuration files. It can be easily integrated with IAM and KMS.
Even though it is supported on-premises servers, it is going to be challenging as AWS needs to push all updates directly and also collect the inventory from each endpoint in corporate data centers. We end up with making many holes on the firewalls and also lot of traffic between AWS and our data centers. This will not only overload your network but also increases your AWS bill.  It was good to have something site agents which can coordinate with agents in the sites and report back to the SSM services in the AWS cloud. This would avoid too much of client connecting directly to the AWS cloud over the link and will be easy to maintain the firewall rules as well. AWS does not recommend keeping many workloads on our premise as they want everything to be moved to the cloud so we may have to rule out anything like that in the future. Below diagram show how the service manager connects to the endpoints.

blog04

Continue reading “Amazon EC2 Systems Manager – Key Points We need to know”

AWS December Announcement: An infrastructure guy should be memorized

download

We have watched many big announcements from AWS this year November and December. It was all part of the AWS Las Vegas Re-invent. You can watch many of the key notes and session from the Youtube easily. You can watch the keynotes from AWS as well.  

I have listed below some of the announcement that AWS made which I think we, infrastructure architects should be noted. It is not the complete list however; it covers only December month announcement as of 10th Dec 2016. Continue reading “AWS December Announcement: An infrastructure guy should be memorized”

What does the AWS | VMware Partnership mean ?

When I heard this for the very first time, I got confused ‘Ehhh What is it? I am sure a few of you had the same puzzle.

At a higher level the largest public cloud company joining hands with largest private cloud offering company.  Alright!!! But what is it in technical terms. This is what I want to know because ….

compoents-of-vmware-cloud-on-aws

As per AWS, it is a native, fully managed VMware environment on the AWS Cloud that can be accessed on an hourly, on-demand basis or in subscription form. It includes the same core VMware technologies that customer runs in their data centers today including vSphere, Virtual SAN and NSX network virtualization platform and is designed to provide a clean, seamless experience.

Continue reading “What does the AWS | VMware Partnership mean ?”