9:07 I just go the email that I can download the private key and certificate to sign the application. I will try this out today. I took several faxes to ChoosenSecurity to prove that AppsDen, LLC is really my company. The issue was that AppsDen belongs to Nouvelles Solutions, Inc (http://n-so.com) which is my consultancy company. Anyhow I received the certificates and this will allow me to code sign the application which in the during the application install shows a yellow question mark icon and a green check mark instead of a redcross. In the end I don’t think most user will know the difference.
So what’s on the agenda today…the biggest task I need before I can release a first public beta version is that I need to configure the pricing for more usage codes. I created a cool aggregation/price calculation engine and used the codes based on my amazon web services usage, but I need to find all the codes for all the different type of services amazon sells. I.e. for Rds I use a small instance and the usage type is InstanceUsage:db.m1.small so now I need to make sure what the usage type is for all the other configurations for Rds. I must do the same for all the other types of services. Also there are different prices for different regions and I didn’t take that into consideration. The best would if I could get all these different usage type codes from amazon. The second best is if people could help me out and send me their usage reports. If I cannot figure these out by myself I will make a “Call For Help” see if I can amazon users to provide me with their reports. Another approach would be for me to subscribe to all the services and check my usage reports. That may cost a little but I could just subscribe to one hour of each services across all zones. I will calculate how much that will cost.
So these are some of the services I currently haven’t used but count against your invoice:
- Spot Instances,
- VPN Connections,
- Elastic Block Store,
- Elastic IP Address,
- Elastic Load Balancing.
There is also reserved instances, but that’s a one of payment so I don’t think I need to calculate the pricing for these. Amazon has a calculator http://calculator.s3.amazonaws.com/calc5.html that shows all the combinations for which I need to figure out the usage codes, but I will also allow to me to determine how much I may paid If I want to use one of each service to get to these usage codes. For example when using the largest ec2 box (High-MEM Quadruple Extra Large) It’s $2.86 for an hour of a linux box,$2.88 with Windows, and $4.08 with Windows and SQL Server. This price varies slightly for each different regions: US-West (Northern California), US-East (Northern Virginia) or Europe (Ireland)
If I look at my ec2 usage log I see entries like this one:
This doesn’t tell me which type of instance or which zone the instance runs in. This instance run on Amazon EC2 (US East - N. Virginia) and is of type m1.small.
So I will now book a different type of instance still with linux and see the usage log is different. Right now I can only book one of two types if I select a 32 bit linux version:
When selecting a 64 bit I get the following choice:
So I’m going ahead with a m1.large. It’s really cool that you can just by a few clicks start a server in the cloud
Ready, set, Launch..The instance is launching…Mmm, why is the Root Device Type “ebs”. It’s running in the us-east-1d zone. All right, I stopped the instance. Let’s see if this entry shows up in the usage log after the next hour.
10:03 I’m also googling to see if I can any information on Usage Types. For Rds, the AmazonDocs show the following values: db.m1.small | db.m1.large | db.m1.xlarge | db.m2.2xlarge | db.m2.4xlarge
A discusion on Amazon’s community site, Amazon RDS DB Instance Sizing Guide () has the following comment:
"For people looking for specific sizes and descriptions:
- db.m1.small (1.7 GB of RAM, $0.11 per hour).
- db.m1.large (7.5 GB of RAM, $0.44 per hour).
- db.m1.xlarge (15 GB of RAM, $0.88 per hour).
- db.m2.2xlarge (34 GB of RAM, $1.55 per hour).
- db.m2.4xlarge (68 GB of RAM, $3.10 per hour).”
This maps some of the Rds usage log entries:
When creating a Rds instance you can specify the type of instance as well as the zone. However the pricing structure, contrary to ec2 instance, is the same regardless of the zone. So I think my Rds pricing logic is now good to go.
Let’s start another small instance in a different zone. I currently have a small instance (m1.small) in US-East (Northern Virginia) Region. Doesn’t seem that I can access another zone at the moment:
Seems that West or Europe is sold out. I’m wrong, it’s just that in the AWS Management Console you must select the zone before starting the instance, now I see:
10:40 A new small instance is starting…Time for some more (de)cafee. After that I will review S3, Sdb, Sqs, and Ec2 pricing logic.
10:51 I’m back. Let’s terminate that small instance.
Let’s see if that large instance appears. Yea, in my UI it appears. When I drill-down to today I see following following aggregation. Notice that I calculate 0.0 for the BoxUsage:m1.large. I can now fix this.
In the usage log file you see following entry:
From EC2’s pricing page
We see for example the price structure for each of the instance type (US-East (Northern Virginia) RegionB):
- $0.085 per Small Instance (m1.small) instance-hour (or partial hour)
- $0.34 per Large Instance (m1.large) instance-hour (or partial hour)
- $0.68 per Extra Large Instance (m1.xlarge) instance-hour (or partial hour)
- $1.20 per Double Extra Large Instance (m2.2xlarge) instance-hour (or partial hour)
- $2.40 per Quadruple Extra Large Instance (m2.4xlarge) instance-hour (or partial hour)
- $0.17 per High-CPU Medium Instance (c1.medium) instance-hour (or partial hour)
- $0.68 per High-CPU Extra Large Instance (c1.xlarge) instance-hour (or partial hour)
So the following instance types are provided by ec2: m1.small, m1.large, m1.xlarge, m2.2xlarge, m2.4xlarge, c1.medium, c1.xlarge. So in addition to the same type than Rds there are two different configuration c1.medium and c1.xlarge which provide High-Cpu machines. It makes sense that these are not provided for Rds with is supposed to be data intensive machines.
This still leaves me figuring out the zones and the windows operating system. I assume it’s a combination like the above BoxUsage:m1.large.zone.os. But I cannot assume, let’s see in an hour what the usage log say about that instance we started in a different zone. In the mean time I’ll try to code sign the application.
11:15 Ok, downloaded the certificate and self signed the application. Now when you want to install it you see the publisher name and that the publisher’s identity has been verified..Yea!
11:32 Ok, lets work on something different. I display the following pie charts for high level summary of service usage. Let’s see if we can make it look somewhat nicer.
Tried with new colors and using out label callout instead of insideWithCallout
Here is insideWithCallout
And with the label inside:
Now with the label inside some of the labels are dropped but it seems to drop the labels for the smallest piece of the pie. This works for me as we still display all the values in a datagrid next to the chart. Now I just need to get some better colors.
Ok I went back to the default colors the component provides and just set the alpha of each pie to 80% which provides for the rounded edge of the pie effect.
I could also “explode” the pie, but that doesn’t look nicer.
12:32 Enough playing with the charting component. I’m getting hungry, but let’s first see if I got the usage data for the small instance we start and stopped over an hour ago. What’s nice with this app you can just click on the “Reload” button and all the usage data for the selected period is just downloaded again from amazon web services. So I am getting the following new data:
So it seems the region is prefixed. Ok, lets start a windows box in Europe see what I get. So from the URL of the aws management console I get the following region codes: us-west-1, eu-west-1 and none for US East which is the default. So let’s create launch a Windows box in Europe:
12:47 Time for lunch.
14:00 Back from Lunch…I was reading up on netbooks. Now back to reality.
Ok, like expected The box usage is prefixed with the “EU-” and the server type (c1.medium) was added at the end. However I don’t see an indication that this box did run Windows, now that would be an issue. After looking at the logs again the operation name indicate the type of os.
So “RunInstances” is linux, “RunInstances:0002” is Windows and I assume “RunInstances:0003” is Windows with SQL Server. I’ve got to try that now. So let’s start a Windows with SQL Server instance and see if this confirms my supposition.
Note if that’s true I will need to change some of my pricing logic as until now it was only based on the UsageType.
15:15 It’s true. And if fact that may also point the price difference I have in calculation so far as I must use both the OperationName and UsageType to uniquely identify an item. So the operation name for this instance is was “RunInstances:0006”. Now I wonder which if anyone has RunInstances:0003 and 0004 and so on?
So what was I thinking before writing this product, that it would be easy? At least if I decipher how this gets calculated then you don’t have too. That’s the value proposition of AwsUsageAnalyzr. I guess I still need to find people that submit their logs so I can cover more ground.
16:11 Refactoring is a little laborious but I cleaned up some of the pricing code a little more but I’m still not done with the EC2 structure. I’m moving to Panera and will continue there.
17:54 All right, I’m back in business. Let’s continue on the EC2 pricing.
All right I mapped most of the items required for EC2 calculation. However the following are still missing regarding EBS $0.11 per 1 million I/O requests and $0.18 per GB-Month of snapshot data stored as I don’t seem to find a correspondence in the log files.
Another interresting asspect is the difference instance types provided for Rds (m1.small, m1.large , m1.xlarge, m2.2xlarge, m2.4xlarge) compared to the ones for EC2 Windows with SQL Server: m1.large, m1.xlarge, m2.2xlarge, m2.4xlarge, c1.xlarge. So the Windows database server doesn’t offer a m1.small but does offer a c1.xlarge. I guess SQLServer cannot run on a small instance and could become cpus intensive ;-)
Le’s look at the price graph for the month:
Note too bad, usually I run at $2.40 a day for my ec2 instances and today it seems to be around $4.22.If we drill down to today we can the price per hour and see what hour where charged most (17, 19 and 21 Amazon time).
So I we look at the usage breakdown for the day:
If we look at the m1.large operation we see the two different instances we started and they add up to a $1.42:
I also have the EU-BoxUsage:c1.medium which Amazon doesn’t report yet in their activity report, only in their usage reports.
Now the EBS calculation seems wrong. Amazon billed the following:
- $0.10 per GB-month of provisioned storage 0.121 GB-Mo 0.01
- $0.10 per 1 million I/O requests 15670 IOs 0.01
And I have the followings:
- EBS:VolumeUsage 0.0088
- EU-EBS:VolumeUsage 0.0065
- USW1-EBS:VolumeUsage 0.0033
Which totals to 0.0186 instead of 0.01
- EU-EBS:VolumeIOStorage 0.0015
- EBS:VolumeIOStorage 0.0002
Which totals to 0.0017 instead of 0.01.
So for the first case I assume that either amazon activity reports have a big delay and may not have the EU data. For the second case it shows that I still need to figure out how and at what stage they round their numbers.
Now what do I need to release a first version for beta? I think the calculation must be more precise. I think the application is already very useful as is, but is must be pretty accurate. Maybe I need to release if to beta tester to find more bugs quicker. I could add a built-in debugging tool that allows user to notify me of inaccuracies. That would be helpful. The licensing tool doesn’t support beta versions, however I could release with a large trial period that could ensure that I have to finish the product before accepting payments. So before that I must just check that I can reset the trial period just in case . Besides the trial period I need a super application icon. Just kidding, although not totally. Then I need to test it on Windows as I store downloaded data from Amazon locally and must ensure that the app has no issues with the file system. Then we should be good to go!
20:30 Lets beef up the Amazon sign-in process. In order to download automatically the usage reports the user must use the application build-in Browser to login in. That way the application can make http requests to amazon website. Before the Amazon web page was embedded on the Amazon Account page but it was obvious what was going on as there was not clear delimitation between the web page and the application. So I just added a Panel in which I embed an HTML component that allows to do the login. It now looks like an embedded browser and hopefully makes the intend cleared.
21:15 That’s it for me today.