third party api integration service example
h2. Disclaimer!
The following document describes Chronatog as a web cron service. The descriptions presented are accurate with one important exception: ACTUAL CRON IS NOT YET IMPLEMENTED. This means you can register as many callbacks as you want, see them, delete them etc… but they will never actually run. Chronatog is currently entirely missing it’s underlying actual cron component.
h1. Chronatog
Chronatog is:
h2. Getting Started: Deploying your own Chronatog
h3. Become a Partner
First you need an Engine Yard Add-on partner account. Once you have one, you can access “https://services.engineyard.com”:https://services.engineyard.com.
h3. Deploy Chronatog
For the rest of these steps, Chronatog will need to be running somewhere with a publicly accessible url.
h3. Save your credentials
In Chronatog, credentials are stored in the database.
Example in script/console:
bc… $ script/console
Chronatog::EyIntegration.save_creds(‘ff4d04dbea52c605’, ‘e301bcb647fc4e9def6dfb416722c583cf3058bc1b516ebb2ac99bccf7ff5c5ea22c112cd75afd28’)
=> #<Chronatog::EyIntegration::EyCredentials id: 1, auth_id: “ff4d04dbea52c605”, auth_key: “e301bcb647fc4e9def6dfb416722c583cf3058bc1b516ebb2ac…”, created_at: “2012-08-22 11:14:48”, updated_at: “2012-08-22 11:14:51”>
h3. Test your connection
To test your connection to services.engineyard.com, you can make a GET request to the registration url. This returns a listing of currently registered services.
Example:
bc… $ script/console
Chronatog::EyIntegration.connection.list_services(registration_url)
=> []
p. Behind the scenes, Chronatog is calling out to @EY::ServicesAPI@.
@list_services@ is a method on @EY::ServicesAPI::Connection@.
h3. Register your Add-on services
To register your Add-on service, you make a POST request to the registration url, passing a JSON body describing your Add-on service. Included in that description are callback URLS, so in order to generate them Chronatog needs to know it’s public-facing url.
Example:
bc… $ script/console
registration_url = http://services.engineyard.com/api/1/partners/96/services
chronatog_url = “https://chronatog.engineyard.com”
registered_service = Chronatog::EyIntegration.register_service(registration_url, chronatog_url)
=> true
p. Behind the scene, Chronatog is calling @register_service@ on a @EY::ServicesAPI::Connection@. The first parameter is the @registration_url@. The second parameter is a hash describing the Add-on service being registered.
In the case of this example it looks something like:
bc… {
:name => “Chronatog”,
:label => “chronatog”,
:description => “Web cron as a service.”,
:service_accounts_url => “https://chronatog.engineyard.com/eyintegration/api/1/customers”,
:home_url => “https://chronatog.engineyard.com/”,
:terms_and_conditions_url => “https://chronatog.engineyard.com/terms”,
:vars => [“service_url”, “auth_username”, “auth_password”]
}
h3. Viewing your Add-on service on cloud.engineyard.com
If your Add-on service registration succeeded, you should see it’s information displayed when you visit @https://services.engineyard.com@. From there you can enable testing of your Add-on service with your Engine Yard Cloud Add-on account.
!https://github.com/engineyard/chronatog/raw/master/images/make_available.png!
Once enabled for testing, you should see your Add-on service available if you navigate to “Add-ons” in the menu bar from @https://cloud.engineyard.com@ (or visit @https://cloud.engineyard.com/addons@).
!https://github.com/engineyard/chronatog/raw/master/images/listing.png!
h3. Verifying requests from Engine Yard
By using the @EY::ApiHMAC::ApiAuth::LookupServer@ middleware in the API controller, Chronatog verifies that each request to it’s API is correctly signed by the requester. The block passed to the middleware is expected to return the @auth_key@ correspondent to the @auth_id@ given. It is then up to @EY::ApiHMAC@ to calculate a signature and verify that it matches the one in the request (@env@).
bc… use EY::ApiHMAC::ApiAuth::LookupServer do |env, auth_id|
EyIntegration.api_creds && (EyIntegration.api_creds.auth_id == auth_id) && EyIntegration.api_creds.auth_key
end
h3. Enabling your service
!https://github.com/engineyard/chronatog/raw/master/images/signup.png!
When you click ‘Sign Up’, EngineYard will make a call to your @service_accounts_url@ to create a service account. In the case of Chronatog, this callback is handled by creating a customer record.
The request will look something like this:
bc… POST https://chronatog.engineyard.com/eyintegration/api/1/customers
{
“name”: “some-account”,
“url”: “http://services.engineyard.com/api/1/partners/80/services/81/service_accounts/85”,
“messages_url”: “http://services.engineyard.com/api/1/partners/80/services/81/service_accounts/85/messages”,
“invoices_url”: “http://services.engineyard.com/api/1/partners/80/services/81/service_accounts/85/invoices”
}
p. Chronatog will handle the callback with the implementation defined in the API controller:
bc… request_body = request.body.read
service_account = EY::ServicesAPI::ServiceAccountCreation.from_request(request_body)
create_params = {
:name => service_account.name,
:api_url => service_account.url,
:messages_url => service_account.messages_url,
:invoices_url => service_account.invoices_url
}
customer = Chronatog::Server::Customer.create!(create_params)
p. As part of handling the callback, a @Customer@ will be created:
bc… #<Chronatog::Server::Customer id: 1, service_id: nil, name: “some-account”, created_at: “2012-08-22 11:14:48”, updated_at: “2012-08-22 11:14:48”, api_url: “http://services.engineyard.com/api/1/partners/2/ser…”, messages_url: “http://services.engineyard.com/api/1/partners/2/ser…”, invoices_url: “http://services.engineyard.com/api/1/partners/2/ser…”, plan_type: “freemium”, last_billed_at: nil>
p. Chronatog returns a JSON response that tells EngineYard some information about the customer.
The code for generating that response:
bc… response_params = {
:configuration_required => false,
:configuration_url => “#{sso_base_url}/customers/#{customer.id}”,
:provisioned_services_url => “#{api_base_url}/customers/#{customer.id}/schedulers”,
:url => “#{api_base_url}/customers/#{customer.id}”,
:message => EY::ServicesAPI::Message.new(:message_type => “status”,
:subject => “Thanks for signing up for Chronatog!”)
}
response = EY::ServicesAPI::ServiceAccountResponse.new(response_params)
content_type :json
headers ‘Location’ => response.url
response.to_hash.to_json
p. Notice @EY::ServicesAPI::Message@ in the code above. The subject text should now appear in the context of the Chronatog service on @https://cloud.engineyard.com@.
What the generated response looks like:
bc… {
“service_account”: {
“url”: “https://chronatog.engineyard.com/eyintegration/api/1/customers/1”,
“configuration_required”: false,
“configuration_url”: “https://chronatog.engineyard.com/eyintegration/sso/customers/1”,
“provisioned_services_url”: “https://chronatog.engineyard.com/eyintegration/api/1/customers/1/schedulers”
},
“message”: {
“message_type”: “status”,
“subject”: “Thanks for signing up for Chronatog!”,
“body”: null
}
}
h3. Visiting Chronatog over SSO
With the service enabled, a “Visit” link should appear. Following this link will redirect to the @configuration_url@ provided in the response to service enablement.
The configuration url provided by Chronatog in this example was:
bc… https://chronatog.engineyard.com/eyintegration/sso/customers/1
p. When EY signs the url it provides additional parameters, such that it looks like this:
p. Chronatog will verify the SSO request with a before filter that looks like this:
bc… before do
if session[“ey_user_name”]
#already logged in
elsif EY::ApiHMAC::SSO.authenticated?(request.url,
Chronatog::EyIntegration.api_creds.auth_id,
Chronatog::EyIntegration.api_creds.auth_key)
then
session[“ey_return_to_url”] = params[:ey_return_to_url]
session[“ey_user_name”] = params[:ey_user_name]
else
halt 401, “SSO authentication failed. Go back.”
end
end
h3. Activating (provisioning a Chronatog Scheduler)
!https://github.com/engineyard/chronatog/raw/master/images/activate.png!
Clicking “Activate” will cause EngineYard to call to your @provisioned_services_url@ to create a provisioned service. In the case of Chronatog, this callback is handled by creating a scheduler.
The request will look something like this:
bc… POST https://chronatog.engineyard.com/eyintegration/api/1/customers/1/schedulers
{
“url”: “http://services.engineyard.com/api/1/service_accounts/104/provisioned_service/109”,
“messages_url”: “http://services.engineyard.com/api/1/partners/99/services/100/service_accounts/104/provisioned_service/109/messages”,
“app”: {
“id”: 106,
“name”: “myapp”
},
“environment”: {
“id”: 107,
“name”: “myenv”,
“framework_env”: “production”,
“aws_region”: “us-east-1”
}
}
p. Chronatog will handle the callback with the implementation defined in the API controller:
bc… request_body = request.body.read
provisioned_service = EY::ServicesAPI::ProvisionedServiceCreation.from_request(request_body)
customer = Chronatog::Server::Customer.find(customer_id)
create_params = {
:api_url => provisioned_service.url,
:environment_name => provisioned_service.environment.name,
:app_name => provisioned_service.app.name,
:messages_url => provisioned_service.messages_url,
:usage_calls => 0
}
scheduler = customer.schedulers.create!(create_params)
p. As part of handling the callback, a @Customer@ will be created:
bc… #<Chronatog::Server::Scheduler id: 1, customer_id: 1, auth_username: “U434828f006ba4b”, auth_password: “P1f49fd4df5588a3adc5f6b18bf”, created_at: “2012-08-22 11:14:51”, updated_at: “2012-08-22 11:14:51”, api_url: “http://services.engineyard.com/api/1/service_accoun…”, environment_name: “myenv”, app_name: “myapp”, messages_url: “http://services.engineyard.com/api/1/partners/112/s…”, decomissioned_at: nil, usage_calls: 0>
p. Chronatog returns a JSON response that tells EngineYard some information about the created scheduler.
The code for generating the response:
bc… response_params = {
:configuration_required => false,
:vars => {
“service_url” => “#{true_base_url}/chronatogapi/1/jobs”,
“auth_username” => scheduler.auth_username,
“auth_password” => scheduler.auth_password,
},
:url => “#{api_base_url}/customers/#{customer.id}/schedulers/#{scheduler.id}”,
:message => EY::ServicesAPI::Message.new(:message_type => “status”,
:subject => “Your scheduler has been created and is ready for use!”)
}
response = EY::ServicesAPI::ProvisionedServiceResponse.new(response_params)
content_type :json
headers ‘Location’ => response.url
response.to_hash.to_json
p. Notice @EY::ServicesAPI::Message@ in the code above. The subject text should now appear in the context of the relevant application and environment on @https://cloud.engineyard.com@.
What the response JSON looks like:
bc… {
“provisioned_service”: {
“url”: “https://chronatog.engineyard.com/eyintegration/api/1/customers/1/schedulers/1”,
“configuration_required”: false,
“configuration_url”: null,
“vars”: {
“service_url”: “https://chronatog.engineyard.com/chronatogapi/1/jobs”,
“auth_username”: “U57566cb822ac0c”,
“auth_password”: “P8348eec86b741dc20b78c5fe1f”
}
},
“message”: {
“message_type”: “status”,
“subject”: “Your scheduler has been created and is ready for use!”,
“body”: null
}
}
h3. Deactivating (de-provisioning a Scheduler)
!https://github.com/engineyard/chronatog/raw/master/images/deactivate.png!
p. Clicking “Deactivate” will cause EngineYard to DELETE to the @url@ provided by Chronatog in the activate/provision step. Chronatog handles the callback by destroying the appropriate scheduler.
The request will look something like this:
bc… POST https://chronatog.engineyard.com/eyintegration/api/1/customers/1/schedulers/1
p. Chronatog will handle the callback with the implementation defined in the API controller:
bc… customer = Chronatog::Server::Customer.find(customer_id)
scheduler = customer.schedulers.find(job_id)
scheduler.decomission!
content_type :json
{}.to_json
h3. Updating API keys
When the scheduler was provisioned we saved the @provisioned_service.url@ into the scheduler @api_url@. We can use this URL to update Engine Yard whenever we change vars.
For example, when we call @reset_auth!@ on a scheduler:
bc… def reset_auth!
self.auth_username = “U”+SecureRandom.hex(7)
self.auth_password = “P”+SecureRandom.hex(13)
EY::ServicesAPI.connection.update_provisioned_service(self.api_url, {
“vars” => { “auth_username” => self.auth_username,
“auth_password” => self.auth_password }})
save!
end
p. We will cause the following request:
bc… PUT http://services.engineyard.com/api/1/service_accounts/104/provisioned_service/109
{
“provisioned_service”: {
“vars”: {
“auth_username”: “U8367e13ec83ac8”,
“auth_password”: “Pe16b696a578e2bfddf1f07e0a2”
}
}
}
h4. Configuration on activation
In the example activation shown, we dint’t require the user to perform any special configuration. However, this is something that the API supports.
The process would be something like this:
h3. Using the provisioned Chronatog service in your app
The Chronatog service has been enabled and provisioned. Values for @service_url@, @auth_username@, and @auth_password@ have been generated and sent back to EngineYard. Now it’s time to make use of the service in your application. Just check-in few changes to your app and deploy!
The public client gem for the Chronatog API is called @chronatog-client@. Add it to your Gemfile like this:
bc… gem ‘chronatog-client’, :require => ‘chronatog/client’
p. To initialize the Chronatog client with the provisioned configs, you’ll also need another gem, @ey_config@:
bc… gem ‘ey_config’
p. With these 2 gems installed, setting up Chronatog can be done like so:
bc… @client = Chronatog::Client.setup!(EY::Config.get(:chronatog, ‘service_url’),
EY::Config.get(:chronatog, ‘auth_username’),
EY::Config.get(:chronatog, ‘auth_password’))
p. @EY::Config@ works by reading the contents of @config/ey_services_config_local.yml@ or @config/ey_services_config_deploy.yml@. In order to develop with a fake version of the Chronatog API locally, you can create @config/ey_services_config_local.yml@ with the contents:
bc… —
chronatog:
service_url: in-memory
auth_username: 123-ignored
auth_password: 456-also-ignored
p. When you deploy, EngineYard will create @config/ey_services_config_deploy.yml@. It might look something like this:
bc… —
chronatog:
service_url: https://chronatog.engineyard.com/chronatogapi/1/jobs
auth_username: U80d6f3acf1a1d5
auth_password: Pce213fec556cac7bee33f028f4
p. The existence of @config/ey_services_config_deploy.yml@ will override all settings in @config/ey_services_config_local.yml@.
h3. Disabling the Chronatog Service
As amazing as our Chronatog web service is, we still need to support user’s deciding they don’t need it anymore and disabling it.
!https://github.com/engineyard/chronatog/raw/master/images/cancel.png!
When the service is disabled, EngineYard will make a DELETE call to the @url@ provided when the service account was first created.
The request will look something like this:
bc… DELETE https://chronatog.engineyard.com/eyintegration/api/1/customers/1
p. Chronatog will handle that request with the implementation defined in the API controller:
bc… customer = Chronatog::Server::Customer.find(customer_id)
customer.bill!
customer.destroy
content_type :json
{}.to_json
p. Notice the call to @customer.bill!@. This causes Chronatog to send a final bill for services before destroying the customer. The request sent looks something like this:
bc… POST http://services.engineyard.com/api/1/partners/80/services/81/service_accounts/85/invoices
{
“invoice”: {
“total_amount_cents”: 27,
“line_item_description”: “For service from 2012/08/21 to 2012/08/22 includes 1 schedulers and 5 jobs run.”
}
}
h3. Billing
Hopefully, normally, customers will use the Chronatog service for a long period of time before canceling. So we need a way to charge them periodically as well.
The mechanism for this is currently a manual process run once a month via script/console.
Simply run:
bc… $ script/console
Chronatog::Server::Customer.all.each(&:bill!)
p. This, of course, will call @bill!@ on all customers, which calculates charges and sends an invoice to the @invoices_url@ for each customer.
The implementation as defined in @Chronatog::EyIntegration::CustomerExtensions@ calculates the total amount owed based on the last time billing was run for each customer (@last_billed_at@ or @created_at@). It then sends an invoice to the @invoices_url@ and sets the @last_billed_at@ to now.
bc… def bill!
#don’t bill free customers
return if plan_type == “freemium”
self.last_billed_at ||= created_at
billing_at = Time.now
#having the awesome service active costs $0.02 per day
total_price = 2 * (billing_at.to_i - last_billed_at.to_i) / 60 / 60 / 24
total_jobs_ran = 0
schedulers.each do |schedule|
#add $0.05 for every time we called a job
usage_price = 5 * schedule.usage_calls
total_jobs_ran += schedule.usage_calls
schedule.usage_calls = 0
schedule.save
total_price += usage_price
end
if total_price > 0
line_item_description = [
“For service from #{last_billed_at.strftime(‘%Y/%m/%d’)}”,
“to #{billing_at.strftime(‘%Y/%m/%d’)}”,
“includes #{schedulers.size} schedulers”,
“and #{total_jobs_ran} jobs run.”,
].join(" ")
invoice = EY::ServicesAPI::Invoice.new(:total_amount_cents => total_price,
:line_item_description => line_item_description)
Chronatog::EyIntegration.connection.send_invoice(self.invoices_url, invoice)
self.last_billed_at = billing_at
save!
end
end
h3. More
TODO: provisioned service SSO.
TODO: using those API keys works. Chronatog automatically updates the status to tell the user they are now using the service. Tell them how many jobs are scheduled.
TODO: using the API to create more than 10 jobs on the free plan and Chronatog sends a notification prompting you to upgrade.
TODO: Examining the monthly billing job Chronatog created in itself and forcing it to run.