HTTPS SSH

README

Project created on event on Sunday, October 23rd, 2016. Source code hosted on https://bitbucket.org/raphox/webhook.

Introduction video https://www.youtube.com/watch?v=UD1OJggvpVI.

The representation of RESTful API by Swagger can be accessed at https://vh-webhook.herokuapp.com/v1/swagger.json . You can download https://github.com/swagger-api/swagger-ui/releases (open the ./dist/index.html) or use http://petstore.swagger.io/ to read our swagger.json.

Description of challenge

As more and more integration takes place between SaaS providers, other SaaS providers and their customers, webhooks have become an invaluable way of sharing events. These events simplify data-synchronization and extensibility.

Project:

HootSuite requirements

Write a webhook calling service that will reliably POST data to destination URLs in the order POST message requests are received.

The service should support the following remote requests via REST

  • register a new destination (URL) returning its id
  • list registered destinations [{id, URL},...]
  • delete a destination by id
  • POST a message to this destination (id, msg-body, content-type): this causes the server to POST the given msg-body to the URL associated with that id.
Behaviour:
  1. If the destination URL is not responding (e.g. the servier is down) or returns a non-200 response, your service should resend the message at a later time
  2. Messages not sent within 24 hours can be be deleted
  3. Messages that failed to send should retried 3 or more times before they are deleted
  4. Message ordering to a destination should be preserved, even when there are pending message retries for that destination

Feel free to add more metadata to the destination (id, URL,) if it helps your implementation

To Consider:
  • is your API using the standard REST-ful conventions for the 4 operations?
  • how can I scale out this service across multiple servers while preserving per-destination ordering?
  • how well does your service support concurrency for multiple destinations while preserving per-destination ordering?
  • how secure is this? should you require HTTPS urls? should the content be signed with something like an HMAC? Should any url be allowed (e.g. one that has or resolves to a private IP address?)

Developpers considerations

We have a project developed with Rails 5 API and all services get and send data following the http://jsonapi.org/ specifications.

Behaviour:
  1. The retry process to send data it is scheduled only when it receives one of these errors: * EOFError * Errno::ECONNRESET * Errno::EINVAL * Net::HTTPBadResponse * Net::HTTPHeaderSyntaxError * Net::ProtocolError * Timeout::Error
  2. Process without success until 24 hours will be remove by ClearOldProcessRequestWorker scheduled with gem sidekiq-cron. This process would not be necessary if we had the PRO version of Sidekiq https://github.com/mperham/sidekiq/wiki/Expiring-Jobs
  3. We are using the sidekiq_options to set retry of schedule.
  4. We have a conditional to prevent this https://gitlab.com/vanhackathon/webhook/blob/master/app/workers/process_request_worker.rb#L21
To Consider:

Developers

Technologies

  • Ruby v2.3.0
  • Rails v5
  • MongoDB
  • Sidekiq + Redis
  • Rspec
  • Swagger