Clients are typically very simple and provide a few simple functions for larger applications to communicate with the server.
Clients are built around a few, simple principles:
The best way to get started implementing a client is to get familiar with an existing client. Either way, we’re going to run through everything that a client needs to implement to be up-to-par with all the other clients.
A client must handle the following events from the server:
| Event | Description |
|---|---|
| /qio/heartbeat | Fired from the server every 60 seconds for some protocols (WebSocket, Raw). Typically, the client should just ignore the message, reset its last receive time, and move on. If the server requests a callback, however, the client MUST fire it as soon as possible. |
| /qio/callback/{id} | Fired when the server is sending a callback to the client; the id parameter is the callback id. This should call the registered callback function registered at this id. |
| /qio/move | Fired when the server is forcing the client to balance to another QuickIO server. The data in the event is the server to move to. This should be as transparent as a reconnect. |
The following events are special events that are fired to alert the client of changes in connection state. These events MUST NEVER be sent to the server in a /qio/on event.
| Event | Description |
|---|---|
| /open | Fire when a new connection to the server is established and the handshake has been completed. |
| /close | Fire whenever a connection to the server is lost or closed (even by a client), but only if /open has been fired. |
| /error | Fire when there is an error connecting to the server, when the connection is lost, or any other error you see fit. The client will only treat this as informational and use it for logging and debugging. Any data included in this event is platform-dependent, but do you best to be thorough. |
A client should implement the following functions:
| Function | Description |
|---|---|
| (constructor) | Creates an object (or control structure) for handling all QuickIO requests. |
| on | Subscribe to an event on the server. This MUST update all internal structures to reflect that the client is now subscribed to the event and will be until off is called. |
| one | Subscribe to the event, receive one event, and unsubscribe |
| off(null) | Unsubscribe from an event on the server, removing all callbacks. |
| off(cb) | Remove a single callback from a subscription, calling unsubscribe on the server if there are no more callbacks. |
| send | Send an event to the server. |
| close | Immediately close the connection to the server. This may be called in an /open callback, in a /close callback, or at any time by the client. |
| reconnect | For clients that exist in multi-threaded environments, this is used to trigger a connection after /open and /close have been subscribed to. It also drops any active connection and reconnects to the cluster immediately. |
While connected to a server, the client MUST send events as quickly as possible to the server. It should perform the following tasks:
When not connected to the server, the client MUST queue up events until it reconnects. The following rules apply to queued events:
Each QuickIO cluster lives behind a single public address, typically a single DNS A record that points to all the servers. Each server, in turn, has a public address that can be requested at the event path /qio/hostname. To connect to a cluster, a client should pick one of the A records at random and try connecting; on failure, it may issue another DNS request for updated hosts or try the next one in its list. Each language has its own ways of doing this, and it’s usually best to let the socket library try to figure everything out. If you are, on the other hand, using bare sockets in C or the standard library in Java, you’re going to have to do this part yourself. Other languages seem to handle this for you.
Since many clients in the wild are behind proxies and “smart” HTTP firewalls that don’t yet support WebSocket, it’s necessary to support HTTP long polling in each client. The client must first attempt to establish a WebSocket connection with the cluster, and if that fails for any reason besides an unreachable network or similar condition, it must immediately fall back to HTTP long polling. If it can establish a connection with the server over HTTP, it must continue using HTTP long polling until either the client or server terminates the connection. Once terminated, if the client is going to attempt to reconnect, it must first attempt WebSocket again, just in case the client changes networks, and the new network supports WebSocket.
QuickIO speaks WebSocket as described in RFC6455. Your client MUST implement the handshake and framing parts of the spec in order to communicate properly with a QuickIO server. QuickIO does NOT implement binary, continuation, ping, or pong frames, so feel free to ignore those (though bear in mind that ping and pong might be implemented in the future if web browsers ever build a standard API for using them).
After going through the standard RFC6455 handshake and upgrade, it is necessary to send the QuickIO handshake as many proxies let all the proper headers through, so it appears that the upgrade succeeds. Without this handshake, it’s impossible to determine if the client can really speak WebSocket. The handshake is very simple: using all RFC6455 framing conventions, simply send the following message:
/qio/ohai
The server will immediately respond with “/qio/ohai”, too, and at this point, all handshakes have finished, and the connection is considered opened. At this point, the client MUST fire an /open event.
If any part of the WebSocket handshake fails, aside from network issues, as mentioned, the client MUST try connection via HTTP. Since many HTTP clients and proxies will attempt to balance HTTP requests over many sockets, it’s possible that they will attempt to send long polling requests to different QuickIO servers in the cluster as they are round-robined behind a single DNS record. In order to ensure that the client is speaking to the same server throughout the lifetime of its HTTP session, it must conduct the opening handshake as follows:
If the server does not respond with a 200, the connection must be failed immediately, and the client must try reconnecting again.
If the server responded with a 200, the body of the response will contain the server’s public address, formatted as an event response. All further communication with the server must use this address.
After receiving any response from the server, including after the handshake, the client must schedule another POST request to run after 0-2000 milliseconds, chosen at random (typically Math.random() * 2000). This is necessary to make sure that there isn’t a stampeding herd attacking the server after HTTP heartbeats.
A client may, at any time, issue a new POST request with newline-separated events, provided that the request is only sent after 0-2000 milliseconds, in the same way that requests are queued up after a poll finishes. Bodies of requests shall contain numerous newline-separated events, and they must be gathered into as few requests as possible (sending 2 requests at the same time is prohibited, the events must be gathered into a single request). Sending a request 0-2000 milliseconds after a previous request is acceptable as there is no way to preempt when events will be fired. The server will respond by completing any other requests from the client and holding onto the newest request until data is ready (the new request will become the long-polling request).
If the client, at any point, sees a non 200 response, it must fail the connection immediately and fire any /close event, as appropriate.
Since this can be a bit complicated, let’s look at a sample HTTP conversation between the client and the server, unnecessary headers omitted:
POST /?sid=16a0dd9a4e554a9f94520c8bfa59e1b9&connect=true HTTP/1.1
Host: quickio.example.com
Content-Type: text/plain
Content-Length: 20
/qio/hostname:1=null
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 62
/qio/callback/1:0={"code":200,"data":"quickio129.example.com"}
POST /?sid=16a0dd9a4e554a9f94520c8bfa59e1b9 HTTP/1.1
Host: quickio129.example.com
Content-Type: text/plain
Content-Length: 0
POST /?sid=16a0dd9a4e554a9f94520c8bfa59e1b9 HTTP/1.1
Host: quickio129.example.com
Content-Type: text/plain
Content-Length: 16
/qio/ping:1=null
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 42
/qio/callback/1:0={"code":200,"data":null}
At this point, there is 1 HTTP request pending at the server, and that will be used to send any new events back to the client. Once this request finishes, the client must send a new request after (Math.random() * 2000) milliseconds.
The client MUST do its best to maintain a connection to the QuickIO cluster until it is told to stop. Between connection failures, it must backoff using the following algorithm, such that backoff increases between successive failures.
onDisconnect(function() {
backoff = Math.min(25600, backoff * 2);
reconnectAfter(backoff);
});
onConnectAndSuccessfulHandshake(function() {
backoff = 100;
});
Once a connection with the server has been re-established, and once all handshakes have been completed, the client must do the following, in this order:
During the lifetime of the client, it will receive a ton of events from the server. Handling them is really rather simple.
Since callbacks are very tightly linked to the server and session they have on the server, they must be explicitly tied to a given connection, typically by giving the connection an ID and associating the callback with that ID. If at any point the application attempts to trigger a callback that is not tied to the current connection, the client must respond to the callback immediately with -1 “disconnected”.
Your average user will most likely be someone connecting to your service from behind some NAT gateway: this presents some interesting problems. In order to make sure the client maintains a connection to the server at all times, even when there is no activity, application-level heartbeats are employed. Each protocol has a different way of handling heartbeats.
By default, a client will receive at least one message every 60 seconds, be it in the form of a callback, broadcast event, or heartbeat.
Heartbeats are implemented such that, if a client hasn’t been sent a message in around 60 seconds (this is variable to within -10 seconds, but a client will never go more than 60 seconds without an event from the server), it will receive a heartbeat.
The best method for implementing a heartbeat is:
HTTP requests will be responded to once every 50 seconds in order to ensure that nasty proxies don’t just time them out. The client must, however, set a timeout of 60 seconds on each request, just to give any response time to traverse the network, and some time to be slow. If the request ever times out, the client must assume the connection has been lost and reconnect.