Re: publish/subscribe data between services

From: Buck Evan <buck_at_yelp.com>
Date: Tue, 8 Mar 2016 09:42:26 -0800

On Tue, Mar 8, 2016 at 1:07 AM, Laurent Bercot <ska-supervision_at_skarnet.org>
wrote:

> On 08/03/2016 02:41, Buck Evan wrote:
>
>> At this point I have a design issue. How do services become aware of each
>> others' port numbers? I've considered using ftrig or envdirs or both to
>> represent and notify this data. Still, I don't see how I can keep the
>> overall state consistent in the case of a single service's restart; I
>> don't
>> believe in general I can assume that its old port number will still be
>> available, which means that I will sometimes need to restart any other
>> services in the "playground" (collection of supervised developer services)
>> that know this port number. This implies dependency resolution, which is
>> too complex for my taste.
>>
>
> With a distributed port number allocation mechanism (i.e. what the kernel
> does) and no dependency resolution, you're going to have race conditions,
> there's no way around it.
>
> The only "correct" way to proceed is to have a centralized authority on
> what port belongs to what service,


Upon re-reading this, I realize that you mean an authority per-machine,
shared across "playgrounds".
I've found a sneaky way around this in my ephemeral-port-reserve script.
It allocates a port 0 with SO_REUSADDR, then forces it into TIME_WAIT.
I then have 60 seconds to bind to that port in my service with no chance
that another service will nab the port (via port 0).
I think this resolves the race you're thinking of?
If so, that's not the problem at hand.

The harder bit (for me) is organizing the communication of this information
to the services that need it.
Even worse, they'll need to be restarted if this information changes,
generally.
Received on Tue Mar 08 2016 - 17:42:26 UTC