Re: publish/subscribe data between services

From: Buck Evan <buck_at_yelp.com>
Date: Tue, 8 Mar 2016 09:56:00 -0800

On Mon, Mar 7, 2016 at 6:41 PM, Patrick Mahoney <pat_at_polycrystal.org> wrote:

> On 2016-03-07 7:41 pm, Buck Evan wrote:
>
> How do services become aware of each others' port numbers?
>>
>> ...
>>
>> I've also considered plopping data into an envdir
>>
>> ...
>>
>> The system I'm replacing solves this by assigning a particular localhost
>> IP
>> to the whole playground, then we may hard-code the port numbers throughout
>> the code base. There's complexity in maintaining the state of assigned
>> localhost IPs that I'd like to factor out.
>>
>
> Not sure if this would be simpler than IP-per-playground, but you
> could look into Linux's network namespaces [1] to assign
> netns-per-playground. Within the netns, you can use statically known
> ports, and you can use `ip netns exec $namespace command...` to run
> arbitrary commands within the namespace. There's a fair amount of
> config to properly create the loopback address within each netns, and
> possibly allow routing through the primary netns (the one attached to
> the LAN) if that's needed.
>
> [1] http://manpages.ubuntu.com/manpages/xenial/en/man8/ip-netns.8.html


Good to know, but not sure if it simplifies, as you said.


> How common is service restart expected to be? What about a two-layered
> supervision tree?

Toplevel has one supervisor-per-playground. Second
> level supervises the services in each single playground. These
> services share a common ./finish script, which kills the whole second
> level supervision tree, resulting in the entire playground exiting
> when any single service exits. Then the toplevel supervisor restarts
> the playground, and you have well-defined places to remove and
> re-create the envdir.


This might help!

There are some lengthy startup routines that I'd like to keep per-service,
but as long as they don't generate shared configuration, they can be kept
at the lower-level supervision tier.

It's not ideal that any and all shared configuration needs to be generated
eagerly before anything starts, but it's better than what I have.

Looking at this, I'm not sure that finish needs to stop the whole
playground, unless you meant to keep the port-0 idea.
If all configuration is generated up-front, a restart doesn't entail any
change in configuration necessarily.
I suppose that disregards the case that another unrelated service has
hopped onto that port though :(
Received on Tue Mar 08 2016 - 17:56:00 UTC

This archive was generated by hypermail 2.3.0 : Sun May 09 2021 - 19:44:19 UTC