Re: Some wishes for s6-log & s6-envdir

From: Laurent Bercot <ska-supervision_at_skarnet.org>
Date: Fri, 27 Feb 2015 14:56:12 +0100

On 26/02/2015 19:37, Olivier Brunel wrote:
> saying I feel this option makes sense on its own, not just to reduce the
> number of executable to call, but because s6-envdir is used to define an
> environment, having a way to guarantee what said environment will be
> seems a good/logical thing.

  Yes, it kinda makes sense. I see how things evolve, and if there are
a few places where I feel it's useful, I'll add the option.


> But you're missing one thing here: I don't want to simply add a prefix,
> I want to do that *after* the whole selection bit happened, and so
> conditionally.
>
> Otherwise I'd then need to maintain the regex to identify the lines in
> both locations: service logger (s6-log) to send the line to stdout/fifo,
> and then in the service reading from that fifo to determine what line
> this is/prefix to use.
>
> I would really like to avoid this. A simple sed one-liner makes sense
> only if I know what the line is. In s6-log I do, because it was selected
> via regex already, but with an external service I don't and need to
> re-implement that whole step again...

  But you are going to perform the selection twice anyway. You're going
to select the line via regex in s6-log, and send it to a fifo with a
prefix, and the service reading from the fifo will select on the prefix.
As long as you're multiplexing data into a single fifo, you cannot
escape parsing the line again.
  Sure, adding a prefix to the line would make your second parsing very
simple, but it would still be a second parsing.


> In s6-log I can select each line via regex and send both to some fifo
> (via 1), but if I didn't add a prefix I don't know which event this is.
> Now I could of course send all lines to that fifo and do the selection
> there, but then we're saying s6-log's selection feature is useless, and
> needs to be reimplemented elsewhere to be used?
>
> The prefix directive was a simple way to benefit from the selection
> feature by giving control over the output (log line, prefixed log line,
> or only the prefix even).
> A more complete solution would be a processor indeed, but, as in Colin's
> example, that has to be an integrated part of/as a processor of s6-log
> to remain a simple one-liner and benefits from the regex selection feature.

  I'm feeling more and more that what you want is a full log processing
flow, with frontends that select lines and deal them to processors that
modify them and pipe them to backends that write them to disk or network.
  And that wouldn't even be completely absurd. That would be powerful and
flexible. And that interests me.

  But s6-log itself was supposed to be mostly a backend, and the "pipe the
line to stdout" thing was added... because you wanted me to. It's on the
verge of an identity crisis; it's getting too close to the "that tool
performs a bit of everything, and isn't very good at it" state that I
absolutely want to avoid. The things s6-log is currently good at is
selecting lines and logging them to autorotated directories: a frontend
job and a backend job. And the backend part is the most important one,
because it's unique and rather tricky; the frontend part is easy to do
and can be put together with some grep and multitee magic. Now, you want
to use s6-log more as a frontend, and between the reasonably good
frontend and the good backend that it already has, you want me to add
mediocre processing. So of course I'm reluctant to do that.

  Rather than piling more stuff into s6-log (whose source is already huge
by my standards), I'd rather break it apart into several executables and
properly design a full log processing chain. Which is doable, but
requires a bit of thought to keep UI complexity under control - and I'm
working on something else atm, that I really would like to finish before
starting another serious project.


> I get that, but one could say that it allows to fully benefit from the
> selection feature of s6-log. One could also point out that even you saw
> an interest in such a prefix, only making it hard-coded and required
> when using 2, and unusable when using 1.

  2 is a special case: if anything gets sent to s6-log's stderr, it is
serious business. It was never meant to be a "normal" log flow. But I
get what you mean.

-- 
  Laurent
Received on Fri Feb 27 2015 - 13:56:12 UTC

This archive was generated by hypermail 2.3.0 : Sun May 09 2021 - 19:44:19 UTC