#42 Connect to UMB and parse brew.sign.rpm
Merged by jkaluza. Opened by cqi.
cqi/freshmaker connect-to-umb  into  master

Download 42.patch

This patch introduces an environment variable to switch freshmaker to an
internal mode to parse and handle messages from UMB specifically. So, in
configuration, all parsers and handlers for handling messages from
Fedora fedmsg are disabled when deploy internally and enable the mode.

  • FRESHMAKER_INSIDE_RH, set it to enable internal mode whatever its
    value is.

Configure fedmsg-hub to use STOMP to connect to UMB and receive
messages. Broker URLs, certificate and private key files can be
configured via several new FRESHMAKER_STOMP_* environment variables.

  • FRESHMAKER_STOMP_URI, URIs of brokers separated by comma.
  • FRESHMAKER_STOMP_SSL_CRT, path to certificate file.
  • FRESHMAKER_STOMP_SSL_KEY, path to private key file.
  • FRESHMAKER_MESSAGING_TOPIC_PREFIX, part of queue name from the
    beginning to word VirtualTopic, e.g.
    /queue/Consumer.msg-client-me.freshmaker_queue.VirtualTopic

Major changes:

  • Make event parsers configurable along with handlers in config
  • Reuse load_classes to load event parsers and handlers from config
  • Modify several config to add internal configuration for event parsers
    and handlers
  • BrewRPMSignEvent is renamed to BrewSignRPMEvent, which is natrual
    to follow the topic name schema brew.sign.rpm
  • Add event parse for BrewSignRPMEvent

Signed-off-by: Chenxiong Qi cqi@redhat.com

rebased

rebased

rebased

+1 for this!

rebased

Add test.

Ready for review.

+1 here, I will wait for the Monday if qwan wants to review that. Feel free to merge on Monday :)

This looks not so reasonable, can we just use something like ZMQ_ENABLED and STOMP_ENABLED to enable/disable connecting to zmq and stomp endpoints?

I think it's unnecessary to change the parsers & handlers in this way, when you need to disable//enable some of the parsers/handlers, you can just edit the config options directly.

I'd prefer to just use PARSERS.

Same as above, we can edit this option directly rather than by checking a variable.

something like:
if ZMQ_ENABLED and STOMP_ENABLED:
# raise error if we can't support enabling ZMQ and STOMP in the same time.
if ZMQ_ENABLED:
# ZMQ specific options
if STOMP_ENABLED:
# STOMP specific options.

I think we need to support specifying the options in config rather than only support getting them from os.environ.

I guess you mean freshmaker's config. How to read freshmaker's config from this file?

Reading these stomp configuration from a config file could be good, but, how much extra benefit to freshmaker in practice?

This file should be read by fedmsg (or probably the fedmsg-hub, I'm not sure) to configure itself. Using environment variables is a convenient way during development normally. It could also be used to configure freshmaker when deploy and run in a server, but in practice, these stomp_* options should be replaced with real value by some deployment scripts, e.g. ansible playbook.

Whatever the way to read options from freshmaker config file, this file is a fedmsg-hub configuration file, I think it is not a good idea to introduce a dependency between fedmsg-hub and freshmaker in the aspect of configuration.

This fedmsg.d/freshmaker.py is just the config I mean, which we'll put it under /etc/fedmsg.d while deploying, yes it's easy to have the variables configured with ansible scripts, but it's not a good idea to assume we will always use ansible for deploying. It's good to have it be configurable via environment variables, but not good to unable to configure in this config file.

Sorry, the comments are for the change in fedmsg.d/freshmaker.py. In my opinion, RH_MODE and FRESHMAKER_INSIDE_RH can just be removed. it makes sense to just update MESSAGING_TOPIC_PREFIX, HANDLERS and PARSERS directly in this config when necessary.

We would then need to have two config.py files in the repository, one for RH and one for Fedora. I don't want to configure all the fields in config.py to have Freshmaker working on Fedora/RH when I need to test something :).

Separate config file for RH is my original thought. However, a bug in init_config need to fix in advance, that is freshmaker does not load and read config from FRESHMAKER_CONFIG_FILE when specify FRESHMAKER_DEVELOPER_ENV.

rebased

2 new commits added

  • Split config files
  • Connect to UMB and parse brew.sign.rpm

2 new commits added

  • Split config files
  • Connect to UMB and parse brew.sign.rpm

2 new commits added

  • Split config files
  • Connect to UMB and parse brew.sign.rpm

What's your opinion on renaming this to just PARSERS?

Compare to config.py, the difference is configuring message topic and disable/enable some handlers/parsers, I personally think it's easy to just update conf/config.py when you need to deploy Freshmaker in different env, I'm wondering does it worth to add another config file for this case.

And I'd suggest squash the 2 commits since the second one is just reverting the changes in the first one with minor updates.

rebased

Commits are squashed. EVENT_PARSERS is renamed to PARSERS.

What's your opinion on renaming this to just PARSERS?

It's ok. Have renamed it.

+1, @qwan, can you also check it?

Pull-Request has been merged by jkaluza

Metadata