New issue
Advanced search Search tips

Issue 804962 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Feature



Sign in to add a comment

Merge backend microservices into one backend service

Project Member Reported by mar...@chromium.org, Jan 23 2018

Issue description

Tricium currently has modules: default, config, driver, gerrit, launcher, static, tracker.

This should be refactored to: default, backend.

This will make switching version more atomic and just simpler to monitor.
 
Currently:
 - the code is organized into directories that correspond to the modules
 - the logging statements in the code also have prefixes like [launcher] etc.
 - the docs talk about the service in terms of these modules, e.g. https://docs.google.com/document/d/15GBhl_rt7JIAa1EvnZTzX5w0BSVjUlLA4bjLyrup6yo/edit#heading=h.4fm7lae0e1zf

We could keep the code organization the same, and probably keep the logging and docs the same while changing the module organization; does that seem like a good idea, or would the docs and logging prefixes start to be confusing if we merged all the backend modules?
Summary: Tricium: merge backend microservices into one backend service (was: Tricium: merge backend modules into one)
Note, the App Engine documentation now refers to modules as "services", and an app composed of several small services uses "microservice architecture".

More thoughts here:

 The advantages of merging backend modules noted above is that switching versions may be simpler and more atomic, and monitoring may be made easier.

But I'm not sure it's so clear --

 1. With the gae.py script, deploying/switching versions is still just one action.
 2. I'm not convinced that monitoring is really different; I think that viewing logs for multiple microservices in Cloud Console is not any harder than viewing logs for one?
 3. Keeping the current structure -- with a directory and init.go and app.yaml for each microservice makes it so that each service's handlers are isolated to within the service's package -- e.g. handlers for each service don't need to be accessible beyond that package. AFAICT, consolidating into into "service" means having a backend package with one init function that must have access to all of the handlers for config, driver, gerrit, launcher and tracker.

More notes:
https://cloud.google.com/appengine/docs/standard/python/microservices-on-app-engine

Blog article about disadvantages of microservices:
http://www.ptone.com/dablog/2015/07/microservices-may-be-the-new-premature-optimization/.

Comment 3 by mar...@chromium.org, Feb 26 2018

- It slows down deployment, as this is N more copies of the code to upload and that's a multiple seconds operation.
- It increases the number of idle services, as cron jobs happen on a 1 minute basis, and each service has to stay alive for it. Not a big deal but still a small downpoint.
- From a clean up perspective, it's much more versions to cleanup.
- I don't really buy into the startup performance point, I would expect all of init() function to complete into the millisecond range. If not, that's actually a saving as this one-time initialization is reused among a smaller set of backend instances.
- Most importantly, it's just simpler to understand: front end, back end.
Personally, I think the frontend/backend organization sounds good, as long as it's clear that "launcher", "driver", "tracker", etc. are not separate modules/services, but just separate collections of handlers. (Kind of like how the Gerrit REST API is organized into groups of handlers like /changes/, /config/, /accounts/, etc.)

What do you think about the requirement that we would add a "backend" package with an init function that lists all of the backend handlers (which would have to be public assuming they're in different Go packages)?

Comment 5 by mar...@chromium.org, Mar 13 2018

lgtm
Labels: Hotlist-CodeHealth
Summary: Merge backend microservices into one backend service (was: Tricium: merge backend microservices into one backend service)
Components: Infra>Platform>Tricium
Components: -Infra>CodeAnalysis
Labels: -Tricium

Sign in to add a comment