Process management for Galaxy servers.

• License: MIT
• Documentation:
• Code:

Overview

Modern Galaxy servers run multiple disparate processes: gunicorn for serving the web application, celery for asynchronous tasks, tusd for fault-tolerant uploads, standalone Galaxy processes for job handling, and more. Gravity is Galaxy's process manager, to make configuring and running these services simple.

Installing Gravity will give you two executables, galaxyctl which is used to manage the starting, stopping, and logging of Galaxy's various processes, and galaxy, which can be used to run a Galaxy server in the foreground.

Quick Start

Installation

Python 3.7 or later is required. Gravity can be installed independently of Galaxy, but it is also a dependency of Galaxy since Galaxy 22.01. If you've installed Galaxy, then Gravity is already installed in Galaxy's virtualenv.

To install independently:

$ pip install gravity

Usage

From the root directory of a source checkout of Galaxy, after first run (or running Galaxy's ./scripts/common_startup.sh), activate Galaxy's virtualenv, which will put Gravity's galaxyctl and galaxy commands on your $PATH:

$ . ./.venv/bin/activate
$ galaxyctl --help
Usage: galaxyctl [OPTIONS] COMMAND [ARGS]...

  Manage Galaxy server configurations and processes.

... additional help output

You can start and run Galaxy in the foreground using the galaxy command:

$ galaxy
Registered galaxy config: /srv/galaxy/config/galaxy.yml
Creating or updating service gunicorn
Creating or updating service celery
Creating or updating service celery-beat
celery: added process group
2022-01-20 14:44:24,619 INFO spawned: 'celery' with pid 291651
celery-beat: added process group
2022-01-20 14:44:24,620 INFO spawned: 'celery-beat' with pid 291652
gunicorn: added process group
2022-01-20 14:44:24,622 INFO spawned: 'gunicorn' with pid 291653
celery                           STARTING
celery-beat                      STARTING
gunicorn                         STARTING
==> /srv/galaxy/var/gravity/log/gunicorn.log <==
...log output follows...

Galaxy will continue to run and output logs to stdout until terminated with CTRL+C.

More detailed configuration and usage examples, especially those concerning production Galaxy servers, can be found in the full documentation.

History

1.0.6

• Support pydantic v1 and v2 by @mvdbeek in

• Fix GalaxyReportsService crash and remove config_type by @jvanbraekel in

• Support running multiple tusds and controlling the value of hooks-http by @natefoo in

1.0.5

• Support pydantic v1 and v2 by @heisner-tillman in

1.0.4

• Supervisor: more readable program name + reread config on each change by @abretaud in

• Remove unneeded supervisorctl update call by @abretaud in

• Minimal path change needed for galaxy PR 16795 by @sveinugu in

1.0.3

• Don't create supervisor conf dir unless necessary, create the gravity data dir as the correct user by @natefoo in

1.0.2

• Pin a minimum package version of gx-it-proxy by @natefoo in

1.0.1

• Added configuration of gx-it-proxy to support path-based proxying by @sveinugu in

1.0.0

Version 1.0.0 represents a significant update to Gravity, its features and functionality. Although Gravity 1.x is intended to be backwards compatible with 0.x, you are strongly encouraged to [read the documentation]() if upgrading to Gravity 1.x or to Galaxy 23.0 (which depends on Gravity 1.x) in order to get the most out of the new features.

• Support systemd as a process manager by @natefoo in

• Full stateless mode when working with single instances and other improvements for 1.0 by @natefoo in

• Multi-unicorn rolling restart and general multi-instance service support by @natefoo in

• Don't clobber other Galaxies' systemd units when managed by different Gravity config files by @natefoo in

• Don't restart tusd on graceful by @natefoo in

• Read job_conf.yml by default if job_config_file is unset by @natefoo in

• Fixes for spaces in the galaxy root path, fix the galaxy entrypoint by @natefoo in

• Update existing env with program env when running exec, rather than the other way around by @natefoo in

• Hide the "exec" ServiceCommandStyle from documentation since it is only used internally by @natefoo in

• Updates for settings documentation generation by @natefoo in

• Set $VIRTUAL_ENV if virtualenv is set in config by @natefoo in

• Always add venv bin dir to $PATH if virtualenv is set by @natefoo in

0.13.6

• Fix graceful method for gunicorn --preload by @Slugger70 in

• Add --version option to get Gravity version by @natefoo in

• Fix stopping of gx-it-proxy by @abretaud in

0.13.5

• If virtualenv is set in the Gravity config, automatically add its bin dir to $PATH if the gx-it-proxy is enabled by @natefoo in

• Support converting settings to command line arguments in a generalized way by @natefoo in

0.13.4

• Fixes for startup test by @natefoo in

• Fix setting environment vars on handlers by @natefoo in

0.13.3

• Don't use gunicorn logging options with unicornherder by @natefoo in

0.13.2

• Don't override PATH in subprocess call by @jdavcs in

• Only send pre create hook by @mvdbeek in

0.13.1

• Set correct default for environment settings by @natefoo in

• Don't catch exceptions in the deregister, register, and rename subcommands by @natefoo in

• processes in the handling dict in the job conf dict is a dict, not a list by @natefoo in

0.13.0

• Add options to enable/disable gunicorn, celery, and celery-beat services by @natefoo in

• Add ability to include gravity config from a separate file and document by @natefoo in

• Only default to preload = true for gunicorn if not using unicornherder by @natefoo in

• Add option to specify tusd path by @natefoo in

• Support setting per-service environment variables by @natefoo in

0.12.0

• Fix typo in log_dir description by @nsoranzo in

• Shortcut individual services fix by @natefoo in

• Add additional options to celery beat / celery workers by @mvdbeek in

0.11.0

• Allow setting supervisor socket path via environment variable by @mvdbeek in

• Automatically switch to non-sample galaxy.yml if it exists by @mvdbeek in

• Add pydantic config schema by @mvdbeek in

• Add --quiet option to galaxy and galaxyctl start by @mvdbeek in

• Add support for yaml job config by @mvdbeek in

• Add --preload support for gunicorn by @mvdbeek in

• Support running tusd by @natefoo in

0.10.0

• Fix for the case where a job_conf.xml exists but no handlers are defined by @natefoo in

• Do not raise error if config file section is empty by @nsoranzo in

• Add tests for static handlers and a defined job_conf.xml with no handlers by @natefoo in

• Fix minor typos in readme by @ic4f in

• Move configuration to gravity key of galaxy.yml file by @mvdbeek in

• Fix for resolved galaxy.yml.sample symlink by @mvdbeek in

• Support managing gx-it-proxy via gravity by @mvdbeek in

0.9

• Gunicorn/fastAPI support, click support, tests by @mvdbeek in

• Don't test on Python 3.6, which is unsupported by @natefoo in

• Update README. Also some various small bugfixes and fixes for other stuff mentioned in the README by @natefoo in

• Add unicornherder support by @natefoo in

• Expose the log following used by start -f as its own subcommand. by @natefoo in

• Better integration with Galaxy's run.sh by @natefoo in

• Use relative paths in supervisord by @natefoo in

• Converted CLI from argparse to click.

• Stole ideas and code from planemo in general.

• Improve the AttributeDict so that it can have "hidden" items (anything that starts with a _) that won't be serialized. Also, it serializes itself and can be created via deserialization from a classmethod. This simplifies using it to persist state data in the new GravityState subclass.

0.8.3

• Merge galaxycfg and galaxyadm commands to galaxy.

0.8.2

• Allow for passing names of individual services directly to supervisorctl via the start, stop, and restart methods.
• Fix a bug where uWSGI would not start when using the automatic virtualenv install method.

0.8.1

• Version bump because I deleted the 0.8 files from PyPI, and despite the fact that it lets you delete them, it doesn't let you upload once they have been uploaded once...

0.8

• Add auto-register to galaxy start if it's called from the root (or subdirectory) of a Galaxy root directory.
• Make galaxycfg remove accept instance names as params in addition to config file paths.
• Use the same hash generated for an instance name as the hash for a generated virtualenv name, so virtualenvs are more easily identified as belonging to a config.
• Renamed from galaxyadmin to gravity (thanks John Chilton).

0.7

• Added the galaxyadm subcommand graceful on a suggestion from Nicola Soranzo.
• Install uWSGI into the config's virtualenv if requested.
• Removed any dependence on Galaxy and eggs.
• Moved project to its own repository from the Galaxy clone I'd been working from.

Older

• Works in progress as part of the Galaxy code.