Index

Administering omnibenchmark

Components

Omnibenchmark is designed as a SaaS. Omnibenchmark's components are modular, and some you can deploy in your own to have extra control. omnibenchmark.org already provides all these components, so this guide only matters if you'd like to replace some of the default components.

• renkulab and gitlab deployment
  • default: renkulab and gitlab deployment from renkulab.io.
  • self: k8s and following renku's admin docs.
• gitlab runners
  • default: provided by renkulab.io.
  • self: registered runners in any architecture (laptops, HPC, GPU-powered machines etc).
• omnibenchmark triplestore
  • default: apache jena from robinsonlab (ask us for details).
  • self: deploy a jena/fuseki to have full control on your triples.
• centralized benchmark listing/json.
  • default: robinsonlab (ask us for details, queried by omb-py).
• bettr deployment
  • default: shiny-server by robinsonlab (ask us for details).
  • self: set up a shiny-server (perhaps using singularity).
• Web dashboard
  • default: omnibenchmark.org from robinsonlab (ask us for details, code).

Besides, omnibenchmark relies on a set of python modules and R packages, which also need to be cross-compatible and compatible with your renkulab deployment, including:

• omnibenchmark python
  • tip: needs to be compatible with renku python
• omni validator
• omni_cli
• bettr

Configuration guides

Mind some of our docs are missing; drafts are listed as ✓, ready-to-use docs as ✅.

1. Services overview ✓
2. Start a new benchmark ✓
3. Set up a triplestore
4. Register a runner ✅
5. Serve bettr ✅
6. Serve a dashboard
7. Deploy renkulab

Contact

To get tokens/authentication details: mark.robinson@uzh.ch, izaskun.mallona@uzh.ch