Skip to content


Administering omnibenchmark


Omnibenchmark is designed as a SaaS. Omnibenchmark's components are modular, and some you can deploy in your own to have extra control. already provides all these components, so this guide only matters if you'd like to replace some of the default components.

  • renkulab and gitlab deployment
  • gitlab runners
    • default: provided by
    • self: registered runners in any architecture (laptops, HPC, GPU-powered machines etc).
  • omnibenchmark triplestore
    • default: apache jena from robinsonlab (ask us for details).
    • self: deploy a jena/fuseki to have full control on your triples.
  • centralized benchmark listing/json.
    • default: robinsonlab (ask us for details, queried by omb-py).
  • bettr deployment
    • default: shiny-server by robinsonlab (ask us for details).
    • self: set up a shiny-server (perhaps using singularity).
  • Web dashboard

Besides, omnibenchmark relies on a set of python modules and R packages, which also need to be cross-compatible and compatible with your renkulab deployment, including:

Configuration guides

Mind some of our docs are missing; drafts are listed as ✓, ready-to-use docs as ✅.

  1. Services overview
  2. Start a new benchmark
  3. Set up a triplestore
  4. Register a runner
  5. Serve bettr
  6. Serve a dashboard
  7. Deploy renkulab


To get tokens/authentication details:,