Metamist

Introduction

Metamist is a database designed for storing de-identified -omics metadata.

Purpose

The project provides an interface to interact with the -omics database via the Python client as well as the GraphQL + HTTP APIs.

Features

• Project-Based Resource Organization: Every resource in Metamist is associated with a specific project.
• Access Control: Access to resources is controlled through a basic permissions system. Users can have one of the following permissions for a project.
  • reader permission allows viewing of all metadata within a project
  • contributor permission allows creating comments on metadata within a project
  • writer permission allows writing of all metadata within a project.

High-Level Architecture

It comprises three key components:

1. System-Versioned MariaDB Database: A robust database system for managing -omics metadata.

2. Python Web API: This component is responsible for:

   • Managing permissions.
   • Storing frequently used queries.
   • Providing a GraphQL/HTTP API for efficient querying of the database.

3. Installable Python Library: Wraps the Python Web API using the OpenAPI generator, facilitating easier interaction with the system.

4. Web client UI A react application that allows viewing the metadata for projects.

Usage

There are a few different ways that you may need to interact with metamist.

Working with production metamist data

If you only need to interact with production metamist data, then you do not need to clone this repository or have a working local installation of metamist. You can install the python client from pypi:

pip install metamist

And use it in your python scripts:

from metamist.graphql import gql, query

def main():
    my_query = gql("""
        {
            myProjects {
                id
                name
            }
        }
    """)

    result = query(my_query)
    print(result)


if __name__ == "__main__":
    main()

Further resources

• GraphiQL UI, for exploration of the GraphQL API
• Python client documentation
• Swagger HTTP API documentation

Testing scripts against a local database

When writing more complex scripts it is a good idea to test them against a local version of metamist. To do this, follow the local installation instructions to set up the API. Setting up the web client is optional and only necessary if you need to view your data in the UI.

Once you have the API set up locally, set the SM_ENVIRONMENT variable to local

export SM_ENVIRONMENT=local

This will tell the metamist python client to point to your local metamist install.

Developing or contributing to metamist

If you are working on developing features, fixing bugs or want to contribute to metamist in other ways then you will need a local installation of metamist. Follow the local installation instructions.

Schema

As of Jan 15, 2024 this schema should reflect the data structure on the tables:

You can also find this at DbDiagram.

The codebase contains the following modules worth noting:

• models -> General data models + enums
• db/python/tables -> Interaction with MariaDB / BigQuery
• db/python/layers -> Logic
• api/graphql : GraphQL API schema and models
• api/routes: HTTP + OpenAPI

And metamist maintains two clients:

• web -> React app that consumes a generated Typescript API + GraphQL
• metamist -> autogenerated Python API

License

This project is licensed under the MIT License. You can see it in the LICENSE file in the root directory of this source tree.