feat(proposal): in-memory database without mlmd service

[Al-Pragliola]

Description

Builds on top of #425

In this proposal, I have added a new datastore as an alternative to mlmd for testing/development/static scenarios. This datastore acts as an in-memory database, it supports all actions possible on mlmd and specified in the openapi spec, and a seeding mechanism to have a common baseline in case of process exit/disturbance.

How Has This Been Tested?

docker build . -f Dockerfile -t docker.io/mr/mr:0.1.0

kind create cluster

kind load docker-image docker.io/mr/mr:0.1.0

kubectl create ns kubeflow

kubectl apply -k "./manifests/kustomize/overlays/inmemory"

kubectl patch deployment -n kubeflow model-registry-deployment --patch '{"spec": {"template": {"spec": {"containers": [{"name": "rest-container", "image": "mr/mr:0.1.0", "imagePullPolicy": "IfNotPresent"}]}}}}'

kubectl port-forward --namespace kubeflow svc/model-registry-service 8080:8080 &

curl -s -X 'POST' "localhost:8080/api/model_registry/v1alpha3/registered_models" -H 'accept: application/json' -H 'Content-Type: application/json' -d '{
      "description": "Iris scikit-learn model",
      "name": "iris"
}'

curl -s 'http://localhost:8080/api/model_registry/v1alpha3/registered_models?pageSize=100&orderBy=ID&sortOrder=DESC' -H 'accept: application/json' | jq

expected output:

{
  "items": [
    {
      "description": "loreum ipsum",
      "id": "1",
      "name": "test"
    },
    {
      "description": "loreum ipsum",
      "id": "2",
      "name": "test2"
    },
    {
      "description": "Iris scikit-learn model",
      "id": "3",
      "name": "iris",
      "state": "LIVE"
    }
  ],
  "nextPageToken": "",
  "pageSize": 1,
  "size": 3
}

Merge criteria:

• All the commits have been signed-off (To pass the DCO check)

• The commits have meaningful messages; the author will squash them after approval or in case of manual merges will ask to merge with squash.
• Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
• The developer has manually tested the changes and verified that the changes work.
• Code changes follow the kubeflow contribution guidelines.

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign ckadner for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[rareddy]

@Al-Pragliola can you extract the non-in-memory changes into a separate PR? Then we can do either in-memory or one using the sqllite version on top.

[Al-Pragliola]

@Al-Pragliola can you extract the non-in-memory changes into a separate PR? Then we can do either in-memory or one using the sqllite version on top.

@rareddy the abstraction is already there #425 , it's planned to be merged after kubeflow 1.10 release https://github.com/kubeflow/model-registry/milestone/2

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.