Merge pull request #63 from kube-HPC/pipeline_versioning

Adir111 · web-flow · commit d1f9d957c66e · 2025-03-11T09:54:46.000+02:00
Pipeline versioning
diff --git a/package.json b/package.json
@@ -4,8 +4,8 @@
   "private": true,
   "version": "2.8.12",
   "scripts": {
-    "start": "rm -rf build && babel-node resources/server.js",
-    "build": "rm -rf build && babel-node resources/build.js",
+    "start": ". $HOME/.nvm/nvm.sh && nvm use 14.21.3 && npm run start:internal",
+    "start:internal": "rm -rf build && babel-node resources/server.js",    "build": "rm -rf build && babel-node resources/build.js",
     "watch": "rm -rf build && babel-node resources/watch.js",
     "docker": "rm -rf build && ./dockerfile/build.sh site",
     "test": "npm run build"
diff --git a/site/learn/API.md b/site/learn/API.md
@@ -8,7 +8,7 @@ sublinks: Restful, Cli
 next: /learn/algorithms/
 ---
 
-Hkube support two types of APIs, Restful and Cli
+HKube support two types of APIs, Restful and Cli
 
 ### Restful API
 
diff --git a/site/learn/Algorithms.md b/site/learn/Algorithms.md
@@ -8,20 +8,20 @@ permalink: /learn/algorithms/
 next: /learn/codeapi/
 ---
 
-There are two ways to integrate your algorithm into Hkube:  
+There are two ways to integrate your algorithm into HKube:  
 1) No code involves, no WebSocket and no Docker.  
 2) Involves code writing, work with WebSocket and Docker.
 
 ## How does it works?
 
-Integrating algorithms into Hkube require 3 steps:  
-1) Implement connectivity with Hkube using WebSocket.  
+Integrating algorithms into HKube require 3 steps:  
+1) Implement connectivity with HKube using WebSocket.  
 2) Build the algorithm image and push it to Docker registry.  
-3) Add the algorithm to Hkube.
+3) Add the algorithm to HKube.
 
-We can do these first two steps for you, so you won't have to deal with WebSocket and Docker. The algorithm (your code) needs a way to communicate with Hkube (receive input, report results and errors)   
-Hkube communicates with algorithms via WebSocket using their full-duplex communication support.  
-All messages between Hkube and algorithm are in JSON format.
+We can do these first two steps for you, so you won't have to deal with WebSocket and Docker. The algorithm (your code) needs a way to communicate with HKube (receive input, report results and errors)   
+HKube communicates with algorithms via WebSocket using their full-duplex communication support.  
+All messages between HKube and algorithm are in JSON format.
 
 ## The easy way
 
@@ -85,10 +85,10 @@ You can do the same using our [API](http://petstore.swagger.io/?url=https://raw.
 - Use WebSocket.
 - Use Docker.
 
-## Events From Hkube to Algorithm
+## Events From HKube to Algorithm
 ---
 
-These events are sent from Hkube to your algorithm.
+These events are sent from HKube to your algorithm.
 
 * [Initialize](#event-initialize)
 * [Start](#event-start)
@@ -220,10 +220,10 @@ Event to inform algorithm that sub pipeline has stopped
 * The "reason" property holds the reason for stopping the sub pipeline.
 
 
-## Events From Algorithm to Hkube
+## Events From Algorithm to HKube
 ---
 
-These events are sent from algorithm to Hkube.
+These events are sent from algorithm to HKube.
 
 * [initialized](#event-initialized)
 * [started](#event-started)
@@ -441,8 +441,8 @@ To finish the last opened tracer span, use this event:
 ## Implement
 ---
 
-Hkube communicates with your algorithm via WebSocket (native WebSocket or socketio).  
-This tutorial explains how to create a websocket client that works with Hkube.
+HKube communicates with your algorithm via WebSocket (native WebSocket or socketio).  
+This tutorial explains how to create a websocket client that works with HKube.
 You can implement the websocket client in any language. (PR are welcomed)
 
 * [Connect](#connect)
@@ -464,7 +464,7 @@ The first thing your algorithm should do is create a websocket client that liste
 
 ## Handle Events
 
-Here we are registering to events from Hkube.  
+Here we are registering to events from HKube.  
 Each event has a specific handler, as described below.
 
 ```hkube-tabs
@@ -473,7 +473,7 @@ Each event has a specific handler, as described below.
 
 ## initialize
 
-The initialize event is the first event that Hkube sends to your algorithm.  
+The initialize event is the first event that HKube sends to your algorithm.  
 The payload of this event includes the pipeline data and the input for your algorithm.  
 You need to store the input in a local variable for later use.  
 > same input as written in the [descriptor](../learn/input/)
@@ -484,8 +484,8 @@ You need to store the input in a local variable for later use.
 
 ## start
 
-The start event is the second event that Hkube sends to your algorithm.  
-As you can see, at the first step of this handler you need to tell Hkube that your algorithm has started.  
+The start event is the second event that HKube sends to your algorithm.  
+As you can see, at the first step of this handler you need to tell HKube that your algorithm has started.  
 Then you let the algorithm do it's work and finally you send the done event with the algorithm result.
 
 ```hkube-tabs
@@ -494,7 +494,7 @@ Then you let the algorithm do it's work and finally you send the done event with
 
 ## stop
 
-Hkube will send this event to your algorithm only if stop request was made by Hkube users.
+HKube will send this event to your algorithm only if stop request was made by HKube users.
 
 ```hkube-tabs
 # { "hkube": true, "schema": "handle-messages-stop" }
@@ -510,15 +510,15 @@ Web Sockets are not auto reconnect, so it's important that you will handle conne
 
 ## Handle Errors
 
-It's highly recommended that you will catch any error in your algorithm and send it to Hkube.  
+It's highly recommended that you will catch any error in your algorithm and send it to HKube.  
 
 ```hkube-tabs
 # { "hkube": true, "schema": "handle-errors" }
 ```
 
 ## Send Event
 
-This is a simple handler for send response back to Hkube.
+This is a simple handler for send response back to HKube.
 
 ```hkube-tabs
 # { "hkube": true, "schema": "send-event" }
@@ -527,4 +527,4 @@ This is a simple handler for send response back to Hkube.
 ## Monitoring Metrics
 Algorithms using Tensorflow can generate metrics for a Tenosrboard view. Later upon request, a Tensorboard webserver will start, serving a dashboard comparing different runs of the algorithm.
 To allow hkube to display your algorithms Tesorboard metrics: In the algorithm code, write your Tensorboard metrics to a folder path set as environment variable ALGO_METRICS_DIR value. 
-To run Tesorboard: In Hkube spec find 'board' api to start a tensorboard web server, visualizing the tensor metrics.
+To run Tesorboard: In HKube spec find 'board' api to start a tensorboard web server, visualizing the tensor metrics.
diff --git a/site/learn/Caching.md b/site/learn/Caching.md
@@ -9,9 +9,9 @@ next: /learn/debug/
 ---
 
 ## Introduction
- Hkube pipeline execution can be long and complex 
+ HKube pipeline execution can be long and complex 
  
-  Hkube Caching allows you to run job from a specific node and getting the data of the predecessors from caching of the previous run, saving time and resources  enable you to monitor or debug the relevant Node without the need to wait for previous prosses to complete
+  HKube Caching allows you to run job from a specific node and getting the data of the predecessors from caching of the previous run, saving time and resources  enable you to monitor or debug the relevant Node without the need to wait for previous prosses to complete
   
   **Note: the Caching is only relevant to the batch pipeline**
 
diff --git a/site/learn/Debug.md b/site/learn/Debug.md
@@ -8,7 +8,7 @@ permalink: /learn/debug/
 next: /learn/tensorboard/
 ---
 ## Introduction
-Hkube enable debugging algorithm on your local IDE as part of a pipeline executed on the cluster
+HKube enable debugging algorithm on your local IDE as part of a pipeline executed on the cluster
 
 ## Use Cases 
  After executing the following job you suspect that the "yellow" node does not calculate its output properly and you wish to debug it in your local IDE  
diff --git a/site/learn/Execution.md b/site/learn/Execution.md
@@ -7,7 +7,7 @@ next: /learn/streaming/
 sublinks: Flow Input, Reference, Batch, Batch Reference, Wait Any, Output
 ---
 
-Hkube allow special signs in the nodes input that designed to define the pipeline execution flow.
+HKube allow special signs in the nodes input that designed to define the pipeline execution flow.
 
 - (@)  - [Reference](#reference)
 - (#)  - [Batch](#batch)
diff --git a/site/learn/InstallHkube.md b/site/learn/InstallHkube.md
@@ -1,6 +1,6 @@
 ---
-title: Install Hkube
-sidebarTitle: Install Hkube
+title: Install HKube
+sidebarTitle: Install HKube
 layout: ../_core/DocsLayout
 category: Learn
 permalink: /learn/install/
@@ -10,13 +10,13 @@ next: /learn/installCLI/
 
 ### Prerequisites
 ---
-Hkube requires a working installation of kubernetes.  
+HKube requires a working installation of kubernetes.  
 There are many options for local k8s installations. e.g  [Minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/) or [microk8s](https://microk8s.io/).  
 
-Hkube is installed using helm. Both helm 2 and the newer [helm 3](https://helm.sh/docs/intro/install/) should work.
+HKube is installed using helm. Both helm 2 and the newer [helm 3](https://helm.sh/docs/intro/install/) should work.
 
 > Make sure kubectl is configured to your cluster.  
-> In order to collect algorithm logs, and create builds, **Hkube** requires that certain pods will run in privileged security permissions. Consult your kubernetes installation to see how to do that.  
+> In order to collect algorithm logs, and create builds, **HKube** requires that certain pods will run in privileged security permissions. Consult your kubernetes installation to see how to do that.  
 
 These instructions assume the use [Minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/) and [helm 3](https://helm.sh/docs/intro/install/)
 
diff --git a/site/learn/InstallHkubeCLI.md b/site/learn/InstallHkubeCLI.md
@@ -1,6 +1,6 @@
 ---
-title: Install Hkube CLI
-sidebarTitle: Install Hkube CLI
+title: Install HKube CLI
+sidebarTitle: Install HKube CLI
 layout: ../_core/DocsLayout
 category: Learn
 permalink: /learn/installCLI/
@@ -9,7 +9,7 @@ next: /learn/api/
 ---
 
 ## Overview
-`hkubectl` is the Hkube command-line tool used for managing and interacting with pipelines, algorithms, and configurations in your Kubernetes cluster.  
+`hkubectl` is the HKube command-line tool used for managing and interacting with pipelines, algorithms, and configurations in your Kubernetes cluster.  
 It supports various commands to streamline the development and deployment of workflows.
 
 ---
@@ -42,8 +42,8 @@ export os="linux" # or macos
 
 ---
 
-### Step 2: Configure Hkube
-1. **Set the Hkube API endpoint:**
+### Step 2: Configure HKube
+1. **Set the HKube API endpoint:**
 ```bash
 hkubectl config set endpoint ${KUBERNETES-MASTER-IP}
 ```
@@ -59,7 +59,7 @@ kubectl get nodes
 ---
 
 > **Note:**  
-> Hkube requires certain pods to run with privileged security permissions.  
+> HKube requires certain pods to run with privileged security permissions.  
 > Consult your Kubernetes installation documentation for guidance.
 
 <script>
diff --git a/site/learn/IntroductionV2.md b/site/learn/IntroductionV2.md
@@ -1,6 +1,6 @@
 ---
-title: Hkube 101
-sidebarTitle: Hkube 101
+title: HKube 101
+sidebarTitle: HKube 101
 layout: ../_core/DocsLayout
 category: Learn
 permalink: /learn/
@@ -40,7 +40,7 @@ next: /learn/advancedPipeline/
 - **Creating your first Algorithm**
 
   - Currently, we support Javascript, Python and Java **autobuild**.
-  - Your algorithm gets it's inputs from Hkube the following way:
+  - Your algorithm gets it's inputs from HKube the following way:
 
   >Java example - see [Documentation](https://raw.githubusercontent.com/kube-HPC/java-wrapper.hkube/master/algorithm-example/src/main/resources/instructions.txt)
   
diff --git a/site/learn/Pipelines.md b/site/learn/Pipelines.md
@@ -7,7 +7,7 @@ sublinks: Raw, Stored
 next: /learn/nodes/
 ---
 
-The basic concept behind the Hkube pipeline is [DAG](https://en.wikipedia.org/wiki/Directed_acyclic_graph)  
+The basic concept behind the HKube pipeline is [DAG](https://en.wikipedia.org/wiki/Directed_acyclic_graph)  
 Directed Acyclic Graph is a graph consisting of [nodes](../learn/nodes/) connected  
 with edges that have a direction: A -> B -> C.  
 
@@ -19,8 +19,8 @@ The DAG structure allowing to:
 * Data flow between the nodes.  
 * Parallel and batch processing.
 
-Hkube store and execute pipelines using Restful API.
-Hkube support three types of pipeline executions: Raw, Stored and Node.
+HKube store and execute pipelines using Restful API.
+HKube support three types of pipeline executions: Raw, Stored and Node.
 
 ### Raw
 
diff --git a/site/learn/Tensorboard.md b/site/learn/Tensorboard.md
@@ -8,12 +8,12 @@ permalink: /learn/tensorboard/
 next: /learn/triggers/
 ---
 ## Introduction
-Hkube allows to view data generated by tensorflow of users algorithm in Tensorboard UI.
+HKube allows to view data generated by tensorflow of users algorithm in Tensorboard UI.
 
 ## Config
 
 #### logdir
-An algorithm using tensorflow generates tensorboard log data. In the algorithm code when creating Tensorboard object a logDir parameter is passed. For integration with Hkube the logDir value passed, must first be obtained from the environment variable : ALGO_METRICS_DIR.
+An algorithm using tensorflow generates tensorboard log data. In the algorithm code when creating Tensorboard object a logDir parameter is passed. For integration with HKube the logDir value passed, must first be obtained from the environment variable : ALGO_METRICS_DIR.
 	 
 ```python
 log_dir = os.environ['ALGO_METRICS_DIR']
diff --git a/site/learn/Triggers.md b/site/learn/Triggers.md
@@ -7,12 +7,12 @@ next: /learn/webhooks/
 sublinks: Cron, Pipelines
 ---
 
-Hkube can run your stored pipelines based on triggers.  
+HKube can run your stored pipelines based on triggers.  
 There are two types of triggers: **Cron** trigger and **Pipelines** trigger.
 
 ### Cron
 
-Hkube can schedule your stored pipelines based on cron pattern.  
+HKube can schedule your stored pipelines based on cron pattern.  
 see this [cron](https://crontab.guru/) editor in order to construct your cron.  
 
 If you want to run your pipeline periodically using cron, you just need to add cron triggers to your stored pipeline.  For example: lets create a pipeline that will run in the 20th and 40th seconds, every Sunday, Monday and Tuesday.
diff --git a/site/learn/Versions.md b/site/learn/Versions.md
@@ -1,31 +1,54 @@
 ---
 title: Versions
+sidebarTitle: Versions
 layout: ../_core/DocsLayout
 category: Learn
 permalink: /learn/versions/
+sublinks: Algorithms, Pipelines
 next: /learn/pipelines/
 ---
 
-Hkube manage versions for algorithms, so you can easily switch between them.  
-There are three cases when a new version will be created:  
-1) First time you insert algorithm with image.  
-2) Each time `algorithm image` has been changed.  
-3) After successful algorithm build.   
+HKube manages versioning for both algorithms and pipelines, allowing you to easily switch between different versions.
+Each version has a unique version ID, creation date, and details about its specification.
 
-Every algorithm points to a specific version, you can switch between versions by using the Dashboard or the API.  
-using the API just `POST` to `/versions/algorithms/apply` with body.  
+## Algorithms
+A new algorithm version is created in the following cases:  
+1) The first time an algorithm is inserted with an image.  
+2) Any time there is a change in the algorithm specification.  
+3) After a successful algorithm build.
+
+Each algorithm points to a specific version, and you can switch between versions using either the Dashboard or the Swagger API.
+
+To switch versions via the Swagger API, send a `POST` request to `/versions/algorithms/apply` with the following JSON body:
 ```json
 {
-    "name"  // the name of the algorithm
-    "image" // the new version you want switch to
-    "force" // should replace or not current algorithm version for running algorithm instances
+    "name": "alg-name",  // The name of the algorithm
+    "version": "version-id", // The version you want to switch to
+    "force": true  // Whether to replace the current algorithm version for running instances
 }
-
 ```
-> using force: true will cause a pipeline failure since Hkube need to shut down the running instances
+> **Note:** Setting "force": true will cause pipeline failures because HKube must shut down running instances.
+
+For more details, see [Algorithm Versions](/spec/#tag/Algorithm-Versions).
+
+After a successful algorithm build, a new version is created. If it is the first build, the new version is automatically activated.  
+For subsequent builds, new versions are created, but you must manually select which version to apply.
 
-see [versions](/spec/#tag/Versions) for more details.
+## Pipelines
+HKube also supports versioning for pipelines, allowing you to track and switch between different pipeline versions.
 
-After successful algorithm build a new version will be created, if it was the first build then the new version will automatically will be activated for this algorithm. the next builds will also create new versions but you will need to manually choose which version you want to apply.
+A new pipeline version is created in the following cases:  
+1) The first time a pipeline is inserted.  
+2) Any time there is a change in the pipeline specification.
 
+Each pipeline points to a specific version, and you can switch between versions using the Dashboard or the Swagger API.
 
+To switch versions via the Swagger API, send a `POST` request to `/versions/pipelines/apply` with the following JSON body:
+```json
+{
+    "name": "pipeline-name",  // The name of the pipeline
+    "version": "version-id" // The version you want to switch to
+}
+
+```
+For more details, see [Pipeline Versions](/spec/#tag/Pipeline-Versions).
diff --git a/site/learn/openshift.md b/site/learn/openshift.md
@@ -1,6 +1,6 @@
 ---
 title: Openshift Install
-sidebarTitle: Install Hkube
+sidebarTitle: Install HKube
 layout: ../_core/DocsLayout
 category: Learn
 permalink: /learn/install/openshift
diff --git a/site/learn/openshift_old.md b/site/learn/openshift_old.md
@@ -1,6 +1,6 @@
 ---
 title: Openshift Install (old)
-sidebarTitle: Install Hkube
+sidebarTitle: Install HKube
 layout: ../_core/DocsLayout
 category: Learn
 permalink: /learn/install/openshift/before_v1.2.189/
