Skip to content

Technologicat/raven

BranchesTags

Repository files navigation

Logo of Raven

Table of Contents

The Raven constellation

As of 08/2025, Raven is now a constellation, no longer a single app:

  • ✅ Raven-visualizer: Visualize research literature

    Screenshot of Raven's main window Screenshot of Raven's wordcloud window

    • Documentation: Visualizer user manual
    • Goal: Take 10k+ studies, find the most relevant ones.
      • Status: Fully operational. Could still use more features; we plan to add some later.
    • Features:
      • GUI app for analyzing BibTeX databases
      • Semantic clustering
      • Automatic per-cluster keyword detection
      • Command-line converters for Web of Science (WOS) and arXiv
      • 100% local, maximum privacy, no cloud services
    • This was the original Raven.

  • ✅ Raven-avatar: AI-animated anime avatar

    Screenshot of Raven-avatar-settings-editor Screenshot of Raven-avatar-pose-editor

    • Documentation: Avatar user manual
    • Goal: Visually represent your LLM as a custom anime character, for PR stunts and for fun.
      • Status: Fully operational standalone tech demo, and Python bindings to integrate the avatar to Python-based GUI apps.
      • JS bindings possible, but not implemented yet. See #2.
    • Features:
      • One static input image into realtime video (THA3 engine).
      • Talking custom anime character with 28 emotional expressions.
      • Lipsync to Raven-server's TTS. Record lipsynced speech (audio + image sequence).
      • Realtime Anime4K upscaler.
      • Realtime video postprocessor with visual effects such as bloom, chromatic aberration, or scanlines.
      • Web API to receive avatar video stream and to control the avatar.

  • ✅ Raven-server: Web API server

    Screenshot of Raven-server

    • Documentation: Server user manual
    • Goal: Run all GPU processing on the server, anywhere on the local network.
      • Status: Fully operational. On the client side, raven-importer and the RAG subsystem of Raven-librarian have no server support yet.
    • Features:
      • AI components for natural language processing (NLP).
      • Speech synthesizer (TTS), using Kokoro-82M.
      • Server side of Raven-avatar.
    • Partially compatible with SillyTavern. Originally developed as a continuation of SillyTavern-extras.
    • Python bindings (client for web API) provided.
      • JS bindings possible, but not implemented yet. See #2.

  • 🚧 Raven-librarian: Scientific LLM frontend (under development)

    • Documentation: under development
    • Goal: Efficiently interrogate a stack of 2k scientific papers. Talk with a local LLM for synthesis, clarifications, speculation, ...
      • Status: A command-line prototype raven-minichat is available.
        • We recommend having raven-server running; this allows the LLM to search the web.
        • A GUI application is planned, but not available yet.
    • Features:
      • 100% local (when using a locally hosted LLM)
      • Natively nonlinear branching chat history - think Loom or SillyTavern-Timelines.
        • Chat messages are stored as nodes in a tree. A chat branch is just its HEAD pointer; the chat app follows the parent links to reconstruct the linear history for that branch.
        • The command-line prototype can create chat branches and switch between them, but not delete them. Complete control for chat branches will be in the GUI app.
      • RAG (retrieval-augmented generation) with hybrid (semantic + keyword) search.
      • Tool-calling (a.k.a. tool use)
        • Currently, a websearch tool is provided.
    • Uses oobabooga/text-generation-webui as the LLM backend through its OpenAI-compatible API.

Install & run

The Raven constellation consists traditional desktop apps. It needs to be installed.

Currently, this takes the form of installing the app and dependencies into a venv (virtual environment). At least at this stage of development, app packaging into a single executable is not a priority.

Raven has been developed and tested on Linux Mint. It should work in any environment that has bash and pdm.

It has been reported to work on Mac OS X, as well as on Windows (with Miniconda).

From source

Raven has the following requirements:

  • A Python environment for running the PDM installer. Linux OSs have one built-in; on other OSs it is possible to use tools such as Miniconda to install one.
  • An NVIDIA GPU for running AI models via CUDA. (This is subject to change in the future.)

❗ Help wanted! If you have an AMD GPU and would be willing to collaborate to get Raven working on it, please chime in. Raven does not directly depend on CUDA, but only on PyTorch and on various AI libraries in the Python ecosystem. ❗

Install PDM in your Python environment

Raven uses PDM to manage its dependencies. This allows easy installation of the app and its dependencies into a venv (virtual environment) that is local to this one app, so that installing Raven will not break your other apps that use machine-learning libraries (which tend to be very version-sensitive).

If your Python environment does not have PDM, you will need to install it first:

python -m pip install pdm

Don't worry; it won't break pip, poetry, or other similar tools.

Install Raven via PDM

Then, to install Raven, in a terminal that sees your Python environment, navigate to the Raven folder.

We will next initialize the new venv, installing the required Python version into it. This Python will be available for PDM venvs, and is independent of Python that PDM itself runs on.

Raven is currently developed against the minimum supported Python version, so we recommend to install that version, like this:

pdm python install --min

The venv will be installed in the .venv hidden subfolder of the Raven folder.

Then, install Raven's dependencies as follows. (If you are a seasoned pythonista, note that there is no requirements.txt; the dependency list lives in pyproject.toml.)

Basic install without GPU compute support

pdm install

This may take a while (several minutes).

Now the installation should be complete.

Install with GPU compute support

❗ Currently this requires an NVIDIA GPU and CUDA. ❗

❗ Using CUDA requires the proprietary NVIDIA drivers, also on Linux. ❗

❗ Currently Raven uses CUDA 12.x. Make sure your NVIDIA drivers support this version. ❗

pdm install --prod -G cuda

If you want to add GPU compute support later, you can run this install command on top of an already installed Raven.

Installing dependencies may take a long time (up to 15-30 minutes, depending on your internet connection), because torch and the NVIDIA packages are rather large (my .venv shows 11.1 GB in total).

Now the installation should be complete.

Install on an Intel Mac with MacOSX 10.x

Installing Raven may fail, if Torch cannot be installed.

On MacOSX, installing torch 2.3.0 or later requires an ARM64 processor and MacOSX 11.0 or later.

If you have an Intel Mac (x86_64) with MacOSX 10.x, to work around this, you can use Torch 2.2.x.

To do this, modify Raven's pyproject.toml in a text editor, so that the line

    "torch>=2.4.0",

becomes

    "torch>=2.2.0,<2.3.0",

Then run pdm install again.

Install on Windows (if Windows Defender gets angry)

Installing Raven does not need admin rights.

  • Raven can be installed as a regular user.
    • We recommend Miniconda as the Python environment.
  • The only exception, that does need admin rights, is installing espeak-ng, so the TTS (speech synthesizer) can use that as its fallback phonemizer.
    • Raven only ever calls espeak-ng from Raven-server's tts module, and only for those inputs for which the TTS's built-in Misaki phonemizer fails.
    • In practice, that is for out-of-dictionary words in English, as well as for some non-English languages.

Using Raven does not need admin rights.

  • All the apps are regular userspace apps that you can run as a regular user.

If you get a permission error when trying to run pdm, try replacing "pdm" with "python -m pdm".

For example, instead of:

pdm install

run the command:

python -m pdm install

This works because PDM is just a Python module. This will be allowed to run if python is allowed to run.

Similarly, Raven apps are just Python modules, and can be run via Python, as follows. Full list as of Raven v0.2.3:

Command                                Replacement

raven-visualizer                  →    python -m raven.visualizer.app
raven-importer                    →    python -m raven.visualizer.importer
raven-arxiv2id                    →    python -m raven.tools.arxiv2id
raven-burstbib                    →    python -m raven.tools.burstbib
raven-wos2bib                     →    python -m raven.tools.wos2bib
raven-pdf2bib                     →    python -m raven.tools.pdf2bib
raven-server                      →    python -m raven.server.app
raven-avatar-settings-editor      →    python -m raven.avatar.settings_editor.app
raven-avatar-pose-editor          →    python -m raven.avatar.pose_editor.app
raven-check-cuda                  →    python -m raven.raven.check_cuda
raven-check-audio-devices         →    python -m raven.raven.check_audio_devices
raven-minichat                    →    python -m raven.librarian.minichat

Check that CUDA works (optional)

Once you have installed Raven with GPU compute support, you can check if Raven detects your CUDA installation:

raven-check-cuda

This command will print some system info into the terminal, saying whether it found CUDA, and if it did, which device CUDA is running on.

It will also check whether the cupy library loads successfully. This library is needed by the spaCy natural language analyzer (so that too can run on GPU).

Example output:

INFO:raven.tools.check_cuda:Raven-check-cuda version 0.2.3
Checking dependencies...
1. PyTorch availability check [SUCCESS] ✅
2. CUDA device availability check [SUCCESS] ✅ (Using NVIDIA GeForce RTX 3070 Ti Laptop GPU)
3. CuPy & CuPyX (for spaCy NLP) [SUCCESS] ✅

System information:
   Python version: 3.10.12
   OS: Linux 6.8.0-109049-tuxedo
   PyTorch version: 2.7.0+cu126

Activate the Raven venv (to run Raven commands such as raven-visualizer or raven-server)

In a terminal that sees your Python environment, navigate to the Raven folder.

Then, activate Raven's venv with the command:

$(pdm venv activate)

Note the Bash exec syntax $(...); the command pdm venv activate just prints the actual internal activation command.

❗ Windows users note: The command $(pdm venv activate) needs the bash shell, and will not work in most Windows command prompts. ❗

Alternatively, you can run the venv activation script directly. You can find the script in .venv/bin/.

❗ For Linux and Mac OS X, the script is typically named .venv/bin/activate; for Windows, typically .venv/bin/activate.ps1 or ./venv/bin/activate.bat. ❗

Whenever Raven's venv is active, you can use Raven commands, such as raven-visualizer.

Activate GPU compute support (optional)

If CUDA support is installed but not working, you can try enabling CUDA (for the current command prompt session) as follows.

With the venv activated, and the terminal in the Raven folder, run the following bash command:

source env.sh

This sets up the library paths and $PATH so that Raven finds the CUDA libraries. This script is coded to look for them in Raven's .venv subfolder.

Choose which GPU to use (optional)

If your machine has multiple GPUs, there are two ways to tell Raven which GPU to use.

If your system permanently has several GPUs connected, and you want to use a different GPU permanently, you can adjust the device settings in raven.server.config, raven.visualizer.config, and raven.librarian.config.

If you switch GPUs only occasionally (e.g. a laptop that sometimes has an eGPU connected and sometimes doesn't), you can use the CUDA_VISIBLE_DEVICES environment variable to choose the GPU temporarily, for the duration of a command prompt session.

We provide an example script run-on-internal-gpu.sh, meant for a laptop with a Thunderbolt eGPU (external GPU), which forces Raven to run on the internal GPU when the external is connected (which is useful e.g. if your eGPU is dedicated for a self-hosted LLM). On the machine where the script was tested, PyTorch sees the eGPU as GPU 0 when available, pushing the internal GPU to become GPU 1. When the eGPU is not connected, the internal is GPU 0.

With the venv activated, and the terminal in the Raven folder, run the following bash command:

source run-on-internal-gpu.sh

Then for the rest of the command prompt session, any Raven commands (such as raven-visualizer) will only see the internal GPU, and "cuda:0" in the device settings will point to the only visible GPU.

Exit from the Raven venv (optional, to end the session)

❗ There is usually no need to do this. You can just close the terminal window. ❗

If you want to exit from the Raven venv without exiting your terminal session, you can deactivate the venv like this:

deactivate

After this command completes, python again points to the Python in your Python environment (where e.g. PDM runs), not to Raven's app-local Python.

If you want to also exit your terminal session, you can just close the terminal window as usual; there is no need to deactivate the venv unless you want to continue working in the same terminal session.

Uninstall

python -m pip uninstall raven-visualizer

Or just delete the venv, located in the .venv subfolder of the Raven folder.

Technologies

Raven builds upon several AI, NLP, statistical, numerical and software engineering technologies:

Note that installing Raven will auto-install dependencies into the same venv (virtual environment). This list is here just to provide a flavor of the kinds of parts needed to build a constellation like this.

License

2-clause BSD.

Acknowledgements

This work was financially supported by the gH2ADDVA (Adding Value by Clean Hydrogen production) project at JAMK, co-funded by the EU and the Regional Council of Central Finland.

JAMK Institute of New Industry

About

Absorb scientific literature

Resources

Readme

License

BSD-2-Clause license
Activity

Stars

2 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

3 tags

Packages

No packages published

Languages