text2image_ir_annotation_tool

This repository contains a web-based annotation tool built in Docker for text-to-image Image Information Retrieval (IR) tasks. Its main purpose is to facilitate the annotation of images with a relevance score for a given text query.

Configuration

First, you need to setup the docker-compose file. The docker-compose.yml file contains the configuration for the web application and the database. You can modify the configuration as needed. We suggest changing the ports to avoid conflicts with other services running on your machine. You can also change the paths that interact with the host machine so that the server can access the image files.

You will also need to configure Auth0 for authentication. The docker instance requires a .env file for configuration. The .env file should contain the following variables:

AUTH0_CLIENT_ID=<your_auth0_client_id>
AUTH0_CLIENT_SECRET=<your_auth0_client_secret>
AUTH0_DOMAIN=<your_auth0_domain>
AUTH0_APP_SECRET_KEY=<your_auth0_app_secret_key>
AUTH0_AUDIENCE=<your_auth0_audience>

Additionally, you can change the rules.md file, located in flask/app/static/markdown/rules.md, to include the guidelines for the annotators. This file contains the rules and guidelines that the annotators should follow while annotating the images.

Running the container

To run the annotation tool, you will need to build and run the Docker container. The following command will build the container and start it:

docker-compose up --build

Adding data

The database used to store information is Redis. You can add data to the database using any preferred method. The only requirement is that the data sent to the database should be in the following format:

There are three main entities in the database: images, articles, and queries. Each entity has its own unique identifier and contains relevant information.

• The images entity contains information about the images, including the image hash, file extension, and a list of articles associated with the image.

{
  "images": {
    "img001": {
      "hash": "img001",
      "extension": "jpg",
      "articles": "[\"articles:art001\"]"
    },
    "img002": {
      "hash": "img002",
      "extension": "jpg",
      "articles": "[\"articles:art002\"]"
    }
  }
}

• The articles entity contains information about the articles, including the title and a list of images associated with the article.

{
  "articles": {
    "art001": {
      "title": "Firefighters in action",
      "images": "[\"images:img001\"]"
    },
    "art002": {
      "title": "Firefighters in a ceremony",
      "images": "[\"images:img002\"]"
    }
  }
}

• The queries entity contains information about the queries, including the query text and a list of ground truth images associated with the query.

{
  "queries": {
    "q001": {
      "query": "Firefighters",
      "ground_truth": "[\"img001\", \"img002\"]",
    },
    "q002": {
      "query": "Police",
      "ground_truth": "[\"img003\", \"img004\"]",
    }
  }
}

Extracting annotations

The annotations are stored in the Redis database in multiple formats. You can extract the annotation by query or by the user.