UTM Redirect Data Receiver

A Moose application for receiving and processing UTM redirect webhook data from utm.fiveonefour.dev.

Overview

This project implements a webhook receiver that:

• Accepts redirect event data from the UTM redirect service
• Processes and enriches the data with extracted UTM parameters
• Stores events in ClickHouse for analytics and reporting

Project Structure

app/
├── ingest/
│   ├── models.ts      # Data models for UTM webhook events
│   └── transforms.ts  # Data transformation logic
└── index.ts           # Main application entry point

Data Flow

1. Webhook Ingestion: Receives POST requests at /ingest/UTMWebhook
2. Data Processing: Transforms raw webhook data and extracts UTM parameters
3. Storage: Stores processed events in ClickHouse's ProcessedUTM table

Getting Started

Prerequisites

• Moose CLI
• Node.js 18+
• ngrok (for local webhook testing)

Installation

1. Clone the repository:

git clone https://github.com/514-labs/redirect-data.git
cd redirect-data

2. Install dependencies:

npm install

3. Configure authentication (see Authentication section below)

4. Start the development server:

moose dev

The server will start on http://localhost:4000

Local Webhook Testing

To receive webhooks from the UTM service locally:

1. Start ngrok to expose your local server:

ngrok http 4000

2. Configure the UTM service at utm.fiveonefour.dev/admin to use your ngrok URL as the analytics endpoint.

Authentication

This service requires bearer token authentication for security. The UTM tracker prototype must include an Authorization: Bearer <token> header when sending webhook data.

Setup

1. Copy the .env.example file to .env:

cp .env.example .env

2. Generate a hashed API key using the Moose CLI:

moose generate hash-token

3. Set the generated hash in your .env file:

MOOSE_INGEST_API_KEY=your_generated_hash_here

4. Configure the UTM tracker prototype to send the same token (before hashing) in the Authorization header:

# In the utm-tracker-prototype .env file:
ANALYTICS_BEARER_TOKEN=your-plain-token-here

How It Works

• The UTM tracker sends: Authorization: Bearer your-plain-token-here
• Moose verifies this against the hashed version in MOOSE_INGEST_API_KEY
• Requests without valid authentication are rejected with a 401 error

Alternative: JWT Authentication

For more advanced use cases, you can use JWT tokens with RSA signatures. See the Moose authentication docs for setup instructions.

Webhook Data Format

The webhook endpoint expects POST requests with the following JSON structure:

{
  "originalUrl": "https://your-domain.com/page?utm_source=twitter",
  "redirectUrl": "https://target-site.com/landing",
  "timestamp": "2025-01-15T10:30:00.000Z",
  "userAgent": "Mozilla/5.0...",
  "utmParams": {
    "utm_source": "twitter",
    "utm_medium": "social",
    "utm_campaign": "launch2025"
  },
  "path": "/page"
}

Data Storage

Processed events are stored in ClickHouse in the ProcessedUTM table with:

Original webhook fields:

• originalUrl, redirectUrl, timestamp, userAgent, path
• utmParams: JSON object with all UTM parameters
• referer: Optional field (if provided by webhook)

Additional generated fields:

• id: Auto-generated unique identifier (format: utm-{timestamp}-{random})
• utm_source, utm_medium, utm_campaign, utm_term, utm_content: Extracted as separate columns for efficient querying

Table configuration:

• Ordered by timestamp for optimal query performance

Deployment

The easiest way to deploy your Moose application is to use Boreal.

Check out the Moose deployment documentation for more details.

Community

Join the Moose community on Slack.

Made by 514

Built with Moose by 514 Labs.