Welcome to Noveum.ai Docs

Welcome to the Noveum.ai documentation! Here you’ll find everything you need to install, configure, and optimize your AI applications using our platform.

We offer tools for monitoring and evaluating AI models across multiple providers. Our documentation is organized to help you get started quickly, while also diving into advanced setup and best practices.

1. Getting Started

Overview

Noveum.ai is composed of two main components:

1. Open-Source AI Gateway: Deployable on Cloudflare Workers, Docker, or Kubernetes. It provides a single, unified route for AI calls and collects real-time metrics (latency, cost, token usage, error rates, etc.).

2. Noveum Platform: Our dashboard and backend that lets you:

   • Curate Datasets from real user logs
   • Run Eval Jobs to compare models
   • Analyze results to find the best model for your cost, speed, or accuracy needs

Check out the Overview section for a more detailed introduction to Noveum.ai.

Quick Start

For a quick start, see our guide on Integrating with the AI Gateway. After a minimal one-line code change, your application calls are routed through the gateway, and Noveum starts capturing insights right away.

Next Steps

1. Set up Datasets using real-world logs: Learn More
2. Run Evaluation Jobs to compare performance, cost, and accuracy: Start Here
3. Interpret your Eval Results and choose the right model: Find Out How
4. Integrate External Providers beyond OpenAI or Anthropic: Add Providers

2. Documentation Structure

Here’s a quick overview of our documentation sections:

• Overview
  Introduction to Noveum.ai and its core features.

• Integrating with AI Gateway
  Step-by-step guide to deploy the open-source gateway and start collecting metrics.

• Datasets
  How to turn logs into curated datasets for more accurate, real-world evaluation.

• Eval Jobs
  Detailed instructions on running evaluation jobs and comparing different models at scale.

• Understanding Eval Jobs Results
  Explanation of accuracy metrics, cost comparisons, and how to interpret the findings.

• Integrating External Providers
  Learn how to add or customize support for new AI providers, or your own custom model endpoints.

3. Basic Example

import openai
 
# Point to Noveum's AI Gateway
openai.api_base = "http://localhost:3000/v1"
openai.api_key = "YOUR_OPENAI_KEY"
 
# Now any request below automatically gathers metrics
response = openai.Completion.create(
    engine="gpt-4",
    prompt="Hello, what's the weather like tomorrow?",
    max_tokens=50
)
 
print(response)

Within minutes, you’ll see metrics in the Noveum dashboard—such as latency, tokens used, and cost per request.

4. Images & Visual References

Visuals often help clarify setup steps. For example, once you deploy the gateway and open the Noveum dashboard, you’ll see an overview similar to:

5. Additional Resources

• OpenTelemetry Documentation – Noveum’s Gateway integrates seamlessly with OpenTelemetry for standardized metrics.
• Cloudflare Workers – Deploy on the edge for ultra-low-latency AI calls.
• Kubernetes Documentation – Scale your gateway in a production-ready cluster.

We hope you enjoy using Noveum.ai. If you have any questions, suggestions, or run into issues, feel free to reach out via our Community Forum or open an issue on our GitHub repository.