OSI Semantic Adapter

This page describes query compatibility for existing OSI (Open Semantic Interchange) semantic models and metrics. The implementation spans two repositories:

• datus-agent: loads existing strict OSI core YAML and makes its metrics available to ask_metrics and other query surfaces.
• datus-semantic-adapter: provides the datus-semantic-osi adapter. It loads OSI YAML, validates the OSI core schema, compiles the document into Datus Semantic IR, and lowers it to an execution backend. The current backend is MetricFlow.

Positioning

Query-only compatibility: existing OSI projects remain queryable, but OSI semantic authoring is retired. Migrate the project to Dosi before making changes, then use semantic_modeling.

In an existing OSI project, OSI is the source format and MetricFlow is the default execution backend that renders SQL and executes metric queries.

existing strict OSI core YAML + DATUS custom_extensions
        |
        v
datus-semantic-osi: validate -> compile IR -> lower to MetricFlow
        |
        v
validate_semantic / query_metrics / ask_metrics

Users and LLM agents should not write MetricFlow YAML fields in OSI mode, such as data_source, measures, measure_proxy, or type_params. The OSI adapter generates those backend artifacts internally. They are disposable execution artifacts, not the source files users maintain.

Installation

Install the OSI adapter from the datus-semantic-adapter repository.

From a released package:

pip install "datus-semantic-osi[metricflow]"

From source:

pip install -e ../datus-semantic-adapter/datus-semantic-core
pip install -e "../datus-semantic-adapter/datus-semantic-metricflow"
pip install -e "../datus-semantic-adapter/datus-semantic-osi[metricflow]"

The metricflow extra installs dependencies required by the MetricFlow execution backend. Static OSI compilation and validation can work without the extra, but metric querying through Datus requires an available execution backend.

Configuration

Configure the semantic layer as osi in agent.yml. The selected semantic layer is global for semantic-model, metric, and metric-query workflows.

agent:
  services:
    datasources:
      starrocks:
        type: starrocks
        host: 127.0.0.1
        port: "9030"
        username: admin
        password: ${STARROCKS_PASSWORD}
        database: ac_manage
        default: true

    semantic_layer:
      osi:
        default: true

execution_backend defaults to metricflow; set it only when you need a different OSI execution backend.

datus-agent uses the active global semantic adapter for query execution. An active osi adapter loads OSI assets; it does not enable semantic authoring.

Legacy node-level semantic_adapter and authoring_format fields are ignored.

Existing Semantic Model Format

Existing OSI projects use a strict OSI core document. The root shape is fixed:

version: 0.2.0.dev0
semantic_model:
  - name: shop
    datasets:
      - name: orders
        source: orders
        primary_key: [order_id]
        fields:
          - name: order_date
            expression:
              dialects:
                - dialect: ANSI_SQL
                  expression: order_date
            dimension:
              is_time: true
            custom_extensions:
              - vendor_name: DATUS
                data: '{"type":"time","time_granularity":"day"}'
          - name: channel
            expression:
              dialects:
                - dialect: ANSI_SQL
                  expression: channel
            dimension: {}
            description: "Order channel"

Key rules:

• Use one canonical dataset per physical table. Do not declare separate datasets for different queries or different metrics over the same table.
• Use OSI core fields, not MetricFlow dimensions.
• Dataset source is a table-name string, not {table: ...}.
• Field roles are structural. A field with a dimension: block is a grouping/filtering dimension; a field without one is a plain row-level expression that documents the column and backs metric expressions. Columns that are only aggregated by metrics (balances, amounts, precomputed rates) are declared as plain fields without the block, and get_dimensions does not list them. Field-level type hints are not part of the authoring contract.
• Physical and logical keys use different evidence. primary_key is written only when source metadata or an explicit data contract declares it. JOIN columns are candidates, not keys. Submit every candidate that will be authored in one validate_semantic_key_candidates call; a candidate may be added as one unique_keys entry only after its complete ordered column list passes the full-table check with no NULL components or duplicate groups.
• A composite primary key that contains the dataset's time dimension (monthly snapshot tables) is valid: the compiler keeps the time dimension and resolves the identifier conflict during lowering.
• Mark time fields with dimension.is_time: true; put the Datus time_granularity hint in custom_extensions.
• Declare relationships under the semantic model object, not inside datasets.

Changing a table's model

To change a table's semantic model, migrate the project to Dosi first and use semantic_modeling so the YAML and Knowledge Base stay synchronized.

Existing Metric Format

Existing OSI metrics live under semantic_model[0].metrics. Business expressions use OSI core expression; Datus execution hints live in custom_extensions.

version: 0.2.0.dev0
semantic_model:
  - name: shop
    datasets:
      - name: orders
        source: orders
        primary_key: [order_id]
        fields: [...]
    metrics:
      - name: revenue
        description: "Total paid order revenue"
        expression:
          dialects:
            - dialect: ANSI_SQL
              expression: "SUM(amount)"
        custom_extensions:
          - vendor_name: DATUS
            data: '{"dataset":"orders","time_dimension":"order_date","unit":"CNY"}'

Supported metric patterns:

Pattern	OSI authoring	Backend lowering
Basic aggregate	SUM, COUNT, COUNT(DISTINCT), AVG, MIN, MAX	MetricFlow measure_proxy
Conditional aggregate	COUNT(DISTINCT CASE WHEN ... THEN id END)	Backing measure + metric
Expression metric	Arithmetic over aggregate results	MetricFlow expr
Ratio	Aggregate division, or DATUS numerator / denominator hints	MetricFlow ratio
Rolling / cumulative	Base aggregate + DATUS window or grain_to_date	MetricFlow cumulative
Period-over-period	Derived metric + input metric offset_window	MetricFlow derived
Joined dimensions	OSI relationship + joined dimension	MetricFlow join identifier

Period-over-period and offset_window

Do not write SQL window functions directly in OSI metric expressions:

LAG(revenue) OVER (...)
ROW_NUMBER() OVER (...)

Model period-over-period metrics as a base metric plus derived metrics. For example:

metrics:
  - name: revenue
    description: "Monthly revenue"
    expression:
      dialects:
        - dialect: ANSI_SQL
          expression: "SUM(amount)"
    custom_extensions:
      - vendor_name: DATUS
        data: '{"dataset":"orders","time_dimension":"order_month"}'

  - name: revenue_previous_month
    description: "Revenue in previous month"
    expression:
      dialects:
        - dialect: ANSI_SQL
          expression: "revenue_previous_month"
    custom_extensions:
      - vendor_name: DATUS
        data: '{"metric_kind":"derived","inputs":[{"name":"revenue","alias":"revenue_previous_month","offset_window":"1 month"}]}'

  - name: revenue_mom_delta
    description: "Revenue month-over-month delta"
    expression:
      dialects:
        - dialect: ANSI_SQL
          expression: "revenue - revenue_previous_month"
    custom_extensions:
      - vendor_name: DATUS
        data: '{"metric_kind":"derived","inputs":[{"name":"revenue"},{"name":"revenue","alias":"revenue_previous_month","offset_window":"1 month"}]}'

OSI metrics describe reusable business semantics, not one-off query SQL. offset_window expresses "the same metric shifted by a prior period" as semantic metadata; the execution backend is responsible for rendering an equivalent query plan.

Multi-table Joins

Multi-table joins are represented with OSI core relationships:

relationships:
  - name: order
    from: order_items
    to: orders
    from_columns: [tenant_id, order_id]
    to_columns: [tenant_id, id]

The target dataset must declare the complete target key:

datasets:
  - name: orders
    source: orders
    unique_keys:
      - [tenant_id, id]

Scalar and composite equality relationships are supported. The two column lists must have the same length and order; every pair becomes one SQL ON predicate. to_columns must exactly match the target dataset's primary_key or one complete unique_keys entry, never a subset. Datus lowers the relationship to correlated MetricFlow identifiers on both sides.

For example, the relationship above produces the logical join:

... JOIN orders
  ON order_items.tenant_id = orders.tenant_id
 AND order_items.order_id = orders.id

The native OSI relationship name is the joined-dimension prefix:

order__status

ask_metrics discovers these queryable dimensions through list_metrics and get_dimensions.

For a scalar relationship, the shorter form remains valid:

relationships:
  - name: orders_to_customers
    from: orders
    to: customers
    from_columns: [customer_id]
    to_columns: [customer_id]

Query Validation

Datus exposes the following validation path for existing OSI assets:

1. validate_semantic(scope="semantic_model") validates a selected semantic model.
2. validate_semantic(scope="all") validates the full semantic layer.
3. query_metrics(..., dry_run=True) checks a metric query by rendering SQL.

OSI mode does not publish or mutate semantic assets. Migrate to Dosi before using semantic_modeling to change a model and reconcile it to the Knowledge Base.

ask_metrics Queries

ask_metrics uses the unified semantic adapter interface. With the OSI adapter, it still calls:

• list_metrics
• get_dimensions
• query_metrics
• validate_semantic

The OSI adapter returns Datus metadata for each metric, such as:

• dataset
• time_dimension
• metric_kind
• expr
• inputs
• offset_window
• window
• grain_to_date
• format
• unit

This metadata helps ask_metrics choose the correct metrics, dimensions, and query parameters.

SQL That Should Not Become Metrics

The legacy OSI metric format does not force these SQL patterns into metrics:

• Detail lists: SELECT col1, col2 ...
• Distinct detail lists: SELECT DISTINCT ...
• Ranking lists: ROW_NUMBER(), RANK(), or DENSE_RANK() producing row-level output
• TopN per group, such as "top N activities per channel"
• Queries whose main output is row-level records rather than aggregate metrics

These patterns may be modeled with derived datasets, materialized views, or a query layer, but they are outside the legacy OSI metric shape.

Current Limits

• The OSI adapter currently defaults to the MetricFlow execution backend.
• Relationship execution supports ordered scalar and composite equality joins. Non-equality predicates still require a pre-joined dataset or query layer.
• SQL window functions cannot be written directly in OSI metric expressions. Use offset_window for period comparisons; ranking and TopN detail queries need a query layer or precomputed dataset.
• When a semantic model has multiple datasets, each metric resolves its owning dataset from qualified column names in its expression (SUM(orders.amount)); a DATUS dataset hint is required only when the expression contains no qualified columns.
• Datus execution information outside OSI core must be encoded in custom_extensions[{vendor_name: DATUS}], not as top-level OSI fields.