Interpretation and Joins

For the physical meaning of CSI, antenna offsets, and model assumptions, read Assumptions and Limitations.

Before joining anything, keep this mental model in mind:

one orchestrator cycle means one rover pose
that pose gets one Qualisys position sample
that same pose then gets one acoustic capture
that same pose then gets one RF measurement cycle

In practice:

one row in exp-<experiment_id>-positions.csv is one rover stop
one RF JSON-line record with the same experiment_id and cycle_id belongs to that stop
in a fully populated reciprocity cycle, that means up to 42 ceiling receiver rows with the same (experiment_id, cycle_id)

In the processed NetCDF:

rover_x, rover_y, rover_z contain the joined Qualisys coordinates
position_available == 1 means the position row was considered usable
csi_real + 1j * csi_imag reconstructs the cable-corrected complex RF quantity
csi_available == 1 means a given host contributed a usable RF record for that experiment/cycle

In the processed acoustic NetCDF:

values stores the microphone waveforms on (experiment_id, cycle_id, microphone_label, sample_index)
the same (experiment_id, cycle_id) pair identifies the acoustic capture recorded at that rover pose
microphone_label selects the microphone channel inside that one capture

The Three Named Axes

Axis	Meaning	Typical question
`experiment_id`	One logical run such as `EXP003` or `EXP005`.	Which measurement campaign am I looking at?
`cycle_id`	One orchestrator loop iteration. This is the physical rover stop axis.	Which stop in the run am I selecting?
`hostname`	One RF receiver host or tile.	Which antenna/tile produced this CSI value?

This means the processed xarray is easiest to picture as:

one outer stack of experiments
inside each experiment, one matrix over cycle_id x hostname
rover coordinates attached to each cycle
CSI values attached to each (cycle_id, hostname) pair

In other words:

rover variables live on (experiment_id, cycle_id)
CSI variables live on (experiment_id, cycle_id, hostname)

Coordinates, Indexes, and Why They Matter

The named coordinates are also the xarray indexes:

Coordinate	What it stores	Why it matters
`experiment_id`	labels like `EXP003`	use `.sel(experiment_id=\"EXP003\")` instead of integer positions
`cycle_id`	orchestrator cycle labels	lets you map CSI and rover pose back to the same physical stop
`hostname`	receiver host labels such as `A05`	lets you select or mask CSI by tile name

Inspect them directly with:

ds.sizes
ds.coords
ds.indexes

The Most Important Structural Detail

cycle_id is a shared axis across the full dataset. That is convenient, but it is also the easiest place to make mistakes.

The important part is:

the dataset may list many cycle_id values
not every experiment uses every listed cycle
not every hostname is populated in every cycle

So a coordinate existing in the dataset does not guarantee that the data at that coordinate is valid. Always treat the masks as authoritative:

csi_available tells you whether CSI exists for one (experiment_id, cycle_id, hostname) location
position_available plus finite rover_x/y/z tells you whether the rover pose is usable for one (experiment_id, cycle_id) location

Make One Measurement Point Tangible

One physical RF measurement point is the pair (experiment_id, cycle_id).

That pair gives you:

one rover position: rover_x, rover_y, rover_z
one availability flag: position_available
one CSI vector across the active hostnames: csi_real, csi_imag, csi_available

Selecting one experiment, then one cycle, then one host shrinks the dataset like this:

Selection	Remaining structure
`ds`	full `(experiment_id, cycle_id, hostname)` dataset
`ds.sel(experiment_id=\"EXP003\")`	one `cycle_id x hostname` slice
`ds.sel(experiment_id=\"EXP003\", cycle_id=123)`	one rover pose plus one CSI vector over `hostname`
`...sel(hostname=\"A05\")`	one scalar CSI value for one tile

Important structural details:

cycle_id is a shared axis across the dataset, not a guarantee that every experiment has data for every listed cycle
always use csi_available to discover which cycles and hostnames are populated
always check position_available and finite rover coordinates before trusting a rover pose
duplicate rover positions may already have been filtered out before the .nc file was written

For a worked walkthrough, open:

Filter Valid RF Snapshots

Start with an opened RF dataset ds. Build separate masks for individual receiver entries and rover positions. Equality checks avoid interpreting a missing flag as true.

import numpy as np

exp = ds.sel(experiment_id="EXP003")
csi_stored = exp["csi_real"] + 1j * exp["csi_imag"]
ap_ok = (
    (exp["csi_available"] == 1)
    & np.isfinite(exp["csi_real"])
    & np.isfinite(exp["csi_imag"])
)
position_ok = (
    (exp["position_available"] == 1)
    & np.isfinite(exp["rover_x"])
    & np.isfinite(exp["rover_y"])
    & np.isfinite(exp["rover_z"])
)

Complete 42-AP snapshots

Checking .all(dim="hostname") alone is insufficient if the file itself contains fewer than 42 hosts. First verify the expected ceiling aperture, then filter cycles using both CSI and position validity:

expected_hosts = {
    f"{column}{row:02d}" for column in "ABCDEFG" for row in range(5, 11)
}
hosts = exp["hostname"].values.astype(str)
if len(hosts) != 42 or set(hosts) != expected_hosts:
    raise ValueError("Expected the 42 ceiling receivers A05–G10")

cycle_ok = position_ok & ap_ok.all(dim="hostname")
cycle_ids = exp["cycle_id"].values[cycle_ok.values]
full = exp.sel(cycle_id=cycle_ids)
y_full = csi_stored.sel(cycle_id=cycle_ids).transpose("cycle_id", "hostname")
rover_xyz_full = np.column_stack([
    full["rover_x"].values,
    full["rover_y"].values,
    full["rover_z"].values,
])
print(f"Retained {len(cycle_ids)} of {int(ap_ok.any('hostname').sum())} populated RF cycles")

For the inspected EXP003 release, this retains cycles 1–527. The other two populated cycles, 528 and 529, each lack B09. Derive the selection from masks rather than these example numbers. Coordinate-only entries elsewhere on the shared cycle axis are also excluded.

Match antenna geometry by hostname

Run the following from a notebook with the repository root on its Python import path. The existing helper reads channel 1 from the published geometry file. For reproducible analysis, retain a copy or identify the geometry revision as explained in Assumptions and Limitations.

from processing.tutorials import csi_plot_utils as csi_utils

antenna_positions = csi_utils.load_antenna_positions()


def geometry_for_hosts(hostnames, positions):
    names = [str(host).upper() for host in hostnames]
    missing = [host for host in names if host not in positions]
    if missing:
        raise ValueError(f"Missing antenna geometry: {missing}")
    xyz = np.asarray([positions[host] for host in names], dtype=float)
    if xyz.shape != (len(names), 3) or not np.isfinite(xyz).all():
        raise ValueError("Expected finite (number_of_hosts, 3) antenna coordinates")
    return xyz


ap_xyz_full = geometry_for_hosts(full["hostname"].values, antenna_positions)

Column m of y_full now corresponds to row m of ap_xyz_full. Do not independently sort or silently drop geometry rows.

A snapshot with an available AP subset

An algorithm that supports a variable number of receivers can retain partial snapshots. Select the hostnames with valid CSI first, and use the identical selection for the observations and AP positions:

cycle_id = 528
if not bool(position_ok.sel(cycle_id=cycle_id).item()):
    raise ValueError("Missing or invalid rover position")

mask = ap_ok.sel(cycle_id=cycle_id)
selected_hosts = exp["hostname"].values[mask.values]
if len(selected_hosts) == 0:
    raise ValueError("No valid RF measurements")

y_subset = csi_stored.sel(cycle_id=cycle_id, hostname=selected_hosts).values
ap_xyz_subset = geometry_for_hosts(selected_hosts, antenna_positions)
snapshot = exp.sel(cycle_id=cycle_id)
rover_xyz_subset = np.array([
    snapshot["rover_x"].item(),
    snapshot["rover_y"].item(),
    snapshot["rover_z"].item(),
])
print("Excluded hosts:", exp["hostname"].values[~mask.values].tolist())

For EXP003/cycle 528, y_subset has shape (41,) and ap_xyz_subset has shape (41, 3), with B09 excluded from both. The method still needs enough appropriately placed APs to solve its localization problem; presence alone does not establish geometric observability. Never fill missing CSI with zero to force a fixed-size vector.

These examples preserve the stored CSI convention and logged rover coordinates. Apply any model-specific conjugation, RF phase-center correction, or quality threshold explicitly after validity filtering, following Assumptions and Limitations. Masks indicate availability, not signal quality; finite values alone do not establish usable phase or a good LoS fit.