Seeing Through Walls: WiFi Human-Activity Recognition on the Versal AI Engine

My ECE Master's at Mississippi State University was in software-defined-radio (SDR) based sensing. One project asked a deceptively simple question: can an ordinary WiFi signal tell what a person is doing in a room?

The answer is yes — and it comes from Channel-State-Information (CSI). Every OFDM WiFi receiver estimates how the channel distorted each subcarrier. When a body moves, the multipath reflections change, and those changes carry a signature of the motion. I built a WiFi-CSI human-activity recognition (HAR) pipeline, benchmarked it against a radar sensing baseline for accuracy, and published the results across several papers (see Publications).

The visual heart of that work is the spectrogram: take the Short-Time Fourier Transform of the CSI time series and each activity paints a distinct time–frequency fingerprint. These are real CSI spectrograms from the dataset — one per activity class:

When I presented that thesis to a former manager, they gently suggested that the whole pipeline could, one day, run in an FPGA — the packet parsing in fabric, the heavy DSP and inference on an accelerator. It was said kindly, almost in passing. But it lodged in my head as a personal challenge I promised myself I'd take on someday.

The napkin sketch of that idea looked like this:

The idea sat on the shelf while life moved on. What finally unblocked it was hardware and time: a VCK190 on the bench and a run of weekends and evenings. Modern tooling — and, honestly, LLMs for grinding through the deep Versal tool-flow issues — made a solo build of this scope tractable.

The pipeline now runs on real hardware. A Raspberry Pi running the nexmon_csi firmware patch turns its WiFi chip into a CSI sensor and packetizes CSI as UDP over Ethernet. That stream enters the VCK190 over an SFP link; the PL parses the UDP/CSI packet and streams the I/Q samples straight into the AI Engine, which computes the motion, breathing and phase-variance features. Only the compact results cross into Linux on the Arm cores, where an on-board web dashboard renders what's happening in the room.

Why this device is the right fit:

• PL streams and parses the Ethernet/UDP packet at line rate.
• The AI Engine is a vector-VLIW array built for FFTs and MAC-heavy DSP — exactly the CSI workload.
• The NoC + DDR and Arm PS handle results and visualization.
• It's a single-chip edge sensor: no PC in the loop.

The full path from WiFi chip to browser. Raw CSI (hundreds of KB/s) never touches the CPU in the target design — only the tiny feature/label vector is DMA'd to Linux. Click any diagram to open it fullscreen and zoom / pan.

No camera, no wearable, often through a wall. The trick is that your body is part of the radio channel. Here is the intuition, in four steps.

Why STFT. Activities are non-stationary, so we take the Short-Time Fourier Transform of the CSI time series — window, DFT, magnitude — producing the time–frequency spectrograms you saw in Act 1. (This directly builds on my paper Effect of the Short-Time Fourier Transform on the Classification of Complex-Valued Mobile Signals.)

In the target architecture the PL owns the packet. RX frames enter as an AXI4-Stream; a custom csi_udp_parser (Vitis HLS) filters the CSI UDP port, strips the Ethernet/IP/UDP headers, extracts the I/Q payload, and streams samples through a csi_mux straight into the AI Engine. This is the real Vivado block design that was built and run on the board:

A simplified view of the same PL datapath, for reference:

The AI Engine runs a three-branch feature graph. Each branch maps directly to a piece of the physics above, and every kernel is a small streaming function connected window-to-window over PLIO:

The graph wiring is pure ADF — kernels connected stream-to-stream on the array:

// motion branch: FIR band-pass -> stats (kernel-to-kernel)
connect<>(in.out[0],    kfir.in[0]);
connect<>(kfir.out[0],  kstats.in[0]);
connect<>(kstats.out[0], out.in[0]);
// breathing branch: windowed-DFT magnitude
connect<>(brt_in.out[0], kdft.in[0]);
connect<>(kdft.out[0],   brt_out.in[0]);
// phase-variance branch: stats -> variance is the phase-var feature
connect<>(phase_in.out[0], kphase.in[0]);
connect<>(kphase.out[0],   phase_out.in[0]);

static inline void stats_core(const float *x, float *out) {
    float s = 0.0f;
    for (int i = 0; i < L; i++) s += x[i];
    float mean = s / (float)L;
    float sd = 0.0f, s2 = 0.0f;
    for (int i = 0; i < L; i++) {
        float d = x[i] - mean;
        sd += d * d;
        s2 += x[i] * x[i];
    }
    out[0] = mean;          // DC / bias
    out[1] = sd / (float)L; // variance  == motion-band / phase-var energy
    out[2] = s2 / (float)L; // power
}

The compact feature vector then feeds a tiny pretrained MLP for the activity label. The design does hand-written AIE kernels here; the vector-VLIW array chews through the FIR taps and DFT butterflies far faster than the scalar A72 could — which is the whole point of the offload.

Talk is cheap — here is the datapath caught in the act on a real VCK190 (board chanterelle10). Four ILA taps were inserted on the live pipeline via the Versal debug automation: ETH_RX (nexmon UDP bytes), CSI_PARSED (parser output), AIE_IN (mux → AI Engine) and AIE_OUT (AIE features → DMA).

The end-to-end numerical check, on hardware:

The VCK190 serves its own dashboard (pure-Python, standard-library only, no CDN — it runs on a bare PetaLinux rootfs). It consumes the AI Engine feature groups and derives presence, activity, breathing rate and fall alerts — and renders a live CSI spectrogram by stacking the streamed brt[33] windowed-DFT magnitude vectors over time (the same STFT view from Act 1, now updating in real time). Below is that dashboard's logic, replaying a representative feature stream right in your browser — empty room → someone walks in → sits and breathes → a fall → stillness:

The widget above is a representative feature-stream replay running the same metrics and thresholds as the on-board dashboard — it is not a live capture. On the board, these tiles update from live AI-Engine features.

The gap between "works in simulation" and "works on silicon" was where the real engineering happened. A few that cost days: