Datamover Test by CycyXX · Pull Request #168 · pulp-platform/pulp-sdk

CycyXX · 2026-04-21T10:28:11Z

Added test suite for the Datamover GVSoC model on the Siracusa platform.

Copilot

Pull request overview

Adds a Datamover validation test suite for the Siracusa/GVSoC environment, including a C test binary, a Python golden-model generator, and a shell runner that iterates many configurations from a CSV.

Changes:

Introduces a Datamover test application (src/datamover_test.c) plus Konark HAL headers/sources to drive the HWPE Datamover.
Adds a Python golden model (datamover_golden_model.py) and a CSV matrix of configurations (datamover_tests.csv) to generate/verify expected outputs.
Adds a runner script (run_datamover_tests.sh), gvtest testset config, documentation, and a large set of pre-generated fixture headers.

Reviewed changes

Copilot reviewed 45 out of 146 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
tests/datamover_test/testset.cfg	gvtest integration for running the test(s).
tests/datamover_test/src/datamover_test.c	Main Datamover test binary: triggers operations and verifies output.
tests/datamover_test/run_datamover_tests.sh	Batch runner to generate golden data, build, run, and check logs.
tests/datamover_test/konark/hal_hwpe.h	HWPE register offsets + access macros and driver prototypes.
tests/datamover_test/konark/hal_hwpe.c	HWPE low-level register driver implementation.
tests/datamover_test/konark/hal_datamover.h	Datamover register definitions and HAL API prototypes.
tests/datamover_test/datamover_tests.csv	Configuration matrix for datamover mode/dimensions.
tests/datamover_test/datamover_golden_model.py	Golden model generator + header writer.
tests/datamover_test/data/pre-generated_tests/data_9x9_transp1.h	Pre-generated fixture header (transpose case).
tests/datamover_test/data/pre-generated_tests/data_9x8_transp1.h	Pre-generated fixture header (transpose case).
tests/datamover_test/data/pre-generated_tests/data_8x8_transp2.h	Pre-generated fixture header (transpose case).
tests/datamover_test/data/pre-generated_tests/data_8x8_transp1.h	Pre-generated fixture header (transpose case).
tests/datamover_test/data/pre-generated_tests/data_8x8_copy_count.h	Pre-generated fixture header (copy/counting).
tests/datamover_test/data/pre-generated_tests/data_8x8_copy.h	Pre-generated fixture header (copy).
tests/datamover_test/data/pre-generated_tests/data_8x7_copy_RA56.h	Pre-generated fixture header (copy with base addrs).
tests/datamover_test/data/pre-generated_tests/data_8x7_copy.h	Pre-generated fixture header (copy).
tests/datamover_test/data/pre-generated_tests/data_8x16_copy_count.h	Pre-generated fixture header (copy/counting).
tests/datamover_test/data/pre-generated_tests/data_8x16_copy.h	Pre-generated fixture header (copy with base addrs).
tests/datamover_test/data/pre-generated_tests/data_8x12_copy.h	Pre-generated fixture header (copy with base addrs).
tests/datamover_test/data/pre-generated_tests/data_8x11_copy_count.h	Pre-generated fixture header (copy/counting).
tests/datamover_test/data/pre-generated_tests/data_8x11_copy.h	Pre-generated fixture header (copy).
tests/datamover_test/data/pre-generated_tests/data_7x7_copy_count.h	Pre-generated fixture header (copy/counting).
tests/datamover_test/data/pre-generated_tests/data_7x7_copy.h	Pre-generated fixture header (copy).
tests/datamover_test/data/pre-generated_tests/data_5x5_copy.h	Pre-generated fixture header (copy).
tests/datamover_test/data/pre-generated_tests/data_5x2_copy.h	Pre-generated fixture header (copy).
tests/datamover_test/data/pre-generated_tests/data_4x65_cim64.h	Pre-generated fixture header (CIM-layout).
tests/datamover_test/data/pre-generated_tests/data_4x2x32_unfold4.h	Pre-generated fixture header (unfold).
tests/datamover_test/data/pre-generated_tests/data_4x28_copy_count.h	Pre-generated fixture header (copy/counting).
tests/datamover_test/data/pre-generated_tests/data_3x3_transp1.h	Pre-generated fixture header (transpose).
tests/datamover_test/data/pre-generated_tests/data_3x3_copy_count.h	Pre-generated fixture header (copy/counting).
tests/datamover_test/data/pre-generated_tests/data_2x2x128_unfold4.h	Pre-generated fixture header (unfold).
tests/datamover_test/data/pre-generated_tests/data_21x19_copy_count.h	Pre-generated fixture header (copy/counting).
tests/datamover_test/data/pre-generated_tests/data_19x21_copy_count.h	Pre-generated fixture header (copy/counting).
tests/datamover_test/data/pre-generated_tests/data_19x21_copy.h	Pre-generated fixture header (copy).
tests/datamover_test/data/pre-generated_tests/data_17x19_copy_count.h	Pre-generated fixture header (copy/counting).
tests/datamover_test/data/pre-generated_tests/data_17x17_copy.h	Pre-generated fixture header (copy).
tests/datamover_test/data/pre-generated_tests/data_16x5_copy.h	Pre-generated fixture header (copy).
tests/datamover_test/data/pre-generated_tests/data_16x16_copy_rand.h	Pre-generated fixture header (copy/random).
tests/datamover_test/data/pre-generated_tests/data_16x16_copy.h	Pre-generated fixture header (copy).
tests/datamover_test/data/pre-generated_tests/data_16x16_cim8_transp1.h	Pre-generated fixture header (CIM+transpose).
tests/datamover_test/data/pre-generated_tests/data_16x16_cim16_transp1_rand.h	Pre-generated fixture header (CIM+transpose/random).
tests/datamover_test/data/pre-generated_tests/data_16x16_cim16_transp1.h	Pre-generated fixture header (CIM+transpose).
tests/datamover_test/data/pre-generated_tests/data_15x15_copy.h	Pre-generated fixture header (copy).
tests/datamover_test/data/pre-generated_tests/data_14x14_copy_count.h	Pre-generated fixture header (copy/counting).
tests/datamover_test/data/pre-generated_tests/data_14x14_copy.h	Pre-generated fixture header (copy).
tests/datamover_test/data/pre-generated_tests/data_11x11_copy.h	Pre-generated fixture header (copy/counting).
tests/datamover_test/README.md	Documentation for running and extending the test suite.
tests/datamover_test/Makefile	Build definition for the Datamover test app + HAL sources.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-21T10:33:28Z

+  0xd5, 0x1c, 0xe8, 0x7e, 0x90, 0x3a, 0x9d, 0x40, 0x9d, 0xfa, 0xbd, 0x39, 0x72, 0xb6, 0x78, 0x3c,
+  0x83, 0x6d, 0x1a, 0x8c, 0xd1, 0x80, 0xe1, 0x0b, 0xbb, 0xe9, 0x1b, 0x6d, 0xa9, 0x82, 0x03, 0x6d,
+  0xf5, 0x12, 0xc3, 0x7c, 0xb0, 0x45, 0x55, 0xb5, 0x88, 0xee, 0x03, 0x31, 0x6b, 0xb2, 0x06, 0x74,//ERROR 1: 0x7406b26b3103ee88 (golden) != 0xfca7b67f716fffe8 (actual) @ 10000288 (31)
+  0xe8, 0xd1, 0x6c, 0x50, 0x7f, 0xa4, 0xa1, 0xa8, 0x32, 0x53, 0x9a, 0xa2, 0x2e, 0x8b, 0xfb, 0xba,
+  0xa6, 0x7c, 0x7d, 0x3e, 0xea, 0xd2, 0xb3, 0xd8, 0x47, 0x76, 0xab, 0x1d, 0x77, 0x0b, 0x2f, 0x80,
+  0xe8, 0x6d, 0xdc, 0x98, 0x4e, 0x4a, 0xb1, 0x02, 0x9e, 0xc1, 0x15, 0x5f, 0xc8, 0xcf, 0xf2, 0x8c,
+  0x98, 0xbd, 0x51, 0x41, 0x78, 0x05, 0x52, 0xa4, 0x3d, 0x1f, 0x88, 0x37, 0xc4, 0x9f, 0x69, 0xaa,//ERROR 2: 0xaa699fc437881f3d (golden) != 0xfe7fffc4ffdc9f7d (actual) @ 100002C8 (39)
+  0x50, 0x8b, 0xdc, 0xec, 0x84, 0xfa, 0x37, 0xf6, 0x79, 0xaf, 0xba, 0x1a, 0x99, 0x7b, 0xeb, 0x8e,


This pre-generated header includes embedded debug artifacts like //ERROR 1: ... (golden) != ... (actual) inside the data initializer. Even if it still compiles, it’s confusing and looks like accidentally committed failure logs. Either regenerate/clean these fixtures or drop them if the test suite is meant to generate data.h on-the-fly.

Copilot · 2026-04-21T10:33:28Z

+def check_output(test, output, is_cluster):
+
+    if is_cluster:
+        if output.find('Hello from cluster_id: 0, core_id: 0') == -1:
+            return (False, "Didn't find output string\n")
+    else:
+        if output.find('Hello from FC') == -1:
+            return (False, "Didn't find output string\n")
+
+    return (True, None)
+
+
+# Called by gvtest to declare the tests
+def testset_build(testset):
+
+    #
+    # Test list decription
+    #
+    testset.new_make_test('hello:fc', flags='build_dir_ext=fc', checker=lambda test, output: check_output(test, output, False))
+    testset.new_make_test('hello:cl', flags='build_dir_ext=cl USE_CLUSTER=1', checker=lambda test, output: check_output(test, output, True))


testset_build registers tests named hello:fc / hello:cl and check_output looks for "Hello from ..." strings. This doesn’t match the Datamover test binary/logs (which print [DM-OK] and Test success), so gvtest will either run the wrong target or always fail its checker. Update the gvtest entries to build/run the datamover test (or rename the suite accordingly) and validate against the datamover success markers.

Copilot · 2026-04-21T10:33:28Z

+    # Test list decription
+    #


Typo in comment: "Test list decription" → "Test list description".

Copilot · 2026-04-21T10:33:29Z

+static inline int datamover_compare_int (uint64_t *actual, uint64_t *golden, int total_elements) {
+  int errors = 0;
+  int len = (int)(total_elements / 8);  // Number of complete uint64_t words
+  int remaining_elements = total_elements % 8;
+  printf("[DM-INFO] Comparing %d 64b-words + %d 8b-elements:\n", len, remaining_elements);
+  printf("[DM-INFO] actual address: %p, golden address: %p\n", (void *)actual, (void *)golden);
+  for (int i=0; i<len; i++) {
+    uint64_t actual_ = *(actual+i);
+    uint64_t golden_ = *(golden+i);
+    // printf("  0x%" PRIx64 " (golden) -- 0x%" PRIx64 " (actual) @ %p (%d)\n", golden_, actual_, (void *)(actual+i), i);
+    if (actual_ != golden_) {
+      printf("ERROR! 0x%" PRIx64 " (golden) @ %p != 0x%" PRIx64 " (actual) @ %p (%d)\n", golden_, (void *)(golden+i), actual_, (void *)(actual+i), i);
+      errors ++;


datamover_compare_int casts uint8_t* buffers to uint64_t* and dereferences them. local_out is aligned, but golden_out / golden_in are generated as uint8_t[] without an alignment guarantee, so this can trigger misaligned 64-bit loads (undefined behavior / potential trap on some RISC-V configs). Compare byte-wise, or ensure the generated arrays are at least 8-byte aligned (and keep the compare implementation consistent with that).

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 10 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-27T13:34:27Z

+static struct pi_cluster_task task[1];
+static struct pi_task events[1];
+
+static void pe_entry(void *arg) {
+  if(pi_core_id() == 0) {
+    glob_errors = run_test();
+  }
+  pi_cl_team_barrier();
+}
+
+static void cluster_entry(void *arg) {
+  pi_cl_team_fork(0, pe_entry, 0);
+}
+
+static int launch_cluster_task() {
+  struct pi_device cluster_dev;
+  struct pi_cluster_conf conf;
+  struct pi_cluster_task task;
+


task and events global arrays are never used, and launch_cluster_task() declares a local struct pi_cluster_task task that shadows the global task[1]. This causes warnings and confusion; remove the unused globals or reuse them consistently (and avoid shadowing).

Copilot · 2026-04-27T13:34:29Z

+  // NOTE: Only supports row_tile_size = BANDWIDTH_ELEMS for now
+  int acq_to = 1000000;
+  int job_id = -1;
+


datamover_cim_layout_reverse() also states it only supports row_tile_size == DATAMOVER_BANDWIDTH_ELEMS but does not enforce it. Add the same input validation here to avoid silent misconfiguration when called with other tile sizes.

Suggested change

if (row_tile_size != DATAMOVER_BANDWIDTH_ELEMS) {

printf("datamover_cim_layout_reverse only supports row_tile_size == DATAMOVER_BANDWIDTH_ELEMS (%u), got %u\n",

DATAMOVER_BANDWIDTH_ELEMS, row_tile_size);

return DATAMOVER_TO;

}

Copilot · 2026-04-27T13:34:30Z

+  // NOTE: Only supports row_tile_size = BANDWIDTH_ELEMS for now
+  int acq_to = 1000000;
+  int job_id = -1;
+


datamover_cim_layout() documents that only row_tile_size == DATAMOVER_BANDWIDTH_ELEMS is supported, but the function does not validate this and will silently misconfigure the HW if another value is passed. Add an explicit check (and return DATAMOVER_ERR) when row_tile_size is unsupported.

Suggested change

if (row_tile_size != DATAMOVER_BANDWIDTH_ELEMS) {

return DATAMOVER_ERR;

}

Copilot · 2026-04-27T13:34:30Z

+
+datamover_status_t datamover_wait_done(uint64_t timeout) {
+  int status = 0;
+  int finished = 0;


datamover_wait_done() declares finished but no longer uses it (the FINISHED polling is commented out). This triggers an unused-variable warning; remove finished (and any other dead locals) or re-enable the FINISHED-based logic behind a compile-time option.

Suggested change

int finished = 0;

…p(1,2,4) and CIM layout conversion, updated datamover_test and HAL for HWPE and Konark, added automated test script run_datamover_tests.sh

…de and removed pre-generated test data [Datamover Test] Implemented multi-byte transpose modes in golden model

CycyXX requested a review from arpansur April 21, 2026 10:28

CycyXX self-assigned this Apr 21, 2026

Copilot AI review requested due to automatic review settings April 21, 2026 10:28

Copilot started reviewing on behalf of CycyXX April 21, 2026 10:28 View session

Copilot AI reviewed Apr 21, 2026

View reviewed changes

CycyXX force-pushed the cdurrer/datamover branch 2 times, most recently from c8b90c7 to 447d608 Compare April 27, 2026 13:21

CycyXX requested a review from Copilot April 27, 2026 13:27

Copilot started reviewing on behalf of CycyXX April 27, 2026 13:28 View session

Copilot AI reviewed Apr 27, 2026

View reviewed changes

CycyXX force-pushed the cdurrer/datamover branch 3 times, most recently from eb00244 to 1bf1fda Compare April 28, 2026 11:01

CycyXX added 3 commits April 28, 2026 13:02

[Datamover Test] Added datamover_test and stimuli to test copy, trans…

fd4f487

…p(1,2,4) and CIM layout conversion, updated datamover_test and HAL for HWPE and Konark, added automated test script run_datamover_tests.sh

[Datamover Test] Implemented unfold and fold modes (MobileViT)

ddeaecd

[Ðatamover Test] Fixed wrong comments and minor issues, cleaned up co…

dead116

…de and removed pre-generated test data [Datamover Test] Implemented multi-byte transpose modes in golden model

CycyXX force-pushed the cdurrer/datamover branch from 1bf1fda to dead116 Compare April 28, 2026 11:02

+  if (row_tile_size != DATAMOVER_BANDWIDTH_ELEMS) {
+    printf("datamover_cim_layout_reverse only supports row_tile_size == DATAMOVER_BANDWIDTH_ELEMS (%u), got %u\n",
+           DATAMOVER_BANDWIDTH_ELEMS, row_tile_size);
+    return DATAMOVER_TO;
+  }

Conversation

CycyXX commented Apr 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants