This CL adds a check during SetUp that an NNAPI driver service is still
alive by pinging the driver service. If it is not alive, the test will
fail during SetUp. Without this quick-fail, the test case would continue
as if the driver were still active, which would result in multiple
EXPECT_* and ASSERT_* statements failing instead of a single, clear
failure message.
Bug: 197035200
Test: mma
Test: presubmit: VtsHalNeuralnetworks*TargetTest
Change-Id: Ib1b75ed20f764055699590581d5ad4e5aff4baae
Do not run validation on "inputs_as_internal" and
"all_tensors_as_inputs" variants.
Bug: 138149072
Bug: 149840439
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I0699ed6703e48b6c4bc0e7a392b79c12770f04c9
Merged-In: I0699ed6703e48b6c4bc0e7a392b79c12770f04c9
(cherry picked from commit 678a10600c)
This CL removes a dependency on the VTS test runner by dynamically
discovering all NN HAL service instances in the gtest binary itself,
and runs through all service instances with parameterized tests.
This CL converts TEST_F cases to TEST_P cases, where the test parameter
is the name of the service instance. For existing TEST_P cases (such as
the generated test cases), the service instance name is made to be the
first test parameter.
This CL enables the NN VTS tests to be more portable, e.g., they can
run directly as a presubmit test.
Fixes: 124540002
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Test: cd $ANDROID_BUILD_TOP/hardware/interfaces/neuralnetworks && atest
Change-Id: I1e301d7c9f9342bb8f35a267bef180f510944b19
Merged-In: I1e301d7c9f9342bb8f35a267bef180f510944b19
(cherry picked from commit 7076f629b7)
This CL does additional NNAPI VTS test cleanup, including consolidating
duplicate functionality. Specifically, this CL:
* consolidates the createPreparedModel function, removing the duplicate
* consolidates the std::out ErrorStatus and DeviceStatus code into Utils
* changes non-null constant pointers to constant references
* removes redudant leading namespace specifiers (V1_0::, ::testing, etc.)
* makes the Valdiation tests free functions
* renames device to kDevice and mTestModel to kTestModel
Bug: N/A
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Change-Id: Ic401bb1f1760cc10384ac0d30c0c93409b63a9c7
Merged-In: Ic401bb1f1760cc10384ac0d30c0c93409b63a9c7
(cherry picked from commit e16af0a44b)
Comparing with v1.1, the converter for 1.2 HIDL model has additional support
for extraParam, dynamic output shape, and zero-sized output.
Modify CompilationCachingTests to use the new test struct.
Bug: 123092187
Bug: 138718240
Test: All VTS
Change-Id: I54ac97f62898e47a338b51cc6d895a0309ab001f
Merged-In: I54ac97f62898e47a338b51cc6d895a0309ab001f
(cherry picked from commit 491b0a8913)
To make it easier to create the next version of NNAPI, this change
removes the following nonsensical dependence:
- NNAPI 1.0 VTS depends on NNAPI 1.1 and 1.2
- NNAPI 1.1 VTS depends on NNAPI 1.2
In particular, I made the following changes:
- split GeneratedTestHarness.cpp into three separate implementations,
- created a restricted version of Callbacks.h for 1.0 and 1.1,
- removed the dependency on frameworks/ml/nn/HalInterfaces.h,
- refactored Android.bp files for more autonomy between 1.0, 1.1, and 1.2,
- consolidated some common code into Utils.h,
- created structure for sharing code between VTS versions (VtsHalNeuralNetworksV1_0_utils).
Bug: 74827824
Bug: 124462414
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_1CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_1TargetTest
Change-Id: I4243d0b5e574255cef1070850f4d0a284f65f54e
Merged-In: I4243d0b5e574255cef1070850f4d0a284f65f54e
(cherry picked from commit 1d6b465997)
We don't have a framework for handcrafting tests resulting in execution
failure. To produce this test, I created
strided_slice_invalid_output_dims.mod.py and used our VTS test generator
to create the corresponding example.cpp and model.cpp files. I put those
under the new directory neuralnetworks/1.2/vts/functional/generated to
distinguish them from the streamlined generated tests.
Bug: 79856511
Bug: 132155416
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: Ib331778605c2e3828195e1e7d1651091873985a0
Merged-In: Ib331778605c2e3828195e1e7d1651091873985a0
(cherry picked from commit 1a18c08052)
This CL adds the following two types of validation tests on the NNAPI
Burst serialized format:
(1) it directly modifies the serialized data (invalidating it) to ensure
that vendor driver services properly validates the serialized
request
(2) it ensures that vendor driver services properly fail when the result
channel is not large enough to return the data
This CL additionally includes miscellaneous cleanups:
(1) having a generic "validateEverything" function
(2) moving the "prepareModel" function that's common across
validateRequest and validateBurst to a common area
Fixes: 129779280
Bug: 129157135
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Change-Id: Ib90fe7f662824de17db5a254a8c501855e45f6bd
Merged-In: Ib90fe7f662824de17db5a254a8c501855e45f6bd
(cherry picked from commit 20f28a24e9)
This CL creates a new suite of tests to enable presubmit tests:
* PresubmitHalNeuralnetworksV1_0TargetTest
* PresubmitHalNeuralnetworksV1_1TargetTest
* PresubmitHalNeuralnetworksV1_2TargetTest
These tests are the same as the VTS tests, with the exception that they
will skip running all tests (but still pass) if the service cannot be
found and its name starts with "service-".
This change does not affect the existing NNAPI VTS tests.
Test: mma
Test: atest
Bug: 124040554
Change-Id: I36a38b66b21fd51d0ca381bb4e05a39266dd353f
(cherry picked from commit ed68233697)
Argument-dependent lookup will only work for operator>> if the operator
is in one of the argument's namespaces. This caused the enumerations to
pretty-print for V1_0, but not for V1_1 or V1_2. This change ensures the
V1_0 namespace is used.
Test: mma
Test: atest VtsHalNeuralnetworksV1_0TargetTest (verified the test output "OFFLINE" for DeviceStatus and "DEVICE_UNAVAILABLE" for ErrorStatus instead of raw byte value representation)
Test: atest VtsHalNeuralnetworksV1_1TargetTest (verified the test output "OFFLINE" for DeviceStatus and "DEVICE_UNAVAILABLE" for ErrorStatus instead of raw byte value representation)
Test: atest VtsHalNeuralnetworksV1_2TargetTest (verified ran and passed tests)
Fixes: 124316129
Change-Id: I764a46e2d87615b1f3da0ab0e6edb134bb533887
(cherry picked from commit 42a35bee10)
Create 1.2 version IPreparedModel, IPreparedModelCallback, and
IExecutionCallback.
Currently the new interfaces are created the same as 1.0 version,
but will have more methods introduced in later CLs.
Bug: 73506513
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Change-Id: Icf4d04c22f88e825d87562f1489377fdf6bf585d
Merged-In: Icf4d04c22f88e825d87562f1489377fdf6bf585d
(cherry picked from commit b5cb8f7632)
This is a copy the v1.1 tests since we don't have any new ops
implemented in v1.2 yet.
Bug: 114365802
Test: mm
Test: NNAPI VTS
Change-Id: Ida7525fcd3ae0fd6f88ff9591e06aba922bdae64
Merged-In: Ida7525fcd3ae0fd6f88ff9591e06aba922bdae64
(cherry-picked from 871be94770)
