This CL also cleans up the validation framework code.
Bug: 67828197
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I84661fb2b8204148788d10425ca0ac986158b15f
The frameworks/ml/nn CL "Improve performance of Burst executions"
changed the signature of a utility function used by NNAPI's VTS tests.
This CL modifies the test accordingly.
Bug: 132073143
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Test: VtsHalNeuralnetworksV1_3TargetTest (with sample-all)
Change-Id: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
This CL removes a dependency on the VTS test runner by dynamically
discovering all NN HAL service instances in the gtest binary itself,
and runs through all service instances with parameterized tests.
This CL converts TEST_F cases to TEST_P cases, where the test parameter
is the name of the service instance. For existing TEST_P cases (such as
the generated test cases), the service instance name is made to be the
first test parameter.
This CL enables the NN VTS tests to be more portable, e.g., they can
run directly as a presubmit test.
Fixes: 124540002
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Test: cd $ANDROID_BUILD_TOP/hardware/interfaces/neuralnetworks && atest
Change-Id: I1e301d7c9f9342bb8f35a267bef180f510944b19
This CL does additional NNAPI VTS test cleanup, including consolidating
duplicate functionality. Specifically, this CL:
* consolidates the createPreparedModel function, removing the duplicate
* consolidates the std::out ErrorStatus and DeviceStatus code into Utils
* changes non-null constant pointers to constant references
* removes redudant leading namespace specifiers (V1_0::, ::testing, etc.)
* makes the Valdiation tests free functions
* renames device to kDevice and mTestModel to kTestModel
Bug: N/A
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Change-Id: Ic401bb1f1760cc10384ac0d30c0c93409b63a9c7
Comparing with v1.1, the converter for 1.2 HIDL model has additional support
for extraParam, dynamic output shape, and zero-sized output.
Modify CompilationCachingTests to use the new test struct.
Bug: 123092187
Bug: 138718240
Test: All VTS
Change-Id: I54ac97f62898e47a338b51cc6d895a0309ab001f
Prior to this CL, HalInterfaces.h polluted the global namespace through
the "using" declarations. To fix b/72880287, those names were put in a
new ::android::nn::hal namespace. This CL fixes the compilation errors
that occur due to this cleanup.
Bug: 72880287
Test: mma
Change-Id: Idb228e37615af7c0343a09f9a51de378e63bdb5a
To make it easier to create the next version of NNAPI, this change
removes the following nonsensical dependence:
- NNAPI 1.0 VTS depends on NNAPI 1.1 and 1.2
- NNAPI 1.1 VTS depends on NNAPI 1.2
In particular, I made the following changes:
- split GeneratedTestHarness.cpp into three separate implementations,
- created a restricted version of Callbacks.h for 1.0 and 1.1,
- removed the dependency on frameworks/ml/nn/HalInterfaces.h,
- refactored Android.bp files for more autonomy between 1.0, 1.1, and 1.2,
- consolidated some common code into Utils.h,
- created structure for sharing code between VTS versions (VtsHalNeuralNetworksV1_0_utils).
Bug: 74827824
Bug: 124462414
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_1CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_1TargetTest
Change-Id: I4243d0b5e574255cef1070850f4d0a284f65f54e
We don't have a framework for handcrafting tests resulting in execution
failure. To produce this test, I created
strided_slice_invalid_output_dims.mod.py and used our VTS test generator
to create the corresponding example.cpp and model.cpp files. I put those
under the new directory neuralnetworks/1.2/vts/functional/generated to
distinguish them from the streamlined generated tests.
Bug: 79856511
Bug: 132155416
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: Ib331778605c2e3828195e1e7d1651091873985a0
This CL adds the following two types of validation tests on the NNAPI
Burst serialized format:
(1) it directly modifies the serialized data (invalidating it) to ensure
that vendor driver services properly validates the serialized
request
(2) it ensures that vendor driver services properly fail when the result
channel is not large enough to return the data
This CL additionally includes miscellaneous cleanups:
(1) having a generic "validateEverything" function
(2) moving the "prepareModel" function that's common across
validateRequest and validateBurst to a common area
Fixes: 129779280
Bug: 129157135
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Change-Id: Ib90fe7f662824de17db5a254a8c501855e45f6bd
This CL adapts the VTS code to the corresponding changes made in the NN
utility library.
Bug: 128319484
Test: mma
Test: atest VtsHalNeuralnetworksV1_0Target
Test: atest VtsHalNeuralnetworksV1_1Target
Test: atest VtsHalNeuralnetworksV1_2Target
Change-Id: I470e8228cde2b75620ad851e4fe408f8e8329e7c
This CL adapts the VTS code to the corresponding changes made in the NN
utility library.
Bug: 119570067
Test: mma
Test: atest VtsHalNeuralnetworksV1_0Target
Test: atest VtsHalNeuralnetworksV1_1Target
Test: atest VtsHalNeuralnetworksV1_2Target
Change-Id: I7cbc1d7025c0352aa1ed29d71dc84c2fcfc20a4f
- Instead of isCachingSupport returning a single boolean, switch to
getNumberOfCacheFilesNeeded returning the number of cache files. This
is to support use cases when driver needs more than one cache file for
each type, or when driver does not need data cache.
- Instead of a separate saveToCache, pass cache info along with
prepareModel_1_2 to save into cache as well as perform compilation.
This is to avoid a potential additional copy of cache files.
Bug: 123780248
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Test: VtsHalNeuralnetworksV1_xTargetTest with a test driver that can
read and write cache entries
Change-Id: I921b7b8ccc3c66af19f6589f7213c6870d6f07bf
FastMessageQueue is a Treble-compliant data structure that enables fast
communication between two processes. The FMQ object itself is an atomic
circular buffer that is optionally synchronized with a futex. However,
FMQ has no notion of ownership or lifetime across processes, so it must
be paired with higher-level constructs to manage the lifetime and
ownership.
The NNAPI is introducing the notion of an "Execution Burst" object (or
more simply a "Burst" object), which is similar to an
ANeuralNetworksExecution, but is intended to be reused across multiple
executions and has lower IPC overheads. It achieves this low IPC
overhead by replacing HIDL HwBinder calls with FMQ messages.
Specifically, it replaces IPreparedModel::executeSynchronously's call
from the client into the service with fmq_sync<FmqRequestDatum> (an FMQ
channel used to pass a serialized Request object) and it replaces
the return from the service into the client with
fmq_sync<FmqResultDatum> (an FMQ channel used to return serialized
result status and OutputShapes information).
Each channel is a unidirectional flow of information with exactly one
producer and exactly one consumer. The channels are created by the NN
runtime and passed to the service via
IPreparedModel::configureExecutionBurst.
This CL tests the Burst in both the execution path and validation path
in the Vendor Test Suite (VTS) in neuralnetworks/1.*/vts/functional/.
The VTS binary--VtsHalNeuralnetworksV1_2TargetTest--can be built and run
as any previous version could.
Bug: 119570067
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I3a36484eff9565c2d028c07c099804a0289f294a
Bug: 119274127
Test: all of the following, with the appropriate android.hardware.neuralnetworks@1.${X}::IDevice/sample-all
VtsHalNeuralnetworksV1_0TargetTest
VtsHalNeuralnetworksV1_0TargetTest
VtsHalNeuralnetworksV1_1CompatV1_0TargetTest
VtsHalNeuralnetworksV1_1CompatV1_0TargetTest
VtsHalNeuralnetworksV1_1TargetTest
VtsHalNeuralnetworksV1_1TargetTest
VtsHalNeuralnetworksV1_2CompatV1_0TargetTest
VtsHalNeuralnetworksV1_2CompatV1_0TargetTest
VtsHalNeuralnetworksV1_2CompatV1_1TargetTest
VtsHalNeuralnetworksV1_2CompatV1_1TargetTest
VtsHalNeuralnetworksV1_2TargetTest
VtsHalNeuralnetworksV1_2TargetTest
Change-Id: Iedfa485b4008d9cec3b81ff4c0ce3ebc0b83c823
Create 1.2 version IPreparedModel, IPreparedModelCallback, and
IExecutionCallback.
Currently the new interfaces are created the same as 1.0 version,
but will have more methods introduced in later CLs.
Bug: 73506513
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Change-Id: Icf4d04c22f88e825d87562f1489377fdf6bf585d
This is a copy the v1.1 tests since we don't have any new ops
implemented in v1.2 yet.
Bug: 114365802
Test: mm
Test: NNAPI VTS
Change-Id: Ida7525fcd3ae0fd6f88ff9591e06aba922bdae64
