Commit Graph

33 Commits

Author SHA1 Message Date
Michael Butler
da1a692880 NNAPI VTS: Add validation for Priority
This CL also cleans up the validation framework code.

Bug: 67828197
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I84661fb2b8204148788d10425ca0ac986158b15f
2020-03-12 11:29:08 -07:00
Michael Butler
e16af0a44b Consolidate NNAPI VTS utility code
This CL does additional NNAPI VTS test cleanup, including consolidating
duplicate functionality. Specifically, this CL:
* consolidates the createPreparedModel function, removing the duplicate
* consolidates the std::out ErrorStatus and DeviceStatus code into Utils
* changes non-null constant pointers to constant references
* removes redudant leading namespace specifiers (V1_0::, ::testing, etc.)
* makes the Valdiation tests free functions
* renames device to kDevice and mTestModel to kTestModel

Bug: N/A
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Change-Id: Ic401bb1f1760cc10384ac0d30c0c93409b63a9c7
2019-08-30 14:01:38 -07:00
Michael Butler
bbe5dad266 Cleanup NNAPI VTS tests
This CL includes the following cleanups:
* namespace compression
* remove "using" from header files
* remove no-op code, default no-op constructors
* clang-formats the code

Bug: N/A
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I023997d8686ca65223858eed3a0881f5444ed2d6
2019-08-28 17:29:40 -07:00
Xusong Wang
9e2b97b83c Update VTS tests with the new test harness.
Bug: 120601396
Test: All VTS
Change-Id: I539e75585b2cc01d153565814491361adfa048be
2019-08-26 14:15:15 -07:00
Slava Shklyaev
1d6b465997 Refactor NNAPI VTS to remove unreasonable dependence between versions
To make it easier to create the next version of NNAPI, this change
removes the following nonsensical dependence:
- NNAPI 1.0 VTS depends on NNAPI 1.1 and 1.2
- NNAPI 1.1 VTS depends on NNAPI 1.2

In particular, I made the following changes:
- split GeneratedTestHarness.cpp into three separate implementations,
- created a restricted version of Callbacks.h for 1.0 and 1.1,
- removed the dependency on frameworks/ml/nn/HalInterfaces.h,
- refactored Android.bp files for more autonomy between 1.0, 1.1, and 1.2,
- consolidated some common code into Utils.h,
- created structure for sharing code between VTS versions (VtsHalNeuralNetworksV1_0_utils).

Bug: 74827824
Bug: 124462414
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_1CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_1TargetTest
Change-Id: I4243d0b5e574255cef1070850f4d0a284f65f54e
2019-07-03 15:58:08 +01:00
Viet Dang
a8f33f7dbf BidirectionalSequenceLSTM op: Fixes VTS test for merge_outputs
Bug: 123644584

Test: VtsHalNeuralnetworksV1_xTargetTest
Change-Id: I8829fd47094ca4af05e5d60da917499ce61acc5e
2019-03-29 10:57:43 +00:00
Xusong Wang
b61ba1ed0b Fix caching interface according to vendor feedback.
- Instead of isCachingSupport returning a single boolean, switch to
  getNumberOfCacheFilesNeeded returning the number of cache files. This
  is to support use cases when driver needs more than one cache file for
  each type, or when driver does not need data cache.

- Instead of a separate saveToCache, pass cache info along with
  prepareModel_1_2 to save into cache as well as perform compilation.
  This is to avoid a potential additional copy of cache files.

Bug: 123780248
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Test: VtsHalNeuralnetworksV1_xTargetTest with a test driver that can
      read and write cache entries
Change-Id: I921b7b8ccc3c66af19f6589f7213c6870d6f07bf
2019-03-21 14:17:08 -07:00
Xusong Wang
8804423ca1 Fix condition in mutateOperationOperandTypeSkip for conv ops.
The filter tensor is not always operand 1 if there are multiple
operations in the model.

Test: 1.2 VTS tests with sample driver
Change-Id: I8925dfd18072ae5579657e8aa13128e7d9f58324
2019-03-13 16:27:05 -07:00
Lev Proleev
923b8c5842 Fix VTS tests.
* Adds a specification of invalid scale and zero point for TENSOR_BOOL8.
This fixes vts failures for comparison ops.
* Removes (FUNDAMENTAL_MIN - 1) from invalid OperationTypes.
FUNDAMENTAL_MIN is equal to 0 and resulting -1 was statically casted to
uint32_t and passed 4294967295 as an invalid OperationType. However, our
validateOperation function interpreted this ID as an extension ID and
didn't fail.
* Adds mutateOperationOperandTypeSkip for QUANTIZE and DEQUANTIZE.
* Adds removeOperandSkip for BIDIRECTIONAL_SEQUENCE_RNN.

Fix: 121130841
Fix: 123247345
Test: VtsHalNeuralnetworksV1_2TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.2::IDevice/sample-all
Change-Id: Iefb502c6b9301d5470eb4cdaa46d398f1a0e512a
2019-01-31 14:00:31 +00:00
Hervé Guihot
bae91697b5 Added TENSOR_QUANT8_SYMM type.
Test: none
Change-Id: I02fc8698b3f80e1ae2a318e5cde593c6d7222bac
2019-01-29 11:45:17 -08:00
Slava Shklyaev
794703db84 Shorten NNAPI enum range names
Test: mma
Change-Id: I5138589acc127336e8c0e72ca7d6b49532a4eeb9
2019-01-28 21:30:40 +00:00
Lev Proleev
1509a2664c mutateOperationOperandTypeSkip for new TRANSPOSE_CONV_2D variant
Bug: 119255406
Test: Vts NNAPI tests.
Change-Id: I63f7fc67cedcc2a9db8c359ff96be3c9403b9a20
2019-01-22 15:20:13 +00:00
Lev Proleev
b0762ccb2f mutateOperationOperandTypeSkip for new GROUPED_CONV_2D variant
Bug: 119255406
Test: Vts NNAPI tests.
Change-Id: I234727f5abba546c1392f7096b7d411d36566bbf
2019-01-21 11:37:28 +00:00
Xusong Wang
d49f665c42 Add new OperandType TENSOR_QUANT16_ASYMM.
Test: NeuralNetworksTest_static
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I8fcd6b30c32f8fbc181d2b43f9ac0b94fdc57e2f
2019-01-18 14:50:17 -08:00
Xusong Wang
a316581b21 Test dynamic output shape in GeneratedTests.
Enable VTS unit test for dynamic output shape deduction.

Only test dynamic output shape for V1_2::IDevice with V1_2::Model.

Bug: 73506513
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Change-Id: I4134e1ec54a15554eb8533134897279651b57da3
2019-01-17 18:39:08 -08:00
Przemyslaw Szczepaniak
47b9141609 mutateOperationOperandTypeSkip for new DEPTHWISE_CONV_2D variant.
Bug: 119255406
Test: Vts NNAPI tests.
Change-Id: Iaeb1ef1095dde51e9324f526a2126ed5d0491586
2018-12-21 11:40:10 +00:00
Przemyslaw Szczepaniak
0c157d30f2 Merge "mutateOperationOperandTypeSkip for new CONV_2D variant." 2018-12-14 16:12:16 +00:00
Xusong Wang
a3aa40af3e Merge "Add 1.2 NN HAL: IPreparedModel & callbacks." 2018-12-13 20:55:48 +00:00
Przemyslaw Szczepaniak
f54f12634c mutateOperationOperandTypeSkip for new CONV_2D variant.
Bug: 119255406
Test: Vts NNAPI tests.
Change-Id: I265ae4968f5fe3589d79e5846a7d640a735a012f
2018-12-12 12:54:06 +00:00
Xusong Wang
b5cb8f7632 Add 1.2 NN HAL: IPreparedModel & callbacks.
Create 1.2 version IPreparedModel, IPreparedModelCallback, and
IExecutionCallback.

Currently the new interfaces are created the same as 1.0 version,
but will have more methods introduced in later CLs.

Bug: 73506513
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Change-Id: Icf4d04c22f88e825d87562f1489377fdf6bf585d
2018-12-11 17:33:15 -08:00
Przemyslaw Szczepaniak
faa59b8a2c Add TENSOR_QUANT8_SYMM_PER_CHANNEL to operand types
Added ExtraParams union for extra Operand parameters.
It's a more sensible approach than adding more fields
to the Operand struct.

Bug: 119249581
Test: NeuralNetworksTest_static
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I59731134cf0ea34cf9e10342686d331da9e9c3b3
2018-12-10 13:08:03 +00:00
Xusong Wang
2256f080d4 Merge "Add new OperandType FLOAT16." 2018-12-08 02:49:25 +00:00
Xusong Wang
7bca34bee7 Add new OperandType FLOAT16.
Bug: 120225191
Test: NeuralNetworksTest_static
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: Iaabebe490303c9c93a34d319c8c594df13fb7899
2018-12-06 16:32:28 -08:00
Michael K. Sanders
5b2615be21 Adds float16 support for RANDOM_MULTINOMIAL.
Bug: 118607845
Test: VtsHalNeuralnetworksV1_2TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.2::IDevice/sample-all
Change-Id: Iac8c0c23a8219702697a7a126e5e2e43ef0461f4
2018-12-06 12:35:10 +00:00
Michael K. Sanders
bbdab2feee Fixes VTS test failures.
Moves comments for type range values into their enums to make it harder
to forget to update them.

Adds missing float16 types for ARGMAX/ARGMIN/CAST.

Test: VtsHalNeuralnetworksV1_2TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.2::IDevice/sample-all
Test: [  PASSED  ] 2519 tests.

Change-Id: Ic7c3df8c8fbff45fe497f304b6b2c7a09e7dc5a6
2018-12-03 17:08:45 +00:00
Lev Proleev
48c8820bac Replace TENSOR_QUANT16_ASYMM with TENSOR_QUANT16_SYMM
* Update doc string
* Update zero point mutation to check for symmetric quantization

Fix: 118671831
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: Id1999c793c839b892cfe45cbb245611b12db2a72
2018-11-15 10:22:50 +00:00
Xusong Wang
5b747ae02b Fix VTS ValidationTest for 1.2 ops.
Fix VTS ValidationTest for 1.2 ops by skipping invalid tests.

Bug: 119425787

Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Change-Id: I61e33d8ff3d9121d9ceb28e667ba6c00bb9951ac
2018-11-13 15:07:57 -08:00
Michael K. Sanders
c785d46eb6 Adds symbolic min/max values for type enums.
This abstracts the boundary values for OperandType and
OperationType to avoid the need to update them in the
model validation functions.

Test: VtsHalNeuralnetworksV1_2TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.2::IDevice/sample-all
Test: VtsHalNeuralnetworksV1_2CompatV1_1TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.2::IDevice/sample-all
Test: VtsHalNeuralnetworksV1_2CompatV1_0TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.2::IDevice/sample-all
Change-Id: I39155148d67215e32b4eb1991b885f65d5eeaca8
2018-11-12 11:43:31 +00:00
Xusong Wang
6433728e2c Fix VTS ValidationTest for 1.2 ops.
Skip L2_NORMALIZATION,  LOCAL_RESPONSE_NORMALIZATION, and SOFTMAX for
addOperationInputTest.

Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Change-Id: I16de09e2eef7610e564f1f49ff067af7efca3784
2018-10-22 13:53:31 -07:00
Michael K. Sanders
19d63453d4 Adds TENSOR_FLOAT16 operand type.
Bug: 113563458
Test: VtsHalNeuralnetworksV1_2TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.2::IDevice/sample-all
Change-Id: If12ceff428e1b1a90ef99b7353f0df60d4ef8010
2018-10-16 12:59:42 +01:00
Lev Proleev
5d7c99527e Add TENSOR_QUANT16_ASYMM to operand types
Add new OperandType::TENSOR_QUANT16_ASYMM.
Add VTS validation for the new type.

Bug: 113561892
Test: NeuralNetworksTest_static
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I4f9ed6a33d5d3ec227e9f335df71954c73edf344
2018-10-16 10:15:44 +01:00
Lev Proleev
abad9eac44 Add new OperandType BOOL.
- Add new enum OperandType::BOOL.
- Add v1.2 Operand, OperandType.
- Add VTS validation tests for BOOL.

Bug: 117423393

Test: NeuralNetworksTest_static
Test: VtsHalNeuralnetworksV1_2TargetTest

Change-Id: I420e2afeb09b881a499eee6b138c1f26e9874f5a
2018-10-08 17:21:45 -07:00
Slava Shklyaev
871be94770 Add VTS tests for NeuralNetworks v1.2
This is a copy the v1.1 tests since we don't have any new ops
implemented in v1.2 yet.

Bug: 114365802
Test: mm
Test: NNAPI VTS
Change-Id: Ida7525fcd3ae0fd6f88ff9591e06aba922bdae64
2018-09-20 11:31:26 +00:00