Commit Graph

50 Commits

Author SHA1 Message Date
Michael Butler
3c8593c0fe Merge "Remove extra tests from NNAPI VTS validation tests" into rvc-dev 2020-03-23 22:07:32 +00:00
Miao Wang
1be73e85a1 Merge "Fix the timing initialization error for failed executeFenced case" into rvc-dev 2020-03-23 20:47:22 +00:00
Miao Wang
ad7678c8cb Fix the timing initialization error for failed executeFenced case
Fixes: 152075771
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I5829397346354ee3fc4a58b0a418197a1eee47cc
2020-03-23 20:46:58 +00:00
Xusong Wang
e71fa3a11a Merge "Add BLOB AHWB tests in VTS." into rvc-dev 2020-03-21 06:38:36 +00:00
Xusong Wang
9c415917e0 Add fenced compute path to memory domain validation test.
Bug: 147777318
Test: 1.3 VTS
Change-Id: I0b731d10384ef2024241af1d908acf3ba760d73f
2020-03-20 14:49:54 -07:00
Xusong Wang
fed2f5213f Add memory domain VTS validation tests.
Bug: 147777318
Test: 1.3 VTS with sample driver
Change-Id: Ia2097345924726d8fb627845fd7438cc3eb35eb6
2020-03-20 13:56:40 -07:00
Michael Butler
678a10600c Remove extra tests from NNAPI VTS validation tests
Do not run validation on "inputs_as_internal" and
"all_tensors_as_inputs" variants.

Bug: 138149072
Bug: 149840439
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I0699ed6703e48b6c4bc0e7a392b79c12770f04c9
2020-03-19 18:37:23 -07:00
Xusong Wang
962fdb26ca Exclude invalid test models from QuantizationCouplingTest.
Bug: 151674996
Test: 1.3 VTS
Change-Id: I381eb6d39926054279f1791bd71d28a963acc0b9
2020-03-16 17:13:41 -07:00
Xusong Wang
41adc5bc11 Add BLOB AHWB tests in VTS.
Bug: 149847930
Test: 1.3 VTS
Change-Id: I9c795dcb7696c843afd12551927463c5529a4b60
2020-03-13 20:04:38 -07:00
Xusong Wang
5882d6d911 Fix VTS of fenced execution with zero-sized output.
Fixes: 151441390
Test: 1.3 VTS
Change-Id: I100329e18c34c377d217af155c2abc8e67078778
Merged-In: I100329e18c34c377d217af155c2abc8e67078778
(cherry picked from commit 9fcccf8b85)
2020-03-13 15:15:39 -07:00
Miao Wang
fa161c7a0a Add tests to make sure executeFenced validate unspecified output shapes
Bug: 148979873
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ib4960c71ca46ca034777cc7b02d7d2885a98691d
2020-03-04 14:56:46 -08:00
Miao Wang
34dfa2f74c Add more tests exercising IPreparedModel::executeFenced API
- executeFenced with device memory
  - executeFenced with depending sync fences

Bug: 148979873
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I772c5c85f75cee56e2af06470c2de5b810f8078d
2020-02-26 13:17:22 -08:00
Michael Butler
9b9a804fe4 Relax NNAPI QoS deadline parameter
Prior to this CL, the NNAPI QoS deadline is a strict deadline: a task
with a deadline must either complete or abort by the deadline. To
avoid as much overhead as possible, it is requested for Android R to
have a relaxed deadline, where the task *may* be aborted after the
deadline has passed or when the task is estimated to take too long.

Because the deadline is now relaxed, the querying method
IDevice::supportsDeadlines can be removed.

Bug: 149766387
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ib9937b5bb95646c2fd82e1aa17f2b0b639e2c420
2020-02-19 10:07:56 -08:00
Slava Shklyaev
cc873aee8f Add NNAPI loop timeout VTS test
Fix: 149554639
Bug: 145906499
Bug: 136735929
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I642772d3c00c12d72380598d9d86743706179d72
2020-02-19 11:10:13 +00:00
Slava Shklyaev
f034bf9c92 Add NNAPI loop timeout API
Bug: 145906499
Bug: 136735929
Test: NNT_static
Change-Id: Ie40c248b174964676985403f9f1a5127b408a00a
2020-02-18 13:37:37 +00:00
Slava Shklyaev
1f98e2e929 Add control flow support to NNAPI VTS tests
See change I98a3edd1.

Bug: 148077633
Bug: 148601177
Bug: 136735929
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I1e436cdba404b68026a45797ac4fb3a34f8be76a
2020-02-12 14:29:27 +00:00
Michael Butler
6a4172cd3d NN 1.3 HAL: ANAPIC review follow up
This CL makes the following three interface changes:
* Removes @1.3::Operand.ExtraParams, because it was the same as
  @1.2::Operand.ExtraParams
* Changes int32_t token to uint32_t for IDevice::allocate and for
  @1.3::Request.MemoryPool
* Renames OptionalTimePoint::nanoseconds to
  OptionalTimePoint::nanosecondsSinceEpoch

This CL also makes test changes in response to the interface changes,
and fixes a minor comment issue in @1.2::types.hal and @1.3::types.hal.

Bug: 148617339
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I95b49ab34b0b79f587dda035ede700b73dc44a38
2020-02-05 03:09:13 +00:00
Miao Wang
3fd70b020e Update sync fence related API for NNAPI
- Allow IPreparedModel::executeFenced to measure gated durations.
 - Allow IPreparedModel::executeFenced to specify an optional deadline
 and aa optional post-gate-release maximum duration.

Bug: 142778241
Bug: 141363565
Test: mm
Change-Id: I1687ec5543f93ed9bb5d7fcf14dbe15ed5b5cd0d
2020-01-28 14:31:57 -08:00
Miao Wang
33173a3e68 Fix the NNAPI vts tests about validateExecuteFenced
- Skip the test if the driver reject executeFenced.
  - remove the -1 fd for hidl_handle as it causes error before reaching
  the driver.

Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Idc3f815040efccbbfa95b70d5d437441d0bd8682
2020-01-23 15:08:15 -08:00
Miao Wang
6604e51458 Merge changes from topic "nn_sync_fence"
* changes:
  Add VTS tests for NNAPI IPreparedModel::executeFenced
  Support sync fence in NNAPI
2020-01-23 08:01:42 +00:00
Miao Wang
2c4e02329b Add VTS tests for NNAPI IPreparedModel::executeFenced
- Validation tests
  - Generated tests to exercise executeFenced path

Bug: 142778241
Test: mm
Change-Id: I509f0b5713fc86885d597940aae5ade0502c97ad
2020-01-22 18:01:58 -08:00
Lev Proleev
8b3f240ea3 Add FILL and RANK ops
Bug: 148050168
Bug: 148049333
Test: NNTest_static and VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Iebdfa600d84e31532807740c21d95cae41c76ad5
2020-01-22 17:16:43 +00:00
Michael Butler
616701d3cd Create VTS tests for QoS in NNAPI
Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: If3ab91cfb3158e4c33e809ff3b149dff47cda76f
2020-01-21 16:02:05 -08:00
Michael Butler
9449a28b2f Update NNAPI 1.3 VTS tests with new types
Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Test: atest VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ie76da9dc9d6993a56bf644cfe20c5f5b421672c9
2020-01-21 16:02:05 -08:00
Slava Shklyaev
a785a3faac Add NNAPI control flow
Bug: 136735929
Bug: 139181916
Test: m
Change-Id: I7a75175f00fc98df626c40ea669021ccd40130e0
2020-01-21 13:39:22 +00:00
Xusong Wang
1f50e54cf8 Add memory domain VTS generated tests.
Bug: 141353602
Bug: 141363565
Test: 1.3 VTS
Change-Id: Ifc7eb3fd6f15e28ba403f02bdf66b4568bddcb64
2020-01-16 16:11:38 -08:00
Xusong Wang
931d5a18bc Memory Domain HAL: Define HAL APIs.
- Add and document memory domain HAL APIs.
- Make necessary changes to the existing VTS codes to make them work
  with V1_3::Request.

Bug: 141353602
Bug: 141363565
Test: mma
Test: NNT_static
Test: 1.3 VTS
Change-Id: Ia32555d4fef149fad4a79728981c5d9cca675a1a
2020-01-16 16:10:13 -08:00
Lev Proleev
baac15d5fd Enable QuantizationCouplingTest in NNAPI VTS
Bug: 137828994
Test: NNTest_static and VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I9b4f20ab0287be9cae9fc1b69a8f64cc8f1996b0
2020-01-09 16:40:41 +00:00
Lev Proleev
6a1d5e405f Add exceptions to the signed quantization coupling tests
Main commit is in frameworks/ml/nn. This one only renames
testModel.hasQuant8AsymmOperands() to
testModel.hasQuant8CoupledOperands().

Bug: 143935412
Bug: 143934188
Bug: 143935353
Bug: 143934467
Bug: 143934184
Test: quantization coupling tests in CTS and VTS
Change-Id: I0f7c331c355fe61ee06605786f09bc1ca7cdef57
2020-01-02 18:34:03 +00:00
Xusong Wang
f4f2a0d6bf Merge "NN HAL: Upgrade IPreparedModel::executeSynchronously to 1.3." 2019-12-11 20:24:12 +00:00
Lev Proleev
56cda83f8c Fix use-after-free error in NNAPI VTS tests
Replace std::initializer_list with std::vector.

Fix: 145705114
Test: VtsHalNeuralnetworksV1_[23]TargetTest
Change-Id: I92cbc56b2b8fbb36b3f3225e7cd0d4b05c84041e
2019-12-09 14:56:09 +00:00
Lev Proleev
9226c1e910 Add quantization coupling test
Quantization coupling test is ran only on models that contain only one
operator and at least one of its inputs has a type of
TENSOR_QUANT8_ASYMM. The test verifies that a model with all the
operands converted to TENSOR_QUANT8_ASYMM_SIGNED will produce the same
result (OK/SKIPPED/FAILED).

Bug: 137828994
Test: VtsHalNeuralNetworksV1_3TargetTest --gtest_filter="*QuantizationCouplingTest*"
Change-Id: I5a2e09b2b1bd790e7e37dde133554e516155a34d
Merged-In: I5a2e09b2b1bd790e7e37dde133554e516155a34d
(cherry picked from commit c88f4c60a8)
2019-11-21 12:06:34 +00:00
Lev Proleev
0d4ba3f33a Combine test parameters into TestConfig structure
Test: VtsHalNeuralnetworksV1_2TargetTest --gtest_filter="GeneratedTests*"
Change-Id: I928aaa42e4745b4a8e0e461046e9632b052d0135
Merged-In: I928aaa42e4745b4a8e0e461046e9632b052d0135
(cherry picked from commit 66f598e10d)
2019-11-21 12:06:31 +00:00
Lev Proleev
3b60d80ff6 Add NNAPI 1.3 OperationType
* Add 1.3 OperationType.
* Remove "using V1_2::OperationType" from 1.3 VTS tests.
* Update current.txt

Test: mma
Change-Id: Ieaede9b7a6cecb16dbcc753f347f7ff04c009f20
Merged-In: Ieaede9b7a6cecb16dbcc753f347f7ff04c009f20
(cherry picked from commit 8774f10b76)
2019-11-21 12:06:27 +00:00
Xusong Wang
1b3f426648 NN HAL: Upgrade IPreparedModel::execute to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I9ca1e28ddc97fe880a72885afe7afb6c93903697
Merged-In: I9ca1e28ddc97fe880a72885afe7afb6c93903697
(cherry picked from commit 62a760c32d)
2019-11-19 12:18:41 -08:00
Xusong Wang
cc47dffa57 NN HAL: Upgrade IPreparedModelCallback::notify to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I56bc7a2fb179a9576036ad0c2aae0e1f41ec4e2c
Merged-In: I56bc7a2fb179a9576036ad0c2aae0e1f41ec4e2c
(cherry picked from commit 68c3234ad3)
2019-11-19 12:18:33 -08:00
Xusong Wang
ebd88ba8d2 NN HAL: Upgrade IPreparedModel::executeSynchronously to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I4b74e8ac031c5d793cf7e5d66190734949367538
2019-11-18 11:20:37 -08:00
Lev Proleev
8774f10b76 Add NNAPI 1.3 OperationType
* Add 1.3 OperationType.
* Remove "using V1_2::OperationType" from 1.3 VTS tests.
* Update current.txt

Test: mma
Change-Id: Ieaede9b7a6cecb16dbcc753f347f7ff04c009f20
2019-11-11 18:11:50 +00:00
Xusong Wang
62a760c32d NN HAL: Upgrade IPreparedModel::execute to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I9ca1e28ddc97fe880a72885afe7afb6c93903697
2019-11-07 17:30:56 -08:00
Xusong Wang
68c3234ad3 NN HAL: Upgrade IPreparedModelCallback::notify to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I56bc7a2fb179a9576036ad0c2aae0e1f41ec4e2c
2019-11-07 17:30:43 -08:00
Michael Butler
648ada5fcf NNAPI VTS update in response to utility function change
The frameworks/ml/nn CL "Improve performance of Burst executions"
changed the signature of a utility function used by NNAPI's VTS tests.
This CL modifies the test accordingly.

Bug: 132073143
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Test: VtsHalNeuralnetworksV1_3TargetTest (with sample-all)
Change-Id: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
Merged-In: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
(cherry picked from commit 5756887a7b)
2019-10-28 13:17:08 -07:00
Lev Proleev
26d1bc8a4e Modify NNAPI VTS tests to run on version 1.3
Bug: 139120468
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I4654dc75c17f8801103015dc1da91663dfa28d52
Merged-In: I4654dc75c17f8801103015dc1da91663dfa28d52
(cherry picked from commit b49dadfb64)
2019-10-17 18:34:51 +01:00
Lev Proleev
13fdfcd44f Copy VTS tests from v1.2 to v1.3
So that it's easier to see what actually has changed in VTS tests for
version 1.3

Bug: 139120468
Test: m
Change-Id: I09797f5f3898501a008186a22dd411b00e9e2c67
Merged-In: I09797f5f3898501a008186a22dd411b00e9e2c67
(cherry picked from commit 3b13b55ac1)
2019-10-17 18:34:51 +01:00
Lev Proleev
9613b4649b Revert "Copy VTS tests from v1.2 to v1.3"
This reverts commit d0e339e1da.

Reason for revert: the topic breaks git_qt-dev-plus-aosp

Change-Id: I68d31526cc7699b9a6c69a1ccc669fdb7605124e
2019-10-10 13:58:56 +00:00
Lev Proleev
64f9eb4bd5 Revert "Modify NNAPI VTS tests to run on version 1.3"
This reverts commit 543606fb92.

Reason for revert: the topic breaks git_qt-dev-plus-aosp

Change-Id: I74859bae687d65d344ed7edc617e1a7d086960a1
2019-10-10 13:54:38 +00:00
Michael Butler
5756887a7b NNAPI VTS update in response to utility function change
The frameworks/ml/nn CL "Improve performance of Burst executions"
changed the signature of a utility function used by NNAPI's VTS tests.
This CL modifies the test accordingly.

Bug: 132073143
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Test: VtsHalNeuralnetworksV1_3TargetTest (with sample-all)
Change-Id: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
2019-10-09 11:15:08 -07:00
Lev Proleev
543606fb92 Modify NNAPI VTS tests to run on version 1.3
Bug: 139120468
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Id9e4d99852da8a3d5167ab7464c0e71885250501
Merged-In: Id9e4d99852da8a3d5167ab7464c0e71885250501
(cherry picked from commit b49dadfb64)
2019-10-09 14:18:49 +01:00
Lev Proleev
d0e339e1da Copy VTS tests from v1.2 to v1.3
So that it's easier to see what actually has changed in VTS tests for
version 1.3

Bug: 139120468
Test: m
Change-Id: Ief294d21349ca6531595612a16fa3ae3382f83ac
Merged-In: Ief294d21349ca6531595612a16fa3ae3382f83ac
(cherry picked from commit 3b13b55ac1)
2019-10-09 14:18:49 +01:00
Lev Proleev
b49dadfb64 Modify NNAPI VTS tests to run on version 1.3
Bug: 139120468
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Id9e4d99852da8a3d5167ab7464c0e71885250501
2019-10-02 13:45:02 +01:00
Lev Proleev
3b13b55ac1 Copy VTS tests from v1.2 to v1.3
So that it's easier to see what actually has changed in VTS tests for
version 1.3

Bug: 139120468
Test: m
Change-Id: Ief294d21349ca6531595612a16fa3ae3382f83ac
2019-10-02 13:45:02 +01:00