Commit Graph

42 Commits

Author SHA1 Message Date
Michael Butler
9c3c8642fb Quick-fail NNAPI VTS test case if driver is dead
This CL adds a check during SetUp that an NNAPI driver service is still
alive by pinging the driver service. If it is not alive, the test will
fail during SetUp. Without this quick-fail, the test case would continue
as if the driver were still active, which would result in multiple
EXPECT_* and ASSERT_* statements failing instead of a single, clear
failure message.

Bug: 197035200
Test: mma
Test: presubmit: VtsHalNeuralnetworks*TargetTest
Change-Id: Ib1b75ed20f764055699590581d5ad4e5aff4baae
2021-08-23 18:27:49 -07:00
Slava Shklyaev
405db5f418 Migrate NNAPI to canonical types
Bug: 160669906
Test: m
Change-Id: Ib5282e5477e7c632485a966b2fef02044ce9fb96
Merged-In: Ib5282e5477e7c632485a966b2fef02044ce9fb96
(cherry picked from commit 8230f8c6f2)
2020-11-02 10:08:49 +00:00
Xusong Wang
91521ab86b Add checks of output shapes when OUTPUT_INSUFFICIENT_SIZE.
Add checks that all returned output dimensions must be at
least as fully specified as the union of the information about the
corresponding operand in the model and in the request.

Bug: 154054474
Test: VTS
Change-Id: I934d084c7665160a98da9828604ce8297fef73b8
Merged-In: I934d084c7665160a98da9828604ce8297fef73b8
(cherry picked from commit d454751e00)
2020-06-02 16:52:56 -07:00
Michael Butler
b1865b6593 Remove extra tests from NNAPI VTS validation tests
Do not run validation on "inputs_as_internal" and
"all_tensors_as_inputs" variants.

Bug: 138149072
Bug: 149840439
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I0699ed6703e48b6c4bc0e7a392b79c12770f04c9
Merged-In: I0699ed6703e48b6c4bc0e7a392b79c12770f04c9
(cherry picked from commit 678a10600c)
2020-03-27 11:44:10 +00:00
Miao Wang
702a91acf5 Fix the timing initialization error for failed executeFenced case
Fixes: 152075771
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I5829397346354ee3fc4a58b0a418197a1eee47cc
Merged-In: I5829397346354ee3fc4a58b0a418197a1eee47cc
(cherry picked from commit ad7678c8cb)
2020-03-27 11:44:08 +00:00
Xusong Wang
75e63ad743 Add BLOB AHWB tests in VTS.
Bug: 149847930
Test: 1.3 VTS
Change-Id: I9c795dcb7696c843afd12551927463c5529a4b60
Merged-In: I9c795dcb7696c843afd12551927463c5529a4b60
(cherry picked from commit 41adc5bc11)
2020-03-27 11:44:06 +00:00
Xusong Wang
2844165bf3 Add fenced compute path to memory domain validation test.
Bug: 147777318
Test: 1.3 VTS
Change-Id: I0b731d10384ef2024241af1d908acf3ba760d73f
Merged-In: I0b731d10384ef2024241af1d908acf3ba760d73f
(cherry picked from commit 9c415917e0)
2020-03-27 11:44:04 +00:00
Xusong Wang
5b0c58d1fb Add memory domain VTS validation tests.
Bug: 147777318
Test: 1.3 VTS with sample driver
Change-Id: Ia2097345924726d8fb627845fd7438cc3eb35eb6
Merged-In: Ia2097345924726d8fb627845fd7438cc3eb35eb6
(cherry picked from commit fed2f5213f)
2020-03-27 11:44:00 +00:00
Xusong Wang
3e64ecc347 Exclude invalid test models from QuantizationCouplingTest.
Bug: 151674996
Test: 1.3 VTS
Change-Id: I381eb6d39926054279f1791bd71d28a963acc0b9
Merged-In: I381eb6d39926054279f1791bd71d28a963acc0b9
(cherry picked from commit 962fdb26ca)
2020-03-20 12:29:16 -07:00
Xusong Wang
f07a957ce2 Fix VTS of fenced execution with zero-sized output.
Fixes: 151441390
Test: 1.3 VTS
Change-Id: I100329e18c34c377d217af155c2abc8e67078778
Merged-In: I100329e18c34c377d217af155c2abc8e67078778
(cherry picked from commit 9fcccf8b85)
2020-03-16 11:08:06 +00:00
Miao Wang
406890769a Add tests to make sure executeFenced validate unspecified output shapes
Bug: 148979873
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ib4960c71ca46ca034777cc7b02d7d2885a98691d
Merged-In: Ib4960c71ca46ca034777cc7b02d7d2885a98691d
(cherry picked from commit fa161c7a0a)
2020-03-16 11:01:09 +00:00
Miao Wang
2d74d7dfae Add more tests exercising IPreparedModel::executeFenced API
- executeFenced with device memory
  - executeFenced with depending sync fences

Bug: 148979873
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I772c5c85f75cee56e2af06470c2de5b810f8078d
Merged-In: I772c5c85f75cee56e2af06470c2de5b810f8078d
(cherry picked from commit 34dfa2f74c)
2020-03-16 11:01:08 +00:00
Michael Butler
ff7d6c5493 Relax NNAPI QoS deadline parameter
Prior to this CL, the NNAPI QoS deadline is a strict deadline: a task
with a deadline must either complete or abort by the deadline. To
avoid as much overhead as possible, it is requested for Android R to
have a relaxed deadline, where the task *may* be aborted after the
deadline has passed or when the task is estimated to take too long.

Because the deadline is now relaxed, the querying method
IDevice::supportsDeadlines can be removed.

Bug: 149766387
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ib9937b5bb95646c2fd82e1aa17f2b0b639e2c420
Merged-In: Ib9937b5bb95646c2fd82e1aa17f2b0b639e2c420
(cherry picked from commit 9b9a804fe4)
2020-02-26 10:36:09 +00:00
Slava Shklyaev
a37c2c84aa Add NNAPI loop timeout VTS test
Fix: 149554639
Bug: 145906499
Bug: 136735929
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I642772d3c00c12d72380598d9d86743706179d72
Merged-In: I642772d3c00c12d72380598d9d86743706179d72
(cherry picked from commit cc873aee8f)
2020-02-26 10:36:07 +00:00
Slava Shklyaev
a96ab7407d Add NNAPI loop timeout API
Bug: 145906499
Bug: 136735929
Test: NNT_static
Change-Id: Ie40c248b174964676985403f9f1a5127b408a00a
Merged-In: Ie40c248b174964676985403f9f1a5127b408a00a
(cherry picked from commit f034bf9c92)
2020-02-19 11:58:35 +00:00
Slava Shklyaev
0fff59b961 Add control flow support to NNAPI VTS tests
See change I98a3edd1.

Bug: 148077633
Bug: 148601177
Bug: 136735929
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I1e436cdba404b68026a45797ac4fb3a34f8be76a
Merged-In: I1e436cdba404b68026a45797ac4fb3a34f8be76a
(cherry picked from commit 1f98e2e929)
2020-02-19 11:58:34 +00:00
Michael Butler
c3e1a29a60 NN 1.3 HAL: ANAPIC review follow up
This CL makes the following three interface changes:
* Removes @1.3::Operand.ExtraParams, because it was the same as
  @1.2::Operand.ExtraParams
* Changes int32_t token to uint32_t for IDevice::allocate and for
  @1.3::Request.MemoryPool
* Renames OptionalTimePoint::nanoseconds to
  OptionalTimePoint::nanosecondsSinceEpoch

This CL also makes test changes in response to the interface changes,
and fixes a minor comment issue in @1.2::types.hal and @1.3::types.hal.

Bug: 148617339
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I95b49ab34b0b79f587dda035ede700b73dc44a38
Merged-In: I95b49ab34b0b79f587dda035ede700b73dc44a38
(cherry picked from commit 6a4172cd3d)
2020-02-12 17:52:34 -08:00
Miao Wang
99753d9e90 Update sync fence related API for NNAPI
- Allow IPreparedModel::executeFenced to measure gated durations.
 - Allow IPreparedModel::executeFenced to specify an optional deadline
 and aa optional post-gate-release maximum duration.

Bug: 142778241
Bug: 141363565
Test: mm
Change-Id: I1687ec5543f93ed9bb5d7fcf14dbe15ed5b5cd0d
Merged-In: I1687ec5543f93ed9bb5d7fcf14dbe15ed5b5cd0d
(cherry picked from commit 3fd70b020e)
2020-01-30 15:23:23 -08:00
Miao Wang
e51e4a9f26 Fix the NNAPI vts tests about validateExecuteFenced
- Skip the test if the driver reject executeFenced.
  - remove the -1 fd for hidl_handle as it causes error before reaching
  the driver.

Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Idc3f815040efccbbfa95b70d5d437441d0bd8682
Merged-In: Idc3f815040efccbbfa95b70d5d437441d0bd8682
(cherry picked from commit 33173a3e68)
2020-01-27 12:59:07 -08:00
Miao Wang
2b5c4cda77 Add VTS tests for NNAPI IPreparedModel::executeFenced
- Validation tests
  - Generated tests to exercise executeFenced path

Bug: 142778241
Test: mm
Change-Id: I509f0b5713fc86885d597940aae5ade0502c97ad
Merged-In: I509f0b5713fc86885d597940aae5ade0502c97ad
(cherry picked from commit 2c4e02329b)
2020-01-27 12:58:50 -08:00
Lev Proleev
53a51cb352 Add FILL and RANK ops
Bug: 148050168
Bug: 148049333
Test: NNTest_static and VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Iebdfa600d84e31532807740c21d95cae41c76ad5
Merged-In: Iebdfa600d84e31532807740c21d95cae41c76ad5
(cherry picked from commit 8b3f240ea3)
2020-01-27 12:58:14 -08:00
Michael Butler
95899b3d39 Create VTS tests for QoS in NNAPI
Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: If3ab91cfb3158e4c33e809ff3b149dff47cda76f
Merged-In: If3ab91cfb3158e4c33e809ff3b149dff47cda76f
(cherry picked from commit 616701d3cd)
2020-01-27 12:51:42 -08:00
Michael Butler
79a41d77c0 Update NNAPI 1.3 VTS tests with new types
Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Test: atest VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ie76da9dc9d6993a56bf644cfe20c5f5b421672c9
Merged-In: Ie76da9dc9d6993a56bf644cfe20c5f5b421672c9
(cherry picked from commit 9449a28b2f)
2020-01-27 12:51:34 -08:00
Slava Shklyaev
f8124a861f Add NNAPI control flow
Bug: 136735929
Bug: 139181916
Test: m
Change-Id: I7a75175f00fc98df626c40ea669021ccd40130e0
Merged-In: I7a75175f00fc98df626c40ea669021ccd40130e0
(cherry picked from commit a785a3faac)
2020-01-27 12:51:09 -08:00
Xusong Wang
e9da9852a5 Add memory domain VTS generated tests.
Bug: 141353602
Bug: 141363565
Test: 1.3 VTS
Change-Id: Ifc7eb3fd6f15e28ba403f02bdf66b4568bddcb64
Merged-In: Ifc7eb3fd6f15e28ba403f02bdf66b4568bddcb64
(cherry picked from commit 1f50e54cf8)
2020-01-24 14:25:32 -08:00
Xusong Wang
b345a4688f Memory Domain HAL: Define HAL APIs.
- Add and document memory domain HAL APIs.
- Make necessary changes to the existing VTS codes to make them work
  with V1_3::Request.

Bug: 141353602
Bug: 141363565
Test: mma
Test: NNT_static
Test: 1.3 VTS
Change-Id: Ia32555d4fef149fad4a79728981c5d9cca675a1a
Merged-In: Ia32555d4fef149fad4a79728981c5d9cca675a1a
(cherry picked from commit 931d5a18bc)
2020-01-21 15:15:39 -08:00
Lev Proleev
3c68b3412f Enable QuantizationCouplingTest in NNAPI VTS
Bug: 137828994
Test: NNTest_static and VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I9b4f20ab0287be9cae9fc1b69a8f64cc8f1996b0
Merged-In: I9b4f20ab0287be9cae9fc1b69a8f64cc8f1996b0
(cherry picked from commit baac15d5fd)
2020-01-16 13:22:33 +00:00
Lev Proleev
673fdcf4e6 Add exceptions to the signed quantization coupling tests
Main commit is in frameworks/ml/nn. This one only renames
testModel.hasQuant8AsymmOperands() to
testModel.hasQuant8CoupledOperands().

Bug: 143935412
Bug: 143934188
Bug: 143935353
Bug: 143934467
Bug: 143934184
Test: quantization coupling tests in CTS and VTS
Change-Id: I0f7c331c355fe61ee06605786f09bc1ca7cdef57
Merged-In: I0f7c331c355fe61ee06605786f09bc1ca7cdef57
(cherry picked from commit 6a1d5e405f)
2020-01-07 13:35:58 +00:00
Xusong Wang
d4a060bb29 NN HAL: Upgrade IPreparedModel::executeSynchronously to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I4b74e8ac031c5d793cf7e5d66190734949367538
Merged-In: I4b74e8ac031c5d793cf7e5d66190734949367538
(cherry picked from commit ebd88ba8d2)
2020-01-06 15:20:14 +00:00
Lev Proleev
56cda83f8c Fix use-after-free error in NNAPI VTS tests
Replace std::initializer_list with std::vector.

Fix: 145705114
Test: VtsHalNeuralnetworksV1_[23]TargetTest
Change-Id: I92cbc56b2b8fbb36b3f3225e7cd0d4b05c84041e
2019-12-09 14:56:09 +00:00
Lev Proleev
9226c1e910 Add quantization coupling test
Quantization coupling test is ran only on models that contain only one
operator and at least one of its inputs has a type of
TENSOR_QUANT8_ASYMM. The test verifies that a model with all the
operands converted to TENSOR_QUANT8_ASYMM_SIGNED will produce the same
result (OK/SKIPPED/FAILED).

Bug: 137828994
Test: VtsHalNeuralNetworksV1_3TargetTest --gtest_filter="*QuantizationCouplingTest*"
Change-Id: I5a2e09b2b1bd790e7e37dde133554e516155a34d
Merged-In: I5a2e09b2b1bd790e7e37dde133554e516155a34d
(cherry picked from commit c88f4c60a8)
2019-11-21 12:06:34 +00:00
Lev Proleev
0d4ba3f33a Combine test parameters into TestConfig structure
Test: VtsHalNeuralnetworksV1_2TargetTest --gtest_filter="GeneratedTests*"
Change-Id: I928aaa42e4745b4a8e0e461046e9632b052d0135
Merged-In: I928aaa42e4745b4a8e0e461046e9632b052d0135
(cherry picked from commit 66f598e10d)
2019-11-21 12:06:31 +00:00
Lev Proleev
3b60d80ff6 Add NNAPI 1.3 OperationType
* Add 1.3 OperationType.
* Remove "using V1_2::OperationType" from 1.3 VTS tests.
* Update current.txt

Test: mma
Change-Id: Ieaede9b7a6cecb16dbcc753f347f7ff04c009f20
Merged-In: Ieaede9b7a6cecb16dbcc753f347f7ff04c009f20
(cherry picked from commit 8774f10b76)
2019-11-21 12:06:27 +00:00
Xusong Wang
1b3f426648 NN HAL: Upgrade IPreparedModel::execute to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I9ca1e28ddc97fe880a72885afe7afb6c93903697
Merged-In: I9ca1e28ddc97fe880a72885afe7afb6c93903697
(cherry picked from commit 62a760c32d)
2019-11-19 12:18:41 -08:00
Xusong Wang
cc47dffa57 NN HAL: Upgrade IPreparedModelCallback::notify to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I56bc7a2fb179a9576036ad0c2aae0e1f41ec4e2c
Merged-In: I56bc7a2fb179a9576036ad0c2aae0e1f41ec4e2c
(cherry picked from commit 68c3234ad3)
2019-11-19 12:18:33 -08:00
Michael Butler
648ada5fcf NNAPI VTS update in response to utility function change
The frameworks/ml/nn CL "Improve performance of Burst executions"
changed the signature of a utility function used by NNAPI's VTS tests.
This CL modifies the test accordingly.

Bug: 132073143
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Test: VtsHalNeuralnetworksV1_3TargetTest (with sample-all)
Change-Id: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
Merged-In: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
(cherry picked from commit 5756887a7b)
2019-10-28 13:17:08 -07:00
Lev Proleev
26d1bc8a4e Modify NNAPI VTS tests to run on version 1.3
Bug: 139120468
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I4654dc75c17f8801103015dc1da91663dfa28d52
Merged-In: I4654dc75c17f8801103015dc1da91663dfa28d52
(cherry picked from commit b49dadfb64)
2019-10-17 18:34:51 +01:00
Lev Proleev
13fdfcd44f Copy VTS tests from v1.2 to v1.3
So that it's easier to see what actually has changed in VTS tests for
version 1.3

Bug: 139120468
Test: m
Change-Id: I09797f5f3898501a008186a22dd411b00e9e2c67
Merged-In: I09797f5f3898501a008186a22dd411b00e9e2c67
(cherry picked from commit 3b13b55ac1)
2019-10-17 18:34:51 +01:00
Lev Proleev
9613b4649b Revert "Copy VTS tests from v1.2 to v1.3"
This reverts commit d0e339e1da.

Reason for revert: the topic breaks git_qt-dev-plus-aosp

Change-Id: I68d31526cc7699b9a6c69a1ccc669fdb7605124e
2019-10-10 13:58:56 +00:00
Lev Proleev
64f9eb4bd5 Revert "Modify NNAPI VTS tests to run on version 1.3"
This reverts commit 543606fb92.

Reason for revert: the topic breaks git_qt-dev-plus-aosp

Change-Id: I74859bae687d65d344ed7edc617e1a7d086960a1
2019-10-10 13:54:38 +00:00
Lev Proleev
543606fb92 Modify NNAPI VTS tests to run on version 1.3
Bug: 139120468
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Id9e4d99852da8a3d5167ab7464c0e71885250501
Merged-In: Id9e4d99852da8a3d5167ab7464c0e71885250501
(cherry picked from commit b49dadfb64)
2019-10-09 14:18:49 +01:00
Lev Proleev
d0e339e1da Copy VTS tests from v1.2 to v1.3
So that it's easier to see what actually has changed in VTS tests for
version 1.3

Bug: 139120468
Test: m
Change-Id: Ief294d21349ca6531595612a16fa3ae3382f83ac
Merged-In: Ief294d21349ca6531595612a16fa3ae3382f83ac
(cherry picked from commit 3b13b55ac1)
2019-10-09 14:18:49 +01:00