Commit Graph

20 Commits

Author SHA1 Message Date
Jordan R Abrahams-Whitehead
745c351b8b vts: Upsize 8bit uniform_int_distribution
uniform_int_distribution<a> for types sizeof(a) < 2 are
not valid by the C++ library standard. Newer versions of LLVM
(particularly spurred on by ChromeOS toolchain changes)
require at least std::uniform_int_distribution<uint16_t>.

This is a required change for rolling LLVM to r458507.
This is necessary, but may not be sufficient to resolve
the issue.

Bug: 231351802
Test: mma
Change-Id: I04c3cc91507f3467c432b9a25effdac3f5fb56f3
2022-05-30 22:30:54 +00:00
Michael Butler
9c3c8642fb Quick-fail NNAPI VTS test case if driver is dead
This CL adds a check during SetUp that an NNAPI driver service is still
alive by pinging the driver service. If it is not alive, the test will
fail during SetUp. Without this quick-fail, the test case would continue
as if the driver were still active, which would result in multiple
EXPECT_* and ASSERT_* statements failing instead of a single, clear
failure message.

Bug: 197035200
Test: mma
Test: presubmit: VtsHalNeuralnetworks*TargetTest
Change-Id: Ib1b75ed20f764055699590581d5ad4e5aff4baae
2021-08-23 18:27:49 -07:00
Xusong Wang
7f40613792 Suppress gtest error for tests without any instance
NNAPI VTS is parameterized by discovered devices. When there is no
device available, the compilation caching and memory domain tests will
not be instantiated, and will cause a gtest error. This patch suppresses
the error by GTEST_ALLOW_UNINSTANTIATED_PARAMETERIZED_TEST macro.

Bug: 168152007
Test: 1.2/1.3 VTS without NNAPI driver
Change-Id: I715ae6f45df66e4a03c12b5d3f38bcc028a9ed7c
Merged-In: I715ae6f45df66e4a03c12b5d3f38bcc028a9ed7c
(cherry picked from commit af9ac3b8a0)
2020-09-28 15:39:01 -07:00
Jim Pollock
9df50c6615 vts: remove deprecated gtest macros
INSTANTIATE_TEST_SUITE_P is the drop-in replacement for
INSTANTIATE_TEST_CASE_P and will enable uprev of gtest.

See
https://github.com/google/googletest/releases/tag/release-1.10.0 for
full context.

BUG=chromium:1091244
Test: mma within hardware/interfaces/neuralnetworks

Change-Id: Ibb44a06f3e9a682847d47f64676b3bb7afdde1d9
2020-08-07 13:20:09 +01:00
Xusong Wang
3dca799497 Revert "Allow compilations in TOCTOU tests to fail with GENERAL_FAILURE."
This reverts commit 362dfd64d5.

Bug: 157489048
Test: 1.2/1.3 VTS TOCTOU tests
Change-Id: Ifdf4a8fab129cd90a3be019c3c598d980b785aaa
Merged-In: Ifdf4a8fab129cd90a3be019c3c598d980b785aaa
(cherry picked from commit 81fa7605cf)
2020-06-26 15:26:47 +01:00
Xusong Wang
0b617ae022 Allow compilations in TOCTOU tests to fail with GENERAL_FAILURE.
A compilation failure is not related to the security aspect of the
TOCTOU test, but it will skip one iteration of security testing. This CL
allows the compilation to fail with GENERAL_FAILURE in TOCTOU tests, and
issues a retry once it happens to ensure enough test coverage.

Bug: 157489048
Test: 1.2/1.3 VTS
Change-Id: Idc88e0365c5d2799187093b6fd7b4abf8f8b463d
Merged-In: Idc88e0365c5d2799187093b6fd7b4abf8f8b463d
(cherry picked from commit 362dfd64d5)
2020-06-26 15:26:40 +01:00
Slava Shklyaev
0fff59b961 Add control flow support to NNAPI VTS tests
See change I98a3edd1.

Bug: 148077633
Bug: 148601177
Bug: 136735929
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I1e436cdba404b68026a45797ac4fb3a34f8be76a
Merged-In: I1e436cdba404b68026a45797ac4fb3a34f8be76a
(cherry picked from commit 1f98e2e929)
2020-02-19 11:58:34 +00:00
Michael Butler
68a868b2ac NNAPI HAL: Remove priority from prepareModelFromCache_1_3
prepareModelFromCache_1_3 uses model cache and data cache that
represent a prepared model. Any argument that contributes in a
meaningful way to the prepared model is implicitly included in
this model cache and data cache. For example, "model" and
"executionPreference" appear in prepareModel_1_3 but not in
prepareModelFromCache_1_3 because they are implicitly included
in the model cache and data cache. In a similar way, because
it could affect the resulting model, "priority" should be
removed from prepareModelFromCache_1_3.

Fixes: 148802784
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I518e493ead8aa97220b16370cce8523b425c378c
Merged-In: I518e493ead8aa97220b16370cce8523b425c378c
(cherry picked from commit c90250ff57)
2020-02-06 11:38:08 +00:00
Michael Butler
79a41d77c0 Update NNAPI 1.3 VTS tests with new types
Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Test: atest VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ie76da9dc9d6993a56bf644cfe20c5f5b421672c9
Merged-In: Ie76da9dc9d6993a56bf644cfe20c5f5b421672c9
(cherry picked from commit 9449a28b2f)
2020-01-27 12:51:34 -08:00
Slava Shklyaev
f8124a861f Add NNAPI control flow
Bug: 136735929
Bug: 139181916
Test: m
Change-Id: I7a75175f00fc98df626c40ea669021ccd40130e0
Merged-In: I7a75175f00fc98df626c40ea669021ccd40130e0
(cherry picked from commit a785a3faac)
2020-01-27 12:51:09 -08:00
Xusong Wang
e9da9852a5 Add memory domain VTS generated tests.
Bug: 141353602
Bug: 141363565
Test: 1.3 VTS
Change-Id: Ifc7eb3fd6f15e28ba403f02bdf66b4568bddcb64
Merged-In: Ifc7eb3fd6f15e28ba403f02bdf66b4568bddcb64
(cherry picked from commit 1f50e54cf8)
2020-01-24 14:25:32 -08:00
Lev Proleev
9226c1e910 Add quantization coupling test
Quantization coupling test is ran only on models that contain only one
operator and at least one of its inputs has a type of
TENSOR_QUANT8_ASYMM. The test verifies that a model with all the
operands converted to TENSOR_QUANT8_ASYMM_SIGNED will produce the same
result (OK/SKIPPED/FAILED).

Bug: 137828994
Test: VtsHalNeuralNetworksV1_3TargetTest --gtest_filter="*QuantizationCouplingTest*"
Change-Id: I5a2e09b2b1bd790e7e37dde133554e516155a34d
Merged-In: I5a2e09b2b1bd790e7e37dde133554e516155a34d
(cherry picked from commit c88f4c60a8)
2019-11-21 12:06:34 +00:00
Xusong Wang
1b3f426648 NN HAL: Upgrade IPreparedModel::execute to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I9ca1e28ddc97fe880a72885afe7afb6c93903697
Merged-In: I9ca1e28ddc97fe880a72885afe7afb6c93903697
(cherry picked from commit 62a760c32d)
2019-11-19 12:18:41 -08:00
Xusong Wang
cc47dffa57 NN HAL: Upgrade IPreparedModelCallback::notify to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I56bc7a2fb179a9576036ad0c2aae0e1f41ec4e2c
Merged-In: I56bc7a2fb179a9576036ad0c2aae0e1f41ec4e2c
(cherry picked from commit 68c3234ad3)
2019-11-19 12:18:33 -08:00
Lev Proleev
26d1bc8a4e Modify NNAPI VTS tests to run on version 1.3
Bug: 139120468
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I4654dc75c17f8801103015dc1da91663dfa28d52
Merged-In: I4654dc75c17f8801103015dc1da91663dfa28d52
(cherry picked from commit b49dadfb64)
2019-10-17 18:34:51 +01:00
Lev Proleev
13fdfcd44f Copy VTS tests from v1.2 to v1.3
So that it's easier to see what actually has changed in VTS tests for
version 1.3

Bug: 139120468
Test: m
Change-Id: I09797f5f3898501a008186a22dd411b00e9e2c67
Merged-In: I09797f5f3898501a008186a22dd411b00e9e2c67
(cherry picked from commit 3b13b55ac1)
2019-10-17 18:34:51 +01:00
Lev Proleev
9613b4649b Revert "Copy VTS tests from v1.2 to v1.3"
This reverts commit d0e339e1da.

Reason for revert: the topic breaks git_qt-dev-plus-aosp

Change-Id: I68d31526cc7699b9a6c69a1ccc669fdb7605124e
2019-10-10 13:58:56 +00:00
Lev Proleev
64f9eb4bd5 Revert "Modify NNAPI VTS tests to run on version 1.3"
This reverts commit 543606fb92.

Reason for revert: the topic breaks git_qt-dev-plus-aosp

Change-Id: I74859bae687d65d344ed7edc617e1a7d086960a1
2019-10-10 13:54:38 +00:00
Lev Proleev
543606fb92 Modify NNAPI VTS tests to run on version 1.3
Bug: 139120468
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Id9e4d99852da8a3d5167ab7464c0e71885250501
Merged-In: Id9e4d99852da8a3d5167ab7464c0e71885250501
(cherry picked from commit b49dadfb64)
2019-10-09 14:18:49 +01:00
Lev Proleev
d0e339e1da Copy VTS tests from v1.2 to v1.3
So that it's easier to see what actually has changed in VTS tests for
version 1.3

Bug: 139120468
Test: m
Change-Id: Ief294d21349ca6531595612a16fa3ae3382f83ac
Merged-In: Ief294d21349ca6531595612a16fa3ae3382f83ac
(cherry picked from commit 3b13b55ac1)
2019-10-09 14:18:49 +01:00