Metamorphic Testing News Today

Simulation testing is considered the supplement to ensure the safety of Autonomous driving (AD) and advanced driving assistance (ADAS's) systems in terms of time and costs. However, it is very difficult and challenging when the simulation results are unexpected. This work presents a simulation-based metamorphic testing (MT) approach to test the ADAS system, implementing the European new car assessment program (Euro NCAP) standards on OpenStreetMap (OSM). We first defined input patterns and ...

The testing of large databases faces the test oracle problem, namely, that it is difficult to verify execution results against expected outcomes. Rigger and Su applied metamorphic testing through query partitioning and ternary logic partitioning techniques to alleviate the challenge. In Part (A) of our project, we conduct an in-depth investigation and have identified a gap between the two techniques. We propose a disjoint partitioning approach to address it. In Part (B), we conduct a comprehensi...

Concerns have been growing over fake news and its impact. Software that can automatically detect fake news is becoming more popular. However, the accuracy and reliability of such fake-news detection software remains questionable, partly due to a lack of testing and verification. Testing this kind of software may face the oracle problem, which refers to difficulty (or inability) of identifying the correctness of the software's output in a reasonable amount of time. Metamorphic testing (MT) h...

Owing to big data, DBMS testing faces the oracle problem, that is, it is difficult to verify execution results against expected outcomes. Rigger and Su applied metamorphic testing to alleviate the challenge. We propose a disjoint-partitioning approach to extend their work. We have conducted an empirical case study on OceanBase, the DBMS associated with the world's fastest online transaction processing system. Even though Ocean- Base has been extensively tested and widely used in the industr...

Context: Simulation-based testing is essential when developing Advanced Driver-Assistance Systems (ADASs) and autonomous driving (AD) systems, producing fast, high-quality test results, at relatively low cost. Simulation testing relies on the quality of the ADAS simulation platform: If the simulation platform is faulty, then the simulation results may be incorrect, and hence useless. However, because of the lack of suitable test oracles — mechanisms to determine the correctness of the software...

Mutation testing and property-based testing are two relatively niche technologies in the Java testers toolkit. Broad adoption is not recommended: they can make well-tested code even more robust but are a waste of time in projects with poor test maturity.

In this paper, we consider the testing of software whose input involves a data stream, such as in video analysis applications. To address the oracle problem in testing this kind of software, existing metamorphic testing (MT) and metamorphic exploration (ME) techniques typically generate both source and follow-up input streams. In contrast, we propose a different strategy: an inplace method. Instead of examining relations among multiple input streams, our method works on a finer granularity by in...

Named entity recognition (NER) is a widely used natural language processing technique; it plays a key role in information extraction from sentences. To be able to test the correctness of NER systems is important, but it is expensive because an automated test oracle is normally unavailable. To address the oracle problem, this study proposes to apply metamorphic testing (MT). The authors conduct a case study with Litigant, an industrial NER system of the Ant Group, and show that MT can effectively...

Metamorphic testing (MT) is recognized as an important quality assurance paradigm for complex systems. MT’s growing popularity is due not only to its effectiveness in alleviating the test oracle and test case generation problems but also to its unique perspective on software testing. The identification of effective metamorphic relations is vital to metamorphic testing. In addition, a testing platform that allows researchers and testers to automate the process of implementing metamorphic relati...

Advanced Driver-Assistance Systems (ADASs) have become increasingly popular. To ensure safety, simulation testing is essential. In this research, we conducted a case study to investigate the fault-detection effectiveness of existing ADAS testing standards: We tested the Lane Keeping Assist System (LKAS), which is a prebuilt ADAS module of MATLAB and Simulink. We first tested LKAS using the European New Car Assessment Programme (Euro NCAP), which contains 40 prebuilt driving scenarios in MATLAB. ...