At each step of a PLM implementation project, we all already encountered any « unexpected and undesirable features » (Also called bugs 🙂 ) due to imperfect tests.

It can be, for example, a regression of the functionalities or an error in the specifications.

That is project life, and we must avoid them as much as possible using various quality means: automated tests are part of them.

Automated tests are a concept allowing, well, to automate tests which were previously performed manually.

They can be started for example every night or after each release.

They are made using a tool which will allow you to:

  • Create test scripts to simulate some use cases of the PLM application
  • Manage the scripts
  • Execute the scripts
  • Maintain them in the future

They can bring an increase of the tests execution speed and can avoid some human errors.

Manual testing or automated testing?

There are criteria on what to test which will be presented later, but even if your tests match these criteria, there is one thing to never forget: The automated tests must be done on a mature and stable application. Otherwise, if there is an overhaul of the automated tested feature, you will have to completely rewrite your scripts.

By testing a mature and stable feature, you will just need a very soft maintenance of your script which will avoid losing the value you created.

All kind of tests are not automatable, the purpose is more to reduce the number of manual tests, especially in the PLM context where we have some very complex features performed by a few experts.

The best way to be efficient is to identify manual tests which could be automatized.

Moreover, the tester profiles will be very different between manual and automated tests: Manual tests will be prepared and executed by a “functional” consultant, while automated tests will be obviously assigned to “technical” consultants or developers. And remember that if you start some automated tests, you will need some preparation before you get results, in opposite with manual tests where you get some immediate results.

Quality manager, guardian of the value?

The quality manager is the stakeholder who will guarantee that the application, at each state of the project, will not lost value.

He must ensure the application meets all necessary requirements before it reach the consumer.

He has different important responsibilities:

  • Understand customer needs & requirements to develop effective quality control processes
  • Devising and reviewing specifications for products and processes
  • Setting requirements for raw material or intermediate products for suppliers and monitoring their compliance

Depending on project’s size, the Quality Manager will be responsible of a team of testers. The quality manager can also be (one of) the tester(s) in the context of small projects.

What should be tested?

One of the best ways to build a test plan covering all business needs is to initiate it from the beginning of the project. Identify the scenarios which will fit the best with the business needs and check how you can get the best value from it with the automated tests.

The 4 main criteria to decide that a test case could be automated are:

  • Test cases which are executed repeatedly
  • Test cases which are time-consuming
  • Critical application features
  • Business value

 

How does it work?

We are going to see in few steps how does the automated tests work.

First of all, you need to decide which testing tool will be the more efficient for your PLM application, then you need to define the scope of application testing and create the planning of your tests.

After that you will be ready to start the development and the test experimentation. When all your tests will be checked, you will be able to deploy and setup a relevant maintenance on your automated tests.

We can identify two kinds of tests: Human/Machine Interface tests & Server tests.

The HMI tests will be able to simulate user clicks and keyboard actions on the application interface.

The server tests will send requests to the server and compare the server’s answer to the expected answer specified in your script.

As we have two different kinds of tests, we can identify different advantages for each of them:

HMI tests advantages:

  • Consider the end-to-end user action
  • No needs to know the technical answer from the server

Server tests advantages:

  • Independent from HMI stability
  • Removes duplicate if the application is used with different platforms (Tablet, smartphone, computer)

When can I implement automated tests?

Most of the time, a relevant choice is to implement the automated tests as soon as the application is considered as stable & mature. This is one of the best ways to give as much value as possible to your project.

When the development has started you can also start to prepare the first test cases scenarios. And then, in parallel, you can start the automated tests development and the manual tests.

As test automation will require a high initial effort, its profitability will increase over time after many test campaigns. The interest of such automation will depend on how long you will be managing the PLM application (both in project and maintenance mode), at least over 2 years duration from our experience.

 

What are the links between Agile & Automated tests?

As we iterate faster thanks to “Agility”, there is more risks regarding the quality of the application.

To answer this problem, the combination of CI/CD (Continuous Integration, Continuous Delivery) pipeline, Agile, and Automated tests can be a very good solution. As CI/CD ensures continuous automation and monitoring throughout the process, it increases the frequency of automated tests distribution.

For our application and ours tests, it means more consistency, more speed, more tests, and therefore more quality.

How to maintain automated tests throughout the project?

The PLM application will evolve during the project, and the automated tests must be adapted. This is a very important part of the process which cannot be avoided.

It can be for example an HMI modification, a database modification, or access rights changes.

We recommend updating the automated tests during the sprint and consider it as a “user story” to be executed by the tester.

 

Author: Thibaut VANVLAENDEREN, Inensia PLM Consultant 

About Polina Todorova